Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > High Efficiency Video Coding (HEVC)

Reply
 
Thread Tools Search this Thread Display Modes
Old 1st February 2024, 04:05   #1  |  Link
sturmen
Registered User
 
Join Date: May 2016
Posts: 8
A Step By Step Guide To Encoding 3D Blu-rays To The Apple Vision Pro

Introduction

This is the workflow I've been using to convert my 3D Blu-ray collection to Apple's new MV-HEVC codec. As of Feb 12, on the Apple Vision Pro these can only be played back in the native Files and Photos applications, but I am optimistic that other, more powerful player apps will be released in the coming months.

Hardware Used
  • Windows PC with a processor capable of decoding MVC (I believe this means an Intel processor that contains an iGPU that supports QuickSync Video)
  • Apple Silicon Mac
  • Blu-ray drive (MakeMKV-compatible)
  • Portable SSD with like 500+ GB of space (optional but recommended)

Software Used
Steps
I recommend saving all files to an external SSD, and simply moving the drive between the computers at the section break.

Section 1: Windows
  1. Rip your Blu-ray with MakeMKV. Make sure to include the stereo video track! It is unchecked by default and needs to be manually selected.
  2. Use FFmpeg to extract the MVC bitstream and audio to separate files.
    Code:
    ffmpeg.exe -i 'Gravity_t00.mkv' -map 0:v -c:v copy -bsf:v h264_mp4toannexb "Gravity.h264" -map 0:a:0 -c:a pcm_s24le "Gravity_5.1_LPCM.mov"
  3. Use FRIM Decode to split the MVC video track into left and right streams. This will take three separate PowerShell windows. Execute the commands in the following order in their own window:
    Code:
    FRIMDecode64.exe -i:mvc Gravity.h264 -o \\.\pipe\lefteye \\.\pipe\righteye
    Code:
    ffmpeg -f rawvideo -s 1920x1080 -r 23.976 -pix_fmt yuv420p -i \\.\pipe\lefteye -c:v libx265 -x265-params keyint=1:min-keyint=1:lossless=1 -vtag hvc1 -movflags +faststart Gravity_left.mov
    Code:
    ffmpeg -f rawvideo -s 1920x1080 -r 23.976 -pix_fmt yuv420p -i \\.\pipe\righteye -c:v libx265 -x265-params keyint=1:min-keyint=1:lossless=1 -vtag hvc1 -movflags +faststart Gravity_right.mov
Section 2: Mac
  1. Combine left and right movies into an MV-HEVC stream.
    Code:
    spatial-media-kit-tool merge -l Gravity_left.mov -r Gravity_right.mov -q 50 --left-is-primary --horizontal-field-of-view 90 -o Gravity_MV-HEVC.mov
  2. (Optional) Transcode the audio to AAC to save space.
    Code:
    ffmpeg -i Gravity_5.1_LPCM.mov -c:a libfdk_aac -b:a 384k Gravity_5.1_AAC.mov
  3. Use MP4Box to remux the audio back into the MV-HEVC file.
    Code:
    mp4box -new -add Gravity_MV-HEVC.mov -add Gravity_5.1_AAC.mov Gravity_AVP.mov

Notes and Explanations, Presented in Q&A Format
  • Which eye is primary? This differs based on the source material. In FFmpeg, look at the input printout of the MKV file. You should see "stereo_mode" under Metadata. "block_lr" means left is primary, "block_rl" means right is primary.
  • What's a good quality value for -q? I don't actually have a good answer for this yet. q=50 is what Apple calls "medium quality" and it seems fine on my 5K Studio Display, but maybe compression artifacts will be glaringly obvious inside the Vision Pro. In my testing, q=50 yields about 2.5 GB per hour of 1080p stereoscopic video. Apple calls q=75 "high quality" and that yields approximately 4x bigger files, which are still about smaller than the size of the source H.264 MVC file.
  • Can this be done with only a Mac? I believe so: MakeMKV does support Macs, and I read about a tool called h264-tools that can decode the MVC stream into the separate files. I haven't used it though.
  • What if my film isn't 1080p at 23.976 fps? Change your FFmpeg commands accordingly. FRIMDecoder should print out the right resolution, dimension, and colorspace. (I420 is the same as yuv420p)
  • Why x265 lossless? I actually use hevc_nvenc lossless, but I didn't want this guide to rely on that. We don't want to use any lossy codecs until the very end, so that eliminates a bunch of codecs, and H.264/HEVC have the advantage of being directly readable by spatial-media-kit-tool, unlike other popular lossless codecs like FFV1 or HuffyYUV. And the final tiebreaker is: lossless H.264 doesn't play in QuickTime whereas HEVC does (useful for spot-checking), so HEVC is the winner.
  • Can spatial-media-kit-tool work on anything other than an Apple Silicon Mac? No. It's actually a pretty thin wrapper around the proper macOS APIs that actually do all the heavy lifting for encoding. We'll have to wait x265 to add MV-HEVC support before we'll see this on any other OS.
  • Why MOV containers? This is again driven by compatibility with the QuickTime framework, which spatial-media-kit-tool heavily relies on. The audio file will also carry through the original chapters from the original MKV into the final MP4box mux, which is nice.
  • Why libfdk_aac with a bitrate of 384 kbps? According to Wikipedia, 384 is "transparent" for 5.1. I've also encountered discs that have 7.1, where I figured 512 kbps ought to be enough. I use libfdk because it supports up to 7.1 channels. If you have 5.1, I recommend you try encoding with the aac_at encoder instead, as it's supposedly even better for the same bitrate.
  • Why MP4box for the final mux? FFmpeg does not preserve the MV-HEVC atoms. See the GitHub readme of SpatialMediaKit.

Last edited by sturmen; 12th February 2024 at 18:36.
sturmen is offline   Reply With Quote
Old 1st February 2024, 18:38   #2  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,817
Good stuff!

I note that one might as well set --keyint 1 for the lossless x265. Long GOP doesn't save that many bits in lossless, and IDR-only can encode a lot faster and decode somewhat faster.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 1st February 2024, 23:13   #3  |  Link
sturmen
Registered User
 
Join Date: May 2016
Posts: 8
Quote:
Originally Posted by benwaggoner View Post
Good stuff!

I note that one might as well set --keyint 1 for the lossless x265. Long GOP doesn't save that many bits in lossless, and IDR-only can encode a lot faster and decode somewhat faster.
As I mentioned in the appendix, I don't actually use libx265 so my memory on the syntax is a little hazy. It would be this, right?
Code:
-c:v libx265 -x265-params keyint=1:min-keyint=1:lossless=1
sturmen is offline   Reply With Quote
Old 3rd February 2024, 08:08   #4  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,817
I don't use ffmpeg for HEVC encoding myself, but it looks right.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 10th February 2024, 19:28   #5  |  Link
CleverTrousers
Registered User
 
Join Date: Feb 2024
Posts: 1
So after all that - what app will you use to play the files?
CleverTrousers is offline   Reply With Quote
Old 11th February 2024, 09:12   #6  |  Link
rwill
Registered User
 
Join Date: Dec 2013
Location: Berlin, Germany
Posts: 375
Quote:
Originally Posted by CleverTrousers View Post
So after all that - what app will you use to play the files?
I think the Apple Vision Pro comes with a Player. Or maybe Quicktime on an Apple Silicon device.
rwill is offline   Reply With Quote
Old 12th February 2024, 18:35   #7  |  Link
sturmen
Registered User
 
Join Date: May 2016
Posts: 8
Quote:
Originally Posted by CleverTrousers View Post
So after all that - what app will you use to play the files?
Right now, you can play them through the Files app, or the Photos app. Neither are ideal, as they don't give you a beautiful theater environment like Apple TV does. But it's early days. Personally, I'm waiting excitedly for Infuse and SKYBOX.
sturmen is offline   Reply With Quote
Old 13th February 2024, 01:16   #8  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,817
They got my files working, thanks for the tips.

And I can make stereo 3D movies now, at least for local playback, using the Apple API.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 15th February 2024, 23:44   #9  |  Link
kagami.h
Registered User
 
Join Date: Feb 2024
Posts: 4
Quote:
Originally Posted by sturmen View Post
Can this be done with only a Mac? I believe so: MakeMKV does support Macs, and I read about a tool called h264-tools that can decode the MVC stream into the separate files. I haven't used it though.
What about reference decoder JMVC? In theory it should be able to decode H.264 MVC.
kagami.h is offline   Reply With Quote
Old 17th February 2024, 01:47   #10  |  Link
sturmen
Registered User
 
Join Date: May 2016
Posts: 8
Quote:
Originally Posted by kagami.h View Post
What about reference decoder JMVC? In theory it should be able to decode H.264 MVC.
I've settled on this workflow that works for me, but if you get a Mac-only workflow going I'm sure there are people who would appreciate a writeup!
sturmen is offline   Reply With Quote
Old 20th February 2024, 11:32   #11  |  Link
vadlerg
Registered User
 
Join Date: May 2005
Posts: 15
On Windows PC BD3D2MK3D to convert 3D BDs or MKV to 3D SBS, TAB or FS MKV the best choice for me.
vadlerg is offline   Reply With Quote
Old 20th March 2024, 01:09   #12  |  Link
cbusillo
Registered User
 
Join Date: Mar 2024
Posts: 5
Converter tool

Hey, I took this method and wrapped it in an easy to use script. I've been using it for a week or so. If you notice any bugs or have any suggestions, please let me know! The goal was to make it as easy as possible for people and also create a script that could be used in automated environments.

https://github.com/cbusillo/BD_to_AVP
cbusillo is offline   Reply With Quote
Old 20th March 2024, 01:13   #13  |  Link
cbusillo
Registered User
 
Join Date: Mar 2024
Posts: 5
Quote:
Originally Posted by kagami.h View Post
What about reference decoder JMVC? In theory it should be able to decode H.264 MVC.
I found a Mac binary of this. While it did work, it was really slow. I found FRIMDecode64 through Wine to be three times faster.
cbusillo is offline   Reply With Quote
Old 25th March 2024, 16:58   #14  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,817
Quote:
Originally Posted by cbusillo View Post
I found a Mac binary of this. While it did work, it was really slow. I found FRIMDecode64 through Wine to be three times faster.
Yeah, reference decoders, like reference encoders, aren't designed for practical real-world performance.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 25th March 2024, 17:36   #15  |  Link
cbusillo
Registered User
 
Join Date: Mar 2024
Posts: 5
Quote:
Originally Posted by benwaggoner View Post
Yeah, reference decoders, like reference encoders, aren't designed for practical real-world performance.
I figured it was worth a try just in case it was as fast as Wine's translation layer. The process does work which is cool, I just figured it wasn't worth being Mac Native for 30% of the speed.
cbusillo is offline   Reply With Quote
Old 26th March 2024, 00:58   #16  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,817
Quote:
Originally Posted by cbusillo View Post
I figured it was worth a try just in case it was as fast as Wine's translation layer. The process does work which is cool, I just figured it wasn't worth being Mac Native for 30% of the speed.
As speed would be the main benefit of being Mac native, nope!

Interesting that Macs can decode MV-HEVC but not MV-AVC. Or...have you tried on M-series Mac? They added a lot of video functionality that wasn't in the x86-64 models.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 26th March 2024, 01:24   #17  |  Link
cbusillo
Registered User
 
Join Date: Mar 2024
Posts: 5
Quote:
Originally Posted by benwaggoner View Post
As speed would be the main benefit of being Mac native, nope!

Interesting that Macs can decode MV-HEVC but not MV-AVC. Or...have you tried on M-series Mac? They added a lot of video functionality that wasn't in the x86-64 models.
This was on my M2 Max. I guess there was no reason for Apple to support an old 3D format that seems to be only used on Blu-ray. MVC AVC up until now, would have been kind of pointless on the Apple hardware. It is neat that the 2D stream just works on non 3D stuff. This was the first time I really played with video encoding and decoding, so I learned a lot of basic stuff.

Edit: Ha, you literally wrote the book on compression and I just recently learned AVC is h264.
cbusillo is offline   Reply With Quote
Old 29th March 2024, 00:07   #18  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,817
Quote:
Originally Posted by cbusillo View Post
This was on my M2 Max. I guess there was no reason for Apple to support an old 3D format that seems to be only used on Blu-ray. MVC AVC up until now, would have been kind of pointless on the Apple hardware. It is neat that the 2D stream just works on non 3D stuff. This was the first time I really played with video encoding and decoding, so I learned a lot of basic stuff.
Quote:
Edit: Ha, you literally wrote the book on compression and I just recently learned AVC is h264.
So I did. Hard to believe the 2nd edition was fifteen years ago now. I keep hoping my publisher will ask me to do a third. Although I'm not sure what the market/audience would be like these days. In 2009 there were still a variety of commercial workstation compression tools. Now is much more automated in the cloud with a lot less scope for individual tuning.

It seems like it'd be easier to target an audience of 200 people who would all pay $1000 than for 4000 people who would pay $50. Of course, my employer may not want me to publish the sorts of things I know that people would pay $1000 for .

"Encoding and ladder design for premium content at global scale" wouldn't exactly leap off the shelves at Barnes & Noble.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 29th March 2024, 00:10   #19  |  Link
cbusillo
Registered User
 
Join Date: Mar 2024
Posts: 5
Or one guy that pays $200,000 for that content. That guy is Netflix. Amazon might not like that.
cbusillo is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 15:16.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.