The Lord of the Rings: The Rings of Power (Amazon) Encoding Quality, Codecs?

[HD MOVIE SOURCE]

Hi,

I've watched the new The Lord of the Rings: The Rings of Power first 2 episodes so far. I've watched them both 3 times on 3 different devices and got 3 different experiences.

One was a 4K Blu-ray Player, Panasonic UB9000. The Amazon app for the player bugs out when playing video using Dolby Vision. The colors flicker. This looks like a player LED or TV LED bug to me. Not sure who needs to fix this, Panasonic or Amazon, but it was unwatchable using Dolby Vision. The frame rates were also very inconsistent using the player even using a wired connection, so it's very poorly optimized using a UB9000. I switched to HDR10 only, and the picture is very good, but still suffer from the frame rate issue and cannot make use of Dolby Atmos so I switched over to my Roku.

The Roku, while the sound and picture aren't as good as the UB9000, at least is stable. I only played this using 1080p on my professionally calibrated 77-inch LGC9 because I don't think HDR looks good on a Roku. Fades and banding are more apparent, and for some reason look smoother using 8-bit SDR. We have Dolby Atmos, but on a Roku, the audio is -10db lower than my reference volume. Overall, a very good experience, but some encoding issues are visible...

--- When the woman and sun are caught in their house and looking through the cupboard. It goes black and there's artifacts all over the screen. I don't even think DVD is this poor, looked more like MPEG1. With that said, the bright scenes looked very good. Complex scenes (to me) looked slightly oversharpened or if I'm using x265 terms. RD and RDOQ is set too high in these scenes. The scene where they're picking snails. In some of the midrange shots, the foliage looks like it's having trouble. This creates inconsistency between shots.
Is Amazon encoding on a scene-by-scene basis? If they are then this might be why there is consistency between shots.
A complex water scene also broke down is partially visible artifacts, and color fades where Galadriel goes into the light on the ship has visible fading and banding.
Is this because at 1080p Amazon using different settings to 4K? Or the Roku Ultra can only use x264 at 1080p? If so that makes sense because the UB9000, even though it had issues looked better.
Bit-depth errors were also noticeable.


My 3rd experience was suggested to me by somebody on Twitter. To use LGC9's Amazon app, its much better than Roku and the Panasonic UB9000. I thought, Okay, let's give it a shot. We now have Dolby Vision without errors, and I audio-returned the Dolby Atmos back to my Denon X6500h. The experience here is excellent. No visible banding on fades, and the horrible-looking black screen artifacts issue was no longer an issue. Oversharppening (or extra detail) on encoder settings still creates a look of slight ringing around hard objects based on the scene. Most of the scenes are close-up shots with the backgrounds soft. These are very easy shots to encode and look gorgeous.



So, 3 different experiences. Is this what scaling of quality is? Are they using 3 different codecs? x264, x265, and my assumption for LG TVs there may be using the new codec AV1? If it's x265 I'd love to know what setting they're using because it's a pretty stellar job that Amazon has done with this show. As soon as a user selects 4K does that mean that they use a different codec? It's actually a shame it doesn't take into account screen size. Because 1080p on a 77inch (streaming) looks not so great, unfortunately.

I'd love to see this show released on 4K Ultra HD Blu-ray at 98 Mbps, going all out with the best settings. Now that would be an incredible experience. Because even though the stream on my 3rd time was very good, it still didn't come close to what a 4K disc could do.


If Amazon is encoding on a scene-by-scene basis, could they re-encode certain scenes? Like imagine scene 5, "number 10" scene looked poor. Could they literally just re-encode those frames with "better" settings, and when you rewatch that scene it now would look much better without having to re-encoding the entire episode?


Anyway, thanks for reading, did anyone else get to see the show and what do you think about their encoding settings? Do you notice anything that they were doing specifically like RD and RDOQ in scene-specific moments? My assumption is that (using x265 logic) sao was not used, intra smoothing was used, and I'm not too sure what fades does, but if it's as simple as it sounds, I assume there using that on the Dolby Vision versions of the streams that for sure. Probably a low qcomp like 0.6 or 0.5 was used because the first 2 episodes aren't that heavy in all-out action. However, if it's scene by scene, then when she climbs the ice rock, they probably used a 0.8 qcomp for that scene. I don't know if they're doing this, but it sounds pretty cool if they are.

Thanks.

[SquallMX]

They are using H265 at 18 Mbps for 4K HDR, and H264 (probably x264) at 7.3 Mbps for 1080p. Most of the time the quality problems are because they use very constrained VBR for better steaming compatibility.

[rwill]

They are using H265 at 18 Mbps for 4K HDR, and H264 (probably x264) at 7.3 Mbps for 1080p. Most of the time the quality problems are because they use very constrained VBR for better steaming compatibility.

Do you mean "bitrate close to vbv maxrate" with constrained? Wouldn't CBR be better in this case? Maybe finally implement Multipass CBR (for better frame size prediciton)?

[SquallMX]

Do you mean "bitrate close to vbv maxrate" with constrained? Wouldn't CBR be better in this case? Maybe finally implement Multipass CBR (for better frame size prediciton)?

For a time using AnyStream you could get the raw files with settings included (now this info is removed), for x264 they use something like --vbv-bufsize 8000 --max-bitrate 10000.

For x265 I dont remember the data for Amazon video files, for HBOMAX back them they used rc=abr / bitrate=18000 / vbv-maxrate=27000 / vbv-bufsize=27000.

Most streaming sites have several bitrates / settings for the same resolution for maximun compatibility across devices.

[excellentswordfight]

Hi,

I've watched the new The Lord of the Rings: The Rings of Power first 2 episodes so far. I've watched them both 3 times on 3 different devices and got 3 different experiences.

One was a 4K Blu-ray Player, Panasonic UB9000. The Amazon app for the player bugs out when playing video using Dolby Vision. The colors flicker. This looks like a player LED or TV LED bug to me. Not sure who needs to fix this, Panasonic or Amazon, but it was unwatchable using Dolby Vision. The frame rates were also very inconsistent using the player even using a wired connection, so it's very poorly optimized using a UB9000. I switched to HDR10 only, and the picture is very good, but still suffer from the frame rate issue and cannot make use of Dolby Atmos so I switched over to my Roku.

The Roku, while the sound and picture aren't as good as the UB9000, at least is stable. I only played this using 1080p on my professionally calibrated 77-inch LGC9 because I don't think HDR looks good on a Roku. Fades and banding are more apparent, and for some reason look smoother using 8-bit SDR. We have Dolby Atmos, but on a Roku, the audio is -10db lower than my reference volume. Overall, a very good experience, but some encoding issues are visible...

--- When the woman and sun are caught in their house and looking through the cupboard. It goes black and there's artifacts all over the screen. I don't even think DVD is this poor, looked more like MPEG1. With that said, the bright scenes looked very good. Complex scenes (to me) looked slightly oversharpened or if I'm using x265 terms. RD and RDOQ is set too high in these scenes. The scene where they're picking snails. In some of the midrange shots, the foliage looks like it's having trouble. This creates inconsistency between shots.
Is Amazon encoding on a scene-by-scene basis? If they are then this is why there is consistency between shots.
A complex water scene also broke down is partially visible artifacts, and color fades where Galadriel goes into the light on the ship has visible fading and banding.
Is this because at 1080p Amazon using different settings to 4K? Or the Roku Ultra can only use x264 at 1080p? If so that makes sense because the UB9000, even though it had issues looked better.
Bit-depth errors were also noticeable.

My 3rd experience was suggested to me by somebody on Twitter. To use LGC9's apps, they are much better than Roku to the Panasonic UB9000. I thought, Okay, let's give it a shot. We now have Dolby Vision without errors, and I audio returned Dolby Atmos back to my Denon X6500h. The experience here is excellent. No visible banding on fades, and the horrible-looking black screen artifacts issue was no longer an issue. Oversharppening (or extra detail) on encoder settings still creates a look of slight ringing around hard objects based on the scene. Most of the scenes are close-up shots with the backgrounds soft. These are very easy shots to encode and look gorgeous.

A bit to much to pick apart and try to answer, but it might be at least worth asking if you are sure that the TV are set to the exact same settings to all input-sources (also for SDR/HDR mode on each source)?

So, 3 different experiences. Is this what scaling of quality is? Are they using 3 different codecs? x264, x265, and my assumption for LG TVs there may be using the new codec AV1? If it's x265 I'd love to know what setting they're using because it's a pretty stellar job that Amazon has done with this show. As soon at the user selects 4K does that mean that they use a different codec? It's actually a shame it doesn't take into account screen size. Because 1080p on a 77inch (streaming) looks pretty poor.
If its HDR you can pretty much assume that its HEVC as I dont belive any premium streaming provider has implemented AV1 for HDR, and if they have different quality tiers for the HDR-ladder I would also assume that the same encoder is used (e.g. x265). I also believe that Amazon is indeed using x265 as their HEVC-encoder.

And yes, when you go from 1080p to 2160p there could be another codec, although it inst necessarily the case. It all depends om how many different formats are encoded for that title and at what resolutions they are available for. But I dont think i´ve read anything about Amazon using AV1 for Prime, so it could be the same as for HDR that you can assume HEVC for all 2160p encodes cause I doubt that the h264 encodes goes up that high.


I'd love to see this show release on 4K Ultra HD Blu-ray at 98 Mbps, going all out with the best settings. Now that would be an incredible experience. Because even though the stream on my 3rd time was very good, it still didn't come close to what a 4K disc could do.

Just wanna point out that just cause UHD-bluray allows for a max bitrate of 100Mbps for the video layer, that doesnt mean that UHD-bluray is at "98/100Mbps", from what i've seen the average is usually in the 50-60Mbps range as it still needs to fit an disc together with a bunch of audio and extras. Still a ton higher than the 15-25Mbps streaming is offering though.

[HD MOVIE SOURCE]

That is true, not all 4K Ultra HD Blu-rays are maxed out. But I've seen a few like Django 4K that is set to around 95 Mbps and look superb, so it certainly can be done if they're willing.

I think you guys are right that it's probably X265, and I'm very impressed with what they've done. I believe the 1080p stream could be x264.

I have seen that Amazon has a download feature, I wonder if that's only to a special folder or if I can actually place the file on a USB and play it from there? That would certainly be very cool.

Just an interesting thought, how big do you think the original files are? At 1200 Mbps per second and the show being about 1 hour that could put it at around 540GB. I'm just assuming it's about that big based on some of the uncompressed digital files I've seen from Netflix.

I appreciate all the feedback so far guys, Did Anyone notice any difference in RD and RDOQ from scene to scene? Maybe too high in foliage scenes?

[excellentswordfight]

Just an interesting thought, how big do you think the original files are? At 1200 Mbps per second and the show being about 1 hour that could put it at around 540GB. I'm just assuming it's about that big based on some of the uncompressed digital files I've seen from Netflix.

Probably Prores or jpeg2k in a IMF package. There are different compression levels for both codecs and bitrate modes are a bit variable so it hard to say exactly. It's most likely not uncompressed or even lossless, both codecs can do visually lossless compression though. I believe 12bit Jpeg2000 800Mbps max is rather common HDR-delivery master for UHD.

Those uncompressed/lossless files from netflix are not delivery masters, you can find netflix delivery specification here: https://drive.google.com/file/d/1_bwbUs4NaF7Y-07_NHwQqsL5L6zPVMPN/view & here https://drive.google.com/file/d/0B9DJydDVOVKKLVdCdlF2cFVDVEE/view?resourcekey=0-GRqfy963JM9jcd-MQcaoBw

edit.
Ultra HD Blu-rays are maxed out. But I've seen a few like Django 4K that is set to around 95 Mbps and look superb
Fair enough, but putting 90min titles on BD-100 discs are surely not that common, 90min movies in general is not that common now days anyway.

so it certainly can be done if they're willing
Well that depends, if the title is very long its not that much to done. Yes you can put fewer audio tracks and less extras (or put extras on a extra disc), but in the end you are working with physical media that has a set size limitation. Last movie I saw on UHD-Bluray was Dune with a runtime of two and half our and I think that title was about 65Mbps on a BD-100 disc, which are to be fair, absolutely fine. At least we are seeing the BD-100 discs to be more common, initially even long titles were put on the 66 discs, and I wouldnt be surprised if they still are in some cases.

[SquallMX]

I have seen that Amazon has a download feature, I wonder if that's only to a special folder or if I can actually place the file on a USB and play it from there? That would certainly be very cool.



Downloaded video files from PrimeVideo are encrypted, and usually are a lower quality tier (3.5-5 Mbps vs 7-9 Mbps).

You can use a free trial of AnyStream to download non encrypted video files from Amazon/Netflix/HBOMax. Trial includes up to 21 downloads. Full product is very pricey.

[HD MOVIE SOURCE]

Downloaded video files from PrimeVideo are encrypted, and usually are a lower quality tier (3.5-5 Mbps vs 7-9 Mbps).

You can use a free trial of AnyStream to download non encrypted video files from Amazon/Netflix/HBOMax. Trial includes up to 21 downloads. Full product is very pricey.

Okay, yeah I just saw that, yeah not worth it.

Just wondering, do you think Amazon encode using Placebo or higher?

[excellentswordfight]

Just wondering, do you think Amazon encode using Placebo or higher?
I doubt they use placebo, the diminishing returns are just to high for placebo, I doubt that you get ROI for all that extra compute time.

I guess that benwaggoner is occupied at IBC, he can probably answer some of your questions as he works for amazon. But given that he is a big promotor of preset slower, that might be a good guess for the baseline. And there are surly some streaming-optimized settings in there as well (e.g. --keyint 96 --min-keyint 96 --rc-lookahead 96 --no-open-gop & vbv limits).

[kolak]

Placebo ? They will use anything but placebo.
You seems to not get what all these services are about ?
They are definitely not about quality. It's all about profit and they do everything to minimise costs. Placebo is pointless preset anyway and used only for "scientific" needs.
With DVD/BD you typically fill disc(s), but studios sometimes even don't bother with this and create suboptimal encode.
It's all done as fast and as cheap as possible today. All what counts is $ (specially for big studios). Quality is fairly good as today's source are rather really good.
Footage from well shot movie with pro cameras like RED/Arri/Sony is rather quite clean and encodes very well.

[Boulder]

It's all done as fast and as cheap as possible today. All what counts is $ (specially or big studios).
This is very much true. Too often the best way to improve the quality is to simply sit farther away from the TV :devil:

[rwill]

It's all done as fast and as cheap as possible today. All what counts is $ (specially or big studios).

It is not all done as fast and cheap as possible, otherwise 'veryfast' style ~1Mbit UHD would be more common.

I think there is a relationship between quality, bitrate and encoding speed.

Lets just say Streaming Services try to hit the optimal point of:

1. Consumers continue to watch because your quality does not give them eye cancer.
2. You don't make a negative due to cost of CDN storage/traffic costs.
3. You don't make a negative due to expensive high compute encoding.

Cross-subsidizing not taken into account, this would too define what you have to/can charge a customer per month. And this makes money in the end.

[benwaggoner]

Alas, I know far too much about the questions discussed here to be allowed to actually answer them ;). I'll make sure people look into the OP's reported issues, which definitely sound like player issues, not encoding issues.

I can say I am extremely proud of the quality we're delivering with Rings of Power, and the effort and innovation the team (myself and others) delivered. As a personal career highlight, the only thing close in the last 33 years was designing the video systems for LEGO Mindstorms 1.0's software back in 1998.

Rest assured that the #1 leadership principle of Amazon is "Customer Obsession."

[benwaggoner]

It is not all done as fast and cheap as possible, otherwise 'veryfast' style ~1Mbit UHD would be more common.

I think there is a relationship between quality, bitrate and encoding speed.

Lets just say Streaming Services try to hit the optimal point of:

1. Consumers continue to watch because your quality does not give them eye cancer.
2. You don't make a negative due to cost of CDN storage/traffic costs.
3. You don't make a negative due to expensive high compute encoding.

Cross-subsidizing not taken into account, this would too define what you have to/can charge a customer per month. And this makes money in the end.
And bear in mind that more expensive encodes can allow for lower bitrates at a given quality level, which both saves on bandwidth costs at top bandwidth and improves quality at lower bandwidths. Since bandwidth costs are dominated by the highest bitrates, spending twice as long to make the top transparent quality stream 5% lower is a very good deal. And encoding costs are amortized across how many times a stream is viewed. If a show is going to be seen thousands of times, encoding costs become rapidly irrelevant. Encoding cost per title would be a much bigger deal for a YouTube, where most streams get single-digit views, and customers don't have "I paid for it" quality expectations.

The bigger concern around encoding times is finishing encoding in time if the source is only available a short time before the content goes live, like for day-after-broadcast TV. But most sources are available days or weeks before the content goes live.

[benwaggoner]

This is very much true. Too often the best way to improve the quality is to simply sit farther away from the TV :devil:
"Turn off the lights" is probably a bigger way, but yeah, distance/screen size ratio is really important. I'd say sitting closer is more often the challenge. Optimal viewing distance from a 4K with high quality content is around the screen size (6-8 feet from a 72" TV).

This can be uncomfortably close to some. The fundamental limit to viewing distance is that people won't let the viewing angles to the sides of the screen get distorted enough to reduce the overall viewing experience. So the fundamental limit on useful detail is determined by the maximum arc-seconds per pixel human vision can perceive. 4K was already close to that limit at acceptable viewing distances, which is why it is nearly impossible to tell the difference between native 4K and 8K encodes on a 8K screen.

Being able to sit closer with good side angles was the one good thing about curved screens ;).

[benwaggoner]

I get surprised by speculation about codecs and bitrates in a world where WireShark exists...

[benwaggoner]

I have seen that Amazon has a download feature, I wonder if that's only to a special folder or if I can actually place the file on a USB and play it from there? That would certainly be very cool.
You definitely cannot just copy an unencrypted download file, no ;)!

Just an interesting thought, how big do you think the original files are? At 1200 Mbps per second and the show being about 1 hour that could put it at around 540GB. I'm just assuming it's about that big based on some of the uncompressed digital files I've seen from Netflix.
Mezzanine files can range in size a lot depending on codec and some on content. For 4K, they start at around 600 Mbps and can go up from there. It's a big improvement from days past. A decade ago MPEG-2 mezzanines were quite common for HD, and even .VOB for longer tail SD.

[rwill]

I remember reading a survey among VOD sevices and they were concerned about bitrate and CDN storage size - not so much about quality. So I guess quality is solved, size is not. I cannot find the survey any longer with my Google skills though.

If money is secondary, encoding time is solved too. Splitting up a video into tiny 1 minute or shorter segments and encoding these in parallel can be done already. Brings a 2 hour movie down to around at least real time in high quality.

I get surprised by speculation about codecs and bitrates in a world where WireShark exists...

No Influencer I know has made a YouTube video about this yet. I have not seen the information being leaked on Twitter too. My associates, who are monitoring TikTok, have picked up nothing as well. So codecs and bandwidth remain a mystery.

[Blue_MiSfit]

Indeed. ProRes is the industry standard for mezz files, which is ~200 Mbps for HD and ~700-1500 Mbps for UHD (depending on format). J2K (usually IMF) can deliver similar or better quality at much lower bitrates.

[benwaggoner]

To be clear, I'm speaking my personal opinion on general principles unless I specify otherwise.

I remember reading a survey among VOD sevices and they were concerned about bitrate and CDN storage size - not so much about quality. So I guess quality is solved, size is not. I cannot find the survey any longer with my Google skills though.
I wouldn't say quality is solved per se. I can find visible artifacts on any service on at least some titles. In particular, encoding grainy content at bitrates load enough for wide distribution is still a work in progress. We're also seeing a lot of VMAF over fitting, where smooth areas and gradients wind up looking suboptimal even if more complex parts of the image are fine.

If money is secondary, encoding time is solved too. Splitting up a video into tiny 1 minute or shorter segments and encoding these in parallel can be done already. Brings a 2 hour movie down to around at least real time in high quality.
Blind segmenting by encoding segments completely independently risks VBV violations if adjoining segments have very high complexity at the stitch point, and you can get visible discontinuities if they happen mid-shot. There are ways around all of those, of course. I've got a couple patents on techniques for it.

No Influencer I know has made a YouTube video about this yet. I have not seen the information being leaked on Twitter too. My associates, who are monitoring TikTok, have picked up nothing as well. So codecs and bandwidth remain a mystery.
Interesting. A lot of that sort of data is available in the clear via Wireshark or whatever, and I'm kind of surprised people don't use those tools more to see what's going on.

People working for the different services generally have a better understanding of what competitors are doing, using those techniques, than I see broadly discussed.

And the systems are pretty darn complex and can evolve pretty rapidly. What codecs and bitstreams are given device will get can be very different from other devices, so it's really hard to generalize. And stuff can vary by content, and live versus VOD.

Even if Amazon gave me permission to try to explain our encoding as much as possible, it would be a whole lot of pages and a whole lot of followup questions I'd have to research myself. You can imagine how many different devices with different features we support. Heck, there are many thousands of unique Android devices alone used for video playback.

[benwaggoner]

Indeed. ProRes is the industry standard for mezz files, which is ~200 Mbps for HD and ~700-1500 Mbps for UHD (depending on format). J2K (usually IMF) can deliver similar or better quality at much lower bitrates.
And then there is ProRes 422 HQ (the baseline these days) all the way up to ProRes 4444 XQ.

I imagine we may start seeing JPEG XL mezzanines in a few years.

[rwill]

No Influencer I know has made a YouTube video about this yet. I have not seen the information being leaked on Twitter too. My associates, who are monitoring TikTok, have picked up nothing as well. So codecs and bandwidth remain a mystery.

Interesting. A lot of that sort of data is available in the clear via Wireshark or whatever, and I'm kind of surprised people don't use those tools more to see what's going on.

Ah I was being sarcastic there. You are right that tools are readily available to find all out about rates and codecs used. Most people I met don't want to bother with them just out of pure interest though and wait for the information to be relayed to them by means of 'someone else did the grunt work' - or on something like an easy to understand silver platter (Social Media).

[kolak]

It is not all done as fast and cheap as possible, otherwise 'veryfast' style ~1Mbit UHD would be more common.

I think there is a relationship between quality, bitrate and encoding speed.

Lets just say Streaming Services try to hit the optimal point of:

1. Consumers continue to watch because your quality does not give them eye cancer.
2. You don't make a negative due to cost of CDN storage/traffic costs.
3. You don't make a negative due to expensive high compute encoding.

Cross-subsidizing not taken into account, this would too define what you have to/can charge a customer per month. And this makes money in the end.

In other words:
Everything is done to minimise costs. You save on bitrate/size up to the point where it's acceptable by consumer (which for typical user is quite low). You won't find a case where it's about quality.
I think todays quality is ok, eg. DV on Netflix is rather good enough.

[Boulder]

"Turn off the lights" is probably a bigger way, but yeah, distance/screen size ratio is really important. I'd say sitting closer is more often the challenge. Optimal viewing distance from a 4K with high quality content is around the screen size (6-8 feet from a 72" TV).

This can be uncomfortably close to some. The fundamental limit to viewing distance is that people won't let the viewing angles to the sides of the screen get distorted enough to reduce the overall viewing experience. So the fundamental limit on useful detail is determined by the maximum arc-seconds per pixel human vision can perceive. 4K was already close to that limit at acceptable viewing distances, which is why it is nearly impossible to tell the difference between native 4K and 8K encodes on a 8K screen.

Being able to sit closer with good side angles was the one good thing about curved screens ;).
To be honest, I'd like to sit closer to the screen. I've got a 65" Sony and sit about 3,5 metres away. The difference between 4K and 720p/1080p is mostly in the perceived sharpness since I definitely don't have 20/20 vision.

If I lived alone, nothing would stop me :devil: Then again, I would probably have to re-encode a lot of my library in the media player domain as I'd notice any encoding issues too easily.

[HD MOVIE SOURCE]

Alas, I know far too much about the questions discussed here to be allowed to actually answer them ;). I'll make sure people look into the OP's reported issues, which definitely sound like player issues, not encoding issues.

I can say I am extremely proud of the quality we're delivering with Rings of Power, and the effort and innovation the team (myself and others) delivered. As a personal career highlight, the only thing close in the last 33 years was designing the video systems for LEGO Mindstorms 1.0's software back in 1998.

Rest assured that the #1 leadership principle of Amazon is "Customer Obsession."

I agree. The Rings of Power is the best quality content I've seen streamed. The quality is so high that I've been wanting to watch other things on Prime now. I honestly haven't taken Prime seriously in the past. I thought the quality was lower than Netflix (obviously), but, Netflix's quality lately has not impressed me. I watched the last season of Ozark and I'm not sure if they were adding grain to smooth areas, but it looked really bad.

After watching The Ring of Power I would consider Prime Video as the best quality streaming service. Now, I do appreciate that movies and maybe some TV Shows have not had the love that The Ring of Power has had. But, still.

Is this the first show to have Dolby Vision on Prime? I did notice superb fades, and color banding was non-existent on my professionally calibrated 77-inch OLED. I mostly watch 4K physical media and had no idea that Prime was capable of this level of quality. It's just a good job all around.

I'd love to see you encode this on 4K Ultra HD Blu-ray where you can use 98 Mbps and really go crazy 😉. Because it already looks stunning at these low bit-rates, and I assume some of the settings reduce high-frequency detail to reduce bit-rates. Very impressed, and keep up the great work.

[benwaggoner]

In other words:
Everything is done to minimise costs. You save on bitrate/size up to the point where it's acceptable by consumer (which for typical user is quite low). You won't find a case where it's about quality.
I think todays quality is ok, eg. DV on Netflix is rather good enough.
It's wildly inaccurate to say everything is just about costs. Providers could just do everything as 600 Kbps 480p AVC, and that would be very cheap ;)!

The user generated content market (YouTube, Facebook). is much more cost-constrained as the vast majority of uploads get single-digit views, and revenue per minute of content is lower (AFAIK).

Customers who pay for content expect good quality for what they paid, and all the streaming services do a lot of innovation and spend a huge amount of money on compute and on salaries to improve quality. OTT also has been leading innovation in new technology like UHD and HDR (Prime Video launched HDR almost a year before UHD Blu-ray). The cost of upgrading production and post to UHD-HDR was massive, all to bring customers better quality, and those investments started before there was huge customer demand for HDR. Netflix's tech blog describes a lot of interesting work, and similar things are happening across the industry, although most other companies don't talk about it in public as much as Netflix does.

The biggest challenge of quality is the lack of good, automated quality metrics that work at scale. When there are dozens of renditions (different codecs and bitrates) per title, having humans do manual QA for even 1% of renditions would be insanely cost prohibitive. If there was an easy button for "encode this with exactly as many bits as it takes to look great, and no more" everyone would use it! But there isn't one. We are lurching towards mediocrity in metrics; VMAF is the least-bad thing we've ever had. But VMAF doesn't even look at chroma! Doing automated optimization for VMAF would turn everything to black and white, among other limitations. I see issues resulting from VMAF over-tuning in all sorts of premium streaming content these days.

And "just crank the bitrate up to 100 Mbps" isn't viable, as most customers can't sustain 100 Mbps reliably for two straight hours. Sure if you have a 1 Gbps connection, live alone, don't use bandwidth for anything else while streaming, and have everything wired up with Gigabit ethernet, no problem. Well, at least for the last mile. Because CDNs and ISPs aren't tuned for that kind of sustained bandwidth for many simultaneous customers as well. Whatever the quality and bitrate of the top stream, lots of customers will get lower than that lots of the time, so getting good bang for the bit is important all the way down the bitrate ladder.

[benwaggoner]

I agree. The Rings of Power is the best quality content I've seen streamed. The quality is so high that I've been wanting to watch other things on Prime now. I honestly haven't taken Prime seriously in the past. I thought the quality was lower than Netflix (obviously)
Why obviously? Prime Video has been a "Prime" target for pirates for many years since my big refactoring project for 1080p (and below) SDR. HDR has been harder for everyone, as all existing metrics have lower subjective correlation in HDR than SDR, for reasons that are easier understood than resolved.

, but, Netflix's quality lately has not impressed me. I watched the last season of Ozark and I'm not sure if they were adding grain to smooth areas, but it looked really bad.
VMAF isn't magic, and too many seem to treat it as if it were ;). Its handling of smooth areas and grain aren't great, for example.

After watching The Ring of Power I would consider Prime Video as the best quality streaming service. Now, I do appreciate that movies and maybe some TV Shows have not had the love that The Ring of Power has had. But, still.
Thank you! I think it's the prettiest video I've worked on in my career. As the Zen of compression is that we do our job most perfectly when it appears we have done nothing, if you watch the top bitrate in Filmmaker Mode on a good TV, you are really seeing almost exactly the image the creatives watched in uncompressed when they made the content.

Is this the first show to have Dolby Vision on Prime? I did notice superb fades, and color banding was non-existent on my professionally calibrated 77-inch OLED. I mostly watch 4K physical media and had no idea that Prime was capable of this level of quality. It's just a good job all around.
There are other Dolby Vision titles on Prime Video. "Jack Ryan" is another popular title with it.

Getting rid of banding is so important (something else VMAF isn't that good at).

I'd love to see you encode this on 4K Ultra HD Blu-ray where you can use 98 Mbps and really go crazy 😉. Because it already looks stunning at these low bit-rates, and I assume some of the settings reduce high-frequency detail to reduce bit-rates. Very impressed, and keep up the great work.
The thing about bitrates is that not enough doesn't look like enough, but more than enough looks the same as enough. Once someone can't tell the difference between and encode and the source, more bits don't make a difference.

Blu-ray is limited by max 1 sec GOPs, max B-frames, and other limitations that reduce its efficiency to work around the limitations of spinning optical media. Streaming doesn't have those constraints, and so can get more bang for the bit. They key is breaking away from constraints that "everyone does that way" which don't need to be done that way. Particularly if one controls the player and encoding stacks, and don't need to ensure compatibility with every random HLS or DASH player out there.

[HD MOVIE SOURCE]

Thanks for all the feedback Ben, much appreciated. We noticed some observations tonight after watching episode 5.

We saw frame rate jumping tonight, and saw it about 5 times, I honestly don't know if it was source-based or Amazon was just under strain or we were having bad internet. I have a wired connection to my LGC9 and I have a 1000 Mbps connection. It measures around 960 Mbps. We've never seen frame rate stutters in any of the other episodes. Do you know if there were any frame rate-adjusted shots in episode 5? I don't have time stamps, but of them was when Galadriel was dueling. Some of the slow-mo shots had frame rate stutters that were not normal 24fps stutters.

I have a question about content access. Because the trailers are freely available on youtube and other video sites. Is it possible to get access to any of the uncompressed 4K trailers?

Main Teaser: https://www.youtube.com/watch?v=ewgCqJDI_Nk
SDCC Trailer: https://www.youtube.com/watch?v=uYnQDsaxHZU
Official Trailer: https://www.youtube.com/watch?v=x8UAUAuKNcU

Is that possible or not? YouTube quality is absolutely awful.

I'd like to encode them, even though they're trailers with a lot of scene cuts, if the quality is uncompressed at native 4K I bet these would look really good. I'm just looking for some content to encode, I'd like to see how close in detail I could get to the main show.

I'll take a look at some other dolby vision content soon, and see how it looks on Prime Video. I didn't know Prime had only just started Dolby Vision content. I don't typically stream, I'm a physical media guy mostly.

[QBhd]

I noticed the frame stutters as well! Very shocked since everything up to this point has been nearly flawless.

QB

[HD MOVIE SOURCE]

I noticed the frame stutters as well! Very shocked since everything up to this point has been nearly flawless.

QB

Yeah, not sure what it is. I'll check it out again whenever I can, if it's in the same places then it has to be the source file right, and not a networking issue.

[benwaggoner]

I'm not sure what you're seeing, but I'm positive it is not an encode issues. The more details you can share, the better for our investigation.

[HD MOVIE SOURCE]

I'm not sure what you're seeing, but I'm positive it is not an encode issues. The more details you can share, the better for our investigation.

If I get time I will take another look at the scenes with frame rate stuttering. It's possible it's an effect applied by post-production, but to my eyes it doesn't look right. I will try find the scenes for you.

[HD MOVIE SOURCE]

I'm not sure what you're seeing, but I'm positive it is not an encode issues. The more details you can share, the better for our investigation.

These are the time stamps for Sword Training

Entire Scene Starts at 21:44 (Sword Training) Episode 5

22:58 - she says not your arms. They fight, the camera scene cuts to above and the frame rate judders/drops every time. In the very next scene, she places the sword to his neck or chest with more frame rate judders. Looks like 15 frames per second to me.

23:42 - panning camera more frame drops.

24:02 - he attempts to strike, the camera pans with frame drops.

22:11 - The sword flies to the ground with panning frame drops.


That's just from this scene, I believe that there were at least 3 more that looked exactly like this in other scenes. If you want to get an idea of what it looks similar to. Think of playing a video game with no v-sync and you see frame Tears, similar to this, but heavier frame Tears.

[benwaggoner]

Excellent. And on what device(s) were you watching?

[TEB]

It's wildly inaccurate to say everything is just about costs. Providers could just do everything as 600 Kbps 480p AVC, and that would be very cheap ;)!

The user generated content market (YouTube, Facebook). is much more cost-constrained as the vast majority of uploads get single-digit views, and revenue per minute of content is lower (AFAIK).

Customers who pay for content expect good quality for what they paid, and all the streaming services do a lot of innovation and spend a huge amount of money on compute and on salaries to improve quality. OTT also has been leading innovation in new technology like UHD and HDR (Prime Video launched HDR almost a year before UHD Blu-ray). The cost of upgrading production and post to UHD-HDR was massive, all to bring customers better quality, and those investments started before there was huge customer demand for HDR. Netflix's tech blog describes a lot of interesting work, and similar things are happening across the industry, although most other companies don't talk about it in public as much as Netflix does.

The biggest challenge of quality is the lack of good, automated quality metrics that work at scale. When there are dozens of renditions (different codecs and bitrates) per title, having humans do manual QA for even 1% of renditions would be insanely cost prohibitive. If there was an easy button for "encode this with exactly as many bits as it takes to look great, and no more" everyone would use it! But there isn't one. We are lurching towards mediocrity in metrics; VMAF is the least-bad thing we've ever had. But VMAF doesn't even look at chroma! Doing automated optimization for VMAF would turn everything to black and white, among other limitations. I see issues resulting from VMAF over-tuning in all sorts of premium streaming content these days.

And "just crank the bitrate up to 100 Mbps" isn't viable, as most customers can't sustain 100 Mbps reliably for two straight hours. Sure if you have a 1 Gbps connection, live alone, don't use bandwidth for anything else while streaming, and have everything wired up with Gigabit ethernet, no problem. Well, at least for the last mile. Because CDNs and ISPs aren't tuned for that kind of sustained bandwidth for many simultaneous customers as well. Whatever the quality and bitrate of the top stream, lots of customers will get lower than that lots of the time, so getting good bang for the bit is important all the way down the bitrate ladder.

Ben, have u ever evaluated SSIMWave for objective quality analyzis?
They gave me a very good rundown on their "tech" on IBC and i was intrigued to actually buy a license for my lab. Both for Live and VOD.

Just saw that IMAX aquired them as well...

[TEB]

I'm not sure what you're seeing, but I'm positive it is not an encode issues. The more details you can share, the better for our investigation.

I have seen alot of frame-rate judder on Amazon Prime, and i also see this on all other streaming services we have on our STB, ott players etc..
It mostly comes down to the lack of a good framerate upsampler in the client/player used. Its really difficult to render 23.98fps on a 50hz TV.
Apple has "solved this" by switching then HDMI sync freq to match the source content and Android is playing the catchup game in ATV12 adding the same API's given that the 3rd party apps can correctly signal what framerate the upcoming playback will utilize.
Application support + HDMI 2.1 + QMS will finally kill this forever.. ;)

[QBhd]

Excellent. And on what device(s) were you watching?

I had the issue on my LG C8 internal Amazon App

QB

[benwaggoner]

I have seen alot of frame-rate judder on Amazon Prime, and i also see this on all other streaming services we have on our STB, ott players etc..
It mostly comes down to the lack of a good framerate upsampler in the client/player used. Its really difficult to render 23.98fps on a 50hz TV.
Apple has "solved this" by switching then HDMI sync freq to match the source content and Android is playing the catchup game in ATV12 adding the same API's given that the 3rd party apps can correctly signal what framerate the upcoming playback will utilize.
Application support + HDMI 2.1 + QMS will finally kill this forever.. ;)
Oh, yeah. On 1st party products like Fire TV we have an option to always set HDMI to the frame rate of the content being played. But there's lot of devices where an app can't control HDMI frame rate, and then it really is up to how well whatever is doing the conversion to do it right.

This is one upside of using a Smart TV, as everything happens under the same hood.

The PAL<>NTSC difference derived from different regions adapting different AC frequencies is definitely an original sin of video technology! I can hardly believe how much hassle it still causes in 2022. Quick summary here: https://en.wikipedia.org/wiki/Utility_frequency#Standardization

[benwaggoner]

Ben, have u ever evaluated SSIMWave for objective quality analyzis?
They gave me a very good rundown on their "tech" on IBC and i was intrigued to actually buy a license for my lab. Both for Live and VOD.

Just saw that IMAX aquired them as well...
SSIMWave has definitely done some innovative stuff that can be quite useful. Nothing that has become essential for my own work, but we have quite a few research scientists who can directly support our efforts.

It's hard for commercial products to scale up to Prime Video needs, as we encode vastly more hours of premium content than anyone else. We have lots of unique needs and opportunities that don't get well captured by products aiming for a broader range of customers.

[quietvoid]

Oh, yeah. On 1st party products like Fire TV we have an option to always set HDMI to the frame rate of the content being played.
I'm not sure if you're aware but even FireTV devices don't properly do integer refresh rates.
So while the OS says it's sending 24.0 fps content (such as Rings of Power), it's actually playing at 23.976.

At least that's for the 4K sticks, there are a lot of complaints about this on forums.

[benwaggoner]

I'm not sure if you're aware but even FireTV devices don't properly do integer refresh rates.
So while the OS says it's sending 24.0 fps content (such as Rings of Power), it's actually playing at 23.976.

At least that's for the 4K sticks, there are a lot of complaints about this on forums.
That's news to me. I use a Fire Cube personally, which seems to do it the right way.

[Boulder]

It was not so long ago that the AppleTV 4K also had the same problem, but it has since been fixed in some tvOS release. I guess that for some manufacturers, true 24p is too rare to worry about..

[benwaggoner]

It was not so long ago that the AppleTV 4K also had the same problem, but it has since been fixed in some tvOS release. I guess that for some manufacturers, true 24p is too rare to worry about..
And it requires all devices on the HDMI chain to support it.

I hope we can kill fractional frame rates during my lifetime. We've pretty much got interlaced deprecated for new premium content.

[HD MOVIE SOURCE]

Excellent. And on what device(s) were you watching?

LGC9 77inch OLED - Using Prime Video from the LG app store.

I don't think it's an issue with the app though because it played the first 4 episodes flawlessly, and I've watched other Prime content on here with flawless 24p playback. From my limited time encoding, I've never seen any encode where it manipulated frame sync or frame rate. Unless the encode wasn't using the source frame rate. But even then it was only in specific spots of the show that is replicable every time. I wonder if they did it in post-production to give more pressure to the scene? Or actually, is just a streaming bug?

I've thought of a movie that did this too, Mad Max Fury Road. It broke 24p playback in many scenes. My thoughts on this is that the native frame rate could not be pulled-down to 24p. I wonder if any scenes were filmed in a frame rate that was not divisible by 24?

[quietvoid]

In the latest FireTV Stick 4K Max update I just got, Dolby Vision CMv4.0 metadata seems to work now.
As Rings of Power's grading makes uses of that, now it works. Very cool. :)

[GEfS]

I don't know if it still applies to APV but APV has 2 video streams for (probably) each resolution. ABR/CBR and VBR. VBR is usually the better one. For most of the scene, VBR is better because it can pump double the average bitrate. And on lower bitrate scene where ABR get higher bitrate, it sometimes still look worse than VBR.
ABR stream is usually set at 10mbps for 1080p and VBR has results of 7-ish mbps and somewhat you can't tell which is which in normal APV player, you have to download it with some shady tool that pirate uses to rip the stream.

[HD MOVIE SOURCE]

The LGC9 actually just received an update. I haven't checked out the offending episode, but I just watched the most recent 2 episodes and frame rates were flawless. I'll try and check out episode 5 again I believe it was for FPS stutters.