Versatile Video Coding (VVC) / H.266: HEVC successor

[birdie]

VVC is not dead yet, Apple is hiring new codec engineers (https://www.linkedin.com/jobs/view/video-codec-algorithm-engineer-at-apple-2743151392).

Expert knowledge in video and image coding principles, algorithms, and techniques

Expert knowledge and experience in video compression standards, such as H.264, HEVC, VVC, VP9, AV1

Experienced in assessing visual quality using both objective metrics and subjective techniques

Excellent software design and debugging skills and solid programming skills in C/C++

Good written and oral communication skills

Familiarity with video processing algorithms such as scaling, noise reduction, tone mapping, etc would be a plus

Familiarity with the latest computer vision and deep learning technologies would be a plus

[birdie]

Oh, here's probably the first 8K content (https://8kassociation.com/industry-info/newsletter/globo-makes-8k-olympics-content-available-vod/) to be offered in VVC.

[FranceBB]

VVenC wants raw yuv not y4m.

If anyone wants y4m to be implemented, please comment and/or upvote this issue: https://github.com/fraunhoferhhi/vvenc/issues/65

[VoodooFX]

I'll post those once I get back to the computer.
Nice, I'm waiting for that source sample.

[FranceBB]

Nice, I'm waiting for that source sample.

The question is: where can I upload it? 'cause 200 frames in v416 from the capture card is over 12 GB. I've re-encoded to FFV1 lossless and now it's around 2GB which means it's gonna be enough to be uploaded on We Transfer.
There we go: https://we.tl/t-WTV68YjuzS

Disclaimer: this sample comes from a losslessly captured film and is meant to be used for academic purposes only like codec testing by engineers in this particular instance, every other use or share across the internet is strictly prohibited and punishable.

The link will expire in 7 days.


H.266 VVC encode in .h266 (there's also a decoded and re-encoded to HuffYUV lossless muxed in .mkv version for those poor souls who can't decode H.266 files directly): https://we.tl/t-gC5K1IGsJj

[VoodooFX]

The question is: where can I upload it? 'cause 200 frames in v416 from the capture card is over 12 GB.
I dunno much about it but few times I was sent huge masters through this site - www.myairbridge.com

[FranceBB]

I dunno much about it but few times I was sent huge masters through this site - www.myairbridge.com

Generally we use Aspera but that's for one side to another, i.e one company to another or one individual to another only. Nonetheless FFV1 is lossless anyway, so it's fine I guess.

[LigH]

WeTransfer may work, or MediaFire.

[birdie]

https://mega.nz/ is great as well. Doesn't have access to your files as they are encrypted before uploading using your key.

[VoodooFX]

Nonetheless FFV1 is lossless anyway, so it's fine I guess.
It's fine. I did few tests with XviD, that first frame looks much better - my 1pass XviD (https://images2.imgbox.com/0d/e3/nK4gakB7_o.png) vs your XviD (https://i.imgur.com/STfIWR4.png). What matrix did you use?
Here is whole file to check (CQM used: EQM v3HR): https://we.tl/t-p0TbcabcDb

EDIT:
What did you use to show that SSIM info on the frame?

[benwaggoner]

Great stuff @FranceBB!

HEVC and VVC are both approaching the SSIM saturation point, and it might be interested to redo the AVC/HEVC/VVC comparison at perhaps 10 Mbps CBR.

The greater variability of SSIM between VVC frames is also interesting. VVC has some impressive motion artifact suppression technologies that might score a lot better psychovisually than SSIM (which is fundamentally designed for still image comparisons, not video). Does the oscillation correlates with B-frames or something? Is it visible in full-speed playback?

[FranceBB]

Great stuff @FranceBB!

HEVC and VVC are both approaching the SSIM saturation point, and it might be interested to redo the AVC/HEVC/VVC comparison at perhaps 10 Mbps CBR.

Yep, re-doing the test at a much lower bitrate is what I also wanna do 'cause it will definitely highlight more things. The 25 Mbit/s were picked 'cause our current UHD playout live H.265 hardware encoder is using 25 Mbit/s for every content, so I wanted the test to be as realistic as possible.


Does the oscillation correlates with B-frames or something? Is it visible in full-speed playback?

What I've noticed is relatively interesting. So basically we all know that any encoder is gonna assign more bits to the I than it will to the P and B, however if we compare it with the H.265 stream, we can see that x265 is spreading those a bit more evenly than VVEnc is for H.266. Let me be more specific: basically, since the final encode is gonna be CBR, I can see how "large" each frame is gonna be in the GOP and what I've noticed is that in H.265 we have like: Intra large, P and B a bit smaller, while in H.266 we have like: Intra very large, P and B much smaller to have the same bitrate in the end in both encodes, which means that either VVEnc better detected the blocks and macroblocks across the scene and was able to use motion compensation in a better way OR it's using some of the new psychovisual features so that it "thinks" that it doesn't need as much bitrate, but it actually does. What I can say, though, is that in motion, if you look closely, you can see the difference as you look at the grain pattern in the back which appears to be blurrier than other frames in the exact same points in which the SSIM score drops, so it looks like something didn't go as expected, but from my experience I feel like this has something to do with the VVEnc implementation rather than H.266 itself. I'm pretty sure that we're gonna see the bitrate much more evenly spread (and much closer to how it's spread in x265) in x266 when it will become available.

[Jamaika]

Yep, re-doing the test at a much lower bitrate is what I also wanna do 'cause it will definitely highlight more things. The 25 Mbit/s were picked 'cause our current UHD playout live H.265 hardware encoder is using 25 Mbit/s for every content, so I wanted the test to be as realistic as possible.

What I've noticed is relatively interesting. So basically we all know that any encoder is gonna assign more bits to the I than it will to the P and B, however if we compare it with the H.265 stream, we can see that x265 is spreading those a bit more evenly than VVEnc is for H.266. Let me be more specific: basically, since the final encode is gonna be CBR, I can see how "large" each frame is gonna be in the GOP and what I've noticed is that in H.265 we have like: Intra large, P and B a bit smaller, while in H.266 we have like: Intra very large, P and B much smaller to have the same bitrate in the end in both encodes, which means that either VVEnc better detected the blocks and macroblocks across the scene and was able to use motion compensation in a better way OR it's using some of the new psychovisual features so that it "thinks" that it doesn't need as much bitrate, but it actually does. What I can say, though, is that in motion, if you look closely, you can see the difference as you look at the grain pattern in the back which appears to be blurrier than other frames in the exact same points in which the SSIM score drops, so it looks like something didn't go as expected, but from my experience I feel like this has something to do with the VVEnc implementation rather than H.266 itself. I'm pretty sure that we're gonna see the bitrate much more evenly spread (and much closer to how it's spread in x265) in x266 when it will become available.
I failed to produce ABR for 10Mbps. Wrong data in VVC come out. I found bitrate low. As a curiosity I added CBR QP 24 for yuv420p10le bt2020 VVC and EVC placebo. The obtained result for codecs with similar codec technology. EVC has twice bitrate in I/P/B frames. This would mean either artificially inflating the result to better advertise the EVC codec or bugs in the VVC codec. Unfortunately for 4K 10Mbps there is lot of grayscale pixel averaging.
https://i.ibb.co/0FTs0JW/wypapeq.png

[benwaggoner]

When a codec gets better bidirectional prediction, you'd expect the I/P to b ratio to get higher. Visually do you see any issues before or after?

SSIM isn't a particularly sensitive metric, and the same QP can look different in different codecs. At this early stage, we need to be relying on visual examination. If we have to use metrics, the newest version of VMAF is the best available for SDR content.

[birdie]

An ffmpeg fork with VVC decoding support: https://github.com/tbiat/FFmpeg

Not a huge fan since it's not exactly clear how it's been patched. Separate patches against the known ffmpeg git snapshot would be better.

[Jamaika]

An ffmpeg fork with VVC decoding support: https://github.com/tbiat/FFmpeg

Not a huge fan since it's not exactly clear how it's been patched. Separate patches against the known ffmpeg git snapshot would be better.
OpenVVC is a bit strange.https://github.com/OpenVVC/OpenVVC/commit/3371be46a526fa5f29ffdcf7836b5f096dabf895

dpb.c:1183:28: warning: passing argument 1 of 'pthread_mutex_lock' discards 'const' qualifier from pointer target type [-Wdiscarded-qualifiers]
1183 | pthread_mutex_lock(&decoded_ctus->ref_mtx);
| ^~~~~~~~~~~~~~~~~~~~~~
In file included from dec_structures.h:5,
from ovdec_internal.h:8,
from ovdpb.h:8,
from dpb.c:10:
c:\msys1200\x86_64-w64-mingw32\include\pthread.h:1039:69: note: expected 'struct pthread_mutex_t_ **' but argument is of type 'struct pthread_mutex_t_ * const*'
1039 | PTW32_DLLPORT int PTW32_CDECL pthread_mutex_lock (pthread_mutex_t * mutex);
| ~~~~~~~~~~~~~~~~~~^~~~~

drv.c:3:45: error: unknown type name 'VVCMV'
3 | fill_mvp_map(struct VVCMVCtx *const mv_ctx, VVCMV mv,
| ^~~~~
drv.c:17:114: error: unknown type name 'VVCMV'
17 | fill_dbf_mv_map_b(struct DBFInfo *const dbf_info, struct VVCMVCtx *const mv_ctx, struct VVCMVCtx *const mv_ctx1, VVCMV mv,
| ^~~~~
drv.c:57:80: error: unknown type name 'VVCMV'
57 | fill_dbf_mv_map(struct DBFInfo *const dbf_info, struct VVCMVCtx *const mv_ctx, VVCMV mv,
| ^~~~~
drv.c:93:8: error: unknown type name 'VVCMergeInfo'
93 | static VVCMergeInfo
| ^~~~~~~~~~~~
drv.c:94:14: error: unknown type name 'VVCLocalContext'
94 | derive_mvp_b(VVCLocalContext *const lc_ctx,
| ^~~~~~~~~~~~~~~
drv.c:95:20: error: unknown type name 'VVCPartSize'
95 | const VVCPartSize *const part_ctx,
| ^~~~~~~~~~~
drv.c:98:14: error: unknown type name 'VVCMV'
98 | VVCMV mvd0, VVCMV mvd1,
| ^~~~~
drv.c:98:26: error: unknown type name 'VVCMV'
98 | VVCMV mvd0, VVCMV mvd1,
| ^~~~~
drv.c:99:14: error: unknown type name 'uint8_t'
99 | uint8_t mvp_idx0, uint8_t mvp_idx1,
| ^~~~~~~
drv.c:1:1: note: 'uint8_t' is defined in header '<stdint.h>'; did you forget to '#include <stdint.h>'?
+++ |+#include <stdint.h>
1 |
drv.c:99:32: error: unknown type name 'uint8_t'
99 | uint8_t mvp_idx0, uint8_t mvp_idx1,
| ^~~~~~~
drv.c:99:32: note: 'uint8_t' is defined in header '<stdint.h>'; did you forget to '#include <stdint.h>'?
drv.c:100:14: error: unknown type name 'uint8_t'
100 | uint8_t inter_dir)
| ^~~~~~~
drv.c:100:14: note: 'uint8_t' is defined in header '<stdint.h>'; did you forget to '#include <stdint.h>'?
drv.c:147:17: error: unknown type name 'VVCLocalContext'
147 | update_mv_ctx_b(VVCLocalContext *const lc_ctx, struct VVCInterCtx *const inter_ctx,
| ^~~~~~~~~~~~~~~
drv.c:148:23: error: unknown type name 'VVCPartSize'
148 | const VVCPartSize *const part_ctx,
| ^~~~~~~~~~~
drv.c:149:23: error: unknown type name 'VVCMV'
149 | const VVCMV mv0, const VVCMV mv1,
| ^~~~~
drv.c:149:40: error: unknown type name 'VVCMV'
149 | const VVCMV mv0, const VVCMV mv1,
| ^~~~~
drv.c:152:17: error: unknown type name 'uint8_t'
152 | uint8_t inter_dir)
| ^~~~~~~


nvcl_hrd.c:5:1: error: expected '=', ',', ';', 'asm' or '__attribute__' before '{' token
5 | {
| ^
nvcl_hrd.c:11:3: warning: data definition has no type or storage class
11 | } OVSubLayerHRD;
| ^~~~~~~~~~~~~
nvcl_hrd.c:11:3: warning: type defaults to 'int' in declaration of 'OVSubLayerHRD' [-Wimplicit-int]
nvcl_hrd.c:36:50: error: unknown type name 'subLayerId'
36 | sublayer_hrd_parameters(OVNVCLReader *const rdr, subLayerId)
| ^~~~~~~~~~
nvcl_hrd.c:55:52: error: unknown type name 'firstSubLayer'
55 | ols_timing_hrd_parameters(OVNVCLReader *const rdr, firstSubLayer, MaxSubLayersVal)
| ^~~~~~~~~~~~~
nvcl_hrd.c:55:67: error: unknown type name 'MaxSubLayersVal'
55 | ols_timing_hrd_parameters(OVNVCLReader *const rdr, firstSubLayer, MaxSubLayersVal)
| ^~~~~~~~~~~~~~~
nvcl_hrd.c:82:1: warning: return type defaults to 'int' [-Wimplicit-int]
82 | general_timing_hrd_parameters(OVNVCLReader *const rdr)
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
nvcl_hrd.c: In function 'general_timing_hrd_parameters':
nvcl_hrd.c:85:5: error: 'ghrd' undeclared (first use in this function)
85 | ghrd->num_units_in_tick = nvcl_read_bits(rdr, 32);
| ^~~~
nvcl_hrd.c:85:5: note: each undeclared identifier is reported only once for each function it appears in

[ksec]

VVenC 1.2.0 released

https://github.com/fraunhoferhhi/vvenc/releases/tag/v1.2.0

major efficiency improvement for faster, minor speedups for fast and medium
allowing separate encoding of first and second pass in 2pRC (using a statistics data file)
improvements to single picture and low-rate rate-control
added support for packed 10 bit YUVs
added a balanced IDR refresh mode (using IDR_W_RADL in first intra period)
improvements to ALF, DMVR, error handling and others
various fixes and cleanups

[birdie]

VVenC 1.2.0 released

https://github.com/fraunhoferhhi/vvenc/releases/tag/v1.2.0

And we still have zero players. I've only found VQProbe (https://vicuesoft.com/vq-probe) but it's an analyzer, not a player, so I've no idea whether it can play audio or play in full screen mode.

[Balling]

An ffmpeg fork with VVC decoding support: https://github.com/tbiat/FFmpeg

Not a huge fan since it's not exactly clear how it's been patched. Separate patches against the known ffmpeg git snapshot would be better.

There is also this. https://patchwork.ffmpeg.org/project/ffmpeg/list/?series=3487

[birdie]

There is also this. https://patchwork.ffmpeg.org/project/ffmpeg/list/?series=3487

This patchset allows ffmpeg to recognize VVC bitstream data, it doesn't contain an actual decoder implementation either directly or though an external library. IOW it's useless.

[contemporarymind]

And we still have zero players. I've only found VQProbe (https://vicuesoft.com/vq-probe) but it's an analyzer, not a player, so I've no idea whether it can play audio or play in full screen mode.

There's a couple proof-of-concept players by now based on vvdec, even for the browser. no audio though...

https://github.com/bitmovin/vvDecPlayer
https://github.com/fraunhoferhhi/vvdecWebPlayer

[Balling]

This patchset allows ffmpeg to recognize VVC bitstream data, it doesn't contain an actual decoder implementation either directly or though an external library. IOW it's useless.

The trace_headers implementation is always the first one! There is no point to implement presenter if bitstream parser is not done. same for AV1 (there is a very simple low overheader parser in ffmpeg not used for decoding) and also there is a parser of HDR10+, no support for presenting HDR10+ yet.

[birdie]

So, MulticoreWare now lists x266 encoder (https://multicorewareinc.com/what-we-deliver/video-codecs/x266-vvc-encoder/) among its products, yet its git repo (https://bitbucket.org/multicoreware/x266/src/master/) continues to be members only, what's going on? Have I missed something?

[Dann0245]

So, MulticoreWare now lists x266 encoder (https://multicorewareinc.com/what-we-deliver/video-codecs/x266-vvc-encoder/) among its products, yet its git repo (https://bitbucket.org/multicoreware/x266/src/master/) continues to be members only, what's going on? Have I missed something?

They listed on their website many months ago.

I don't know but they seem to need more time for the first version of x266 than x265, and there is no any player support yet.

[birdie]

The latest MSU codecs comparison: https://compression.ru/video/codec_comparison/2021/main_report.html at FullHD.

TLDR: H.266 shines except for high-speed/RT encoding where H.265 implementations are still better.

Looks like you can test their codec for free: https://www.huaweicloud.com/intl/en-us/product/mpc.html - I want to try this.

[Selur]

So the conclusion of that comparision is that in theory you get the best qulits with these three encoders: HW266, S266_vX, Tencent266 but since they are closed source and not available to try or buy you can't use them.

[benwaggoner]

They listed on their website many months ago.

I don't know but they seem to need more time for the first version of x266 than x265, and there is no any player support yet.
The complexity jump in new tools is a lot bigger in absolute terms than from H.264 to H.265, so that's more new code to write. Relative complexity jump is about on par.

[MoSal]

So the conclusion of that comparision is that in theory you get the best qulits with these three encoders: HW266, S266_vX, Tencent266 but since they are closed source and not available to try or buy you can't use them.

On the other hand, it's the first time I hear about xin26x which could develop into a good encoder suite if the developers ever get to a stage where some effort is spent on subjective tuning.

[RanmaCanada]

So the conclusion of that comparision is that in theory you get the best qulits with these three encoders: HW266, S266_vX, Tencent266 but since they are closed source and not available to try or buy you can't use them.

Sadly it was the same thing with HEVC when it was being tested. You can't buy literally 90% of the codecs or encoders they used, so it's geared more towards industrial aspects like security camera or broadcasting.

[foxyshadis]

What I've noticed is relatively interesting. So basically we all know that any encoder is gonna assign more bits to the I than it will to the P and B, however if we compare it with the H.265 stream, we can see that x265 is spreading those a bit more evenly than VVEnc is for H.266. Let me be more specific: basically, since the final encode is gonna be CBR, I can see how "large" each frame is gonna be in the GOP and what I've noticed is that in H.265 we have like: Intra large, P and B a bit smaller, while in H.266 we have like: Intra very large, P and B much smaller to have the same bitrate in the end in both encodes, which means that either VVEnc better detected the blocks and macroblocks across the scene and was able to use motion compensation in a better way OR it's using some of the new psychovisual features so that it "thinks" that it doesn't need as much bitrate, but it actually does. What I can say, though, is that in motion, if you look closely, you can see the difference as you look at the grain pattern in the back which appears to be blurrier than other frames in the exact same points in which the SSIM score drops, so it looks like something didn't go as expected, but from my experience I feel like this has something to do with the VVEnc implementation rather than H.266 itself. I'm pretty sure that we're gonna see the bitrate much more evenly spread (and much closer to how it's spread in x265) in x266 when it will become available.
Rate control in general always seems to be one of the biggest bugbears of any prototype encoder. SSIM is a flawed measure but oscillating that much is absolutely going to be visible, the same way Xvid and early versions of x264 worked. It's a painful amount of iteration, trying to optimize for a whole spectrum of use cases and completely unknown upcoming features, so I can understand why it doesn't get as much love as say, a new SIMD optimization that can be directly benched against the C as an immediate improvement.

[benwaggoner]

Rate control in general always seems to be one of the biggest bugbears of any prototype encoder. SSIM is a flawed measure but oscillating that much is absolutely going to be visible, the same way Xvid and early versions of x264 worked. It's a painful amount of iteration, trying to optimize for a whole spectrum of use cases and completely unknown upcoming features, so I can understand why it doesn't get as much love as say, a new SIMD optimization that can be directly benched against the C as an immediate improvement.
Yep. Rate Control is one of the most complex and nuanced features of real-world encoders. It's got a lot of psychovisual aspects, not just pure engineering. And choices made in each frame can propagate through the rest of the GOP, and getting that right is super complex. One of the secret weapons of x264 was cu-tree (ctu-tree in x265)

[Dann0245]

I hope Multicoreware could talk more about the current state of x266 encoder at IBC 2021!

https://multicorewareinc.com/news/meet-us-at-ibc-2021/

or even better, release x266 0.1 version.

[birdie]

MediaTek has announced Pentonic 2000 SoC (https://www.prnewswire.com/news-releases/mediatek-announces-new-pentonic-smart-tv-family-with-new-pentonic-2000-for-flagship-8k-120hz-tvs-301428273.html) which supports VVC. Finally!

[Jamaika]

For me this is the paranoia of the techno of the Asiatic world. Buy 8K 55 "- 98" for your street business as an advertisement. There are no winters so knock the wall over the TV. 4K magnified to 8K has pixels. Oh dear, then we'll add neo qled or evo oled so as not to scare the client off the street.
What for a home user 4K below 48 ". It does not matter most of this technology. Will the creators change old movies on vvc? They will not. Here only fans of mass events benefit.
What about professional 8K cameras when webcam or youtube reigns? For VIPs.
https://www.samsung.com/pl/tvs/qled-tv/highlights/
https://www.lg.com/pl/oled-tv/2021/self-lit-pixels?cmpid=2021HQSEM_OLED-TV_PL_Google_OLED_Core-mid_Broad_OLED_k0002-added0611_pc_2021&gclid=EAIaIQobChMI5Iv6s_mm9AIVh94YCh2dqwY5EAAYAiAAEgLi7PD_BwE&gclsrc=aw.ds
I don't know what the Panansonic has in hiding, but in Central Europe it isn't taken into account.
Sony and Sharp used to be the leaders. Where are they on the silk road?

[Dann0245]

To play VVC file with VLC.

Download O266 lib, libo266dec.dll file (by tencent): https://multimedia.tencent.com/resources/vvc-player

Put libo266dec.dll file into dir: VLC\plugins\codec

then play *.266 file

But you may see the framerate is incorrect, you need encode with new VVenC build, which has have new --fps switch for exact framerate.

[benwaggoner]

For me this is the paranoia of the techno of the Asiatic world. Buy 8K 55 "- 98" for your street business as an advertisement. There are no winters so knock the wall over the TV. 4K magnified to 8K has pixels. Oh dear, then we'll add neo qled or evo oled so as not to scare the client off the street.
What for a home user 4K below 48 ". It does not matter most of this technology. Will the creators change old movies on vvc? They will not. Here only fans of mass events benefit.
Optical disc development ended with UHD Blu-ray, so 4K HDR HEVC. But streaming services absolutely adopt new codecs when they have sufficient installed base and compression efficiency improvements to merit it. And encoding is done from a high quality mezzanine, so the content creators don't need to do anything for customers to get the benefit of a new codec.

We won't be watching much H.264/HEVC/VP9 in 10 years!

What about professional 8K cameras when webcam or youtube reigns?[/QUOTE]
Used heavily for premium content creation. YouTube is a slice of the market, but user-generated content as a business pales in comparison to that for premium TV and movies.

[FranceBB]

What about professional 8K cameras when webcam or youtube reigns?

Reign where? Red Monstro 8K cameras have been around for quite some time now ehehehehe
I mean, a major licensors can afford to spend 60k for the camera corpse and a bit more for the lenses etc.
Over here (and keep in mind that we're broadcasters, so our productions are smaller - although we have made some documentaries and tv series and we might have a surprise for Christmas) we're still living with 4K as we're largely based on Canon C300 for documentaries and Arri Alexa LF for TV Series, so I think the major licensors, the big names that populate the market, would have no problem spending the extra buck, going the extra mile and start producing 8K contents soon-ish.

[benwaggoner]

Reign where? Red Monstro 8K cameras have been around for quite some time now ehehehehe
I mean, a major licensors can afford to spend 60k for the camera corpse and a bit more for the lenses etc.
Over here (and keep in mind that we're broadcasters, so our productions are smaller - although we have made some documentaries and tv series and we might have a surprise for Christmas) we're still living with 4K as we're largely based on Canon C300 for documentaries and Arri Alexa LF for TV Series, so I think the major licensors, the big names that populate the market, would have no problem spending the extra buck, going the extra mile and start producing 8K contents soon-ish.
Do you mean shooting in 8K and post in 4K, or 8K through to a mezzanine file?

Shooting in 8K, sure. That's been happening for a while, and a number of movies and TV shows shot with 8K cameras are already available.

Post and distribution in 8K is a whole other matter, and I'm not aware of any real-world titles using it or even seriously contemplating doing so. The extra costs far outweigh any potential added market value. 4x more pixels mean new monitors (and there aren't any DisplayPort 8K HDR monitors), 4x the storage, 4x the RAM, 4x longer rendering, projectors can't be used for previews, and a whole lot of other hassles. Plus, since 4K and 8K look identical for the 99.9% of premium content that's at 24p, determining that 8K is actually working is a huge challenge.

Valuable 8K is a lot more likely at 120p. But that's pushed the delta up to 20x storage, 20x render time, another generation of studio monitors, etcetera. Moore's Law is a wonderful thing, so even if 20x is impossible today it'll be no big deal in 10-12 years.

[excellentswordfight]

Do you mean shooting in 8K and post in 4K, or 8K through to a mezzanine file?

Shooting in 8K, sure. That's been happening for a while, and a number of movies and TV shows shot with 8K cameras are already available.

Post and distribution in 8K is a whole other matter, and I'm not aware of any real-world titles using it or even seriously contemplating doing so. The extra costs far outweigh any potential added market value. 4x more pixels mean new monitors (and there aren't any DisplayPort 8K HDR monitors), 4x the storage, 4x the RAM, 4x longer rendering, projectors can't be used for previews, and a whole lot of other hassles. Plus, since 4K and 8K look identical for the 99.9% of premium content that's at 24p, determining that 8K is actually working is a huge challenge.

Valuable 8K is a lot more likely at 120p. But that's pushed the delta up to 20x storage, 20x render time, another generation of studio monitors, etcetera. Moore's Law is a wonderful thing, so even if 20x is impossible today it'll be no big deal in 10-12 years.
The part that made Moore's Law such a wonderful thing is dead, and that is that it factored in price. Yes we are still seeing an rather impressive increase in transistor count, but it now comes at a huge price as the cost per transistor for modern advanced nodes is nowhere near what Moore described. This is extra obvious when you look at GPUs (even if we exclude the inflated prices atm), when moor's law was alive and well you could pretty much just wait for semiconductor improvements and get a huge performance increase from just the transistor increase alone at the same manufacturing cost, now we even see new nodes with worse transistor price then prior generations, we can just expect the prices to continue to climb if we want the performance keep scaling from transistors alone.

There are ofc other tech areas that improves on this in other ways, just look at what apple is doing with their silicon and ecosystem, and what you are able to do on those machines in terms of video compared to what was needed for that a couple of years ago. Not that their more recent chips are not impressive when it comes to transistors as well, I think that Apple is in a rather unique situation here; they are so profitable elsewere and has such a huge volume so it wouldnt surprise me if they actually have relatively low margins on these parts.

[FranceBB]

just look at what apple is doing with their silicon and ecosystem, and what you are able to do on those machines in terms of video compared to what was needed for that a couple of years ago. Not that their more recent chips are not impressive when it comes to transistors as well, I think that Apple is in a rather unique situation here; they are so profitable elsewere and has such a huge volume so it wouldnt surprise me if they actually have relatively low margins on these parts.

What is Apple doing? They're largely based on ARM CPUs with x86 emulation. Even if the emulation is well done, they will never ever be as efficient as running an x86 application natively and so far ARM isn't really good for video encoding, it just isn't.

[benwaggoner]

The part that made Moore's Law such a wonderful thing is dead, and that is that it factored in price. Yes we are still seeing an rather impressive increase in transistor count, but it now comes at a huge price as the cost per transistor for modern advanced nodes is nowhere near what Moore described. This is extra obvious when you look at GPUs (even if we exclude the inflated prices atm), when moor's law was alive and well you could pretty much just wait for semiconductor improvements and get a huge performance increase from just the transistor increase alone at the same manufacturing cost, now we even see new nodes with worse transistor price then prior generations, we can just expect the prices to continue to climb if we want the performance keep scaling from transistors alone.

There are ofc other tech areas that improves on this in other ways, just look at what apple is doing with their silicon and ecosystem, and what you are able to do on those machines in terms of video compared to what was needed for that a couple of years ago. Not that their more recent chips are not impressive when it comes to transistors as well, I think that Apple is in a rather unique situation here; they are so profitable elsewere and has such a huge volume so it wouldnt surprise me if they actually have relatively low margins on these parts.
Yeah, no doubt that Moore's Law predictions have changed a bunch. We are still seeing good compute/$ improvements over time, but it comes from an aggregation of improvements, not just more transistors and higher clock speeds anymore.

For the film industry, it can be as much about what's possible as how much it costs. Lots of tools that can do 4Kp24 today simply don't have any way to scale to 8Kp120 at any costs. You can squint and imagine how a cluster of beefy servers chock full of a6000 GPUs and clever tiling software could do the same thing. But unless that software and hardware is available, it just isn't possible. And regressing from real-time previews to "I'm going to render the shot now, should be done after lunch" is a huge deal.

It's the same way you can't buy a gaming rig that can do 8Kp120 for 8x (or 80x) the cost of one that can do 4Kp60. There is only so much parallelism that can be taken advantage of by the underlying software architecture before an Amdahl's law limit gets hit. SLI was never 2x the perf, nor was TLI.

[excellentswordfight]

What is Apple doing? They're largely based on ARM CPUs with x86 emulation. Even if the emulation is well done, they will never ever be as efficient as running an x86 application natively and so far ARM isn't really good for video encoding, it just isn't.
I'm referring to transistors here, were Apple has an CPU architecture that has an very impressive IPC without spending relatively speaking an huge amount of transistors to achieve it. I.e. we can still see impressive technical advancements without Moores Law, and Apples SoC is very much impressive, regardless of how good (or bad) it is for the sw-encoders available atm.

And sw-encoding is not the only factor in the workflow if 8k will become valid or not, I have tried the M1 for video editing 4k HEVC & Prores files, and what that SoC achieves at this powerdraw is simply astounding. And its not just the CPU-cores I'm refering to, it's all the other stuff on the SoC as well in combination with stuff like Metal (or what Intel is doing with oneAPI) that really displays that there is a promising future even without Moores Law.

Yeah, no doubt that Moore's Law predictions have changed a bunch. We are still seeing good compute/$ improvements over time, but it comes from an aggregation of improvements, not just more transistors and higher clock speeds anymore.

Yeah the breakdown of Dennard scaling (https://en.wikipedia.org/wiki/Dennard_scaling) isnt helping either :/

[benwaggoner]

What is Apple doing? They're largely based on ARM CPUs with x86 emulation. Even if the emulation is well done, they will never ever be as efficient as running an x86 application natively and so far ARM isn't really good for video encoding, it just isn't.
The amount of x86 emulation on the M1 processors is dropping rapidly, with lots of native apps already out and more coming.

x265 itself got a bunch of ARM optimization check-ins this year that have more than doubled encoding fps on M1.

Talking to colleagues deep in this stuff, there isn't anything fundamentally better about the x86-64 instruction set for encoding. It's more than Intel and AMD build big beefy CPUs with lots and lots of cores and cache and SIMD.

[benwaggoner]

I'm referring to transistors here, were Apple has an CPU architecture that has an very impressive IPC without spending relatively speaking an huge amount of transistors to achieve it. I.e. we can still see impressive technical advancements without Moores Law, and Apples SoC is very much impressive, regardless of how good (or bad) it is for the sw-encoders available atm.

And sw-encoding is not the only factor in the workflow if 8k will become valid or not, I have tried the M1 for video editing 4k HEVC & Prores files, and what that SoC achieves at this powerdraw is simply astounding. And its not just the CPU-cores I'm refering to, it's all the other stuff on the SoC as well in combination with stuff like Metal (or what Intel is doing with oneAPI) that really displays that there is a promising future even without Moores Law.
Yeah, making a big desktop-caliber SoC allowed for integration of RAM and high performance GPU in a single chip. And the tight coupling between HW and SW that Apple delivers means that the full capabilities of the new design are leveraged.

A metaphor would be Xbox Series/PS5-only games. They can rely on SSD storage, DirectStorage like SSD to GPU transfers, etc within a tightly defined architecture. Like the M1, they can outperform a general purpose computer setup at a fraction of the cost.

[FranceBB]

Lots of tools that can do 4Kp24 today simply don't have any way to scale to 8Kp120 at any costs.

By the way, not really related, but as much as I wanted the proposal for broadcasting PAL 8K to be 100fps and for NTSC 8K to be 120fps, it looks like it's going to be the same as the "old" 4K specs, so 50p and 60p. There's still an argument about 10bit VS 12bit but looks like that one is gonna be scrapped as well and it will still be 10bit. :(
Nothing official, just rumours, but still...



x265 itself got a bunch of ARM optimization check-ins this year that have more than doubled encoding fps on M1.

I actually noticed it (no I'm not an Apple user, I just remember seeing the commits and something related in the x265 topic).


Talking to colleagues deep in this stuff, there isn't anything fundamentally better about the x86-64 instruction set for encoding. It's more than Intel and AMD build big beefy CPUs with lots and lots of cores and cache and SIMD.

Really? I thought ARM were meant for mobile use and had fewer transistors and those transistors were actually made to use the smallest amount of energy as possible at the expense of performances. I also thought they had different registers made just for stuff that you use in a mobile like TLS computations for websites and some ai things for assistants etc. Of course something like an encoder could make use of some of those registers as well for its calculations, hence speeding things up with the manually written intrinsics like x265 has done, but I thought the ARM registers were no match for the x86_64 ones (SSE/AVX). If what you're saying is true, this opens the door to lots of new scenarios. :eek:

[birdie]

Yet another topic where people are talking about a Moore's "Law" when in reality it's never been anything but an observation.

[birdie]

Bitmovin has released its Annual Video Developer Report for 2021 (https://www.streamingmedia.com/Articles/ReadArticle.aspx?ArticleID=150364).

[benwaggoner]

Yet another topic where people are talking about a Moore's "Law" when in reality it's never been anything but an observation.
Indeed. And one which has required a lot of reformulation over the years to keep meaningful.

But the general point that we can anticipate increasing compute/$ for at least a couple more decades is important, even if the rate of improvement is slowing down. Digital media stuff consistently gets some of the biggest generation-on-generation performance/efficiency improvements, as we can leverage a whole lot of SIMD and parallelism.

[ksec]

Bitmovin has released its Annual Video Developer Report for 2021 (https://www.streamingmedia.com/Articles/ReadArticle.aspx?ArticleID=150364).

We conducted the survey from July 7th through August 22nd, 2021. 538 video developers and industry experts from 65 countries participated.

While the Bitmovin Developer Survey is open for everyone to participate and we are posting it industry-wide, we want to preface that the results might be slightly skewed towards the Bitmovin customer base.

Not really useful in terms of the whole industry "codec usage" but for me there were two interesting point.

1. Quality is not even the top 5 features respondent care about. Which is what I have been saying for sometime, if you want higher quality, go higher bitrate or better codec than H.264. It is no longer much a tech issue, but a cost issue.

2. It is not only me and a small group of people screaming for better low latency video. Live Streaming is the one that makes money, and also the one which requires most attention from codec usage.

[benwaggoner]

I also noted that, for all the pessimism around it, HEVC had a bigger year-on-year market share growth than AV1.

I was pleasantly surprised to see as much AC-4 and MPEG-H as there was as well.