Alliance for Open Media codecs

[nhw_pulsar]

Originally Posted by benwaggoner

It hasn't been trained on still images or HDR content or VVC style motion artifacts or AV1 grain synthesis

As I am only trained on still image artifacts, I am rather fascinated by people who talk about motion artifacts, as I have no knowledge on it.

So VVC has its own style of motion artifacts? If you have time, could you or another expert let us know if VVC style motion artifacts are visually more pleasant than AV1 motion artifacts for example?

Cheers,
Raphael

[benwaggoner]

Originally Posted by nhw_pulsar

As I am only trained on still image artifacts, I am rather fascinated by people who talk about motion artifacts, as I have no knowledge on it.

So VVC has its own style of motion artifacts? If you have time, could you or another expert let us know if VVC style motion artifacts are visually more pleasant than AV1 motion artifacts for example?

VVC has some really great tools to keep high QP inter prediction from revealing underlying block structure, which keeps highly compressed motion looking a lot more natural.

AV1 has some nice features in the same area as well, which provide good visible benefits at high compression ratios. I've not compared AV1 and VVC them in depth in this regard. It's kind of hard to separate any one tool out of the general codec and encoder alchemy*. Bit-for-bit, VVC is clearly a stronger codec than either AV1 or HEVC, although the ecosystem is still 1-2 years from broad deployment.

*A good 20+ years ago Touradj Ebrahimi told me that making a codec is "10% science, 20% alchemy, and 70% SUN worship."

In that we start with some good theory about how to get improvements with new tools, kind of mash them together in different ways and see what seems to work, and then simulate the heck of it to tune quant tables and symbols for optimum entropy coding and all that.

Obviously, that was from a post MPEG-4 part 2, pre H.264, SPARC for high performance compute era. But still some deep truths in there. Today it'd be machine learning instead of SUN worship.

[megalol]

Originally Posted by Jamaika

GCC 11.3.1 20221229 without _GLIBCXX_HAS_GTHREADS, SIMD AVX
libavif 0.11.1.0-62f8095 / libwebp-->avif b80553d (aom v3.5.0-9c91575, svt-av1 1.4.1-ad82cde) / libheif-->avif 96a114f (aom v3.5.0-9c91575, svt-av1 1.4.1-ad82cde)
https://www.sendspace.com/file/4kd4ud

Hi, thanks, useful tools but its has very big problem that converted to avif webp images with av1enc_avx.exe looks much darker (tried with older version posted here also and different settings). If I do the same with libvips or imagemagick for example then it has the same brightness.

[foxyshadis]

Originally Posted by megalol

Hi, thanks, useful tools but its has very big problem that converted to avif webp images with av1enc_avx.exe looks much darker (tried with older version posted here also and different settings). If I do the same with libvips or imagemagick for example then it has the same brightness.

av1enc is a gstreamer test application that assumes a number of properties that often don't apply to images. If you think it should, report the error to gstreamer. (But I'm pretty sure I've seen this in their tracker.) I wouldn't touch it with a six-foot pole for avif, even if it's great for streaming and playback.

avifenc or heifenc are proper image encoders that offer the full suite of detection and override options. libvips and ImageMagick both use libheif, though in a different way than these command-line tools.

[megalol]

Originally Posted by foxyshadis

av1enc is a gstreamer test application that assumes a number of properties that often don't apply to images. If you think it should, report the error to gstreamer. (But I'm pretty sure I've seen this in their tracker.) I wouldn't touch it with a six-foot pole for avif, even if it's great for streaming and playback.

avifenc or heifenc are proper image encoders that offer the full suite of detection and override options. libvips and ImageMagick both use libheif, though in a different way than these command-line tools.

I wouldn't use av1enc either if there would be direct webp->avif support in avifenc because I agree that its best tool for avif.

[LigH]

New uploads: (MSYS2; MinGW32 / MinGW64: GCC 12.2.0)

AOM _v3.6.0-252-gdfbaa9891

rav1e 0.6.1-4-gf178e97

dav1d 1.1.0-0-g9593e62

SVT-AV1 v1.4.1-79-gaef9ee9e

avif 0.11.1-86d66f0
dav1d [dec]:1.1.0-0-g9593e62, aom [enc/dec]:3.6.0-252-gdfbaa9891, rav1e [enc]:0.6.1 (p20230214-4-gf178e97)

[hajj_3]

Youtube is adding AV1 livestream uploading using RTMP+ - https://www.tomshardware.com/news/av...ing-to-youtube

[FranceBB]

Eh, for streamers, not for those who actually watch those streams, which is my use case...
This means that although you'll be able to stream in AV1 and save some bitrate yourself instead of streaming in H.264, unfortunately YouTube will still re-encode to VP9...
In other words, although it's a good thing for streamers, it's not really going to affect viewers that much... at least not in my use case.
This is because my use case is pretty peculiar: we stream a 20 Mbit/s H.264 FULL HD 25p 8bit live feed containing news which Google re-encodes to a crappy low bitrate VP9, thus nullifying the whole thing.
I'm sure this is the case for plenty of other news broadcasters who picked YouTube to stream their contents for free.
What I'd like to see is AV1 actually being served to viewers in live streams so that they'll be able to enjoy a much better quality.
Anyway, allowing streamers to actually stream in AV1 is a first step...

The only side-effect of YouTube's implementation is that videos will still get transcoded to VP9. This means AV1 live streams will be re-coded to YouTube's VP9 codec.

[birdie]

Originally Posted by FranceBB

Eh, for streamers, not for those who actually watch those streams, which is my use case...
This means that although you'll be able to stream in AV1 and save some bitrate yourself instead of streaming in H.264, unfortunately YouTube will still re-encode to VP9...
In other words, although it's a good thing for streamers, it's not really going to affect viewers that much... at least not in my use case.
This is because my use case is pretty peculiar: we stream a 20 Mbit/s H.264 FULL HD 25p 8bit live feed containing news which Google re-encodes to a crappy low bitrate VP9, thus nullifying the whole thing.
I'm sure this is the case for plenty of other news broadcasters who picked YouTube to stream their contents for free.
What I'd like to see is AV1 actually being served to viewers in live streams so that they'll be able to enjoy a much better quality.
Anyway, allowing streamers to actually stream in AV1 is a first step...

YouTube's offline VP9 encoder is actually really good but the one they use for streaming is outright horrible. Dunno why.

[FranceBB]

Originally Posted by birdie

YouTube's offline VP9 encoder is actually really good but the one they use for streaming is outright horrible. Dunno why.

Ah, so it wasn't just me and my eyes. Thanks for confirming.
It's a shame, really...

[benwaggoner]

Originally Posted by birdie

YouTube's offline VP9 encoder is actually really good but the one they use for streaming is outright horrible. Dunno why.

Real-time encoding has real-time requirements. They can't use the "split the source up across lots of temporarily idle Google servers" tactic, but dedicate fixed compute for the duration of the session. If low latency is also a goal, that can limit use of lookahead and out-of-order encoding. High quality live encoding requires a lot of dedicated fast cores, so the stream needs to have enough revenue potential to make up for the much higher $/minute costs, and those costs go up quickly with better quality. The VP9 encoders never had that great multithreading, so that can also limit best results.

Real professional-grade live transcoding can get expensive quickly: https://aws.amazon.com/medialive/pricing/.

[birdie]

Originally Posted by benwaggoner

Real-time encoding has real-time requirements. They can't use the "split the source up across lots of temporarily idle Google servers" tactic, but dedicate fixed compute for the duration of the session. If low latency is also a goal, that can limit use of lookahead and out-of-order encoding. High quality live encoding requires a lot of dedicated fast cores, so the stream needs to have enough revenue potential to make up for the much higher $/minute costs, and those costs go up quickly with better quality. The VP9 encoders never had that great multithreading, so that can also limit best results.

Real professional-grade live transcoding can get expensive quickly: https://aws.amazon.com/medialive/pricing/.

That makes a lot of sense, thanks. AV1 is also very hard to parallelize (unless as you said encoding chunks of video separately offline), and I wonder why after VP9 Google did not address this major shortcoming. It seemingly bit them in the buttocks.

[benwaggoner]

Originally Posted by birdie

That makes a lot of sense, thanks. AV1 is also very hard to parallelize (unless as you said encoding chunks of video separately offline), and I wonder why after VP9 Google did not address this major shortcoming. It seemingly bit them in the buttocks.

VP9 was mostly on YouTube, where they did single-threaded encoding of short chunks distributed on otherwise idle Google cloud instances. That mode is all about throughput per watt, which single-threaded encoding is best at. Live is all about throughput per minute, so multithreading is really important.

VP9 was particularly poor for multithreading as frame encoding and decoding were (accidentally, I gather) serialized due to how in-loop filtering was structured. Shared entropy coding context across frames also limited decoder threading. VP9 was definitely a codec designed to run on a fast x86-64 core without a lot of consideration for multithreaded hardware or software implementations. AV1 is somewhat better, but is still more complex to decode than even VVC.

HEVC's Wavefront Parallel Processing is great and elegant, allowing for one thread per 64 pixels tall of the video. WPP yields better compression efficiency than slices, tiles, and frame threading. The more traditional IPBb reference frame hierarchy of H.264 and HEVC makes for more straightforward frame threading, although there is always some risk of quality regressions.
Some frame threading is feasible in AV1 doing tricks with golden frames. SVT-AV1 is a good demonstration of an AV1 encoder that can do a lot of threading (although quality isn't great). It'd be interesting to compare SVT-HEVC and SVT-AV1's parallelization.

Fragment/GOP level threading has challenges due to higher latency and making sure VBV state is compliant across fragment boundaries. But the encoders are well optimized for that as that's YouTube's primary encoding model.

I hope AV2 focuses more on architectural support for highly parallelized encoding and decoding.

[Beelzebubu]

Originally Posted by birdie

AV1 is also very hard to parallelize

This statement is unsupported by facts. SVT-AV1 for encoding and dav1d for decoding demonstrate the opposite.

[Beelzebubu]

Originally Posted by benwaggoner

frame encoding and decoding were (accidentally, I gather) serialized due to how in-loop filtering was structured.

It's just how big the deblock filter is in VP9: writing 7 pixels on each side, making the luma delay 7+64 lines, but because downsampled chroma has the same filter length, it has the same delay (i.e. 7+32 lines in memory), the ultimate delay is 14+64=80 lines. In AV1, the line buffer is 8+128=136 (luma) lines (or 4+64 for downsampled chroma). Note that this is not less than VP9. HEVC is only a few lines better than VP9 in this respect, and VVC is no different from AV1.

Originally Posted by benwaggoner

Shared entropy coding context across frames also limited decoder threading.

... but can be disabled by setting the frame-parallel bit in the header, and helps several percent (!!) in compression efficiency when enabled. In fact, this "tool" was (from what I hear) proposed for HEVC - but rejected. I can't comprehend why, and have heard similar (depressed) comments about this feature from other HEVC/MPEG contributing members. It should have made it in.

[birdie]

Originally Posted by Beelzebubu

This statement is unsupported by facts. SVT-AV1 for encoding and dav1d for decoding demonstrate the opposite.

And SVT-AV1 is nowhere near close to libaom (almost single-threaded) in terms of quality. Right.

[benwaggoner]

Originally Posted by Beelzebubu

... but can be disabled by setting the frame-parallel bit in the header, and helps several percent (!!) in compression efficiency when enabled. In fact, this "tool" was (from what I hear) proposed for HEVC - but rejected. I can't comprehend why, and have heard similar (depressed) comments about this feature from other HEVC/MPEG contributing members. It should have made it in.

IIRC it made the decoders too complex, and added a lot of latency for random access. It's also something that mostly helped with easy to encode content while not helping hard content much which improvements are most valuable.
I believe a feature like that could be architected in a way that doesn't cause those issues (only follow context of frames that need to be referenced to decode the given frame anyway).

[takla]

Originally Posted by FranceBB

Eh, for streamers, not for those who actually watch those streams, which is my use case...
This means that although you'll be able to stream in AV1 and save some bitrate yourself instead of streaming in H.264, unfortunately YouTube will still re-encode to VP9...
In other words, although it's a good thing for streamers, it's not really going to affect viewers that much... at least not in my use case.
This is because my use case is pretty peculiar: we stream a 20 Mbit/s H.264 FULL HD 25p 8bit live feed containing news which Google re-encodes to a crappy low bitrate VP9, thus nullifying the whole thing.
I'm sure this is the case for plenty of other news broadcasters who picked YouTube to stream their contents for free.
What I'd like to see is AV1 actually being served to viewers in live streams so that they'll be able to enjoy a much better quality.
Anyway, allowing streamers to actually stream in AV1 is a first step...

I seriously hate how youtube re-encodes all videos. I really wish they'd have an option that leaves the video as is. I understand that they do this for bandwidth savings, obviously, but they could simply have strict bitrate requirements in place and probe the stream...

Originally Posted by birdie

YouTube's offline VP9 encoder is actually really good but the one they use for streaming is outright horrible. Dunno why.

You can pick 1 of 3 speed settings in the youtube streamer dashboard. The setting with the highest delay offers the 'best' image quality. By default it is set to low latency, which is worse but most people don't change it.

[RanmaCanada]

Originally Posted by Beelzebubu

This statement is unsupported by facts. SVT-AV1 for encoding and dav1d for decoding demonstrate the opposite.

SVT-AV1 is garbage. No matter how many times I try to use anything but, I can't get it to work with multi-threading and it just uses 1 core (SVT-AV1 is the only encoder that uses all my cores, for clarity). This shows just how broken the codec is. I've argued with bluesword a lot over this and the only answers I ever get is "use these" never "this is why it's a single threaded encoder".

There is far too much gatekeeping on the documentation, and far too much "why isn't this working".

If they really want this to take over HEVC and in the future VVC, things need to fixed and addressed. Blindly defending it without addressing it's massive shortcomings and serious problems won't make it any better.

[hajj_3]

Originally Posted by takla

I seriously hate how youtube re-encodes all videos. I really wish they'd have an option that leaves the video as is.

recently youtube has been testing a high bitrate 1080p setting option.