x266 VVC Encoder

[birdie]

Development is already underway (https://multicorewareinc.com/x266-a-state-of-the-art-open-source-code-for-vvc-encoding-by-multicoreware-inc/):

MulticoreWare announces the formation of x266 consortium and development of an open source code for VVC encoding is already underway. Similarly, MulticoreWare’s x265 consortium led the development of the open source code x265 for HEVC based encoding and become the mmost widely used HEVC encoding software.

Application areas of x266 will include encoding High Definition(HD) and Ultra High Definition(UHD- 4K and 8K) video for broadcasting, live streaming, Over The Top(OTT) video distribution, Adaptive Bit Rate(ABR) streaming, video storage and archiving, surveillance, screen content sharing, video teleconferencing, video games, eSports, immersive media such as 360-degree video, Virtual Reality (VR) etc.

x266 will be able to encode video with Standard, High Dynamic Range and Wide Color Gamut. It will also offer a user flexibility to trade-off speed of encoding with coding efficiency. Initial version of the x266 software will support single layer coding of 10 bits video with 4:2:0 chroma format. More advanced versions will support multilayer coding, higher than 10 bits depth video and higher than 4:2:0 chroma formats.

[FranceBB]

Great news!! :D

[microchip8]

Awesome!

[Greenhorn]

Hopefully, this will get off to a good start; the availability of a genuinely good encoder seems like it'll be critical for VVC.

[ksec]

Hopefully x265 and x266 being developed by the same team would speed things up a little bit.

What that means is that we could see EVC, VVC and AV2 Encoder all appearing within a year.

[hajj_3]

What that means is that we could see EVC, VVC and AV2 Encoder all appearing within a year.

AV2? That would be a bad idea as it would make it less likely for AV1 to receive mass adoption. They should wait 2.5yrs more to ratify AV2.

Also lots of patents will have expired over 5yrs since AV1 was ratified it they wait.

[ksec]

AV2? That would be a bad idea as it would make it less likely for AV1 to receive mass adoption. They should wait 2.5yrs more to ratify AV2.

Also lots of patents will have expired over 5yrs since AV1 was ratified it they wait.

Yes..... and the originally schedule for AV2 is 2020. And AV1 was suppose to be 2017....

They sure should take their time to ratify, but who knows.

[FranceBB]

Hopefully x265 and x266 being developed by the same team would speed things up a little bit.

Yep, that's the idea.
I really hope it will be ready for 2021. That would be great.

(On the other hand, it would mean that now I don't have any excuses not to buy a new CPU...)

Speaking of new hardware, what about decoding support? Is there anything in program? Like new gen GPUs?

[birdie]

To be honest I'm a lot more excited by VVC than AV1. The former looks like a codec which will be embraced by the scene and which will be viable for end users. AV1 is so computationally expensive it's suitable only for Google with their insane compute resources. SVT-AV1 is fast and ... blurry at all presets. Of course, patents and everything, only it doesn't matter for your personal archival purposes.

Also, AV1/H.265/VP9 basically offer nothing/very little vs. H.264 in terms of transparent near-lossless compression where H.264 is still the king. I wonder if H.266 could change that.

[hajj_3]

(On the other hand, it would mean that now I don't have any excuses not to buy a new CPU...)

Now is a bad time to buy a new cpu. AMD's zen 3 cpu's are going to be released soon, i'd wait until then before buying.

Speaking of new hardware, what about decoding support? Is there anything in program? Like new gen GPUs?

VVC was only ratified recently, it will likely take 1.5-2.5yrs for hardware decoders to ship in apu's or gpu's. So not likely until 2022.

[tnti]

Hope that there is something new in the rate control algorithm of X266.

[Cary Knoop]

Is VCC specced to support floating point code values?

[benwaggoner]

Hope that there is something new in the rate control algorithm of X266.
Like what?

[tnti]

Like what?

You should understand Alibaba's S265 encoder.

[benwaggoner]

You should understand Alibaba's S265 encoder.
What about it? Is there a particular rate control feature you'd like to see adopted elsewhere? If so, can you provide details?

[tnti]

What about it? Is there a particular rate control feature you'd like to see adopted elsewhere? If so, can you provide details?

https://www.livevideostack.cn/news/taobao-hevc-h265-cdn-rtc/
Sorry, there is no English version of the PPT.

[benwaggoner]

https://www.livevideostack.cn/news/taobao-hevc-h265-cdn-rtc/
Sorry, there is no English version of the PPT.
Can you offer a summary? I can't read Chinese, nor can most of us here.

[quietvoid]

Does anyone have more info about this repo here: https://github.com/chenm001/x266
Looks like just a research implementation but curious.

[FranceBB]

Does anyone have more info about this repo here: https://github.com/chenm001/x266
Looks like just a research implementation but curious.

The last release says "on Oct 21, 2017" when the standard wasn't even remotely a thing and was so far away from being finalized. I don't think it's something official nor from Multicoreware...

[tnti]

Does anyone have more info about this repo here: https://github.com/chenm001/x266
Looks like just a research implementation but curious.

Min Chen temporarily has no time to maintain this open source project.
I asked about this in July 2020.:sly:

[chenm001]

Min Chen temporarily has no time to maintain this open source project.
I asked about this in July 2020.:sly:

I have enough time since September 2020, I focus on my x266 encoder and decoder after that.

[chenm001]

Does anyone have more info about this repo here: https://github.com/chenm001/x266
Looks like just a research implementation but curious.

Thank you interesting of my projects.

That is my research tree, the goal is a hybrid video processing system since HEVC is a little complicated and WC almost can't be implemented as a real world product.
So I think software as a framework with hardware acceleration components is a good idea.

However, my previous employer is a China company, they almost prohibit any spare time work, either related to company products or unrelated.
As a result, my project has been frozen for almost 2 years, it is unfreezing now because I am unemployed since September 2020, I can continue to contribute to my projects.

[chenm001]

The last release says "on Oct 21, 2017" when the standard wasn't even remotely a thing and was so far away from being finalized. I don't think it's something official nor from Multicoreware...

It is not related to Multicoreware, it is my research tree...

[FranceBB]

It is not related to Multicoreware, it is my research tree...

Oh, ok, got it. I was asking 'cause I know Multicoreware was working on x266 and I would have been surprised to find the repository on GitHub out of the blue, just like that.


However, my previous employer is a China company, they almost prohibit any spare time work, either related to company products or unrelated.
As a result, my project has been frozen for almost 2 years, it is unfreezing now because I am unemployed since September 2020, I can continue to contribute to my projects.

Oh... Sorry to hear that, man...
I hope you'll find something else.
By the way, I didn't know that Chinese companies had such strict regulations... but I guess I should have expected that, given that they banned pretty much everything on the internet...
Anyway, take care, I'm sure you'll find another job.

[chenm001]

Oh, ok, got it. I was asking 'cause I know Multicoreware was working on x266 and I would have been surprised to find the repository on GitHub out of the blue, just like that.



Oh... Sorry to hear that, man...
I hope you'll find something else.
By the way, I didn't know that Chinese companies had such strict regulations... but I guess I should have expected that, given that they banned pretty much everything on the internet...
Anyway, take care, I'm sure you'll find another job.

Thank you.
I guess most China companies want t share nothing, no idea.

btw: Based on the Multicoreware scramble for x266 registering trademarks, I maybe rename my x266 to m266 or similar.
However, the project name is not important, I will still focus on make a really world high performance encoder and decoder products.

[tnti]

Thank you.
I guess most China companies want t share nothing, no idea.

btw: Based on the Multicoreware scramble for x266 registering trademarks, I maybe rename my x266 to m266 or similar.
However, the project name is not important, I will still focus on make a really world high performance encoder and decoder products.

According to past practice, it should be named T26X.：:stupid:

[soresu]

Hopefully x265 and x266 being developed by the same team would speed things up a little bit.

What that means is that we could see EVC, VVC and AV2 Encoder all appearing within a year.

AV2 is in deep dev mode at the moment but nowhere near finished at this point from what I gather from the commits on the AOM experimental branch.

I'd be surprised if we saw it even in 2022, more likely 2023 at the earliest?

[soresu]

To be honest I'm a lot more excited by VVC than AV1. The former looks like a codec which will be embraced by the scene and which will be viable for end users. AV1 is so computationally expensive it's suitable only for Google with their insane compute resources. SVT-AV1 is fast and ... blurry at all presets. Of course, patents and everything, only it doesn't matter for your personal archival purposes.

Also, AV1/H.265/VP9 basically offer nothing/very little vs. H.264 in terms of transparent near-lossless compression where H.264 is still the king. I wonder if H.266 could change that.

From what I gather on the AV1 discord libaom has come on in leaps and bounds in both speed and memory consumption since SVT AV1 came out guns blazing on the speed front.

As for near lossless compression - I don't think that this is an aspect of the codec standard so much as the implementation.

x264 had an insane level of focus on replacing XviD at all levels, and it did eventually do that by about 2010/11 timeframe from what I remember - I think that this focus was the real reason that x264 is still the high bitrate king of the OSS encoders.

I've seen benchmarks that show x265 isn't even the best H265/HEVC encoder out there (HW265 was it?), so it seems there is definitely room for improvement.

II think that the rav1e people are focused on being a true open source x264 successor in terms of quality in their endgame, although it will probably take until well past the release of AV2 to reach that goal.

[benwaggoner]

"Best" is a very subjective word! For streaming, "transparent near-lossless compression" is a nice aspiration, but the focus is on "best quality within available bandwidth." Near lossless is more for content usable as source for additional reencoding, like ProRes, as a term of art.

The Beamr HEVC encoder is general thought of the highest quality option these days. Not sure what HW265 is.

Does rav1e have a team focused on psychovisual optimization algorithms? I wasn't sure what's happening with them after all the Mozilla layoffs.

It's amazing how influential x265's "Rate Factor" has been; we see --crf equivalents or straight --crf popping up all over the place these days. With the complexity of AV1 and VVC, I want to think that there are deeper ways to take advantage of the new codec features. Actual QP is less important any many ways given the novel ways that artifacts are suppressed. VVC's ability to encode high-QP motion without any block pattern becoming apparent is a game changer.

[Losko]

I stumbled upon this:
https://github.com/fraunhoferhhi/vvenc
Pure C++, open source (but I didn't read the license yet) and cross platform: looks promising!

[benwaggoner]

I stumbled upon this:
https://github.com/fraunhoferhhi/vvenc
Pure C++, open source (but I didn't read the license yet) and cross platform: looks promising!
Yeah, definitely more practical performance than the reference encoder. Although the efficiency hit from multithreading is quite high. It's probably better for making test streams than in demonstrating potential bitrate savings versus other codecs, at least so far. Fraunhofer has a history of making good quality encoders.

[PCU]

Friends, what format do you think Google will use in the future: H.266 or AV1?
Please compare: Elecard/MainConcept H.266 with x266.

[nevcairiel]

With Google, you mean YouTube? I don't think they are likely to adopt a brand new MPEG codec, they have avoided HEVC entirely and only begrudgingly offer H.264 as a fallback when VP9 isn't available.

[PCU]

With Google, you mean YouTube? I don't think they are likely to adopt a brand new MPEG codec, they have avoided HEVC entirely and only begrudgingly offer H.264 as a fallback when VP9 isn't available.

What about: Facebook, Twitter, Instagram?

[birdie]

What about: Facebook, Twitter, Instagram?

As far as I know there are no representatives of these companies here on these forums, so unlikely anyone could give you a definitive answer.

[FranceBB]

As far as I know there are no representatives of these companies here on these forums, so unlikely anyone could give you a definitive answer.

Which is a shame. :(
On the other hand, we know that Facebook is still stuck on H.264 and it won't allow anything over HD 30p.
So for instance I generally record simple 3840x2160 60p videos in H.265 100 Mbits with my phone. If I upload them to Facebook they will be downscaled to 1280x720 using a simple Bicubic as resizing kernel, frames will be dropped to 30p and it will be re-encoded to low bitrate H.264.
This is nowhere near what a user expects in 2021.
We've been complaining a lot about YouTube and the fact that videos are low bitrate, 8bit only and full of banding, but at least you can upload up to 8K 60p there and if you upload HLG or PQ it will be preserved and passed through to supported devices (while showing a puny LUT conversion on SDR ones). I mean, this is at least something. Facebook on the other hand is simply ridiculous, but I think they don't care as 95% of their users is using Mobile Phones on relatively poor networks and tiny displays so they don't care... :(

[ksec]

Facebook is pretty much Anti-MPEG, or at least certain SVP seems to be this way. Twitter doesn't really care about these sort of thing. They have never even participate in the input of any codec discussion, on or offline AFAIK. Instagram tends to follow Facebook, although both Facebook and Instagram is supporting JPEG XL rather than AVIF so at least they seems to be pragmatic about it rather than ideological.

At the end of the day I think ( or I would like to think ) it really is some market force and business decisions. With Apple now supporting VP9 via partial hardware acceleration, and I would imagine better hardware acceleration in the future ( No one has tested the new Hardware Decoder Block on A15 yet ), may be VP9 HDR will be fully hardware accelerated.

Note: Some "site" decide to label Apple as supporting AV1 hardware acceleration and gave zero fact or evidence that is the case, and somehow someone continuously edit this information to be included as fact on wiki.

Modern Internet Video Giant like Facebook and Youtube now have different set of priorities most may not realise. Bandwidth Cost is falling, and it is falling faster than we expected. With enough peering, bandwidth is quite literally free. ( Cloudflare's CEO made this point very very clear. ) It is storage and compute that are now the cost centre. Storage prices hasn't moved at all. And Compute hasn't gotten faster anywhere near the complexity of a new codec. Introducing a new codec means adding another storage cost for every video they have. For Netflix, they can afford to re-encode their whole catalog every three months just to gain on additional image quality via encoder improvement. Their priorities are very different to Facebook or Youtube. But Netflix doesn't like MPEG. Or at least their engineers doesn't like it due to patents and royalty.

Basically we might very well end up with something similar to what we currently have. H.264 and HEVC / VP9.

I would love to be wrong though since I quite like what VVC is showing so far. But the patent pool situation from VVC, MPEG-LA has yet to announce their pricing, 10 months after their initial announcement.

That is from Internet Video Companies perspective. From broadcasting, I gather the industry is quite excited especially those from Asia.

[benwaggoner]

Friends, what format do you think Google will use in the future: H.266 or AV1?
Please compare: Elecard/MainConcept H.266 with x266.
Google was a founder of AOM and primary contributor of AV1, which started with the libvpx codebase and VP9 design. They've been vocally opposed to patent-bearing codecs for years now, and specifically coded out Chrome's support for passing HEVC decode to a system decoder.

Google's YouTube has always been the primary user of VP 7, 8, and 9, and now AV1, which they've used to drag devices into supporting their players to get access to new YouTube quality tiers.

I think it is very safe to assume that Google will continue to support AV1, and will work hard to make AV2 the next big codec. It would be a profound about-face for them to even allow VVC to be decodable in Chrome, let alone actually publish content using it.

[PCU]

Google was a founder of AOM and primary contributor of AV1, which started with the libvpx codebase and VP9 design. They've been vocally opposed to patent-bearing codecs for years now, and specifically coded out Chrome's support for passing HEVC decode to a system decoder.

Google's YouTube has always been the primary user of VP 7, 8, and 9, and now AV1, which they've used to drag devices into supporting their players to get access to new YouTube quality tiers.

I think it is very safe to assume that Google will continue to support AV1, and will work hard to make AV2 the next big codec. It would be a profound about-face for them to even allow VVC to be decodable in Chrome, let alone actually publish content using it.

Thanks.
What about Netflix & others? discs are dead these days.

[PCU]

Who do you think abandons a royalty-free codec and pays for a codec like H.266? http://www.emoticonr.com/design/yahoo/devil.gif

[birdie]

https://multicorewareinc.com/news/join-us-at-ces-2022/

Let’s Get Together at CES2022 this January in
Las Vegas



Join us at CES to learn more about our

Vision & Non-Vision (Radar, LiDAR, IMU, GPS, etc.) Sensors
Video Solutions (x266™, x265 & Ultraziq)
Human Behavior Intelligence (HBI) platform leveraging the latest AI & ML technologies &
LipSync – Video QC Tool

MulticoreWare is expertise in developing algorithms for various technologies including Camera (RGB, LWIR, ToF), Radar, Lidar, and IMUs to solve challenges in ADAS / AV sensor technology with R&D focus and on optimization of existing algorithms for edge deployment.

MulticoreWare’s industry-leading video codec products (x266™/x265/Ultraziq) have been deployed in live streaming or VOD services across many broadcast customers, video streaming services, movie studios, postproduction facilities, video system developers worldwide.

HuBe.ai – a highly scalable AI/API/SaaS Platform for all aspects of Human Behavior. With HuBe.ai, customers can accelerate the application development to track human behavior or augment existing solutions with similar functionalities to detect and identify distress/anomaly, assess people effectively in terms of communication & behavior, etc.

We also offer ML Engineering Services serving a wide group of customers with solutions like Hardware Platform Compilers & Toolchains, SDK Libraries, and Algorithm & Data Engineering solutions.

The first public unveiling of x266.

[PCU]

Guys, has anyone ever tested the x266 encoder to see if it is better than AV1 encoders or not?
Do you think the visual quality of H.266 is better than AV1?

[LigH]

It is hard to test a codec which is not available to the public.

You can test Google AV1 vs. Fraunhofer VVC. But Multicoreware x266 is not available yet, IIRC.

[FranceBB]

It is hard to test a codec which is not available to the public.

Multicoreware x266 is not available yet.

Yep, precisely.
The repository is still private and there are no builds around, let alone the source code, so no one tested it.

has anyone ever tested the x266 encoder to see if it is better than AV1 encoders or not?


The only one we did test was VVEnc by Fraunhofer, which was good enough to defeat x264 and x265 at 25 Mbit/s in UHD (you can see my tests in the VVC topic with screenshots and SSIM and the difference - especially between H.265 and H.266 was really really small, but that might be because x265 has been around for quite some time and VVEnc has not).


Do you think the visual quality of H.266 is better than AV1?


I haven't tested it against AV1, but it should be better than AV1 on paper.
Perhaps that's gonna be my next test when I'll have time.

[PCU]

Yep, precisely.
The repository is still private and there are no builds around, let alone the source code, so no one tested it.



The only one we did test was VVEnc by Fraunhofer, which was good enough to defeat x264 and x265 at 25 Mbit/s in UHD (you can see my tests in the VVC topic with screenshots and SSIM and the difference - especially between H.265 and H.266 was really really small, but that might be because x265 has been around for quite some time and VVEnc has not).



I haven't tested it against AV1, but it should be better than AV1 on paper.
Perhaps that's gonna be my next test when I'll have time.

http://www.emoticonr.com/design/yahoo/thumbs-up.gif

It is hard to test a codec which is not available to the public.

You can test Google AV1 vs. Fraunhofer VVC. But Multicoreware x266 is not available yet, IIRC.

Is the Fraunhofer free to test?

[birdie]

http://www.emoticonr.com/design/yahoo/thumbs-up.gif

Is the Fraunhofer free to test?

Would be great if you at least (https://en.wikipedia.org/wiki/Versatile_Video_Coding) visited the Wikipedia article on the matter where all the available H.266 codecs are already listed:

https://github.com/fraunhoferhhi/vvenc
https://vcgit.hhi.fraunhofer.de/jvet/VVCSoftware_VTM

Yes, they are "free" to test - what more can one ask for where the source code is freely available.

[PCU]

Would be great if you at least (https://en.wikipedia.org/wiki/Versatile_Video_Coding) visited the Wikipedia article on the matter where all the available H.266 codecs are already listed:

https://github.com/fraunhoferhhi/vvenc
https://vcgit.hhi.fraunhofer.de/jvet/VVCSoftware_VTM

Yes, they are "free" to test - what more can one ask for where the source code is freely available.

Yes, I saw the source code, but I meant exactly that, it's so weird that Fraunhofer put the source on Github, I just said I mean, can this source be used in a free program?

[LigH]

See Jamaika's XEVE (https://forum.doom9.org/showthread.php?p=1959462#post1959462) as a custom All-In-One encoder application (similar to ffmpeg) for an example...

[birdie]

Yes, I saw the source code, but I meant exactly that, it's so weird that Fraunhofer put the source on Github, I just said I mean, can this source be used in a free program?

https://vcgit.hhi.fraunhofer.de/jvet/VVCSoftware_VTM/-/blob/master/COPYING

[kurkosdr]

Google was a founder of AOM and primary contributor of AV1, which started with the libvpx codebase and VP9 design. They've been vocally opposed to patent-bearing codecs for years now, and specifically coded out Chrome's support for passing HEVC decode to a system decoder.

Google's YouTube has always been the primary user of VP 7, 8, and 9, and now AV1, which they've used to drag devices into supporting their players to get access to new YouTube quality tiers.

I think it is very safe to assume that Google will continue to support AV1, and will work hard to make AV2 the next big codec. It would be a profound about-face for them to even allow VVC to be decodable in Chrome, let alone actually publish content using it.

If I may add, the majority of websites have historically avoided patented formats that come with "content fees" so they won't have to pay said "content fees" for every picture or video they distribute (which can be a problem considering how thin per-content profit margins are for web content available as free-to-view/ad-supported). It's the reason why there was almost no web content encoded in JPEG2000, Mpeg2 or Mpeg4 ASP in the past. Websites just used good old JPEG for pictures and for video they used Cinepak (and after that WMV or Quicktime, often both in order to effortlessly serve both Windows and Mac).

In addition, browser vendors that offer their browsers as free downloads (such as Mozilla and Google) avoid patented codecs so they won't have to pay a "decoder fee" with every download. Sure, they could use the system codecs where available, but the "where available" part causes an inconsistent experience. Also, blocking system codecs is a form of "gatekeeping" to keep patented codecs out of the web so they won't have to be dragged into paying "decoder fees" later.

If you think about it, H.264 was the exception to the rule, the only ISO video compression standard to be widely used in the web. Firstly because it blew everything out of the water at the time in terms of efficiency, secondly because it had hardware acceleration in almost every computer (if not full, at least assisted) back when full software decoding was still an issue on low-powered PCs, and thirdly because it was imposed on browser vendors by the proprietary Adobe Flash plugin (which meant browser vendors couldn't "gatekeep" it out of their browsers).

None of these conditions apply today. Someone needs to break it to H.265 and H.266 patent holders that they aren't getting the kind of windfall they did from H.264 patents. This includes Apple.