Grand Challenge on Open-Source HEVC Encoding (x265 BDR Efficiency Improvements)

[foxyshadis]

ACM Multimedia Systems has put out an open call for patches improving core x265 picture quality, with prizes of $5,000 and $2,500 for the top entries.

Each entry will be judged according to the following criteria:

* Evaluations will focus on 4K content; both HDR and non-HDR sequences will be used.
* For PSNR and SSIM-based evaluations, we will measure Bjontegaard metrics with bitrates of 10, 12, 14, 16 Mbps for 24p content, and 18, 20, 22, 24 Mbps for 60p content with realistic VBV settings. For perceptual evaluations, we will use the SSIMPlus quality metric.
* Content used for evaluation will be a number of cinematic and sports content similar to https://4kmedia.org/lg-new-york-hdr-uhd-4k-demo/ and https://4kmedia.org/samsung-travel-with-my-pet-hdr-uhd-4k-demo/.
* Run-time computational complexity of the proposed algorithm, assuming Intel Sky Lake-generation or newer Xeon CPU for assessing the potential of assembly optimization of the proposed technique. Assembly optimization is not expected.


https://2020.acmmmsys.org/x265_challenge.php

[RanmaCanada]

Why Xeon? Why not Threadripper? Then they could address the inability of x265 to saturate all cores without distributed encoding :P

(I do know you are not in control of this, just complaining haha)

[nevcairiel]

Why Xeon? Why not Threadripper? Then they could address the inability of x265 to saturate all cores without distributed encoding :P


You did read that the challenge is primarily about quality, right? :P
The computational complexity of the newly proposed algorithms is to be evaluated as part of the process - its not about making x265 faster.

[RanmaCanada]

You did read that the challenge is primarily about quality, right? :P
The computational complexity of the newly proposed algorithms is to be evaluated as part of the process - its not about making x265 faster.

As I said, I was complaining. The fact that they mention Xeon and Intel only, with ignoring the current leader in cores and performance, just shows the bias. How many people had a 12-16 core system before this year? Not many and AMD has been selling every single one they make, unlike Intel and their HEDT processors that sit and gather dust. Heck it wasn't until AMD brought out their 8 core systems that it made it to mainstream. If anything, with the amount of cores that are available to mainstream users, they should be focusing on improving performance as quality is probably as good as it's going to get with the "limited" processor usage.

Unless of course they've decided to just say screw it, and push for more utilization in the next codec that will replace HEVC. The current standard is producing studio quality encodes if you have enough time to wait. That wait should be shortened. :P That's all haha.

[nevcairiel]

Still missing the point entirely. The challenge is specifically about increasing quality. Performance or utilization may be your pet peeve, but it has no place in this challenge. They didn't have to mention any hardware at all, they only mentioned that which it will be tested on. You are entirely focusing on one insignificant side-note, ignoring the entire primary purpose of the challenge.

[benwaggoner]

Are they really only looking at PSNR and SSIM? That's going to preclude a lot of psychovisual techniques where the real juice is left to squeeze. Particularly in HDR, where the subjective/objective correlations with PSNR and SSIM are even worse than with SDR.

They mention SSIMPlus, which is better. But we can't ever trust any existing objective metric to properly evaluate a novel psychovisual algorithm.

[sonnati]

Moreover, why SSIMwave that's not freely available ? How to estimate the improvement ?

[nevcairiel]

They did say they are going to provide access to SSIMPlus to all participants. Its not like perfect metrics grow on trees.

[Lypheo]

Are they really only looking at PSNR and SSIM? That's going to preclude a lot of psychovisual techniques where the real juice is left to squeeze. Particularly in HDR, where the subjective/objective correlations with PSNR and SSIM are even worse than with SDR.

Why’s that? Wasn’t the PQ EOTF explicitly designed to match the human visual system’s contrast sensitivity? I would’ve thought that would improve the correlation of mathematical distance and perceived distortion.

[benwaggoner]

Why’s that? Wasn’t the PQ EOTF explicitly designed to match the human visual system’s contrast sensitivity? I would’ve thought that would improve to correlation of mathematical distance and perceived distortion.
I don't have a particularly compelling theory about this. But it's what we've seen comparing the MOS scores with objective metrics with SDR and HDR; lower correlation in HDR.


My personal guess is this may reflect a bunch of implicit psychovisual optimization in existing encoders or codec standards as they were heavily tuned to maximize PSNR and also MOS. The same algorithms and underlying assumptions just don't work as well in HDR.

There is a lot of implicit curve fitting in codec design based on metrics. All the MPEG codecs are highly optimized for a fixed QP delivering maximum mean PSNR across frames of a short clip with good MOS. There's no fundamental reason why a codec needs that property, but the way we've designed them means tools that don't deliver on this get excluded.

AV1 is the first codec that has VMAF baked into some of its underlying tables and such, which is a new and different thing. And results in AV1 encodes having VMAF scores higher than their MOS scores would merit.

[foxyshadis]

Why Xeon? Why not Threadripper? Then they could address the inability of x265 to saturate all cores without distributed encoding :P

(I do know you are not in control of this, just complaining haha)

They mostly only care that your amazing new algorithm runs in fps, maybe spf at worst, but not hpf (hours per frame). A cpu that's twice as fast also won't change the relative ranking of the entries.

[vpupkind]

As I said, I was complaining. The fact that they mention Xeon and Intel only, with ignoring the current leader in cores and performance, just shows the bias.

AMD is mentioned in the current version of the text

[benwaggoner]

They mostly only care that your amazing new algorithm runs in fps, maybe spf at worst, but not hpf (hours per frame). A cpu that's twice as fast also won't change the relative ranking of the entries.This is a QUALITY improvement effort. It's not about compute performance so architecture really isn't relevant.

Sent from my SM-T837V using Tapatalk