Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > New and alternative video codecs

Reply
 
Thread Tools Search this Thread Display Modes
Old 16th November 2021, 18:22   #81  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,725
Quote:
Originally Posted by foxyshadis View Post
Downside: If you don't target hardware in some fashion, even if only to make it FPGA or GPU-capable if not a full fixed chip, you end up with no mobile adoption, and mobile is the majority of the market now. AV1 got extraordinarily lucky that HEVC licensing imploded, otherwise it would never have existed as a consumer product because it requires herculean efforts to turn into hardware even with several advisors tweaking the final design. Cameras will want to offload as much encoding as possible, galleries and social media apps will want to offload as much decoding as possible, and even games will nearly always only ever use anything natively supported.

It doesn't need to be like MPEG, where almost every decision is based on hardware, but I hope at some point you have a plan for hardware engineers to at least take a look and point out where something that works in software is going to be nigh-impossible in hardware before it's too late.
So true. Just like I worry about the global CO2 impact of using AV1 SW decoders when a HW decoder is available, I imagine the net impact of doing lot of SW image decode could hurt device battery life in an all AVIF world.

Also, all that new assembly code is new surface area for bugs and exploits. Decode in HW has a lot lower (but non-zero) risk of exploits. A huge share of device remote exploits have been on media features.

Quote:
I have a hard time squaring that idea with the fact that ISO-BMFF and Matroska have proven to be incredibly robust and versatile for images, without more than a few bytes of overhead compared to every vaunted custom container. With each generation it gets more confusing why the wheel keeps being reinvented but worse, if it's not just NIH. Then it has to be extended and re-extended until it finally has most of the features they could have had for free.

Let's face it, the Exif and thumbnail will always be orders of magnitude less efficient than container overhead.
Yeah, I can't imagine a scenario where a custom wrapper would yield a material reduction in file size in apples-to-apples comparisons.

And MP4 container parsing is battle-hardened code. While wrapper code isn't as risky as actual DSP SW, those still have been a fertile area for exploits.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 16th November 2021, 21:27   #82  |  Link
nhw_pulsar
Registered User
 
Join Date: Apr 2017
Posts: 170
Quote:
Originally Posted by foxyshadis View Post
Downside: If you don't target hardware in some fashion, even if only to make it FPGA or GPU-capable if not a full fixed chip, you end up with no mobile adoption, and mobile is the majority of the market now. AV1 got extraordinarily lucky that HEVC licensing imploded, otherwise it would never have existed as a consumer product because it requires herculean efforts to turn into hardware even with several advisors tweaking the final design. Cameras will want to offload as much encoding as possible, galleries and social media apps will want to offload as much decoding as possible, and even games will nearly always only ever use anything natively supported.

It doesn't need to be like MPEG, where almost every decision is based on hardware, but I hope at some point you have a plan for hardware engineers to at least take a look and point out where something that works in software is going to be nigh-impossible in hardware before it's too late.
Wow, sadly this is just another major hurdle when you are an individual and you have created a codec, to penetrate the codec market, because this is prohibitive to afford hardware experts that will tell you what is possible and what is not possible in hardware, and so what to code.

What also worries me is when you say: "before it's too late", because for example in NHW, I did not develop for hardware, and so this worries me that there are maybe main design choices/errors that prevent the codec from being feasible in hardware....
nhw_pulsar is offline   Reply With Quote
Old 19th November 2021, 16:28   #83  |  Link
skal
Registered User
 
Join Date: Jun 2003
Posts: 121
Quote:
Originally Posted by benwaggoner View Post
Also, all that new assembly code is new surface area for bugs and exploits.
?! there's nothing really scary about assembly code, all things considered. I've seen more harm done by "malloc(-1)"'s than any SIMD code in media processing.

Quote:
Originally Posted by benwaggoner View Post
Decode in HW has a lot lower (but non-zero) risk of exploits. A huge share of device remote exploits have been on media features.
Bad drivers and HW interaction have caused quite some harm. And they are very hard to retro-fix or fuzz. It's actually pretty easy to make drivers freeze on bad corrupt bitstreams these days.

Anyways, complex containers are extremely difficult to get right. Stagefright exploit comes to mind:
https://www.kb.cert.org/vuls/id/924951.
WebP's RIFF container was very simple, and yet we had 1 exploit (only!) in 10 years. Not bad, but not zero.

If you're not receiving user media files uploaded from the wild, it's easy to think a container format is simple and safe.
VLC community has great horror stories about containers. Even after so many years of bug fixing, i still wouldn't trust ffmpeg to decode any MP4 file without proper sandboxing.

Regarding HW, in the past, we tried to reuse VP8 hardware decoding for WebP lossy, and it wasn't terribly faster. And actually quite brittle because of driver failures.
https://groups.google.com/a/webmproj...m/Ctl5wEoalGwJ

All in all, there's a real need for less flexible and easier-to-implement container, without any low-usage complex feature.
It's not just about size reduction, but also making sure it's easy for implementations to get right.
skal is offline   Reply With Quote
Old 6th February 2022, 08:38   #84  |  Link
Jamaika
Registered User
 
Join Date: Jul 2015
Posts: 688
I created the newest file av1enc with
Library encoder: av1enc 0.0.1 8_12bit c++ [01 Feb 2022] {boost, mingw}
libWebP 1.2.2 c [20 Jan 2022]
aom 3.3.0 c [03 Feb 2022]
libavif 0.9.3 c [28 Jan 2022]
jpegxl 0.7.0 c++ [04 Feb 2022]
brotli 1.0.9 c [06 Dec 2021]
libyuv 1.8.10 c++ [03 Feb 2022]
boost 1.78.0 c++ [02 Dec 2021]
libJPEG-turbo 2.1.3 8bit c [01 Feb 2022] {assembler SIMD}
libPNG 1.6.38 c [12 Mar 2021] {zlib}
libTIFF 4.3.0 c [05 Feb 2022] {zlib, libdeflate, zstd}
giflib 5.2.1 c [24 Jun 2019]
zlib 1.2.11.1 c [01 Jan 2022]
I can see that the names of some functions have changed.
Code:
  opt.Add("-pass <int>", "number of passes [1..2..3]");
Code:
  cpi->source =
      av1_scale_if_required(cm, cpi->unscaled_source, &cpi->scaled_source,
                            cm->features.interp_filter, 0, false, false);
  if (cpi->unscaled_last_source != NULL) {
    cpi->last_source = av1_scale_if_required(
        cm, cpi->unscaled_last_source, &cpi->scaled_last_source,
        cm->features.interp_filter, 0, false, false);
  }
I am changing to:
Code:
  cpi->source =
      av1_realloc_and_scale_if_required(cm, cpi->unscaled_source, &cpi->scaled_source,
                            cm->features.interp_filter, 0, false, false, cpi->oxcf.border_in_pixels, cpi->oxcf.tool_cfg.enable_global_motion);
  if (cpi->unscaled_last_source != NULL) {
    cpi->last_source = av1_realloc_and_scale_if_required(
        cm, cpi->unscaled_last_source, &cpi->scaled_last_source,
        cm->features.interp_filter, 0, false, false, cpi->oxcf.border_in_pixels, cpi->oxcf.tool_cfg.enable_global_motion);
  }
I add fix:
https://github.com/AOMediaCodec/libavif/pull/838
I also don't know why avif is no longer compatible with av1enc.
The file sizes for av1 and avif are different. I don't think command functions pass.
av1enc.exe -q 90 -420 -size 1563x1558 -effort 5 -thread 4 image_21447_24bit.jpg -d image_21447_24bitw1.av1 1239149kb
av1enc.exe -avif -q 90 -420 -size 1563x1558 -effort 5 -thread 4 image_21447_24bit.jpg -d image_21447_24bitw1.avif 931201kb

Codec works
av1enc.exe -q 90 -420 -size 1563x1558 -effort 5 -tune butteraugli -thread 4 image_21447_24bit.jpg -d image_21447_24bitw2.av1
Codec doesn't work tune butteraugli
av1enc.exe -avif -q 90 -420 -size 1563x1558 -effort 5 -tune butteraugli -thread 4 image_21447_24bit.jpg -d image_21447_24bitw2.avif

https://www.sendspace.com/file/iryn52

I add defined
Code:
  opt.Add("-tune <metric>", "choose metric for tuning. "
#if CONFIG_TUNE_VMAF
                            "One of: psnr, ssim, vmaf"
#elif CONFIG_TUNE_BUTTERAUGLI
                            "One of: psnr, ssim, butteraugli"
#elif CONFIG_TUNE_VMAF && CONFIG_TUNE_BUTTERAUGLI
                            "One of: psnr, ssim, vmaf, butteraugli"
#else
                            "One of: psnr, ssim"
#endif // CONFIG_TUNE_VMAF
         );
Code:
#if defined(WP2_HAVE_LIBGAV1)
  opt.Add("-libgav1", "use libgav1 decoder");
#endif
Code:
#if CONFIG_TUNE_VMAF
      } else if (!strcmp(argv[c], "vmaf") || !strcmp(argv[c], "VMAF")) {
        params.tuning = WP2::ParamsAV1::Tuning::kVmaf;
#elif CONFIG_TUNE_BUTTERAUGLI
      } else if (!strcmp(argv[c], "butteraugli") || !strcmp(argv[c], "BUTTERAUGLI")) {
        params.tuning = WP2::ParamsAV1::Tuning::kButteraugli;
#endif
Code:
#if defined(WP2_HAVE_LIBGAV1)
    } else if (!strcmp(argv[c], "-libgav1")) {
      params.use_libgav1_decoder = true;
#endif
Code:
  const int tuning =
      (params.tuning == ParamsAV1::Tuning::kPsnr)   ? AOM_TUNE_PSNR
      : (params.tuning == ParamsAV1::Tuning::kSsim) ? AOM_TUNE_SSIM
#if CONFIG_TUNE_VMAF
      : (params.tuning == ParamsAV1::Tuning::kVmaf) ? AOM_TUNE_VMAF_MAX_GAIN
#elif CONFIG_TUNE_BUTTERAUGLI
      : (params.tuning == ParamsAV1::Tuning::kButteraugli) ? AOM_TUNE_BUTTERAUGLI
#endif
                                                    : 0;
Code:
  const char* kTuningStr[] = {"PSNR", "SSIM"
#if CONFIG_TUNE_VMAF
                             , "VMAF"
#elif CONFIG_TUNE_BUTTERAUGLI
                             , "BUTTERAUGLI"
#endif
                             };
libavif
Code:
static const struct aomOptionEnumList tuningEnum[] = { //
    { "psnr", AOM_TUNE_PSNR },                         //
    { "ssim", AOM_TUNE_SSIM },                         //
#if CONFIG_TUNE_VMAF
    { "vmaf", AOM_TUNE_VMAF_MAX_GAIN },                //
#elif CONFIG_TUNE_BUTTERAUGLI
    { "butteraugli", AOM_TUNE_BUTTERAUGLI },           //
#endif
    { NULL, 0 }
};

Last edited by Jamaika; 6th February 2022 at 21:55.
Jamaika is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 20:42.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.