WebP2 image codec

[skal]

Hi,

libwebp2 has been open-sourced (here), and i see some users reporting compilation issues (example).

Best is to start this thread proper and consolidate the reports here!

skal/

(tests welcome, too. Keep in mind the code is changing on a ~weekly basis, though!)

[Jamaika]

Errors can probably be added on the google forum. I create non-professional codecs in GCC. On Windows there is Visual Studio and others that have assembler and simd support.
Codec WebP2 works without AOM codecs. So all other AOM add-ons ie AVIF, DV1D, GAV1 will also not work under GCC.

[skal]

Originally Posted by Jamaika

Errors can probably be added on the google forum.

Yes, compile errors with details should be reported there:

https://bugs.chromium.org/p/webp2/issues/list

Originally Posted by Jamaika

I create non-professional codecs in GCC. On Windows there is Visual Studio and others that have assembler and simd support.
Codec WebP2 works without AOM codecs. So all other AOM add-ons ie AVIF, DV1D, GAV1 will also not work under GCC.

These pre-compiled binaries links can be posted here, indeed.

[skal]

Btw, for definiteness here are some command line examples if you want to test encoding / decoding:

cwp2 cat.png -o cat.wp2 -d cat.wp2.png -q 44

avifenc --min 40 --max 40 -s 2 cat.png cat.avif
avifdec cat.avif cat.avif.png

cwebp cat.png -o cat.webp -q 16
dwebp cat.webp -o cat.webp.png

====

[benwaggoner]

So, this is an enhanced encoder for the VP8 "WebP" format?

Do we care about WebP? I've not really seen exciting real-world implementations that were enough better than JPEG to merit the extra work, or close enough to HEIF H.264 that building a JavaScript implementation of that leveraging HW H.264 decoders wouldn't make more sense at scale. And AVIF is certainly going to outperform WebP while addressing the same licensing goals.

[VoodooFX]

Originally Posted by benwaggoner

Do we care about WebP?

I see up to ~50% better compression over PNG in my use scenarios.

[IgorC]

Originally Posted by benwaggoner

Do we care about WebP?

I see WebP quite ofent in internet.

Chrome/Firefox/Edge support WebP.
Such popular sites in Latin America as www.mercadolibre.com uses WebP.

Originally Posted by benwaggoner

I've not really seen exciting real-world implementations that were enough better than JPEG ...

This is far from reality (as many other of your statements regarding VP8/VP9/AV1)

WebP has its advantages on certain range of bitrates https://www.reddit.com/r/AV1/comment...if_comparison/

[skal]

Originally Posted by benwaggoner

So, this is an enhanced encoder for the VP8 "WebP" format?

Do we care about WebP? .

Hmm... if you missed the last 10 years of WebP development, i made a recap presentation here: http://shorturl.at/BEMV8

And yes, it's used a lot.

[Jamaika]

Originally Posted by skal

Btw, for definiteness here are some command line examples if you want to test encoding / decoding:

cwp2 cat.png -o cat.wp2 -d cat.wp2.png -q 44

avifenc --min 40 --max 40 -s 2 cat.png cat.avif
avifdec cat.avif cat.avif.png

cwebp cat.png -o cat.webp -q 16
dwebp cat.webp -o cat.webp.png
====

My test

dwp2.exe -v image_21447.wp2 -mt -o image_21447_64bit.jpg
Time to decode picture: 0.415s
Decoded image_21447.wp2. Dimensions: 1563 x 1558. Transparency: no.
Could not save to 'image_21447_64bit.jpg'
Error: unsupported feature
Status: WP2_STATUS_UNSUPPORTED_FEATURE

dwp2.exe -v image_21447.wp2 -mt -o image_21447_64bit.webp
Time to decode picture: 0.399s
Decoded image_21447.wp2. Dimensions: 1563 x 1558. Transparency: no.
Could not save to 'image_21447_64bit.webp'
Error: unsupported feature
Status: WP2_STATUS_UNSUPPORTED_FEATURE

avifenc and avifdec are a different library, unless something else has been added in the meantime.
Strange settings help av1enc. 'O' is not output and 'D' is not decode. What does not working 'o' and 'd' mean?

av1enc.exe -444 -size 1563x1558 -O image_21447_64bit.png
aom_codec_decode() failed.
Error compressing to AV1

Troublesome creating avif because you have to enter size image and chroma subsampling every time. Why are photos in webp2 and addons only 8bit? Additionally, to generate avif you need to add avif which is saved as av1.
To produce the avif with png you must add the 'D'. Why?

av1enc.exe -avif -420 -size 1563x1558 -D image_21447_64bit.avif -O image_21447_64bit.png
674650 35.965336   (99.000000 35.446957 39.947750 34.281830) [ 2.22 bpp ] [ 1563 x 1558 ] [73.223 / 0.632 secs]
Saved decoded image image_21447_64bit_dec.png
Saved encoded file: image_21447_64bit.av1 (674650 bytes) <-- ????

but

av1enc.exe -avif -444 -size 1563x1558 -D image_21447_64bit.avif -O image_21447_64bit.png
ERROR: Failed to encode: Decoding of color planes failed
Error while computing the distortion.

I will just add that when compiling libavif you should include the definition of IO. AVIF_CODEC_AOM, AVIF_CODEC_AOM_DECODE, AVIF_CODEC_AOM_ENCODE. This is an exception. Without this the accessory will not work.
File decoding RAW converted as TIFF RAW_CANON_EOS_5DS.cr2 8688x5792
Long waiting time. On my old computer thread 4 it is about 10 minutes.

cwp2.exe RAW_CANON_EOS_5DS.tiff -summary -lossless -mt -o RAW_CANON_EOS_32bit.wp2
Assertion failed: std::abs(progress_ + kProgressEnd - 1.f) < 0.001f, file progress_watcher.cc, line 52

Does WEBP2 codec handle metadata? Who knows that?

[skal]

Originally Posted by Jamaika

My test

...
Could not save to 'image_21447_64bit.jpg'
Error: unsupported feature
Status: WP2_STATUS_UNSUPPORTED_FEATURE

That's expected: you cannot save to a lossy format (jpg, webp), since it would defeat the purpose of showcasing WP2's compression capabilities.

[skal]

Originally Posted by Jamaika

...
aom_codec_decode() failed.

Sorry, i'm not an avifenc/avifdec specialist, just tried it quickly on Unix.

Originally Posted by Jamaika

Why are photos in webp2 and addons only 8bit?

If you are interesting in non-8bit input/output, you can have a look at some tools in the 'extras' directory. For instance 'ewp2'. They can handle Y4M format.

Originally Posted by Jamaika

File decoding RAW converted as TIFF RAW_CANON_EOS_5DS.cr2 8688x5792
Long waiting time. On my old computer thread 4 it is about 10 minutes.

Yes, lossless coding can be quite slow for such large photos. Note that compressing losslessly photo material is not the main use-case case WebP2 is targetting!

Originally Posted by Jamaika

Assertion failed: std::abs(progress_ + kProgressEnd - 1.f) < 0.001f, file progress_watcher.cc, line 52

This is a know problem that has been fixed in the latest sync patch.
You may want to recompile at HEAD to fix it (commit df4ec90a2befb79ed45cf8810e8ac93da7a47297).

Originally Posted by Jamaika

Does WEBP2 codec handle metadata? Who knows that?

Yes. By default, cwp2 will copy EXIF, XMP and ICC metadata.

[Scope]

WebP 2 df4ec90, MinGW 64, GCC 10.2.0

dwp2

[ 90%] Building CXX object CMakeFiles/dwp2.dir/examples/dwp2.cc.obj
C:/msys64/home/Scope/libwebp2/examples/dwp2.cc: In function 'std::string {anonymous}::PrintSummary(const WP2::BitstreamFeatures&, size_t, bool)':
C:/msys64/home/Scope/libwebp2/examples/dwp2.cc:207:37: error: unknown conversion type character 'z' in format [-Werror=format=]
  207 |     str += SPrintf("file size:     %zu\n", input_data_size);
      |                                     ^
[ 91%] Building CXX object CMakeFiles/extras.dir/extras/extras.cc.obj
cc1plus.exe: all warnings being treated as errors
make[2]: *** [CMakeFiles/dwp2.dir/build.make:82: CMakeFiles/dwp2.dir/examples/dwp2.cc.obj] Error 1
make[1]: *** [CMakeFiles/Makefile2:261: CMakeFiles/dwp2.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....

CMakeError.log

Performing C++ SOURCE FILE Test WP2_HAVE_FLAG___SSE4_2__ failed with the following output:
Change Dir: C:/msys64/home/Scope/libwebp2/build/CMakeFiles/CMakeTmp

Run Build Command(s):C:/msys64/usr/bin/make.exe cmTC_f9854/fast && /usr/bin/make  -f CMakeFiles/cmTC_f9854.dir/build.make CMakeFiles/cmTC_f9854.dir/build
make[1]: Entering directory '/home/Scope/libwebp2/build/CMakeFiles/CMakeTmp'
Building CXX object CMakeFiles/cmTC_f9854.dir/src.cxx.obj

/C/msys64/mingw64/bin/g++.exe   -DWP2_HAVE_FLAG___SSE4_2__ -o CMakeFiles/cmTC_f9854.dir/src.cxx.obj -c /C/msys64/home/Scope/libwebp2/build/CMakeFiles/CMakeTmp/src.cxx
C:/msys64/home/Scope/libwebp2/build/CMakeFiles/CMakeTmp/src.cxx: In function 'int main()':
C:/msys64/home/Scope/libwebp2/build/CMakeFiles/CMakeTmp/src.cxx:4:9: error: invalid use of 'this' in non-member function
    4 |         this is not valid code
      |         ^~~~
make[1]: *** [CMakeFiles/cmTC_f9854.dir/build.make:85: CMakeFiles/cmTC_f9854.dir/src.cxx.obj] Error 1
make[1]: Leaving directory '/home/Scope/libwebp2/build/CMakeFiles/CMakeTmp'
make: *** [Makefile:140: cmTC_f9854/fast] Error 2

Source file was:
      int main(void) {
        #if !defined(__SSE4_2__)
        this is not valid code
        #endif
        return 0;
      }

[skal]

Originally Posted by Scope

WebP 2 df4ec90, MinGW 64, GCC 10.2.0
...

Thanks for the report, opened an issue for it.

[Jamaika]

Originally Posted by skal

Yes. By default, cwp2 will copy EXIF, XMP and ICC metadata.

If I'm wasting my time I'm a bit of a piss off the creators.
Converting DNG to WEBP2 with metadata ICC, XMP, EXIF

TIFFFetchNormalTag: Warning, Incorrect value for "RichTIFFIPTC"; tag ignored.
Dimension:     5212 x 3468
animation:     no
quality hint:  15
preview:       no
preview color: 102, 85, 51
rotated by:    0 degrees clockwise
transfer:      0
ICC:           no   <-- ???
XMP/EXIF:      no   <-- ???
output size:  31860621 (14.10 bpp)    [RAW_CANON_EOS_32bit.wp2]

What should be the config for libtiff? The library includes RichTIFFIPTC.

[skal]

Originally Posted by Jamaika

TIFFFetchNormalTag: Warning, Incorrect value for "RichTIFFIPTC"; tag ignored.
...
What should be the config for libtiff? The library includes RichTIFFIPTC.

Hard to say without more details, the source image, compile flags, exact libtiff library version used, etc.

Best is to open an issue on the bug tracker, with all the infos. But it looks like a libtiff issue, honestly...

[Jamaika]

Everything is here. The rest for those willing.
New codecs, untestet with bugs fixed 25.11.2020
https://www.sendspace.com/file/dbx3jx
Added:
https://chromium.googlesource.com/co...8ac93da7a47297
https://gitlab.com/libtiff/libtiff/-...81479c267febaa
https://github.com/m-ab-s/aom/commit...339afb1c3baaf5
https://github.com/AOMediaCodec/liba...ee872c78c0bdb9
https://github.com/libjpeg-turbo/lib...e2cc49cde3b1ac
https://github.com/webmproject/libwe...085206947e98f2
https://github.com/dcnieho/FreeGLUT/...c90462ad0c6f50
https://github.com/glennrp/libpng/co...4b0666549b18e6
https://github.com/madler/zlib/commi...44df991c499252
GCC 10.2.1 http://msystem.waw.pl/x265/mingw-gcc1021-20200926.7z

[skal]

latest submit 7c0dceb should fix the %zu problem.

It also introduced some ~10-15% speed-up on x86 contribution (thanks Ilya!) and a '-progress' option to cwp2/dwp2 to report the enc/dec progress.


thx!

[Scope]

WebP 2 7c0dceb, MinGW 64, GCC 10.2.0

[ 89%] Built target ansz
C:/msys64/home/Scope/libwebp2/examples/example_utils.cc: In function 'std::string WP2::PrintSummary(const WP2::BitstreamFeatures&, size_t, bool)':
C:/msys64/home/Scope/libwebp2/examples/example_utils.cc:157:20: error: unknown conversion type character 'l' in format [-Werror=format=]
  157 |     str += SPrintf("file size:     %" PRIu64 "\n", (uint64_t)input_data_size);
      |                    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
cc1plus.exe: all warnings being treated as errors
make[2]: *** [CMakeFiles/example_utils.dir/build.make:82: CMakeFiles/example_utils.dir/examples/example_utils.cc.obj] Error 1
make[1]: *** [CMakeFiles/Makefile2:463: CMakeFiles/example_utils.dir/all] Error 2
make: *** [Makefile:160: all] Error 2

This may be related to Microsoft's printf https://stackoverflow.com/questions/...er-windows-64x

CMakeError.log

Performing C++ SOURCE FILE Test WP2_HAVE_FLAG___SSE4_2__ failed with the following output:
Change Dir: C:/msys64/home/Scope/libwebp2/build/CMakeFiles/CMakeTmp

Run Build Command(s):C:/msys64/usr/bin/make.exe cmTC_f12f3/fast && /usr/bin/make  -f CMakeFiles/cmTC_f12f3.dir/build.make CMakeFiles/cmTC_f12f3.dir/build
make[1]: Entering directory '/home/Scope/libwebp2/build/CMakeFiles/CMakeTmp'
Building CXX object CMakeFiles/cmTC_f12f3.dir/src.cxx.obj

/C/msys64/mingw64/bin/g++.exe   -DWP2_HAVE_FLAG___SSE4_2__ -o CMakeFiles/cmTC_f12f3.dir/src.cxx.obj -c /C/msys64/home/Scope/libwebp2/build/CMakeFiles/CMakeTmp/src.cxx
C:/msys64/home/Scope/libwebp2/build/CMakeFiles/CMakeTmp/src.cxx: In function 'int main()':
C:/msys64/home/Scope/libwebp2/build/CMakeFiles/CMakeTmp/src.cxx:4:9: error: invalid use of 'this' in non-member function
    4 |         this is not valid code
      |         ^~~~
make[1]: *** [CMakeFiles/cmTC_f12f3.dir/build.make:85: CMakeFiles/cmTC_f12f3.dir/src.cxx.obj] Error 1
make[1]: Leaving directory '/home/Scope/libwebp2/build/CMakeFiles/CMakeTmp'
make: *** [Makefile:140: cmTC_f12f3/fast] Error 2

Source file was:

      int main(void) {
        #if !defined(__SSE4_2__)
        this is not valid code
        #endif
        return 0;
      }

[Jamaika]

I would also add this in vwp2.cc:

#if defined(__unix__) || defined(__CYGWIN__)
  } else if (key == 'x' || key == 'X') {
    Modify(wp2dbg_value_, 2u, key == 'x');
    setenv("WP2DBG", (wp2dbg_value_ == 0) ? "0" : "1", 1);
    if (!EncodeAndDecode()) return;
    AddInfo(SPrintf("export WP2DBG=%d", wp2dbg_value_));
#endif

I would also add this in av1enc.cc:

#if defined(__unix__) || defined(__CYGWIN__)
  for (size_t i = 0; i < std::max((size_t)1, env_vars.size()); ++i) {
#endif
...
#if defined(__unix__) || defined(__CYGWIN__)
      if (!env_vars.empty()) {
        setenv(env_vars[i].first.c_str(), env_vars[i].second.c_str(), 1);
        if (env_vars.size() > 1) {
          // Only append it to the file name if there are several env vars.
          suffix = "_" + env_vars[i].first;
          if (!env_vars[i].second.empty()) suffix += "_" + env_vars[i].second;
        }
      }
#endif
...
#if defined(__unix__) || defined(__CYGWIN__)
      if (!env_vars.empty()) {
        unsetenv(env_vars[i].first.c_str());
      }
#endif
...
#if defined(__unix__) || defined(__CYGWIN__)
  }
#endif

[skal]

Originally Posted by Jamaika

I would also add this in vwp2.cc:
...

looks like we need a replacement for setenv/getenv under Windows env. Thanks for pointing this out!