Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > New and alternative video codecs

Reply
 
Thread Tools Search this Thread Display Modes
Old 22nd November 2020, 19:34   #1  |  Link
skal
Registered User
 
Join Date: Jun 2003
Posts: 118
WebP2 image codec

Hi,

libwebp2 has been open-sourced (here), and i see some users reporting compilation issues (example).

Best is to start this thread proper and consolidate the reports here!

skal/

(tests welcome, too. Keep in mind the code is changing on a ~weekly basis, though!)
skal is offline   Reply With Quote
Old 22nd November 2020, 21:52   #2  |  Link
Jamaika
Registered User
 
Join Date: Jul 2015
Posts: 774
Errors can probably be added on the google forum. I create non-professional codecs in GCC. On Windows there is Visual Studio and others that have assembler and simd support.
Codec WebP2 works without AOM codecs. So all other AOM add-ons ie AVIF, DV1D, GAV1 will also not work under GCC.
Jamaika is offline   Reply With Quote
Old 22nd November 2020, 22:22   #3  |  Link
skal
Registered User
 
Join Date: Jun 2003
Posts: 118
Quote:
Originally Posted by Jamaika View Post
Errors can probably be added on the google forum.
Yes, compile errors with details should be reported there:

https://bugs.chromium.org/p/webp2/issues/list

Quote:
Originally Posted by Jamaika View Post
I create non-professional codecs in GCC. On Windows there is Visual Studio and others that have assembler and simd support.
Codec WebP2 works without AOM codecs. So all other AOM add-ons ie AVIF, DV1D, GAV1 will also not work under GCC.
These pre-compiled binaries links can be posted here, indeed.
skal is offline   Reply With Quote
Old 23rd November 2020, 15:48   #4  |  Link
skal
Registered User
 
Join Date: Jun 2003
Posts: 118
Btw, for definiteness here are some command line examples if you want to test encoding / decoding:

cwp2 cat.png -o cat.wp2 -d cat.wp2.png -q 44

avifenc --min 40 --max 40 -s 2 cat.png cat.avif
avifdec cat.avif cat.avif.png

cwebp cat.png -o cat.webp -q 16
dwebp cat.webp -o cat.webp.png

====
Attached Images
    
skal is offline   Reply With Quote
Old 23rd November 2020, 21:33   #5  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 3,461
So, this is an enhanced encoder for the VP8 "WebP" format?

Do we care about WebP? I've not really seen exciting real-world implementations that were enough better than JPEG to merit the extra work, or close enough to HEIF H.264 that building a JavaScript implementation of that leveraging HW H.264 decoders wouldn't make more sense at scale. And AVIF is certainly going to outperform WebP while addressing the same licensing goals.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 23rd November 2020, 21:44   #6  |  Link
VoodooFX
Registered User
 
Join Date: Sep 2008
Posts: 177
Quote:
Originally Posted by benwaggoner View Post
Do we care about WebP?
I see up to ~50% better compression over PNG in my use scenarios.
VoodooFX is offline   Reply With Quote
Old 23rd November 2020, 23:02   #7  |  Link
IgorC
Registered User
 
Join Date: Apr 2004
Posts: 1,319
Quote:
Originally Posted by benwaggoner View Post
Do we care about WebP?
I see WebP quite ofent in internet.

Chrome/Firefox/Edge support WebP.
Such popular sites in Latin America as www.mercadolibre.com uses WebP.

Quote:
Originally Posted by benwaggoner View Post
I've not really seen exciting real-world implementations that were enough better than JPEG ...
This is far from reality (as many other of your statements regarding VP8/VP9/AV1)

WebP has its advantages on certain range of bitrates https://www.reddit.com/r/AV1/comment...if_comparison/

Last edited by IgorC; 23rd November 2020 at 23:06.
IgorC is offline   Reply With Quote
Old 23rd November 2020, 23:39   #8  |  Link
skal
Registered User
 
Join Date: Jun 2003
Posts: 118
Quote:
Originally Posted by benwaggoner View Post
So, this is an enhanced encoder for the VP8 "WebP" format?

Do we care about WebP? .
Hmm... if you missed the last 10 years of WebP development, i made a recap presentation here: http://shorturl.at/BEMV8

And yes, it's used a lot.
skal is offline   Reply With Quote
Old 24th November 2020, 07:40   #9  |  Link
Jamaika
Registered User
 
Join Date: Jul 2015
Posts: 774
Quote:
Originally Posted by skal View Post
Btw, for definiteness here are some command line examples if you want to test encoding / decoding:

cwp2 cat.png -o cat.wp2 -d cat.wp2.png -q 44

avifenc --min 40 --max 40 -s 2 cat.png cat.avif
avifdec cat.avif cat.avif.png

cwebp cat.png -o cat.webp -q 16
dwebp cat.webp -o cat.webp.png
====
My test
Code:
dwp2.exe -v image_21447.wp2 -mt -o image_21447_64bit.jpg
Time to decode picture: 0.415s
Decoded image_21447.wp2. Dimensions: 1563 x 1558. Transparency: no.
Could not save to 'image_21447_64bit.jpg'
Error: unsupported feature
Status: WP2_STATUS_UNSUPPORTED_FEATURE
Code:
dwp2.exe -v image_21447.wp2 -mt -o image_21447_64bit.webp
Time to decode picture: 0.399s
Decoded image_21447.wp2. Dimensions: 1563 x 1558. Transparency: no.
Could not save to 'image_21447_64bit.webp'
Error: unsupported feature
Status: WP2_STATUS_UNSUPPORTED_FEATURE
avifenc and avifdec are a different library, unless something else has been added in the meantime.
Strange settings help av1enc. 'O' is not output and 'D' is not decode. What does not working 'o' and 'd' mean?
Code:
av1enc.exe -444 -size 1563x1558 -O image_21447_64bit.png
aom_codec_decode() failed.
Error compressing to AV1
Troublesome creating avif because you have to enter size image and chroma subsampling every time. Why are photos in webp2 and addons only 8bit? Additionally, to generate avif you need to add avif which is saved as av1.
To produce the avif with png you must add the 'D'. Why?
Code:
av1enc.exe -avif -420 -size 1563x1558 -D image_21447_64bit.avif -O image_21447_64bit.png
674650 35.965336   (99.000000 35.446957 39.947750 34.281830) [ 2.22 bpp ] [ 1563 x 1558 ] [73.223 / 0.632 secs]
Saved decoded image image_21447_64bit_dec.png
Saved encoded file: image_21447_64bit.av1 (674650 bytes) <-- ????
but
Code:
av1enc.exe -avif -444 -size 1563x1558 -D image_21447_64bit.avif -O image_21447_64bit.png
ERROR: Failed to encode: Decoding of color planes failed
Error while computing the distortion.
I will just add that when compiling libavif you should include the definition of IO. AVIF_CODEC_AOM, AVIF_CODEC_AOM_DECODE, AVIF_CODEC_AOM_ENCODE. This is an exception. Without this the accessory will not work.
File decoding RAW converted as TIFF RAW_CANON_EOS_5DS.cr2 8688x5792
Long waiting time. On my old computer thread 4 it is about 10 minutes.
Code:
cwp2.exe RAW_CANON_EOS_5DS.tiff -summary -lossless -mt -o RAW_CANON_EOS_32bit.wp2
Assertion failed: std::abs(progress_ + kProgressEnd - 1.f) < 0.001f, file progress_watcher.cc, line 52
Does WEBP2 codec handle metadata? Who knows that?

Last edited by Jamaika; 24th November 2020 at 08:35.
Jamaika is offline   Reply With Quote
Old 24th November 2020, 07:47   #10  |  Link
skal
Registered User
 
Join Date: Jun 2003
Posts: 118
Quote:
Originally Posted by Jamaika View Post
My test
Code:
...
Could not save to 'image_21447_64bit.jpg'
Error: unsupported feature
Status: WP2_STATUS_UNSUPPORTED_FEATURE
That's expected: you cannot save to a lossy format (jpg, webp), since it would defeat the purpose of showcasing WP2's compression capabilities.
skal is offline   Reply With Quote
Old 24th November 2020, 12:27   #11  |  Link
skal
Registered User
 
Join Date: Jun 2003
Posts: 118
Quote:
Originally Posted by Jamaika View Post
...
aom_codec_decode() failed.
Sorry, i'm not an avifenc/avifdec specialist, just tried it quickly on Unix.

Quote:
Originally Posted by Jamaika View Post
Why are photos in webp2 and addons only 8bit?
If you are interesting in non-8bit input/output, you can have a look at some tools in the 'extras' directory. For instance 'ewp2'. They can handle Y4M format.

Quote:
Originally Posted by Jamaika View Post
File decoding RAW converted as TIFF RAW_CANON_EOS_5DS.cr2 8688x5792
Long waiting time. On my old computer thread 4 it is about 10 minutes.
Yes, lossless coding can be quite slow for such large photos. Note that compressing losslessly photo material is not the main use-case case WebP2 is targetting!

Quote:
Originally Posted by Jamaika View Post
Code:
Assertion failed: std::abs(progress_ + kProgressEnd - 1.f) < 0.001f, file progress_watcher.cc, line 52
This is a know problem that has been fixed in the latest sync patch.
You may want to recompile at HEAD to fix it (commit df4ec90a2befb79ed45cf8810e8ac93da7a47297).

Quote:
Originally Posted by Jamaika View Post
Does WEBP2 codec handle metadata? Who knows that?
Yes. By default, cwp2 will copy EXIF, XMP and ICC metadata.
skal is offline   Reply With Quote
Old 24th November 2020, 12:54   #12  |  Link
Scope
Registered User
 
Join Date: Feb 2020
Posts: 16
WebP 2 df4ec90, MinGW 64, GCC 10.2.0

dwp2
Code:
[ 90%] Building CXX object CMakeFiles/dwp2.dir/examples/dwp2.cc.obj
C:/msys64/home/Scope/libwebp2/examples/dwp2.cc: In function 'std::string {anonymous}::PrintSummary(const WP2::BitstreamFeatures&, size_t, bool)':
C:/msys64/home/Scope/libwebp2/examples/dwp2.cc:207:37: error: unknown conversion type character 'z' in format [-Werror=format=]
  207 |     str += SPrintf("file size:     %zu\n", input_data_size);
      |                                     ^
[ 91%] Building CXX object CMakeFiles/extras.dir/extras/extras.cc.obj
cc1plus.exe: all warnings being treated as errors
make[2]: *** [CMakeFiles/dwp2.dir/build.make:82: CMakeFiles/dwp2.dir/examples/dwp2.cc.obj] Error 1
make[1]: *** [CMakeFiles/Makefile2:261: CMakeFiles/dwp2.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
CMakeError.log
Code:
Performing C++ SOURCE FILE Test WP2_HAVE_FLAG___SSE4_2__ failed with the following output:
Change Dir: C:/msys64/home/Scope/libwebp2/build/CMakeFiles/CMakeTmp

Run Build Command(s):C:/msys64/usr/bin/make.exe cmTC_f9854/fast && /usr/bin/make  -f CMakeFiles/cmTC_f9854.dir/build.make CMakeFiles/cmTC_f9854.dir/build
make[1]: Entering directory '/home/Scope/libwebp2/build/CMakeFiles/CMakeTmp'
Building CXX object CMakeFiles/cmTC_f9854.dir/src.cxx.obj

/C/msys64/mingw64/bin/g++.exe   -DWP2_HAVE_FLAG___SSE4_2__ -o CMakeFiles/cmTC_f9854.dir/src.cxx.obj -c /C/msys64/home/Scope/libwebp2/build/CMakeFiles/CMakeTmp/src.cxx
C:/msys64/home/Scope/libwebp2/build/CMakeFiles/CMakeTmp/src.cxx: In function 'int main()':
C:/msys64/home/Scope/libwebp2/build/CMakeFiles/CMakeTmp/src.cxx:4:9: error: invalid use of 'this' in non-member function
    4 |         this is not valid code
      |         ^~~~
make[1]: *** [CMakeFiles/cmTC_f9854.dir/build.make:85: CMakeFiles/cmTC_f9854.dir/src.cxx.obj] Error 1
make[1]: Leaving directory '/home/Scope/libwebp2/build/CMakeFiles/CMakeTmp'
make: *** [Makefile:140: cmTC_f9854/fast] Error 2

Source file was:
      int main(void) {
        #if !defined(__SSE4_2__)
        this is not valid code
        #endif
        return 0;
      }
Scope is offline   Reply With Quote
Old 24th November 2020, 16:58   #13  |  Link
skal
Registered User
 
Join Date: Jun 2003
Posts: 118
Quote:
Originally Posted by Scope View Post
WebP 2 df4ec90, MinGW 64, GCC 10.2.0
...

Thanks for the report, opened an issue for it.
skal is offline   Reply With Quote
Old 25th November 2020, 07:20   #14  |  Link
Jamaika
Registered User
 
Join Date: Jul 2015
Posts: 774
Quote:
Originally Posted by skal View Post
Yes. By default, cwp2 will copy EXIF, XMP and ICC metadata.
If I'm wasting my time I'm a bit of a piss off the creators.
Converting DNG to WEBP2 with metadata ICC, XMP, EXIF
Code:
TIFFFetchNormalTag: Warning, Incorrect value for "RichTIFFIPTC"; tag ignored.
Dimension:     5212 x 3468
animation:     no
quality hint:  15
preview:       no
preview color: 102, 85, 51
rotated by:    0 degrees clockwise
transfer:      0
ICC:           no   <-- ???
XMP/EXIF:      no   <-- ???
output size:  31860621 (14.10 bpp)    [RAW_CANON_EOS_32bit.wp2]
What should be the config for libtiff? The library includes RichTIFFIPTC.

Last edited by Jamaika; 25th November 2020 at 07:44.
Jamaika is offline   Reply With Quote
Old 25th November 2020, 08:13   #15  |  Link
skal
Registered User
 
Join Date: Jun 2003
Posts: 118
Quote:
Originally Posted by Jamaika View Post
Code:
TIFFFetchNormalTag: Warning, Incorrect value for "RichTIFFIPTC"; tag ignored.
...
What should be the config for libtiff? The library includes RichTIFFIPTC.
Hard to say without more details, the source image, compile flags, exact libtiff library version used, etc.

Best is to open an issue on the bug tracker, with all the infos. But it looks like a libtiff issue, honestly...
skal is offline   Reply With Quote
Old 25th November 2020, 18:15   #17  |  Link
skal
Registered User
 
Join Date: Jun 2003
Posts: 118
latest submit 7c0dceb should fix the %zu problem.

It also introduced some ~10-15% speed-up on x86 contribution (thanks Ilya!) and a '-progress' option to cwp2/dwp2 to report the enc/dec progress.


thx!
skal is offline   Reply With Quote
Old 25th November 2020, 18:54   #18  |  Link
Scope
Registered User
 
Join Date: Feb 2020
Posts: 16
WebP 2 7c0dceb, MinGW 64, GCC 10.2.0

Code:
[ 89%] Built target ansz
C:/msys64/home/Scope/libwebp2/examples/example_utils.cc: In function 'std::string WP2::PrintSummary(const WP2::BitstreamFeatures&, size_t, bool)':
C:/msys64/home/Scope/libwebp2/examples/example_utils.cc:157:20: error: unknown conversion type character 'l' in format [-Werror=format=]
  157 |     str += SPrintf("file size:     %" PRIu64 "\n", (uint64_t)input_data_size);
      |                    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
cc1plus.exe: all warnings being treated as errors
make[2]: *** [CMakeFiles/example_utils.dir/build.make:82: CMakeFiles/example_utils.dir/examples/example_utils.cc.obj] Error 1
make[1]: *** [CMakeFiles/Makefile2:463: CMakeFiles/example_utils.dir/all] Error 2
make: *** [Makefile:160: all] Error 2
This may be related to Microsoft's printf https://stackoverflow.com/questions/...er-windows-64x

CMakeError.log
Code:
Performing C++ SOURCE FILE Test WP2_HAVE_FLAG___SSE4_2__ failed with the following output:
Change Dir: C:/msys64/home/Scope/libwebp2/build/CMakeFiles/CMakeTmp

Run Build Command(s):C:/msys64/usr/bin/make.exe cmTC_f12f3/fast && /usr/bin/make  -f CMakeFiles/cmTC_f12f3.dir/build.make CMakeFiles/cmTC_f12f3.dir/build
make[1]: Entering directory '/home/Scope/libwebp2/build/CMakeFiles/CMakeTmp'
Building CXX object CMakeFiles/cmTC_f12f3.dir/src.cxx.obj

/C/msys64/mingw64/bin/g++.exe   -DWP2_HAVE_FLAG___SSE4_2__ -o CMakeFiles/cmTC_f12f3.dir/src.cxx.obj -c /C/msys64/home/Scope/libwebp2/build/CMakeFiles/CMakeTmp/src.cxx
C:/msys64/home/Scope/libwebp2/build/CMakeFiles/CMakeTmp/src.cxx: In function 'int main()':
C:/msys64/home/Scope/libwebp2/build/CMakeFiles/CMakeTmp/src.cxx:4:9: error: invalid use of 'this' in non-member function
    4 |         this is not valid code
      |         ^~~~
make[1]: *** [CMakeFiles/cmTC_f12f3.dir/build.make:85: CMakeFiles/cmTC_f12f3.dir/src.cxx.obj] Error 1
make[1]: Leaving directory '/home/Scope/libwebp2/build/CMakeFiles/CMakeTmp'
make: *** [Makefile:140: cmTC_f12f3/fast] Error 2

Source file was:

      int main(void) {
        #if !defined(__SSE4_2__)
        this is not valid code
        #endif
        return 0;
      }

Last edited by Scope; 25th November 2020 at 19:34.
Scope is offline   Reply With Quote
Old 25th November 2020, 19:55   #19  |  Link
Jamaika
Registered User
 
Join Date: Jul 2015
Posts: 774
I would also add this in vwp2.cc:
Code:
#if defined(__unix__) || defined(__CYGWIN__)
  } else if (key == 'x' || key == 'X') {
    Modify(wp2dbg_value_, 2u, key == 'x');
    setenv("WP2DBG", (wp2dbg_value_ == 0) ? "0" : "1", 1);
    if (!EncodeAndDecode()) return;
    AddInfo(SPrintf("export WP2DBG=%d", wp2dbg_value_));
#endif
I would also add this in av1enc.cc:
Code:
#if defined(__unix__) || defined(__CYGWIN__)
  for (size_t i = 0; i < std::max((size_t)1, env_vars.size()); ++i) {
#endif
...
#if defined(__unix__) || defined(__CYGWIN__)
      if (!env_vars.empty()) {
        setenv(env_vars[i].first.c_str(), env_vars[i].second.c_str(), 1);
        if (env_vars.size() > 1) {
          // Only append it to the file name if there are several env vars.
          suffix = "_" + env_vars[i].first;
          if (!env_vars[i].second.empty()) suffix += "_" + env_vars[i].second;
        }
      }
#endif
...
#if defined(__unix__) || defined(__CYGWIN__)
      if (!env_vars.empty()) {
        unsetenv(env_vars[i].first.c_str());
      }
#endif
...
#if defined(__unix__) || defined(__CYGWIN__)
  }
#endif

Last edited by Jamaika; 25th November 2020 at 19:59.
Jamaika is offline   Reply With Quote
Old 25th November 2020, 20:22   #20  |  Link
skal
Registered User
 
Join Date: Jun 2003
Posts: 118
Quote:
Originally Posted by Jamaika View Post
I would also add this in vwp2.cc:
...
looks like we need a replacement for setenv/getenv under Windows env. Thanks for pointing this out!
skal is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 19:07.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, vBulletin Solutions Inc.