Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > VP9 and AV1

Reply
 
Thread Tools Search this Thread Display Modes
Old 7th August 2020, 06:35   #2241  |  Link
foxyshadis
ангел смерти
 
foxyshadis's Avatar
 
Join Date: Nov 2004
Location: Lost
Posts: 9,558
Quote:
Originally Posted by nhw_pulsar View Post
Hello @foxyshadis,

Hope that I am not trolling too much, I of course agree on the technical side with you and the other impressive reference members here, but I contacted you about Xiph as you seem to know well Monty and this organization.

Do you think Xiph can be interested in the NHW Project? Unfortunately I can not have contact with them, and maybe just like Alliance for Open Media, Xiph is not interested in NHW because it does not work for any image resolution? And that's why my submissions at Xiph and AOM are ignored? I thought that NHW could be a good project for Xiph... (that's only my opinion of course), and certainly a better fit than AOM, but maybe Xiph also only supports excellent codecs and they don't estimate that NHW is one of them?

Many thanks.
Cheers,
Raphael
If you have something that pushes the state of the art, especially if it can be dropped in to a small code segment, not the whole codebase, and you are willing to give it away patent-free and can verify that no one else has patents on it, AOM wants to hear from you.

But they had to deal with getting things encoded and decoded in a reasonable time. AV1 seems slow, but it's miles ahead of what it could have been. Like MPEG, it chops out anything that isn't fast enough to make the cut, and maybe a refinement will make it next generation.
foxyshadis is offline   Reply With Quote
Old 7th August 2020, 09:14   #2242  |  Link
nhw_pulsar
Registered User
 
Join Date: Apr 2017
Posts: 171
Quote:
Originally Posted by foxyshadis View Post
If you have something that pushes the state of the art, especially if it can be dropped in to a small code segment, not the whole codebase, and you are willing to give it away patent-free and can verify that no one else has patents on it, AOM wants to hear from you.

But they had to deal with getting things encoded and decoded in a reasonable time. AV1 seems slow, but it's miles ahead of what it could have been. Like MPEG, it chops out anything that isn't fast enough to make the cut, and maybe a refinement will make it next generation.
Many thanks for your answer.

Yes, I think there are new ideas/processings in the NHW Project that can give interesting "state-of-the-art" results, I don't think they are patended because I never saw them described in the Internet nor in the litterature, and so I am totally willing to give them to AOM patent-free.

The "big" problem is that these new ideas/processings are completely tailored for wavelet coding and wavelet decomposition, I don't think they are adaptable/transposable to DCT AV1 codebase for example... And so that's maybe why AOM always answered me that they were not interested in NHW?

Cheers,
Raphael
nhw_pulsar is offline   Reply With Quote
Old 9th August 2020, 17:59   #2243  |  Link
nhw_pulsar
Registered User
 
Join Date: Apr 2017
Posts: 171
Hello,

Just a quick reply, it seems that wavelets are not well-suited for current highly-efficient video codecs with block-based motion compensation/estimation, and so I don't think AOM wants to include then NHW in one of its video codec...

However NHW seems well-suited for an image codec, because it has state-of-the-art results for 0.4bpp to 2bpp which is the Internet range (NHW is not good for extreme compression for now, which can also be a problem for a video codec...), it is also very fast which is an advantage for mobile devices...

Again I am totally open to give my technology to AOM for free, and maybe they'll review it, but for now, all the answers I had from AOM, Google, are: "sorry, we are not interested" or "sorry, we don't have time to study your work"... This is very brief... @foxyshadis, I am very sorry for my impoliteness, maybe you would have contact within AOM and maybe you could inform me what's blocking with NHW? What would need to be changed/improved? Because it would help me a lot to have such advice, and to eventually know what to improve and maybe then become of consideration/interest for AOM?

Cheers,
Raphael

Last edited by nhw_pulsar; 9th August 2020 at 22:24.
nhw_pulsar is offline   Reply With Quote
Old 11th August 2020, 00:27   #2244  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,751
Quote:
Originally Posted by foxyshadis View Post
From the Graveyard of Dead Tools post, it just never worked as well as spatial-domain, since it was another NP-hard idea. It's notable that most of the dead tool ideas came from audio coding, which is Monty's real wheelhouse, but Xiph still managed to push the state of the art and conjure up a real codec; I'm still waiting for a good intra paint plugin for Photoshop, because that tool is amazing.
Anyone who has some idea they are sure is brilliant in video coding needs to read that Graveyard of Dead Tools post to see how all sorts of smart ideas wind up not being of practical advantage. It's a good reinforcer of humility.

++ on the intra paint plugin idea!
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is online now   Reply With Quote
Old 11th August 2020, 10:16   #2245  |  Link
nhw_pulsar
Registered User
 
Join Date: Apr 2017
Posts: 171
Quote:
Originally Posted by benwaggoner View Post
Anyone who has some idea they are sure is brilliant in video coding needs to read that Graveyard of Dead Tools post to see how all sorts of smart ideas wind up not being of practical advantage. It's a good reinforcer of humility.

++ on the intra paint plugin idea!
Yes, I have also theoretical ideas for a wavelet video codec, but I also fear that they turn out of no practical advantage.

Very quickly, I wanted to rectify my previous post because I completely forgot that an engineer from Google told me that NHW has serious aliasing and discoloration artifacts that must be corrected.For aliasing, I thought about a post-processing function in the decoder which will detect aliasing and remove it from the decoded Y luma comp, but I must admit that I am ultra lazy and also demotivated for now... For discoloration, it can be corrected but I want to do this with Chroma from Luma technique because it will also save quite a lot of bits.

So yes NHW has some drawbacks, and there is a reason why the industry has chosen AVIF and JPEG XL as the new image compression standards.I think they have certainly evaluated the pros and the cons of the different solutions/codecs, and so made that choice, and I totally respect it of course because they are a lot more skilled than me to evaluate it.

Just to finish, if I can advertise my skills, I think I have a good knowledge of wavelet coding, and if you would have such projects, I am very interested and could work on it with a freelance contract for example... Image/video compression is a passion for me (and also as I struggle hard with jobs here), and I would like to live of it now...

I will try not to pollute that much the AOM thread now.

Cheers,
Raphael

Last edited by nhw_pulsar; 11th August 2020 at 10:23.
nhw_pulsar is offline   Reply With Quote
Old 13th August 2020, 20:05   #2246  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,751
Quote:
Originally Posted by nhw_pulsar View Post
So yes NHW has some drawbacks, and there is a reason why the industry has chosen AVIF and JPEG XL as the new image compression standards.I think they have certainly evaluated the pros and the cons of the different solutions/codecs, and so made that choice, and I totally respect it of course because they are a lot more skilled than me to evaluate it.
There is also a HUGE advantage to technologies that get broadly implemented in HW decoders. The long term trend is absolutely towards using IDR frames of video codecs for still image encoding to maximize decode speed and reliability. JPEG in software is okay because it is very simple and fast to decode. But with more complex and efficient image coding, decoding complexity goes up and HW has an advantage. While an individual frame isn't such a big deal, but doing things like generating lots of thumbnails from JPEG can be quite slow even on fast computers today.[/QUOTE]

Quote:
Just to finish, if I can advertise my skills, I think I have a good knowledge of wavelet coding, and if you would have such projects, I am very interested and could work on it with a freelance contract for example... Image/video compression is a passion for me (and also as I struggle hard with jobs here), and I would like to live of it now...
For an individual contributor, the real money is in better implementation of standards than in trying to create new standards or formats. Figuring out how to tune video encoders for better still images would be a valuable offer as a contractor. While the bitstream is the same, there's lots of stuff that an encoder does to optimize for moving images that isn't appropriate for still images. Once interframe coherancy is irrelevant, lots of different choices become optimal. For example, x264's --tune stillimage mode really:

Code:
- stillimage (psy tuning):
--aq-strength 1.2
--deblock -3:-3
--psy-rd 2.0:0.7
And even those didn't get much emperical testing

Given the huge increase in tools available in AV1, HEVC, and VVC, I'm sure optimal tunings would be correspondingly more complex. And improving content adaption is a huge deal. Coding a pure natural image photograph is very different from encoding a screen shot, which is different from an iamge that combines rendered text, graphics, and natural photography.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is online now   Reply With Quote
Old 13th August 2020, 20:47   #2247  |  Link
nhw_pulsar
Registered User
 
Join Date: Apr 2017
Posts: 171
Quote:
Originally Posted by benwaggoner View Post
There is also a HUGE advantage to technologies that get broadly implemented in HW decoders. The long term trend is absolutely towards using IDR frames of video codecs for still image encoding to maximize decode speed and reliability. JPEG in software is okay because it is very simple and fast to decode. But with more complex and efficient image coding, decoding complexity goes up and HW has an advantage. While an individual frame isn't such a big deal, but doing things like generating lots of thumbnails from JPEG can be quite slow even on fast computers today.
Yes, I agree with you and HW decoders have an advantage.But I still wanted to emphasize that NHW is extremely fast to encode/decode, and I even think that software NHW decoder will be very faster than hardware HEVC, AV1, VVC decoders.For example with the same level of (software) optimization, NHW is around 15x faster to decode than x265 (optimized HEVC)!


Quote:
Originally Posted by benwaggoner View Post
For an individual contributor, the real money is in better implementation of standards than in trying to create new standards or formats. Figuring out how to tune video encoders for better still images would be a valuable offer as a contractor. While the bitstream is the same, there's lots of stuff that an encoder does to optimize for moving images that isn't appropriate for still images.
Yes, it could be very interesting to tune video encoders for better still images, because I generally find that they lack of neatness, at least as a still image.And I have also developed processings that enhance neatness and that are not related to wavelet coding, and so transposable to any compression scheme.Yes neatness is very subjective, but really for me, despite NHW has far and far worse PSNR and SSIM scores than x265, AVIF, I still find that its results are visually more pleasant.So I do think that psychovisual tuning for still image is very important, and it would be great to work on it.
-For the little story, I did not intend to create a new standard, it's just I had very interesting course at university on wavelets in 2004-2005, and I absolutely did not have knowledge on DCT, and so naturally I orientated towards wavelets and played at home with them to try to see how far they can go...-

Many thanks again for your answer and your time Sir.
Cheers,
Raphael

Last edited by nhw_pulsar; 13th August 2020 at 20:54.
nhw_pulsar is offline   Reply With Quote
Old 21st August 2020, 05:46   #2248  |  Link
foxyshadis
ангел смерти
 
foxyshadis's Avatar
 
Join Date: Nov 2004
Location: Lost
Posts: 9,558
Quote:
Originally Posted by nhw_pulsar View Post
Many thanks for your answer.

Yes, I think there are new ideas/processings in the NHW Project that can give interesting "state-of-the-art" results, I don't think they are patended because I never saw them described in the Internet nor in the litterature, and so I am totally willing to give them to AOM patent-free.

The "big" problem is that these new ideas/processings are completely tailored for wavelet coding and wavelet decomposition, I don't think they are adaptable/transposable to DCT AV1 codebase for example... And so that's maybe why AOM always answered me that they were not interested in NHW?

Cheers,
Raphael
Unfortunately, before they're willing to consider the technical merits of your idea, you have to prove the legal merits, namely that it is not patented, that it's different enough from similar patents, or that you own a patent to the technology that you'll willing to sign over to AOMedia. "Haven't seen it before" isn't enough of a guarantee, because there are just way too many niche things in journals and patents out there.

AV1 does have a few pieces that really were designed specifically to benefit non-standard use cases, like still images and desktop streaming, so it's not entirely done for. But they won't take anything for AV2 that doesn't pass the patent minefield.

And yeah, you'll have to make some attempt at integrating the tool to prove it can help some use case, otherwise it's just another idea on the Mount Everest of ideas.
foxyshadis is offline   Reply With Quote
Old 21st August 2020, 08:37   #2249  |  Link
nhw_pulsar
Registered User
 
Join Date: Apr 2017
Posts: 171
Quote:
Originally Posted by foxyshadis View Post
Unfortunately, before they're willing to consider the technical merits of your idea, you have to prove the legal merits, namely that it is not patented, that it's different enough from similar patents, or that you own a patent to the technology that you'll willing to sign over to AOMedia. "Haven't seen it before" isn't enough of a guarantee, because there are just way too many niche things in journals and patents out there.

AV1 does have a few pieces that really were designed specifically to benefit non-standard use cases, like still images and desktop streaming, so it's not entirely done for. But they won't take anything for AV2 that doesn't pass the patent minefield.
Yes, you're right and I completely understand the very deep imperatives of AOM concerning patents.Just, for example, searching the whole US patents database website for prior art/patents will be quite of a hard task for me... and also I don't have the money to pay for a patent lawyer for that unfortunately... Do you think I can receive some help for that task? Also the thing that will be hard to defend is that most of the main ideas of NHW were done in 2007-2008, but from 2007 to 2012, NHW was close-source, it's just from 2012 that it was open-source, but really the main ideas are of 2007/2008.

>AV1 does have a few pieces that really were designed specifically to benefit non-standard use cases

That's great news, and it gives me a little hope now with AOM, thank you for letting me know, even if I know that it will be very difficult.But first how can I clear the patent concern?

Quote:
Originally Posted by foxyshadis View Post
And yeah, you'll have to make some attempt at integrating the tool to prove it can help some use case, otherwise it's just another idea on the Mount Everest of ideas.
Quite frankly, it will be very diffcult for me to integrate NHW or some of his tools in the AV1 code... For the tools, the main idea not related to wavelets that comes to mind, is to perform a pre-sharpening (with laplacian kernel, quite old technique) of the Y comp (at the very begining just after colorspace conversion) to enhance neatness of the results, but I don't know if it'll work with AV1, because I have read that DCT quantization naturally tends to sharpen image...

Cheers,
Raphael
nhw_pulsar is offline   Reply With Quote
Old 21st August 2020, 23:26   #2250  |  Link
nhw_pulsar
Registered User
 
Join Date: Apr 2017
Posts: 171
@foxyshadis (and to the other members),

I have read your post today that SVT-AV1 will become the AV1 "production" encoder because of its reasonable complexity, and you also wrote: "aomenc will continue as a research codec for AV2 development."

So AV2 will be based on aomenc and so I guess encoding time won't certainly be the problem.I have even read that a AOM founding member researcher said that for AV2, they are deeply devoted to really introduce ML/AI, for example for the good representation(/segmentation) of objects/shapes and better understand their motion and so further improve compression.

So from my understanding, AV2, based on aomenc, will be an experimental research codec that will further compress over AV1 and VVC and so will have exceptional PSNR and SSIM scores at the expense of a very "huge" encoder(/decoder) complexity/time.When I try to think about NHW in that picture, it seems contrary actually, because NHW strong point is extremely fast encoder/decoder with very good visual aspect but poor PSNR and SSIM scores.

So I start to have big doubts again that AOM could be interested in NHW for AV2, plus all the negative answers I had from AOM these last years, all this makes me very pessimistic again...

@foxyshadis, could you confirm what you said and do you really think AOM could be interested in NHW for few non-standard use cases like still image?

It would be great if you could give me your point of view for NHW and AOM codecs, even if it would be severe and negative (would still help me).

Cheers,
Raphael
nhw_pulsar is offline   Reply With Quote
Old 25th August 2020, 14:46   #2251  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 6,753
New uploads: (MSYS2; MinGW32 / MinGW64: GCC 10.2.0)

AOM v2.0.0-762-g7e235b0d9

rav1e 0.3.0 (325ae51 / 2020-08-25)

dav1d 0.7.1 (d0e50cac / 2020-08-25)
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Old 26th August 2020, 07:10   #2252  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 6,753
avif-0.8.1 p20200818-10-g325ae515 (MSYS2/MinGW, GCC 10.2.0, current rust for rav1e library)
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Old 28th August 2020, 08:46   #2253  |  Link
hajj_3
Registered User
 
Join Date: Mar 2004
Posts: 1,120
Chrome 85 can now view .AVIF files. Microsoft Edge 85 cannot view .AVIF files which is a little strange as they both use chromium, i guess microsoft decided not to adopt that functionality right away.
hajj_3 is offline   Reply With Quote
Old 28th August 2020, 09:15   #2254  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 6,753
Vivaldi 3.3.2022.6 is based on Chrome/85.0.4183.84 and displays the Netflix samples.

Firefox 80.0 does not.
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Old 29th August 2020, 00:22   #2255  |  Link
birdie
Artem S. Tashkinov
 
birdie's Avatar
 
Join Date: Dec 2006
Posts: 337
Quote:
Originally Posted by LigH View Post
Vivaldi 3.3.2022.6 is based on Chrome/85.0.4183.84 and displays the Netflix samples.

Firefox 80.0 does not.
It does if you set image.avif.enabled to True.
birdie is offline   Reply With Quote
Old 29th August 2020, 08:39   #2256  |  Link
foxyshadis
ангел смерти
 
foxyshadis's Avatar
 
Join Date: Nov 2004
Location: Lost
Posts: 9,558
Quote:
Originally Posted by birdie View Post
It does if you set image.avif.enabled to True.
Nice, flipping that right now! Looking at the bug history (particularly #1625363), it looks like turning it on by default was held up first because they wanted to switch to dav1d, and now because they're overhauling the entire media handling stack and it's sort of in limbo until that's done. At least there's an easy way to test it out, though.
foxyshadis is offline   Reply With Quote
Old 31st August 2020, 07:53   #2257  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 6,753


You are right, birdie.
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Old 1st September 2020, 17:55   #2258  |  Link
Yups
Registered User
 
Join Date: Sep 2011
Posts: 362
Quote:
Originally Posted by GTPVHD View Post
https://images.anandtech.com/doci/16...0901173558.jpg

Nvidia confirms AV1 hardware decoding on Ampere GPUs.
Quote:

Stream and playback videos at 8K HDR using the new AV1 decoder on GeForce RTX 30 Series GPUs, which enables more efficient playback of 8K HDR YouTube videos. With much more efficient compression than H.265, AV1 dramatically lowers the internet bandwidth requirements you need to stream high resolution video.

Good news for AV1 in general having AV1 support from Intel and Nvidia in their next gen GPUs. Only AMD is missing, we have to wait for RNDA2 if it supports AV1 as well.
Yups is offline   Reply With Quote
Old 1st September 2020, 18:19   #2259  |  Link
hajj_3
Registered User
 
Join Date: Mar 2004
Posts: 1,120
great news
hajj_3 is offline   Reply With Quote
Old 1st September 2020, 18:34   #2260  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,751
Quote:
Originally Posted by Yups View Post
Good news for AV1 in general having AV1 support from Intel and Nvidia in their next gen GPUs. Only AMD is missing, we have to wait for RNDA2 if it supports AV1 as well.
This is a big deal!

AMD does typically lag NVidia for a year or more in video capabilities, so I wouldn't make any assumptions about their next gen before they announce something.

At this point, it's really Qualcomm and AMD needed. Although given the much slower replacement rate of PCs these days, it'll still be at least a decade before it'll be safe to assume all PCs would have HW AV1 decode. IIRC, it was around 2012 before it was quite rare to have a PC without HW H.264.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is online now   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 00:05.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.