Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > VP9 and AV1

Reply
 
Thread Tools Search this Thread Display Modes
Old 27th July 2020, 20:47   #2321  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 3,284
Quote:
Originally Posted by mzso View Post
"Or it could be that we luckily hit upon the right essential transform that balances spatial and temporal prediction better than available alternatives."

I highly doubt it. I think it's more likely the illusion of familiarity and the blinders that come with it.

"Some of this could be because of the momentum of R&D around the traditional stuff."

It certainly seems like that alternatives got only limited efforts on them.
Yeah. It's really hard to disprove the hypothesis that there could be better fundamental ways of encoding.

That said, wavelets sure got a lot of attention for image and motion coding. Good for images, but no one figured out an efficient motion compensation strategy for it.

Daala had a lot of really intriguing notions, but the most interesting stuff in it never really got to a promising proof of concept. Sure, maybe with 10 years of 1000 engineers something could be found. Any alternative transforms have to compete with decades of refinement of block-based frequency coding.

A lot of promising ideas get figured out how to port into a block-based structure. For example, HEVC's transform skip mode can make anime, graphics, and text way easier to encode at low bitrates and high quality. So new features, like have been seen in VVC and AV1, can get included as tools. Arguably, once you have 64x64 or bigger blocks, you've pretty much got all the advantages of wavelet coding already, within a block based model. And intra-frame prediction brings a lot of the potential value of fractal encoding.

One exciting thing (to me at least) about Daala that didn't make it into AV1 was doing frequency-domain prediction, so there was no need to rasterize a frame that wasn't going to get displayed, and dithering didn't need to be included in quantization. It didn't work out for reasons I don't quite recall.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 29th July 2020, 20:17   #2322  |  Link
nhw_pulsar
Registered User
 
Join Date: Apr 2017
Posts: 14
Hello,

Thank you for your nice comment about wavelets for images (as I have also made a wavelet image codec, called NHW...).

You said: "but no one figured out an efficient motion compensation strategy for it". Do you think this is this aspect that prevents organizations such as Alliance for Open Media from starting and supporting a wavelet codec?

Cheers,
Raphael
nhw_pulsar is offline   Reply With Quote
Old 1st August 2020, 11:13   #2323  |  Link
foxyshadis
ангел смерти
 
foxyshadis's Avatar
 
Join Date: Nov 2004
Location: Lost
Posts: 9,509
Quote:
Originally Posted by benwaggoner View Post
Arguably, once you have 64x64 or bigger blocks, you've pretty much got all the advantages of wavelet coding already, within a block based model.
I remember in the late 90's, first working with codecs in code, thinking that 8x8 must be some kind of fundamental limit of DCT, and wavelets must be superior since they can go from 128x128 all the way to 4096x4096 in JPEG2000. No, it turns out engineers were just excited about the new hotness instead of extending the old battleaxe, DCT, plus it would take at least until SSE2 to really be able to optimize transforms larger than 8x8.

I still think that *lets, curvelets, ridgelets, etc, could help further reduce still images/I-frames, but all the new prediction modes have really put a huge dent in how residuals look.

Quote:
Originally Posted by benwaggoner View Post
One exciting thing (to me at least) about Daala that didn't make it into AV1 was doing frequency-domain prediction, so there was no need to rasterize a frame that wasn't going to get displayed, and dithering didn't need to be included in quantization. It didn't work out for reasons I don't quite recall.
From the Graveyard of Dead Tools post, it just never worked as well as spatial-domain, since it was another NP-hard idea. It's notable that most of the dead tool ideas came from audio coding, which is Monty's real wheelhouse, but Xiph still managed to push the state of the art and conjure up a real codec; I'm still waiting for a good intra paint plugin for Photoshop, because that tool is amazing.
__________________
There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order.
foxyshadis is offline   Reply With Quote
Old 2nd August 2020, 10:28   #2324  |  Link
nhw_pulsar
Registered User
 
Join Date: Apr 2017
Posts: 14
Hello @foxyshadis,

Hope that I am not trolling too much, I of course agree on the technical side with you and the other impressive reference members here, but I contacted you about Xiph as you seem to know well Monty and this organization.

Do you think Xiph can be interested in the NHW Project? Unfortunately I can not have contact with them, and maybe just like Alliance for Open Media, Xiph is not interested in NHW because it does not work for any image resolution? And that's why my submissions at Xiph and AOM are ignored? I thought that NHW could be a good project for Xiph... (that's only my opinion of course), and certainly a better fit than AOM, but maybe Xiph also only supports excellent codecs and they don't estimate that NHW is one of them?

Many thanks.
Cheers,
Raphael
nhw_pulsar is offline   Reply With Quote
Old 3rd August 2020, 07:43   #2325  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 6,157
Dead tools and video codecs and wavelets ... hmm ... I believe I still have a copy of Rududu.
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Old 3rd August 2020, 09:33   #2326  |  Link
nhw_pulsar
Registered User
 
Join Date: Apr 2017
Posts: 14
I did not test Rududu video codec, but I have tested the latest Rududu Image codec (RIC) and it is very good.If I remember correctly, RIC is kind of enhanced and state-of-the-art SPIHT, which is a different technology from NHW.-For the little story, when Rududu author released RIC in march 2008, I was totally blown away by its very impressive results on objective metrics like PSNR and by its very good precision, and then I realized that I could not be at that level of PSNR and precision with NHW, and so then I definitely decided to orientate NHW towards neatness and visual aspect.-

To come back on-topic, yes possibly in the late 90's with JPEG2000, wavelets were the hotness, but frankly since 2001, DCT block-based intra prediction+residual coding is really the main research focus of the industry.Wavelet compression research has been abandoned for years (by industry) actually, the last release of Dirac was in 2008, the last release of Rududu was also in 2008, Snow is around 2008, and most of the main ideas of NHW were also made in 2008...

Who believes organizations like AOM could restart wavelet compression technology today?

Cheers,
Raphael
nhw_pulsar is offline   Reply With Quote
Old 5th August 2020, 20:59   #2327  |  Link
GTPVHD
Registered User
 
Join Date: Mar 2008
Posts: 255
https://www.intc.com/investor-relati...h/default.aspx

Tiger Lake Virtual Launch Event on September 2, 2020.

AV1 fixed-function hardware decoding coming soon to PC.
GTPVHD is offline   Reply With Quote
Old 7th August 2020, 01:25   #2328  |  Link
Yups
Registered User
 
Join Date: Sep 2011
Posts: 213
A new slide appeared on imgur with the media capabilities of Tigerlake-U.





Previously it was 4k60 and here it's 8k30 AV1.

Last edited by Yups; 7th August 2020 at 07:02.
Yups is offline   Reply With Quote
Old 7th August 2020, 05:15   #2329  |  Link
NikosD
Registered User
 
Join Date: Aug 2010
Location: Athens, Greece
Posts: 2,761
Quote:
Originally Posted by Yups View Post
A new slide appeared on imgur with the media capabilities of Tigerlake-U.
They have increased the speed - from 8K30 to 8K60 - of HEVC/VP9 decoder too.
They have added 12bit HEVC/VP9 decoding.
Also, that SCC of the table means Screen Content Coding and it's a HEVC profile/extension, optimized for screen captured content.
It could be used by streaming apps/services like YouTube, Skype, Zoom, Netflix etc but I don't know the real use of this extension.
And it's the first time I see this in the supported features of any decoder.
__________________
Win 10 x64 (19041.388) - Core i3-9100F - nVidia 1660 (451.67)
HEVC decoding benchmarks
H.264 DXVA Benchmarks for all

Last edited by NikosD; 7th August 2020 at 05:22.
NikosD is offline   Reply With Quote
Old 7th August 2020, 06:35   #2330  |  Link
foxyshadis
ангел смерти
 
foxyshadis's Avatar
 
Join Date: Nov 2004
Location: Lost
Posts: 9,509
Quote:
Originally Posted by nhw_pulsar View Post
Hello @foxyshadis,

Hope that I am not trolling too much, I of course agree on the technical side with you and the other impressive reference members here, but I contacted you about Xiph as you seem to know well Monty and this organization.

Do you think Xiph can be interested in the NHW Project? Unfortunately I can not have contact with them, and maybe just like Alliance for Open Media, Xiph is not interested in NHW because it does not work for any image resolution? And that's why my submissions at Xiph and AOM are ignored? I thought that NHW could be a good project for Xiph... (that's only my opinion of course), and certainly a better fit than AOM, but maybe Xiph also only supports excellent codecs and they don't estimate that NHW is one of them?

Many thanks.
Cheers,
Raphael
If you have something that pushes the state of the art, especially if it can be dropped in to a small code segment, not the whole codebase, and you are willing to give it away patent-free and can verify that no one else has patents on it, AOM wants to hear from you.

But they had to deal with getting things encoded and decoded in a reasonable time. AV1 seems slow, but it's miles ahead of what it could have been. Like MPEG, it chops out anything that isn't fast enough to make the cut, and maybe a refinement will make it next generation.
__________________
There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order.
foxyshadis is offline   Reply With Quote
Old 7th August 2020, 09:14   #2331  |  Link
nhw_pulsar
Registered User
 
Join Date: Apr 2017
Posts: 14
Quote:
Originally Posted by foxyshadis View Post
If you have something that pushes the state of the art, especially if it can be dropped in to a small code segment, not the whole codebase, and you are willing to give it away patent-free and can verify that no one else has patents on it, AOM wants to hear from you.

But they had to deal with getting things encoded and decoded in a reasonable time. AV1 seems slow, but it's miles ahead of what it could have been. Like MPEG, it chops out anything that isn't fast enough to make the cut, and maybe a refinement will make it next generation.
Many thanks for your answer.

Yes, I think there are new ideas/processings in the NHW Project that can give interesting "state-of-the-art" results, I don't think they are patended because I never saw them described in the Internet nor in the litterature, and so I am totally willing to give them to AOM patent-free.

The "big" problem is that these new ideas/processings are completely tailored for wavelet coding and wavelet decomposition, I don't think they are adaptable/transposable to DCT AV1 codebase for example... And so that's maybe why AOM always answered me that they were not interested in NHW?

Cheers,
Raphael
nhw_pulsar is offline   Reply With Quote
Old 9th August 2020, 17:59   #2332  |  Link
nhw_pulsar
Registered User
 
Join Date: Apr 2017
Posts: 14
Hello,

Just a quick reply, it seems that wavelets are not well-suited for current highly-efficient video codecs with block-based motion compensation/estimation, and so I don't think AOM wants to include then NHW in one of its video codec...

However NHW seems well-suited for an image codec, because it has state-of-the-art results for 0.4bpp to 2bpp which is the Internet range (NHW is not good for extreme compression for now, which can also be a problem for a video codec...), it is also very fast which is an advantage for mobile devices...

Again I am totally open to give my technology to AOM for free, and maybe they'll review it, but for now, all the answers I had from AOM, Google, are: "sorry, we are not interested" or "sorry, we don't have time to study your work"... This is very brief... @foxyshadis, I am very sorry for my impoliteness, maybe you would have contact within AOM and maybe you could inform me what's blocking with NHW? What would need to be changed/improved? Because it would help me a lot to have such advice, and to eventually know what to improve and maybe then become of consideration/interest for AOM?

Cheers,
Raphael

Last edited by nhw_pulsar; 9th August 2020 at 22:24.
nhw_pulsar is offline   Reply With Quote
Old 11th August 2020, 00:27   #2333  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 3,284
Quote:
Originally Posted by foxyshadis View Post
From the Graveyard of Dead Tools post, it just never worked as well as spatial-domain, since it was another NP-hard idea. It's notable that most of the dead tool ideas came from audio coding, which is Monty's real wheelhouse, but Xiph still managed to push the state of the art and conjure up a real codec; I'm still waiting for a good intra paint plugin for Photoshop, because that tool is amazing.
Anyone who has some idea they are sure is brilliant in video coding needs to read that Graveyard of Dead Tools post to see how all sorts of smart ideas wind up not being of practical advantage. It's a good reinforcer of humility.

++ on the intra paint plugin idea!
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 11th August 2020, 10:16   #2334  |  Link
nhw_pulsar
Registered User
 
Join Date: Apr 2017
Posts: 14
Quote:
Originally Posted by benwaggoner View Post
Anyone who has some idea they are sure is brilliant in video coding needs to read that Graveyard of Dead Tools post to see how all sorts of smart ideas wind up not being of practical advantage. It's a good reinforcer of humility.

++ on the intra paint plugin idea!
Yes, I have also theoretical ideas for a wavelet video codec, but I also fear that they turn out of no practical advantage.

Very quickly, I wanted to rectify my previous post because I completely forgot that an engineer from Google told me that NHW has serious aliasing and discoloration artifacts that must be corrected.For aliasing, I thought about a post-processing function in the decoder which will detect aliasing and remove it from the decoded Y luma comp, but I must admit that I am ultra lazy and also demotivated for now... For discoloration, it can be corrected but I want to do this with Chroma from Luma technique because it will also save quite a lot of bits.

So yes NHW has some drawbacks, and there is a reason why the industry has chosen AVIF and JPEG XL as the new image compression standards.I think they have certainly evaluated the pros and the cons of the different solutions/codecs, and so made that choice, and I totally respect it of course because they are a lot more skilled than me to evaluate it.

Just to finish, if I can advertise my skills, I think I have a good knowledge of wavelet coding, and if you would have such projects, I am very interested and could work on it with a freelance contract for example... Image/video compression is a passion for me (and also as I struggle hard with jobs here), and I would like to live of it now...

I will try not to pollute that much the AOM thread now.

Cheers,
Raphael

Last edited by nhw_pulsar; 11th August 2020 at 10:23.
nhw_pulsar is offline   Reply With Quote
Old 11th August 2020, 16:46   #2335  |  Link
GTPVHD
Registered User
 
Join Date: Mar 2008
Posts: 255
https://nvidianews.nvidia.com/news/n...o-jensen-huang
https://www.nvidia.com/en-us/geforce/special-event/

Nvidia launching new GPUs on Sept 1st, time to see if they support AV1 fixed-function hardware decoding.
GTPVHD is offline   Reply With Quote
Old Yesterday, 14:09   #2336  |  Link
GTPVHD
Registered User
 
Join Date: Mar 2008
Posts: 255
https://www.anandtech.com/show/15973...m-the-bottom/6
https://images.anandtech.com/doci/15973/Intel-3_25.jpg

Intel officially confirms AV1 fixed-function hardware decoding.
GTPVHD is offline   Reply With Quote
Old Yesterday, 20:05   #2337  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 3,284
Quote:
Originally Posted by nhw_pulsar View Post
So yes NHW has some drawbacks, and there is a reason why the industry has chosen AVIF and JPEG XL as the new image compression standards.I think they have certainly evaluated the pros and the cons of the different solutions/codecs, and so made that choice, and I totally respect it of course because they are a lot more skilled than me to evaluate it.
There is also a HUGE advantage to technologies that get broadly implemented in HW decoders. The long term trend is absolutely towards using IDR frames of video codecs for still image encoding to maximize decode speed and reliability. JPEG in software is okay because it is very simple and fast to decode. But with more complex and efficient image coding, decoding complexity goes up and HW has an advantage. While an individual frame isn't such a big deal, but doing things like generating lots of thumbnails from JPEG can be quite slow even on fast computers today.[/QUOTE]

Quote:
Just to finish, if I can advertise my skills, I think I have a good knowledge of wavelet coding, and if you would have such projects, I am very interested and could work on it with a freelance contract for example... Image/video compression is a passion for me (and also as I struggle hard with jobs here), and I would like to live of it now...
For an individual contributor, the real money is in better implementation of standards than in trying to create new standards or formats. Figuring out how to tune video encoders for better still images would be a valuable offer as a contractor. While the bitstream is the same, there's lots of stuff that an encoder does to optimize for moving images that isn't appropriate for still images. Once interframe coherancy is irrelevant, lots of different choices become optimal. For example, x264's --tune stillimage mode really:

Code:
- stillimage (psy tuning):
--aq-strength 1.2
--deblock -3:-3
--psy-rd 2.0:0.7
And even those didn't get much emperical testing

Given the huge increase in tools available in AV1, HEVC, and VVC, I'm sure optimal tunings would be correspondingly more complex. And improving content adaption is a huge deal. Coding a pure natural image photograph is very different from encoding a screen shot, which is different from an iamge that combines rendered text, graphics, and natural photography.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old Yesterday, 20:47   #2338  |  Link
nhw_pulsar
Registered User
 
Join Date: Apr 2017
Posts: 14
Quote:
Originally Posted by benwaggoner View Post
There is also a HUGE advantage to technologies that get broadly implemented in HW decoders. The long term trend is absolutely towards using IDR frames of video codecs for still image encoding to maximize decode speed and reliability. JPEG in software is okay because it is very simple and fast to decode. But with more complex and efficient image coding, decoding complexity goes up and HW has an advantage. While an individual frame isn't such a big deal, but doing things like generating lots of thumbnails from JPEG can be quite slow even on fast computers today.
Yes, I agree with you and HW decoders have an advantage.But I still wanted to emphasize that NHW is extremely fast to encode/decode, and I even think that software NHW decoder will be very faster than hardware HEVC, AV1, VVC decoders.For example with the same level of (software) optimization, NHW is around 15x faster to decode than x265 (optimized HEVC)!


Quote:
Originally Posted by benwaggoner View Post
For an individual contributor, the real money is in better implementation of standards than in trying to create new standards or formats. Figuring out how to tune video encoders for better still images would be a valuable offer as a contractor. While the bitstream is the same, there's lots of stuff that an encoder does to optimize for moving images that isn't appropriate for still images.
Yes, it could be very interesting to tune video encoders for better still images, because I generally find that they lack of neatness, at least as a still image.And I have also developed processings that enhance neatness and that are not related to wavelet coding, and so transposable to any compression scheme.Yes neatness is very subjective, but really for me, despite NHW has far and far worse PSNR and SSIM scores than x265, AVIF, I still find that its results are visually more pleasant.So I do think that psychovisual tuning for still image is very important, and it would be great to work on it.
-For the little story, I did not intend to create a new standard, it's just I had very interesting course at university on wavelets in 2004-2005, and I absolutely did not have knowledge on DCT, and so naturally I orientated towards wavelets and played at home with them to try to see how far they can go...-

Many thanks again for your answer and your time Sir.
Cheers,
Raphael

Last edited by nhw_pulsar; Yesterday at 20:54.
nhw_pulsar is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 22:33.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, vBulletin Solutions Inc.