Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > VP9 and AV1

Reply
 
Thread Tools Search this Thread Display Modes
Old 12th May 2021, 18:15   #1  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 3,751
Definitive link for AOM AV1 encoder parameter documentation?

I spent some time trying to find the definitive link for description of what the myriad AV1 encoder options do. Most seem to be common across implementations.

I found https://github.com/master-of-zen/Av1...ders/aomenc.md and https://ffmpeg.org/ffmpeg-codecs.html#librav1e. But where is the definitive kept-current source of that information?
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 12th May 2021, 21:53   #2  |  Link
Funky080900
Registered User
 
Join Date: Aug 2019
Posts: 9
Best I could find is the SVT-AV1 encoder guide. Might be relevant to aomenc too, since SVT-AV1 and aomenc share many setting.
Funky080900 is offline   Reply With Quote
Old 13th May 2021, 03:16   #3  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 3,751
Quote:
Originally Posted by Funky080900 View Post
Best I could find is the SVT-AV1 encoder guide. Might be relevant to aomenc too, since SVT-AV1 and aomenc share many setting.
Yeah, I can find many things that document parameters derived from AOM, but there's no good way to tell if or how they'd modified those parameters or their behavior.

I hope someday there will be an AV1 equivalent of https://x265.readthedocs.io/en/master/cli.html. Which is the best encoder documentation that I've not written myself .
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 23rd May 2021, 18:07   #4  |  Link
RanmaCanada
Registered User
 
Join Date: May 2009
Posts: 207
Quote:
Originally Posted by benwaggoner View Post
Yeah, I can find many things that document parameters derived from AOM, but there's no good way to tell if or how they'd modified those parameters or their behavior.

I hope someday there will be an AV1 equivalent of https://x265.readthedocs.io/en/master/cli.html. Which is the best encoder documentation that I've not written myself .
I myself have been trying and trying to find said documentation and from what I can see, there is nothing that is definitive, nor helpful to someone who is trying to migrate over to AV1 from different codecs. I see a lot of gatekeeping, or snippets of information, but no one really going out of their way to explain things. It's possible I may be looking in the wrong areas, as I want to try using AV1, but the documentation is extremely lacking, and that makes it frustrating as I don't want to wait literally days to test out parameters. Attempted to encode a movie earlier and it gave me a timeline of 122 days! on my 5800x.
RanmaCanada is offline   Reply With Quote
Old 23rd May 2021, 21:37   #5  |  Link
Marsu42
Huba Huba
 
Marsu42's Avatar
 
Join Date: Aug 2005
Location: Palumbian Jungle
Posts: 91
Quote:
Originally Posted by RanmaCanada View Post
Attempted to encode a movie earlier and it gave me a timeline of 122 days! on my 5800x.
Personally, I've gotten encoding guidelines from reddit's r/AV1 - beside enthusiasts that have figured out the settings some devs write there (like BlueSwordM):

https://www.reddit.com/r/AV1/comment...thesis_how_it/

https://www.reddit.com/r/AV1/comment...cav1libaomav1/
__________________
"The innocent have nothing to fear" :stupid:
Marsu42 is offline   Reply With Quote
Old 25th May 2021, 01:46   #6  |  Link
RanmaCanada
Registered User
 
Join Date: May 2009
Posts: 207
And still no documention as to why AV1 doesn't respect --threads. No matter what I do I can not get it to use more than 1 thread.. I would love to use the encoder, but there is still too much "magic" or "try settings and figure it out yourself" and when it's single threaded (because I can't get --threads to work on any builds, maybe I'm cursed!), that takes far, FAR too long. Maybe I'm spoiled because of the awesome documentation that x265 and x264 had.
RanmaCanada is offline   Reply With Quote
Old 25th May 2021, 03:28   #7  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 3,751
I suppose it is the Modern Way of Things, but I really appreciate a description of what parameters do with input from the people who implemented it. "Read the source" is gallows humor, not an actual plan .

Although only Media Cleaner Pro's manuals and the x265 readthedocs have really met that bar in the last 25 years. And those projects happened because the project leaders personally took responsibility to write a lot of the documentation themselves. The CEO of Terran Interactive (who made Media Cleaner Pro) liked to do product planning by writing the manual for the new version and having the developers implement that.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 27th May 2021, 05:00   #8  |  Link
RanmaCanada
Registered User
 
Join Date: May 2009
Posts: 207
Yes. I think it is pretty stupid that all these companies want to make this the next big codec, but are refusing to explain to people how to use it. I guess it's just more proof that AV1 was never intended for the masses, and was just a way for the media cartels to stop paying royalties, like Disney is attempting to do with authors.
RanmaCanada is offline   Reply With Quote
Old 27th May 2021, 11:17   #9  |  Link
Marsu42
Huba Huba
 
Marsu42's Avatar
 
Join Date: Aug 2005
Location: Palumbian Jungle
Posts: 91
Quote:
Originally Posted by benwaggoner View Post
I suppose it is the Modern Way of Things, but I really appreciate a description of what parameters do with input from the people who implemented it. "Read the source" is gallows humor, not an actual plan .
Well, I have certainly read the source to figure out the params, and we should rename RTFM to RTFS :-)

Quote:
Originally Posted by RanmaCanada View Post
Yes. I think it is pretty stupid that all these companies want to make this the next big codec, but are refusing to explain to people how to use it. I guess it's just more proof that AV1 was never intended for the masses, and was just a way for the media cartels to stop paying royalties
To be fair, at least AOM doesn't argue otherwise and a multi-codec-world is the future which includes the patented (and more efficient) mpeg codecs like VVC.
__________________
"The innocent have nothing to fear" :stupid:
Marsu42 is offline   Reply With Quote
Old 28th May 2021, 00:13   #10  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 3,751
Quote:
Originally Posted by RanmaCanada View Post
Yes. I think it is pretty stupid that all these companies want to make this the next big codec, but are refusing to explain to people how to use it. I guess it's just more proof that AV1 was never intended for the masses, and was just a way for the media cartels to stop paying royalties, like Disney is attempting to do with authors.
AV1 was really driven by companies focused on user generated content playing in web browsers: YouTube and Facebook. Those are all about bulk encoding in the cloud, not high-touch consumer tweaking like we can do with x264 and x265.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 28th May 2021, 00:27   #11  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 3,751
Quote:
Originally Posted by Marsu42 View Post
To be fair, at least AOM doesn't argue otherwise and a multi-codec-world is the future which includes the patented (and more efficient) mpeg codecs like VVC.
Indeed. Although their assertion that VP9 is 40% better than H.264 and equivalent to HEVC is flat-out wrong for every scenario I've tested. Maybe PSNR at fixed QP or something, but certainly not in subjective MOS testing of subjectively-tuned encodes.

There's no doubt that, with a good Film Grain Synthesis implementation, AV1 can outperform HEVC for grainy content at lower bitrates. But that's really the only case where I've seen a reliable reduction in bitrate versus HEVC when compared at the same bitrate and encoding time. And the FGS implementations aren't quite to the point where they can be used automatically, but that's orthogonal to codec bitstream itself, and I believe will be continuously improved.

Of course, the FGS being orthogonal, decoders which support it with AV1 could easily support FGS for HEVC and VVC too. I quite like the AV1 FGS implementation, and is probably technology that will be reused in other contexts. It's certainly much better than the H.264 FGS model that was available with HD-DVD players (although never used in any actual discs AFAIK).
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 4th June 2021, 12:50   #12  |  Link
Beelzebubu
Registered User
 
Join Date: Feb 2003
Location: New York, NY (USA)
Posts: 86
Quote:
Originally Posted by RanmaCanada View Post
And still no documention as to why AV1 doesn't respect --threads. No matter what I do I can not get it to use more than 1 thread.. I would love to use the encoder, but there is still too much "magic" or "try settings and figure it out yourself" and when it's single threaded (because I can't get --threads to work on any builds, maybe I'm cursed!), that takes far, FAR too long. Maybe I'm spoiled because of the awesome documentation that x265 and x264 had.
Threads in aom is implemented using tiling, so you need tiling active in order for --threads to do anything. --tile-columns and --tile-rows are both in log2 units, and have a value of 0 as default. For example, to get 4 tiles & threads, use --tile-columns=2 --tile-rows=0 --threads=4 or --tile-columns=1 --tile-rows=1 --threads=4.
Beelzebubu is offline   Reply With Quote
Old 10th June 2021, 05:23   #13  |  Link
RanmaCanada
Registered User
 
Join Date: May 2009
Posts: 207
Quote:
Originally Posted by Beelzebubu View Post
Threads in aom is implemented using tiling, so you need tiling active in order for --threads to do anything. --tile-columns and --tile-rows are both in log2 units, and have a value of 0 as default. For example, to get 4 tiles & threads, use --tile-columns=2 --tile-rows=0 --threads=4 or --tile-columns=1 --tile-rows=1 --threads=4.
And this is why better documentation is needed. But then what do the tile settings mean and do? All we get is "you should use this command line!". I've even tried reading reddit threads and people just say "do this" with no explanation as to why...

Thank you.
RanmaCanada is offline   Reply With Quote
Old 10th June 2021, 19:18   #14  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 3,751
Quote:
Originally Posted by RanmaCanada View Post
And this is why better documentation is needed. But then what do the tile settings mean and do? All we get is "you should use this command line!". I've even tried reading reddit threads and people just say "do this" with no explanation as to why...
Exactly!

And I imagine some implementations would add frame-level or other kinds of threading. With a single definitive resources for aomenc, other encoders could explain their changes relative to that so we could actually track what's happening.

Self-documenting code is a myth, and doubly so in compression.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 11th June 2021, 16:42   #15  |  Link
Beelzebubu
Registered User
 
Join Date: Feb 2003
Location: New York, NY (USA)
Posts: 86
Quote:
Originally Posted by benwaggoner View Post
other encoders could explain their changes relative to that
But that suggests that other AV1 encoders are based on aomenc.
Beelzebubu is offline   Reply With Quote
Old 11th June 2021, 16:51   #16  |  Link
Beelzebubu
Registered User
 
Join Date: Feb 2003
Location: New York, NY (USA)
Posts: 86
Quote:
Originally Posted by RanmaCanada View Post
And this is why better documentation is needed.
I agree with the problem statement. I just don't think it's easily solved. I try to help with individual questions as I did here, but anything more than that would probably require much more effort than I'm willing to put into it... Libaom lacks the activist/enthusiastic volunteer contributor community that x264 had back in the day.

Quote:
Originally Posted by RanmaCanada View Post
But then what do the tile settings mean and do?
--tile-columns=N means exp2(N) tile columns, and --tile-rows=M means exp2(M) tile rows. So, values of 0,1,2,3,4 for either one would imply 0,2,4,8,16 tile columns or rows. The total number of tiles in a file is equal to tile_rows * tile_columns, so to get 4 tiles, you can use 2 tile columns and 2 tile rows (--tile-columns=1 --tile-rows=1), or 4 tile columns and 1 tile row (--tile-columns=2 --tile-rows=0), or 1 tile column and 4 tile rows (--tile-columns=0 --tile-rows=2).
Beelzebubu is offline   Reply With Quote
Old 11th June 2021, 21:13   #17  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 3,751
The readthedocs.io system that is used for x265 documentation looks reasonably simple to get an outline in and fill out as needed. It used to allow user editing, which would be appropriate here.

https://x265.readthedocs.io/en/master/

It didn't start anything close to as comprehensive as that.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 12:51.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, vBulletin Solutions Inc.