Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > New and alternative video codecs

Reply
 
Thread Tools Search this Thread Display Modes
Old 1st February 2019, 20:13   #1421  |  Link
Mr_Khyron
Member
 
Mr_Khyron's Avatar
 
Join Date: Nov 2002
Location: Sweden
Posts: 85
SVT-AV1 encoder!

https://github.com/OpenVisualCloud/SVT-AV1
Quote:
Welcome to the GitHub repo for the SVT-AV1 encoder! To see a list of feature request and view what is planned for the SVT-AV1 encoder, visit our Trello page: http://bit.ly/SVT-AV1 Help us grow the community by subscribing to our SVT-AV1 mailing list

Quote:
Hardware

The SVT-AV1 Encoder library supports the x86 architecture

CPU Requirements

In order to achieve the performance targeted by the SVT-AV1 Encoder, the specific CPU model listed above would need to be used when running the encoder. Otherwise, the encoder runs on any 5th Generation Intel® Core™ processor, (Intel® Xeon® CPUs, E5-v4 or newer).

RAM Requirements

In order to run the highest resolution supported by the SVT-AV1 Encoder, at least 48GB of RAM is required to run a 4k 10bit stream multi-threading on a 112 logical core system. The SVT-AV1 Encoder application will display an error if the system does not have enough RAM to support this. The following table shows the minimum amount of RAM required for some standard resolutions of 10bit video per stream:
Resolution Minimum Footprint (GB)
4k 48gb
1080p 16gb
720p 8gb
480p 4gb
Mr_Khyron is offline   Reply With Quote
Old 1st February 2019, 20:17   #1422  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 5,800
so an Intel only encoder?
__________________
Hybrid here in the forum, homepage
Selur is offline   Reply With Quote
Old 1st February 2019, 20:20   #1423  |  Link
nevcairiel
Registered Developer
 
Join Date: Mar 2010
Location: Hamburg/Germany
Posts: 9,628
Quote:
Originally Posted by Selur View Post
so an Intel only encoder?
It should be able to run on any AVX2 CPU.

But the entire series of SVT encoders (SVT-HEVC is also a thing) is designed specifically for a use-case of running them on powerful datacenter systems with loads of memory and CPU cores.
__________________
LAV Filters - open source ffmpeg based media splitter and decoders
nevcairiel is offline   Reply With Quote
Old 1st February 2019, 20:47   #1424  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 2,794
Quote:
Originally Posted by Mr_Khyron View Post
Wow, that's a LOT of RAM for 4K. But if it's somewhat proportional to number of cores, no biggie. Any 112 logical core system is going to have >> 48 GiB RAM. The biggest c5 instance today is 72 logical threads and 144 GiB RAM.

I don't think there's ever been an encoder that could usefully use anything like 112 cores except via GOP-level parallelism. But hey, it's Intel.

I've not been able to find much detailed documentation about the SVT HEVC or AV1 projects. Do they mean "Scalable Video" ala SVC and SHVC with enhancement layers, mainly used in videoconferencing? Or scalable in the sense of scaling with hardware?

Leveraging the new low-level encoder SDK from Intel offers some interesting potential for very fast initial estimates for encoding, leaving the CPU to focus more on refinement. There isn't an AV1 encoder in the current Intel CPUs, obviously, but perhaps some VP9 functionality added in Kaby/Coffee Lake can be leveraged. Certainly things like weighted prediction and coarse motion vectors could be reused to some degree. SVT HEVC has a full 8-bit HEVC encoder implementation to leverage in Skylake-S+ and 10-bit in Kaby/Coffee.

Unfortunately there aren't any Xeon processors with VP9 encoding yet. The best available is the 8/16 core i9-9900K. I don't see any public roadmap for when AV1 might be added. Ice Lake? I see that has an all new HEVC encoder at least. Although given tape-out schedules and how recent the AV1 bitstream was finalized, a full fixed-function implementation might not be there before Tiger Lake. (all just personal speculation fueled by Wikipedia).

I am very curious to see what comes out of the next generation of GPU-assisted software-defined encoding. Having it all on-die instead avoid the PCI bus latency challenges of past GPU+CPU implementations.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 1st February 2019, 20:50   #1425  |  Link
nevcairiel
Registered Developer
 
Join Date: Mar 2010
Location: Hamburg/Germany
Posts: 9,628
Quote:
Originally Posted by benwaggoner View Post
I've not been able to find much detailed documentation about the SVT HEVC or AV1 projects. Do they mean "Scalable Video" ala SVC and SHVC with enhancement layers, mainly used in videoconferencing? Or scalable in the sense of scaling with hardware?
Hardware. Its not producing "scalable video".
__________________
LAV Filters - open source ffmpeg based media splitter and decoders
nevcairiel is offline   Reply With Quote
Old 2nd February 2019, 00:26   #1426  |  Link
TomV
VP Strategy, Beamr
 
Join Date: Jan 2018
Location: Palo Alto, CA
Posts: 43
Quote:
Originally Posted by benwaggoner View Post
I've not been able to find much detailed documentation about the SVT HEVC or AV1 projects. Do they mean "Scalable Video" ala SVC and SHVC with enhancement layers, mainly used in videoconferencing? Or scalable in the sense of scaling with hardware?
No. Intel bought eBrisk (they already owned a good chunk of eBrisk, thanks to the Altera acquisition, because Altera had invested in eBrisk), and then open sourced their HEVC encoder. Then they started focusing on AV1, and now they've open sourced that encoder. The HEVC encoder is fast, but the video quality is not competitive. I'm not sure if it can beat x264 under equal conditions. It certainly can't beat x265 or Beamr 5 under any conditions.
TomV is offline   Reply With Quote
Old 2nd February 2019, 00:26   #1427  |  Link
TomV
VP Strategy, Beamr
 
Join Date: Jan 2018
Location: Palo Alto, CA
Posts: 43
Quote:
Originally Posted by nevcairiel View Post
Hardware. Its not producing "scalable video".
Not hardware. Software.
TomV is offline   Reply With Quote
Old 2nd February 2019, 00:43   #1428  |  Link
nevcairiel
Registered Developer
 
Join Date: Mar 2010
Location: Hamburg/Germany
Posts: 9,628
Quote:
Originally Posted by TomV View Post
Not hardware. Software.
You should read the context of the question that answer was to. ;-)

To make sure its not lost again, let me paraphrase:
Q: Scalable Video, or Scaling with Hardware?
A: Hardware.
__________________
LAV Filters - open source ffmpeg based media splitter and decoders

Last edited by nevcairiel; 2nd February 2019 at 00:50.
nevcairiel is offline   Reply With Quote
Old 2nd February 2019, 02:47   #1429  |  Link
TomV
VP Strategy, Beamr
 
Join Date: Jan 2018
Location: Palo Alto, CA
Posts: 43
Quote:
Originally Posted by nevcairiel View Post
You should read the context of the question that answer was to. ;-)

To make sure its not lost again, let me paraphrase:
Q: Scalable Video, or Scaling with Hardware?
A: Hardware.
OK... I see. Exactly what they were thinking when they used this acronym, which, as Ben points out, is confusingly similar to Scalable Video Coding and Scalable HEVC Video Coding, I don't know. Nothing to see here... move along.
TomV is offline   Reply With Quote
Old 3rd February 2019, 17:47   #1430  |  Link
hajj_3
Registered User
 
Join Date: Mar 2004
Posts: 874
Intel SVT-AV1 benchmarks: https://twitter.com/fg118942/status/1092045469981671424
hajj_3 is offline   Reply With Quote
Old 3rd February 2019, 20:54   #1431  |  Link
soresu
Registered User
 
Join Date: May 2005
Location: Swansea, Wales, UK
Posts: 56
Is it just me or is rav1e actually pulling out ahead of VP9 at some bitrates on the graph? If so thats a nice milestone for rav1e, given the timeframe.
soresu is offline   Reply With Quote
Old 4th February 2019, 19:41   #1432  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 2,794
Quote:
Originally Posted by hajj_3 View Post
Is there more documentation on what's actually being tested and graphed here? Based on the parameters, it doesn't appear to be controlled for encoding speed. And odd to use --tune ssim for x264/x265 for VMAF, which is a superior objective metric than SSIM.

I wish tests would provide the actual per-frame VMAF scores instead of just a mean. For real-world duration stuff, variability of quality can hurt subjective quality in a way that VMAF itself won't capture. Keyframe strobing on one frame every 5 seconds doesn't drag down the mean much, but it can be a very annoying artifact viewers can clap along to.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 4th February 2019, 20:31   #1433  |  Link
Nintendo Maniac 64
Registered User
 
Nintendo Maniac 64's Avatar
 
Join Date: Nov 2009
Location: Northeast Ohio
Posts: 372
Quote:
Originally Posted by benwaggoner View Post
I wish tests would provide the actual per-frame VMAF scores instead of just a mean.
I presume you mean (pun not intended) in a manner similar to frame-time graphs used in GPU performance benchmarks, or at least just also showing a 1% low? For a similar reason, they came about since showing the average frame rate hides any uneven frame delivery which is much more important to game playability.

The only thing is that such graphs would tend to be limited to having a single bitrate or quality setting since the bottom axis in such a situation would be time rather than bitrate.

...which might very well be why people don't do it - because they want to show a single graph with various differing bitrates rather than a really detailed graph but only at a single bitrate or quality setting.


EDIt: A 1% low graph would at least let you do this, but it would still also require making a second graph (unless you don't even care about the mean at all, in which case you could just graph a 1% low and call it a day).
__________________
____HTPC____  | __Desktop PC__
2.93GHz Xeon x3470 (4c/8t Nehalem) | 4.5GHz Pentium G3258 (2c/2t Haswell)
Radeon HD5870  | Intel iGPU      
2x2GB+2x1GB DDR3-1333 | 4x4GB DDR3-1600       

Win7 x64

Last edited by Nintendo Maniac 64; 4th February 2019 at 20:38.
Nintendo Maniac 64 is offline   Reply With Quote
Old 4th February 2019, 20:37   #1434  |  Link
nevcairiel
Registered Developer
 
Join Date: Mar 2010
Location: Hamburg/Germany
Posts: 9,628
We get SSIM graphs with per-frame curves, so its not that of a "new" idea to also do that for VMAF or the likes.
__________________
LAV Filters - open source ffmpeg based media splitter and decoders
nevcairiel is offline   Reply With Quote
Old 5th February 2019, 00:56   #1435  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 2,794
Quote:
Originally Posted by Nintendo Maniac 64 View Post
I presume you mean (pun not intended) in a manner similar to frame-time graphs used in GPU performance benchmarks, or at least just also showing a 1% low? For a similar reason, they came about since showing the average frame rate hides any uneven frame delivery which is much more important to game playability.

The only thing is that such graphs would tend to be limited to having a single bitrate or quality setting since the bottom axis in such a situation would be time rather than bitrate.

...which might very well be why people don't do it - because they want to show a single graph with various differing bitrates rather than a really detailed graph but only at a single bitrate or quality setting.


EDIt: A 1% low graph would at least let you do this, but it would still also require making a second graph (unless you don't even care about the mean at all, in which case you could just graph a 1% low and call it a day).
Having a harmonic mean of the worst 0.1%, 1%, 10% would be quite useful, yes.

But the actual VMAF output is just per-frame scores, so anyone publishing a mean VMAF already has the data. Even if they don't want to plot the data, they could still make the log files available for download.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 5th February 2019, 03:24   #1436  |  Link
fg118942
Registered User
 
Join Date: Aug 2018
Posts: 3
Quote:
Originally Posted by benwaggoner View Post
Having a harmonic mean of the worst 0.1%, 1%, 10% would be quite useful, yes.

But the actual VMAF output is just per-frame scores, so anyone publishing a mean VMAF already has the data. Even if they don't want to plot the data, they could still make the log files available for download.
Log files and encoded videos are here.
https://www.dropbox.com/s/nbnlsicvsl..._60fps.7z?dl=0

I am encoding it with tune ssim because I followed the instructions in this article.
https://www.streamingmedia.com/Artic...ok-127133.aspx

I may not be able to answer difficult questions as I am not good at English.

Last edited by fg118942; 6th February 2019 at 10:12. Reason: To indicate the existence of video
fg118942 is offline   Reply With Quote
Old 5th February 2019, 16:57   #1437  |  Link
TomV
VP Strategy, Beamr
 
Join Date: Jan 2018
Location: Palo Alto, CA
Posts: 43
Quote:
Originally Posted by hajj_3 View Post
x264 and x265 preset slower is not the right preset to use versus aomenc --cpu-used = 0 and SVT-AV1 enc-mode 0. This test should compare with x264, x265 --preset placebo. Better yet, forget objective metrics. Just show us the video, so we can judge for ourselves the bit rates that produce matching subjective quality.
TomV is offline   Reply With Quote
Old 6th February 2019, 10:03   #1438  |  Link
kanaka
Registered User
 
Join Date: Jan 2019
Posts: 13
Quote:
Originally Posted by SmilingWolf View Post
I tested avif format with this photo
https://personal.sron.nl/~pault/imag...test_small.png
Avif file was different from source... (text wasn't readable), so i removed
Code:
--color-primaries=bt709 --transfer-characteristics=bt709 --matrix-coefficients=bt709
and result was ok. Why did you put this color profile?

I'm thinking about conversion my 12bit raw photos to avif. Is is possible? What pix_format shoul I use?
kanaka is offline   Reply With Quote
Old 6th February 2019, 11:46   #1439  |  Link
fg118942
Registered User
 
Join Date: Aug 2018
Posts: 3
Quote:
Originally Posted by TomV View Post
x264 and x265 preset slower is not the right preset to use versus aomenc --cpu-used = 0 and SVT-AV1 enc-mode 0. This test should compare with x264, x265 --preset placebo. Better yet, forget objective metrics. Just show us the video, so we can judge for ourselves the bit rates that produce matching subjective quality.
I thought that the point was right so I added placebo data.

Also, the video encoded with SVT-AV1 has been uploaded here.
https://www.dropbox.com/s/nbnlsicvsl..._60fps.7z?dl=0
fg118942 is offline   Reply With Quote
Old 6th February 2019, 18:27   #1440  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 5,790
New uploads: (MSYS2; MinGW32: GCC 7.4.0 / MinGW64: GCC 8.2.1)

AOM v1.0.0-1299-g54eabb5c8

rav1e 0.1.0 (2cec0f9 / 2019-02-06)

dav1d 0.1.1 (caca572 / 2019-02-06)
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 08:35.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.