Alliance for Open Media codecs

[hydra3333]

My Cpu is a Ryzen 1700 and when i play the UHD clips all cores are at %40 and i get between 8-16 fps
A dummy's question - may one enquire : I have mpc-hc, how to I tell the fps ? I see it stutters at the higher bitrates but am unsure how you measure "i get between xx-yy fps" ? (yes I have "View Statistics" ticked)

[NikosD]

https://ark.intel.com/products/186605/Intel-Core-i9-9900K-Processor-16M-Cache-up-to-5-00-GHz-

Hopefully someone can test the 9900K soon.
What did you mean ?
Intel has already paid a company for misleading benchmarks.

Crooks...They should pay a huge penalty for these dirty, low tricks.
https://www.techspot.com/article/1722-misleading-core-i9-9900k-benchmarks/

[Nintendo Maniac 64]

A dummy's question - may one enquire : I have mpc-hc, how to I tell the fps ? I see it stutters at the higher bitrates but am unsure how you measure "i get between xx-yy fps" ? (yes I have "View Statistics" ticked)

To be honest, I don't really know how to do so either, so I do it the manual way via trial and error - I import the videostream into mkvtoolnix, set the frame rate to something (like 20p), export to an mkv with the new framerate, and then see if can playback this new mkv smoothly without any stutter.

If it does stutter, then I do the exact same process again but with a lower framerate (like 15p).

If it plays without any stutter, then I still do the exact same process again but with a higher framerate.


I keep doing this trial-and-error process until I find the highest framerate that doesn't result in any stuttering (note that I only use integer fps values though, like 24p, 25p, 26p, etc - testing fractional framerates as well would be WAY too time consuming with this method).

This method does have the benefit however of working on any media player or browser or the like.


And yes, technically without using a variable refresh rate display or a bajillion different custom resolutions, you're going to always have some visual stutter due to many of the the tested framerates not being an exact multiple of your display's refresh rate, but the sort of stuttering caused by inadequate video decoder performance tends to be way worse than any sort of telecine judder or the like.

[olduser217]

https://code.fb.com/video-engineering/facebook-video-adds-av1-support/

Facebook is working on to add AV1 support too.
The browsers that supported AV1 is able to play the AV1 version of embedded video (but highest resolution only 360p).

[benwaggoner]

To be honest, I don't really know how to do so either, so I do it the manual way via trial and error - I import the videostream into mkvtoolnix, set the frame rate to something (like 20p), export to an mkv with the new framerate, and then see if can playback this new mkv smoothly without any stutter.

If it does stutter, then I do the exact same process again but with a lower framerate (like 15p).

If it plays without any stutter, then I still do the exact same process again but with a higher framerate.
That's a pretty good process given the tools available today.

I do note that it might somewhat underestimate SW decoder performance requirements for real-world content significantly.

Slowing down 60p to 30p will result in a stream that may be easier to decode than the same content natively captured at 30p. This is because twice as much motion happens between 30p frames, so there's more prediction and motion vectors per frame to process. Also, the bitrate will drop by half in a 60-30 conversion, when real-world a 30p might be 70-80% the bitrate of a 60p for the same spatial quality (since twice as much change per frame is being captured).

Of course, if real-world decoder characteristics are understood, rate control techniques like VBV can cap the worst-case decoding times, although at the potential risk of capping maximum quality for difficult segments. We got a little spoiled from the last decade-ish of relatively ubiquitous H.264 HW decoding :).

[marcomsousa]

ffmpeg -benchmark -i Stream2_AV1_4K_22.7mbps.webm -f null -
Video: wrapped_avframe, yuv420p, 3840x2160 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 25 fps, 25 tbn, 25 tbc (default)
frame= 3604 fps= 16 q=-0.0 Lsize=N/A time=00:02:24.16 bitrate=N/A speed=0.622x
bench: utime=1069.000s stime=12.891s rtime=231.970s
bench: maxrss=856880kB


Since the video was 25 fps, in benchmark give that my PC is only capable to decode at 15-16 fps (speed=0.622x) at with this 22.7mbps video.

CPU: Intel Core i7-8550U
Decoder: ffmpeg-20181007-0a41a8b-win64 - libaom-av1 1.0.0-691-gbb8157b89

[MoSal]

frame= 3604 fps= 26 q=-0.0 Lsize=N/A time=00:02:24.16 bitrate=N/A speed=1.02x

CPU: Intel Core i7-7700k (60-65% utilization).
Decoder: libaom (1.0.0.r749.g955242e6a6, -DCONFIG_LOWBITDEPTH=1).

[hydra3333]

Cough,
frame= 3604 fps= 14 q=-0.0 Lsize=N/A time=00:02:24.16 bitrate=N/A speed=0.577x
bench: utime=1170.531s stime=6.344s rtime=249.925s
bench: maxrss=852156kB

CPU: Intel i7-i3820
Decoder: ffmpeg version a day or two old, N-92147-gf85fa100db ; libaom 1.0.0-708-gdf7131064 commit df7131064bf37fb5c7ee427ba564c31a2ed8bbbe (the one before it conflicts with libvpx) without DCONFIG_LOWBITDEPTH
Commandline: "ffmpeg.exe" -benchmark -i Stream2_AV1_4K_22.7mbps.webm -f null -

[marcomsousa]

Youtube already support AV1 when upload videos.

Then I uploaded (av1 video) to YouTube, the results:

YouTube successfully recognized the video and re-encoded it to h.264 and VP9.

YouTube did not display the original in AV1 and dit also not re-encode to AV1.

Source (https://www.reddit.com/r/AV1/comments/9n0h7o/i_uploaded_an_4k_60fps_av1_video_to_youtube_this/)

[Pushman]

frame= 3604 fps= 11 q=-0.0 Lsize=N/A time=00:02:24.16 bitrate=N/A speed=0.435x

ffmpeg version N-92132-g0a41a8bf29 Copyright (c) 2000-2018 the FFmpeg developers
[libaom-av1 @ 000001d804fecb00] 1.0.0-691-gbb8157b89

CPU: Intel i3-4170

[clsid]

GraphStudioNext has a performance test feature which can be used to measure how many fps a DirectShow decoder can deliver.

[Clare]

Does aomenc support multithread yet? I have tile-columns=4 and row-mt=1 but it still only using 1 core.

[marcomsousa]

Does aomenc support multithread yet? I have tile-columns=4 and row-mt=1 but it still only using 1 core.

you forget --threads=8?

aomenc -v -w 1920 -h 1080 --cpu-used=0 --target-bitrate=1500 --threads=8 --profile=0 --aq-mode=0 --lag-in-frames=25 --auto-alt-ref=1 --tile-columns=4 --row-mt=1 -o test15.webm test1.y4m
This use all CPU.

Tune to you logical cores.

[mandarinka]

Can a modern system do that for HEVC? HEVC decode is going to be more inherently parallelizable due to WPP. And I'm not aware of any software decoders that can do a realtime 2160p60 HEVC on any hardware I've looked at.

I think it was recently mentioned here that FFmpeg doesn't do WPP simultaneously in addition to frame threading. I'm also aware of it not scaling very well, maybe this is the reason. (Doesn't OpenHEVC support doing this?)
But since Nevcariel says it works on some PCs, I guess it is a matter of CPU and RAM bandwidth. And perhaps single-thread per-core performance. Many slower cores might not cut it due to bw/scaling issues, but fewer ones on 4,0-4,5 GHz like those Kaby Lake/Coffee Lake chips could?

FFmpeg's HEVC decoder isn't yet/atm optimised thoroughly, there is some intrinsics optimizations from openhevc missing (LAV Video decoder has them though) and there is probably some other pickable fruit too. There just wasn't motivation on the side of devs it seems (preferences for the google/royalty-free formats etc).

[Clare]

you forget --threads=8?

aomenc -v -w 1920 -h 1080 --cpu-used=0 --target-bitrate=1500 --threads=8 --profile=0 --aq-mode=0 --lag-in-frames=25 --auto-alt-ref=1 --tile-columns=4 --row-mt=1 -o test15.webm test1.y4m
This use all CPU.

Tune to you logical cores.

Ooops thanks now it's working.

aomenc --threads=8 --cpu-used=4 --tile-columns=4 --row-mt=1 --passes=2 --pass=2 --bit-depth=10 --input-bit-depth=10 --end-usage=q --cq-level=28 --fpf=Chimera_DCI4k2398p_HDR_P3PQ.log -o Chimera_DCI4k2398p_HDR_P3PQ.ivf Chimera_DCI4k2398p_HDR_P3PQ.y4m

Edit: it bursted on all core for 30 seconds but went back to one core afterwards :(

[easyfab]

for --row-mt=1 I think you need to wait that https://aomedia-review.googlesource.com/c/aom/+/72801 is merged.

[Clare]

for --row-mt=1 I think you need to wait that https://aomedia-review.googlesource.com/c/aom/+/72801 is merged.

I already patched my build with this. It seems that as soon as the first frame is finished rendering, it drops back to one core.

[Nintendo Maniac 64]

Slowing down 60p to 30p will result in a stream that may be easier to decode than the same content natively captured at 30p. This is because twice as much motion happens between 30p frames, so there's more prediction and motion vectors per frame to process. Also, the bitrate will drop by half in a 60-30 conversion, when real-world a 30p might be 70-80% the bitrate of a 60p for the same spatial quality (since twice as much change per frame is being captured).

Yep indeed, this is why I specifically use YouTube's 30fps encodes when dealing with a frame rate that's less than 60fps as it's better to under-estimate performance and then be pleasantly surprised to find out that real performance is better.

[benwaggoner]

I think it was recently mentioned here that FFmpeg doesn't do WPP simultaneously in addition to frame threading
That is going to be way more dependent on the encoder used than ffmpeg itself. x265 uses WPP AND frame-threads by default if you have multiple cores, and I doubt ffmpeg would disable that. Turning off WPP silently would be a big problem, as WPP also has significant decoder impact as well, particularly with multithreaded software decode.

It's an ongoing challenge with all encoder to make sure that the right flags are allowed by products that incorporate them. Making sure that commonly used tools like ffmpeg integrate libaom (or a superior alternative) well is pretty darn important, as that's what lots of reviewers and evaluators will use.

Getting good, actionable documentation into ffmpeg and in general is also important. Listing options without explaining why one might want to use it and its pros/cons isn't really documentation. x265.readthedocs.io is the gold standard here, and I don't even have a runner up.

[benwaggoner]

Yep indeed, this is why I specifically use YouTube's 30fps encodes when dealing with a frame rate that's less than 60fps as it's better to under-estimate performance and then be pleasantly surprised to find out that real performance is better.
Yeah, your approach is the best I can think of until it becomes more feasible to personally encode test content that makes use of a realistic array of AV1 features.

Testing fast encodes risks skipping features, making decoding simpler for a SW decoder than real-world competitive quality AV1 encodes would be. Some examples from past codecs where faster encoder modes simplify impact decoder performance that can be turned off for encoder performance include:

Weighted prediction
In-loop deblocking or SAO
Number of reference frames
Number of B-frames

[Kurosu]

There just wasn't motivation on the side of devs it seems (preferences for the google/royalty-free formats etc).
I'd rather say that most capable devs had enough of (big) corps freeloading and/or are paid to do something else. In a sense, it is actually a good thing that they learnt such a lesson, as it indicates a maturing and more professional community. And if AoM is ready to play fair in that regard, then it's "not-AoM" loss.

As for HEVC, the mentioned WPP+frame threading could be available today. All in all, there's likely 30% speed-up left.

Source: said capable devs.

[Monarc]

ffmpeg -benchmark -i Stream2_AV1_4K_22.7mbps.webm -f null -
Video: wrapped_avframe, yuv420p, 3840x2160 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 25 fps, 25 tbn, 25 tbc (default)
frame= 3604 fps= 16 q=-0.0 Lsize=N/A time=00:02:24.16 bitrate=N/A speed=0.622x
bench: utime=1069.000s stime=12.891s rtime=231.970s
bench: maxrss=856880kB


Since the video was 25 fps, in benchmark give that my PC is only capable to decode at 15-16 fps (speed=0.622x) at with this 22.7mbps video.

CPU: Intel Core i7-8550U
Decoder: ffmpeg-20181007-0a41a8b-win64 - libaom-av1 1.0.0-691-gbb8157b89

on my Intel(R) Core(TM) i5-3550 CPU @ 3.30GHz

ffmpeg -benchmark -i Stream2_AV1_4K_22.7mbps.webm -f null -

ffmpeg version n4.0.2
[libaom-av1 @ 0x55e28fe6f180] 1.0.0-759-g90a15f4f28

frame= 3604 fps=6.2 q=-0.0 Lsize=N/A time=00:02:24.16 bitrate=N/A speed=0.246x
video:1886kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
bench: utime=579.607s
bench: maxrss=602748kB



ffmpeg -threads 4 -benchmark -i Stream2_AV1_4K_22.7mbps.webm -f null -

frame= 3604 fps= 16 q=-0.0 Lsize=N/A time=00:02:24.16 bitrate=N/A speed=0.643x
video:1886kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
bench: utime=656.087s
bench: maxrss=737080kB

[benwaggoner]

I'd rather say that most capable devs had enough of (big) corps freeloading and/or are paid to do something else. In a sense, it is actually a good thing that they learnt such a lesson, as it indicates a maturing and more professional community. And if AoM is ready to play fair in that regard, then it's "not-AoM" loss.
Also, multithreaded decode speed isn't THAT important a feature for ffmpeg, which is mainly for transcoding. Lots of operations that include decode are going to be encoder-bound. And the other operations likely have work to do on unused cores. Multithreaded decode uses more memory and sometimes CPU, so an export-bound operation may actually be a little faster with a single-threaded decoder.

You are right that lots of ffmpeg and other open-source encoder/tool development is corporate funded, and so a lot of its development is driven by what companies want enough to pay for. And HEVC source is still pretty uncommon outside of some high end contribution streams from live events.

As for HEVC, the mentioned WPP+frame threading could be available today. All in all, there's likely 30% speed-up left.

Source: said capable devs.
That's reasonable. Decoders hit the point of asymptotic speed increases a lot sooner than encoders as there is only one "right" answer. AV1 is going to have >>30% decoder perf headroom still. I am curious how the greater variety of tools will impact optimal decoder performance in AV1 versus HEVC. AV1 is MUCH better designed for both HW and multithreaded SW decoders than VP9 and earlier, which had limitations in the loop filter and frame signaling that made them almost single-threaded. Which was generally fine for a PC web browser of the era, but had some real limitations for battery operated devices or those with lots of slower cores.

We'll probably have a good sense of the fundamental real-world decoder perf differences between HEVC and AV1 in 2019.

[Kurosu]

Also, multithreaded decode speed isn't THAT important a feature for ffmpeg, which is mainly for transcoding
True, the decoding speed increase is no longer useful for a lot of companies. Security is often more important.

AV1 is MUCH better designed for both HW and multithreaded SW decoders than VP9 and earlier, which had limitations in the loop filter and frame signaling that made them almost single-threaded.
One of those bottlenecks in thread scaling, what I'd call entropic state sharing across frames, is still there in AV1. This is to me something that was decided irrespective of SW decoding. Another sign that indeed the interest in it is only transitory.

We'll probably have a good sense of the fundamental real-world decoder perf differences between HEVC and AV1 in 2019.
And in 2020, hopefully, new products will all have HW decoding of AV1, so that would then no longer matter.

[Mjpeg]

New article from Streaming Media:
http://www.streamingmedia.com/Articles/Editorial/Featured-Articles/HEVC-VP9-AV1-and-VVC-Presenting-a-Codec-Update-in-11-Charts-127956.aspx

On the second page there is an intriguing slide from Bitmovin that with a pure software encoder version of libaom, they attained a speedup for cpu-used=0 from 1000x of VP9 in the early days down to 40x today with improvements ongoing. The claim from AOM was always that there was lots of room for optimization work once the spec was settled and ... well maybe that's happening. The proof will be when those speedups are in mainline FFmpeg and show up in tests on this very thread!

The slide also quotes software decode of 3.4x of VP9 single-threaded.

[utack]

New article from Streaming Media:
http://www.streamingmedia.com/Articles/Editorial/Featured-Articles/HEVC-VP9-AV1-and-VVC-Presenting-a-Codec-Update-in-11-Charts-127956.aspx


Regarding the "poor results for AV1".
The BBC paper used single pass encode
The parameters “--passes=1” and “--lag-in-frames=0” were set to run AV1 in single pass mode
without the possibility of looking ahead in the video sequence before encoding
The Harmonic paper used a fixed GOP of less than a single second of video material
--min-gf-interval=16 --max-gf-interval=16
The Ateme paper does not mention any configuration for the encoder

That is basically three results that are only good for academic publication count and the gutter in the real world


On another note, there is a 5 minute 720p clip with super diverse content out there now on youtube where VP9 and AV1 version have basically the same file size:
https://www.youtube.com/watch?v=WaWnLiffxJ4
You can do some direct comparison of quality

[marcomsousa]

I already patched my build with this. It seems that as soon as the first frame is finished rendering, it drops back to one core.


Another fix was merged
Fix allocation of workers for enc row-mt

Workers allocated for row based multi-threading of encoder are
now evaluated as minimum of num of threads and total number of
sb rows in the frame to be encoded.

Change-Id: I07501f43514f1ee45dd6637fe56411432930396c

[dapperdan]

New article from Streaming Media:
http://www.streamingmedia.com/Articles/Editorial/Featured-Articles/HEVC-VP9-AV1-and-VVC-Presenting-a-Codec-Update-in-11-Charts-127956.aspx

On the second page there is an intriguing slide from Bitmovin that with a pure software encoder version of libaom, they attained a speedup for cpu-used=0 from 1000x of VP9 in the early days down to 40x today with improvements ongoing.

Might be worth noting that the slide is from a Google employee, the Bitmovin employee was just in the audience.

I'm intrigued to hear more about the AI strategies they are using and wonder how portable those are once the "trick" has been revealed by the AI.

[Tommy Carrot]

On the second page there is an intriguing slide from Bitmovin that with a pure software encoder version of libaom, they attained a speedup for cpu-used=0 from 1000x of VP9 in the early days down to 40x today with improvements ongoing.
The speed improvements are nice, but many of those optimizations came at the cost of the quality. I'd estimate the compression efficiency dropped about 2-3% since the earlier versions.

[soresu]

Personally, considering that BBC is directly involved in development of VVC, and given that they joined the AV1 party relatively late in the development cycle, I'm inclined to find their so called research analysis somewhat suspect.

The Streaming Media article mentions their credibility on the issue, coming from a member of AOMedia, without taking into account just how late they joined.

At the very least it seems a conflict of interest to post such a negative paper so early in the post standard development of AV1, even more so considering that VVC is 2 years away from even being standardised.

Does anyone here have any idea what level of patent/proposals BBC have in the VVC working group?

Edit: Sorry if this is on the wrong thread, it concerns both AV1 and VVC so I wasnt sure which to drop it in.

[Clare]

AV1 Image File Format (AVIF) https://people.xiph.org/~negge/AVIF2018.pdf

Mile-High Video Workshop videos http://mile-high.video/files/mhv2018/

with topics such as:

Video Encoding and HEVC
Into the Depths: The Technical Details behind AV1
AV1 vs. HEVC: Perceptual Evaluation of Video Encoders
Codec Comparison from TCO and Compression Efficiency Perspective
VVC – The Next-Generation Video Standard of the Joint Video Experts Team
Pushing Encoding Quality and Speed with x265
Massively Parallel Encoding

[benwaggoner]

The speed improvements are nice, but many of those optimizations came at the cost of the quality. I'd estimate the compression efficiency dropped about 2-3% since the earlier versions.
A 2.5x perf improvement with only a 2-3% efficiency tradeoff is a pretty good optimization option. That's ballpark to a x26? preset ratio in perf and quality.

And things have to get LOTS faster to do the psychovisual and rate control tuning to get AV1 into something that can be practically compared to other encoders/bitstreams. Quality won't matter within an order of magnitude of the current speeds. It's not like anyone is actually delivering in volume 1080p encoded with --preset placebo either!

Quality @ Bitrate @ Perf!

[SmilingWolf]

According to my tests on my F.Y.C clip (1280x720, 480 frames) aomenc right now has got roughly the same encoding quality @ bpp of 1 month ago, with a deviation within the 1%, but is also somewhere between 14-25% faster than 2 months ago for --cpu-used=4
Which means that clip now encodes, in single thread at CQ 20, in 66 minutes rather than 87 on my i7-4770.

aomenc 1.0.0-359-g1bc580401:
# aomenc --frame-parallel=0 --tile-columns=0 --auto-alt-ref=2 --cpu-used=4 --passes=2 --threads=1 --lag-in-frames=25 --end-usage=q --cq-level=20 -o test.av1.cq20.webm orig.i420.y4m
Pass 1/2 frame 480/481 92352B 1539b/f 36899b/s 21265 ms (22.57 fps)
Pass 2/2 frame 480/480 5995490B 99924b/f 2395780b/s 5245119 ms (0.09 fps)

aomenc 1.0.0-775-g7f492af02:
# aomenc --frame-parallel=0 --tile-columns=6 --auto-alt-ref=1 --cpu-used=4 --tune=psnr --passes=2 --threads=1 --end-usage=q --cq-level=20 -o test.av1.cq20.webm orig.i420.y4m
Pass 1/2 frame 480/481 92352B 1539b/f 36899b/s 17764 ms (27.02 fps)
Pass 2/2 frame 480/480 6092708B 101545b/f 2434645b/s 3969928 ms (0.12 fps)

[mzso]

AV1 Image File Format (AVIF) https://people.xiph.org/~negge/AVIF2018.pdf


Funnily enough there's barely any mention of AVIF. Like 8 pages.

[benwaggoner]

Funnily enough there's barely any mention of AVIF. Like 8 pages.
Well, everything about intra coding is relevant to AVIF.

It's interesting to see the use of VMAF for still images. I question its relevance, as VMAF was only calibrated on >=24p moving images. Has anyone published any data suggesting VMAF is useful for measuring still images?

Also, the image comparisons versus HEVC don't show the command line. HEVC doesn't have --preset stillimage like x264, but I've tried an adaption of those parameters in x265. I'd expect it would improve the quality of the HEVC (HEIF?) image significantly. I saw some of the new Adobe tools that came out the last few days also include an HEIF exporter, which would be interesting to compare. Looks like fixed QP is being used for both AV1 and HEVC, which is going to be suboptimal for both codecs. Particularly for anything that includes mixed natural/synthetic imagery.

It amused me that it ended with a frames-per-minute graph that doesn't reference frame size, nor whether this is for still only or for temporal encoding. The idea of a still image taking 38 seconds to encode is going to horrify anyone using an image server.

[marcomsousa]

AV1 Image File Format (AVIF) https://people.xiph.org/~negge/AVIF2018.pdf


Presentation explained
https://www.youtube.com/watch?v=On9VOnIBSEs

[marcomsousa]

Google released today Chrome 70

Chrome 70 adds an AV1 decoder (it's enabled by default) to Chrome Desktop x86-64 based on the official bitstream specification. At this time, support is limited to “Main” profile 0 (https://aomediacodec.github.io/av1-spec/#profiles) and does not include encoding capabilities. The supported container is MP4 (ISO-BMFF (https://aomediacodec.github.io/av1-isobmff)) (see From raw video to web ready for a brief explanation of containers (https://developers.google.com/web/fundamentals/media/manipulating/files#how_are_media_files_put_together)).

To try AV1:

Update your stable Chome just type in the url: chrome://settings/help and restart Chrome
Go to the YouTube TestTube page (https://www.youtube.com/testtube).
Select "Prefer AV1 for SD" or "Always Prefer AV1" to get the desired AV1 resolution. Note that at higher resolutions, AV1 is more likely to experience playback performance issues on some devices.
Try playing YouTube clips from the AV1 Beta Launch Playlist (https://www.youtube.com/playlist?list=PLyqf6gJt7KuHBmeVzZteZUlNUQAVLwrZS).
Confirm the codec av01 in "Stats for nerds".

[dapperdan]

And things have to get LOTS faster to do the psychovisual and rate control tuning to get AV1 into something that can be practically compared to other encoders/bitstreams. Quality won't matter within an order of magnitude of the current speeds. It's not like anyone is actually delivering in volume 1080p encoded with --preset placebo either!


Netflix already use VP9, which is lacking good rate control and psychovisual tuning, but still delivering higher quality (VMAF) at lower bitrates than their other streams (according to their blog posts) and they have said they'd roll out AV1 when it was within 4-10x slower than VP9. Presumably based on getting higher quality that makes that tradeoff worthwhile for them at that point.
Bitmovin claims their latest encoder release is twice as fast as stock AV1 and 20x slower than VP9 so things don't seem to be that far off AV1 being delivered on a large scale.

They also have the extra bonus of being able to encode their in-house stuff without film grain and add it later, a feature they were keen to have added to AV1.

[mzso]

Netflix already use VP9, which is lacking good rate control

Is that still true? My ipression these days from youtube are that the bitrates are quite stable. (Might be wrong)


They also have the extra bonus of being able to encode their in-house stuff without film grain and add it later, a feature they were keen to have added to AV1.

Stupidest feature ever. They should have implemented an optional "hellyeahIwantnoiseLOL" decoder feature instead.

[LigH]

Don't miss the detail that bitrate optimization for streaming mainly means ABR — and the smaller the assumed decoding buffer and allowed preload time, the closer it gets to CBR, which will cause quality fluctuations. Oh, the joy of the VBV model.

[Adonisds]

SVP (http://svp-team.com/).

Note that there are several variants of it:
the old SVP 3.1.7 is free but only works on Windows and with programs like MPC-HC; also it might only work with 32bit media players.
the full-featured version of SVP 4 is only free on Linux and works with VLC and mpv.
the basic yet free version of SVP 4 which only works on Windows and with programs like MPC-HC much like v3.1.7, but has considerably reduced configuration options compared to both the full-featured version and the old 3.1.7 version.
the paid full-featured "Pro" version on Windows and Mac is largely identical to the free version of SVP 4 on Linux, though on Windows it also works with MPC-HC as well as VLC and/or mpv.

One thing to keep in mind is that interpolating to refresh rates that are exact multiples of the source framerate will provide a smoother result with fewer artifacts - e.g. 30fps interpolated to 120Hz (4x) is better than 30fps interpolated to 144Hz (4.8x) - this is most easily accomplished with something like like MPC-HC's or madVR's built-in automatic resolution changer which can be used to change your refresh rate depending on a given video frame rate (though you may need to create a custom resolution in order to access certain refresh rates on your display).

Thanks

[Phanton_13]

Is that still true? My ipression these days from youtube are that the bitrates are quite stable. (Might be wrong) Yes, you have to take in consideration that even if the bitrate is quite stable you can can have a bad distribution of it. I found that in vp9 if you limit the maximun quantification between 24-38 the overall quality is boosted without much variation on final bitrate/size (max value to be used depends of bitrate and resolution), basically you prevent the encoding rate control to be fucked sometimes with its corresponding drop in quality.

[benwaggoner]

Netflix already use VP9, which is lacking good rate control and psychovisual tuning, but still delivering higher quality (VMAF) at lower bitrates than their other streams (according to their blog posts) and they have said they'd roll out AV1 when it was within 4-10x slower than VP9. Presumably based on getting higher quality that makes that tradeoff worthwhile for them at that point.
VMAF has a much better subjective correlation than SSIM, but it is pretty far from perfectly subjectively correlated with double-blind subjective measurements. The risk of any new metric is over-optimizing an encoder for metric scores instead of actual subjective experience. I saw plenty of places where it got things wrong in the previous version; I've not worked extensively with the latest update, though, which has doubtless improved.

Netflix is also going from H.264 to VP9/AV1, so they don't need to beat or even meet HEVC quality to get a worthwhile improvement, of course.

Netflix IS using HEVC for all UHD and HDR encoding AFAIK.

I'm not sure where VP9 versus H.264 quality stands today. I'm running an encoding challenge, and would love to have someone provide best-effort VP9 and even AV1 samples for comparison.

https://forum.doom9.org/showthread.php?t=175776

Bitmovin claims their latest encoder release is twice as fast as stock AV1 and 20x slower than VP9 so things don't seem to be that far off AV1 being delivered on a large scale.
That would suggest that stock AV1 is only 40x slower than VP9, which is not my understanding at all. Perhaps they are talking the highest speed mode of their encoder, which would involve some quality degradation. Quality @ Perf is the important thing.

Getting good multithreading into AV1 is going to be really important, since time to market matters for a lot of content. Being able to encode something 8x faster on 16 cores probably doesn't matter for Netflix or YouTube given their chunking and lack of day-after-broadcast content. But it's critical for other markets, and essential for live encoding. Some of the VP9 and AV1 comparisons with x264 and x265 were artificially limited to 1-2 cores. Which makes sense if optimizing for absolute volume of minutes encoded, but understates the speed advantages of x26? for latency-critical tasks.

They also have the extra bonus of being able to encode their in-house stuff without film grain and add it later, a feature they were keen to have added to AV1.
That is a pretty huge feature! It was optional for H.264 (only required in HD-DVD decoders, but I don't know of anything authored with it). Random noise is mathematically uncompressible, so this kind of noise synthesis is an extremely promising way to improve quality and compressibility of the most challenging content.

...and will also reveal how much of the apparent detail in film comes from the grain. Older films, particularly, look really soft without the grain, and often just don't have much spatial detail.

But don't underestimate the challenge of the removing grain part; parameterizing it and then reconstructing it on playback are the easy parts. It's way more feasible now than 12 years ago, but it isn't trivial of something that can run 100% automated without messing up sometimes.

Unfortunately production workflows put in film grain much earlier than the encoding stage, so it's already baked in way before it gets to an encoder.

[Mr_Khyron]

http://www.socionext.com/en/pr/sn_pr20180831_01e.pdf
AV1 Hardware Accelerated Encoder Solution
There will be a demonstration of the world’s first hardware-based implementation of an AV1 encoding system.
AV1 is a new, advanced video data compression standard established by the Alliance for Open Media, a consortium
that includes Socionext, and is capable of compressing data about 30% more effectively than HEVC, whilst keeping the same image quality.
A hardware encoder already?! :cool:

[utack]

Random noise [...] will also reveal how much of the apparent detail in film comes from the grain.

Same frame with (from libaom) and without grain (from dav1d)
http://screenshotcomparison.com/comparison/122531
Biggest difference is in the river if you ask me
It is probably even more important when pushing the bitrate even lower in a scene like this

[Audionut]

Unfortunately production workflows put in film grain much earlier than the encoding stage, so it's already baked in way before it gets to an encoder.

Sounds like production workflows need overhauling. :devil:

I can't recall the specifics, but I do recall some sort of overview on bandwidth statistics, showing that some large percentage of worldwide bandwidth is Netflix.

Grain = bandwidth!

[benwaggoner]

Sounds like production workflows need overhauling. :devil:
Yeah, I'll get on that once I wean the industry off 24p :).

I can't recall the specifics, but I do recall some sort of overview on bandwidth statistics, showing that some large percentage of worldwide bandwidth is Netflix.
Yeah, the majority of internet traffic is now some sort of video. The public numbers use a wide variety of methodology, and have been known to get it wrong by a factor of 2x.

This is one reason we need to forever push the boundaries of compression efficiency. Once we get to transparent compression quality, we need to keep making the required bandwidth to reach that ever lower. Over my 25 years doing digital video, we've seen ~20% improvement every year in the bandwidth required to achieve a given level of quality. Decent 1080p today takes fewer bits than ugly 320x176 did back in 1995.

[benwaggoner]

http://www.socionext.com/en/pr/sn_pr20180831_01e.pdf

A hardware encoder already?! :cool:
It's a hardware appliance. From the description, it's a fast CPU plus some acceleration. Probably source decode, preprocessing, coarse motion search, frame type selection, weighted prediction, that kind of thing.

ASICs and even GPU-based compression was left behind with HEVC; there are so many different tools a modern codec can use (>2x more in AV1 than HEVC), and GPU's aren't great at the tight inner loops and rapid mode determination required to get good use out of a modern bitstream. Fast individual cores with strong SIMD and L1/2/3 cache run rings around hardware that best works with a one way waterfall cascade of operations.

And CABAC is all about having a few very fast cores. That needs to be done on CPU.

Heck, lots of HW-on-GPU and ASIC encoders don't even have good B-frame support.

[benwaggoner]

Same frame with (from libaom) and without grain (from dav1d)
http://screenshotcomparison.com/comparison/122531
Biggest difference is in the river if you ask me
It is probably even more important when pushing the bitrate even lower in a scene like this
Good example. And also a good illustration of the challenges with film grain removal. The degrained version also took out the ripples from the water, which were NOT grain. Small details that move stochastically can be very hard to discriminate from grain. It requires some flavor of bidirectional optical flow analysis. Turbulence can be a lot like grain mathematically, but not at all visually.

[Selur]

According to the Wiki (https://en.wikipedia.org/wiki/AV1#Profiles)
av1 high profile should support 4:2:0 with 8bit and 10bit, but:
ffmpeg -y -loglevel fatal -threads 8 -i "F:\TestClips&Co\files\test.avi" -map 0:0 -an -sn -vf zscale=rangein=tv:range=tv -pix_fmt yuv420p -vsync 0 -f yuv4mpegpipe - | aomenc --passes=1 --pass=1 --target-bitrate=1500 --end-usage=vbr --profile=1 --cpu-used=3 --undershoot-pct=0 --overshoot-pct=0 --buf-sz=6 --buf-initial-sz=4 --buf-optimal-sz=5 --drop-frame=0 --kf-min-dist=0 --kf-max-dist=250 --auto-alt-ref=1 --arnr-maxframes=7 --arnr-strength=5 --noise-sensitivity=0 --sharpness=0 --static-thresh=0 --tune-content=default --tile-columns=0 --tile-rows=0 --min-gf-interval=0 --max-gf-interval=0 --threads=2 --width=640 --height=352 --i420 --input-bit-depth=8 --bit-depth=8 --row-mt=0 --cdf-update-mode=1 -o "E:\Temp\18_13_01_6710_01.ivf" -
ffmpeg -y -loglevel fatal -threads 8 -i "F:\TestClips&Co\files\test.avi" -map 0:0 -an -sn -vf zscale=rangein=tv:range=tv -pix_fmt yuv420p10le -strict -1 -vsync 0 -f yuv4mpegpipe - | aomenc --passes=1 --pass=1 --target-bitrate=1500 --end-usage=vbr --profile=1 --cpu-used=3 --undershoot-pct=0 --overshoot-pct=0 --buf-sz=6 --buf-initial-sz=4 --buf-optimal-sz=5 --drop-frame=0 --kf-min-dist=0 --kf-max-dist=250 --auto-alt-ref=1 --arnr-maxframes=7 --arnr-strength=5 --noise-sensitivity=0 --sharpness=0 --static-thresh=0 --tune-content=default --tile-columns=0 --tile-rows=0 --min-gf-interval=0 --max-gf-interval=0 --threads=2 --width=640 --height=352 --i420 --input-bit-depth=10 --bit-depth=10 --row-mt=0 --cdf-update-mode=1 -o "E:\Temp\18_14_27_4610_01.ivf" -
give me:
Profile 1 requires 4:4:4 color format

also av1 professional profile should support 4:2:0 with 8bit, 10bit, 12bit, but:
ffmpeg -y -loglevel fatal -threads 8 -i "F:\TestClips&Co\files\test.avi" -map 0:0 -an -sn -vf zscale=rangein=tv:range=tv -pix_fmt yuv420p -vsync 0 -f yuv4mpegpipe - | aomenc --passes=1 --pass=1 --target-bitrate=1500 --end-usage=vbr --profile=2 --cpu-used=3 --undershoot-pct=0 --overshoot-pct=0 --buf-sz=6 --buf-initial-sz=4 --buf-optimal-sz=5 --drop-frame=0 --kf-min-dist=0 --kf-max-dist=250 --auto-alt-ref=1 --arnr-maxframes=7 --arnr-strength=5 --noise-sensitivity=0 --sharpness=0 --static-thresh=0 --tune-content=default --tile-columns=0 --tile-rows=0 --min-gf-interval=0 --max-gf-interval=0 --threads=2 --width=640 --height=352 --i420 --input-bit-depth=8 --bit-depth=8 --row-mt=0 --cdf-update-mode=1 -o "E:\Temp\18_16_01_5510_01.ivf" -
and
ffmpeg -y -loglevel fatal -threads 8 -i "F:\TestClips&Co\files\test.avi" -map 0:0 -an -sn -vf zscale=rangein=tv:range=tv -pix_fmt yuv420p10le -strict -1 -vsync 0 -f yuv4mpegpipe - | aomenc --passes=1 --pass=1 --target-bitrate=1500 --end-usage=vbr --profile=2 --cpu-used=3 --undershoot-pct=0 --overshoot-pct=0 --buf-sz=6 --buf-initial-sz=4 --buf-optimal-sz=5 --drop-frame=0 --kf-min-dist=0 --kf-max-dist=250 --auto-alt-ref=1 --arnr-maxframes=7 --arnr-strength=5 --noise-sensitivity=0 --sharpness=0 --static-thresh=0 --tune-content=default --tile-columns=0 --tile-rows=0 --min-gf-interval=0 --max-gf-interval=0 --threads=2 --width=640 --height=352 --i420 --input-bit-depth=10 --bit-depth=10 --row-mt=0 --cdf-update-mode=1 -o "E:\Temp\18_16_44_1810_01.ivf" -
give me:
Profile 2 bit-depth < 10 requires 4:2:2 color format


From the looks of it:
Main at least supports:

420 with 8 and 10 bit

High at least supports

444 with 8 and 10 bit

Professional at least supports

420 with 12 bit
422 with 8,10,12 bit
444 with 8,10,12 bit


-> is the wiki wrong of is this a missing feature or bug in aomenc?

Some reliably info would be nice.
(using: av1 - AOMedia Project AV1 Encoder 1.0.0-810-gc9c806a80)

Cu Selur