Google VP9 "Next Generation Open Video" information posted

[Jamaika]

I meant x264 vs VP9. Can't compare crf codecs because the files are different bitrates. Different amounts of data in frames B and P. This doesn't mean that it is less worse. Codecs may have better conversion algorithms.

[mzso]

I meant x264 vs VP9. Can't compare crf codecs because the files are different bitrates. Different amounts of data in frames B and P. This doesn't mean that it is less worse. Codecs may have better conversion algorithms.

I wanted settings to roughly the same quality, nothing more.

[Clare]

So VP9 seems pretty mature these days, right?
I usually use these settings generally to encode AVC:

ffmpeg -i -g 30 -vcodec libx264 -preset slower -crf 18 ki.mkv

What would be roughly identical quality wise with VP9? (with sparing a bit of space, otherwise it's pointless) Are there any convenient presets for encoding speed as for x264?

Can I encode full range 4:4:4 videos like I do with libx264?
ffmpeg -i -pix_fmt yuvj444p -vf scale=out_color_matrix=bt709 -g 30 -vcodec libx264 -preset slower -crf 18 -x264opts colorprim=bt709:transfer=bt709:colormatrix=bt709 out.mkv
(Which I normally do for screencasts)

There is no direct relation between x264's and vp9's crf, it depends on the video. I'd say x264 crf 18 would be similar to a crf between 29 and 33 for vp9.

For encoding speed the main setting is '-cpu-used', between -8 and +8. Higher values speed up the encode at the cost of quality. 1 is the default, usually 1 or 2 is used. You also need to enable multithreading with thread 8 and tile-columns.

For yuv444p, you need to set the profile to 1.

So something like:
ffmpeg -y -i in.y4m -pix_fmt yuv444p -c:v libvpx-vp9 -b:v 0 -profile 1 -crf [29-33] -quality good -threads 8 -cpu-used 2 -tile-columns 4 -an out.mkv

I don't know about colormatrix or full range settings.

[mzso]

Is there a decent guide to the settings at least. Seems a lot more convoluted than libx264. There's "quality", "speed", "cpu-used ", where libx264 had the convenient "preset" options.

Also the video is encoded much faster if I increase crf, which is odd, and also unlike libx264. At the same complexity it should be the same as on higher quality (aka bitrate) settings.

[LigH]

I wanted settings to roughly the same quality, nothing more.

:rolleyes:

During the last years, people have explained repetitively why that won't work well (mainly because you can't really measure a subjective concept like "quality" with objective numbers). The only recommendable strategy is to try to create the same size, as good as it gets, and then compare the quality subjectively (looking at them and deciding which kind of loss looks more annoying).

[Jamaika]

@Clare
Try this:
ffmpeg -y -i in.y4m -c:v libvpx-vp9 -b:v 0 -crf 20 -quality good -threads 8 -cpu-used 2 -tile-columns 4 -arnr-maxframes 7 -arnr-strength 5 -an out.webm
vpxenc --good -p 1 --threads=8 --cpu-used=2 --tile-columns=4 --end-usage=q --cq-level=20 -o out.webm in.y4m
It is some idea. Number of pframes the same. Almost the same, but not the same.
https://www.sendspace.com/filegroup/fhv%2FFn%2BeEFcAMUSn5DQPxA
What bothers me more? Changed framerate in not the newest vpxenc.
Frame rate : 30.036 FPS
Edit:
http://forum.doom9.org/showthread.php?p=1784385#post1784385

[mzso]

:rolleyes:

During the last years, people have explained repetitively why that won't work well (mainly because you can't really measure a subjective concept like "quality" with objective numbers). The only recommendable strategy is to try to create the same size, as good as it gets, and then compare the quality subjectively (looking at them and deciding which kind of loss looks more annoying).

That's why I said roughly, so that I had a starting point

There is no direct relation between x264's and vp9's crf, it depends on the video. I'd say x264 crf 18 would be similar to a crf between 29 and 33 for vp9.

For encoding speed the main setting is '-cpu-used', between -8 and +8. Higher values speed up the encode at the cost of quality. 1 is the default, usually 1 or 2 is used. You also need to enable multithreading with thread 8 and tile-columns.

For yuv444p, you need to set the profile to 1.

So something like:
ffmpeg -y -i in.y4m -pix_fmt yuv444p -c:v libvpx-vp9 -b:v 0 -profile 1 -crf [29-33] -quality good -threads 8 -cpu-used 2 -tile-columns 4 -an out.mkv

I don't know about colormatrix or full range settings.

Thanks for the tips!
Without specifying tile-columns multi-threading doesn't work? (The options "quality" (good, best, realtime), "speed" don't do anything useful?)


On a different note:

It looks like I actually can encode full range:
ffmpeg -i infile -pix_fmt yuv444p -vf scale=out_color_matrix=bt709:out_range=pc -g 30 -vcodec libvpx-vp9 -b:v 0 -quality good -threads 4 -cpu-used 1 -crf 35 vp9-fr.webm
But how do I add correct metadata so that it appears normally? Limited ranges is specified by ffmpeg, because madvr claims: "limited range (claims upstream)", also "matrix bt.709 (best guess)", so that's not specified at all.

I used "-pix_fmt yuvj444p -vf scale=out_color_matrix=bt709 [...] -x264opts colorprim=bt709:transfer=bt709:colormatrix=bt709 out.mkv" with libx264 and it appeared normally.

Edit: Nevermind, found it: Just need to add -pix_fmt yuv444p -vf scale=out_color_matrix=bt709:out_range=pc -color_range pc -colorspace bt709

[Clare]

Is there a decent guide to the settings at least. Seems a lot more convoluted than libx264. There's "quality", "speed", "cpu-used ", where libx264 had the convenient "preset" options.

Also the video is encoded much faster if I increase crf, which is odd, and also unlike libx264. At the same complexity it should be the same as on higher quality (aka bitrate) settings.

It is more convoluted, I hope they make a simpler encoder for AV1. I don't know what is the "speed" settings from ffmpeg, there's nothing similar in vpxenc manual.

[nevcairiel]

Is there a decent guide to the settings at least. Seems a lot more convoluted than libx264. There's "quality", "speed", "cpu-used ", where libx264 had the convenient "preset" options.

"speed" is just an (legacy) alias for "cpu-used", which is your primary knob for performance/quality trade-off. Its a bit oddly named for sure.
cpu-used=0 is the slowest and highest quality mode, going up to 9 for vp9 as the fastest mode.

On top of that, you have the quality mode, which is either "good" or "best" (or "realtime" which is only useful for what the name suggests it would be).
"best" is often overkill and even the vpx docs don't recommend using it, its akin to a placebo mode in libx264.

So in short, start with quality=good (which is also the default, so may as well skip it) and try a few cpu-used modes to dial in your performance/quality tradeoff.

Various explanations for the options are here, if it wasn't linked above already:
http://www.webmproject.org/docs/encoder-parameters/

Note that not all options map 1:1 if you use ffmpeg as a wrapper to the encoder, but most probably do.

[mzso]

@nevcairiel
Thanks!

[Clare]

@Clare
Try this:
ffmpeg -y -i in.y4m -c:v libvpx-vp9 -b:v 0 -crf 20 -quality good -threads 8 -cpu-used 2 -tile-columns 4 -arnr-maxframes 7 -arnr-strength 5 -an out.webm
vpxenc --good -p 1 --threads=8 --cpu-used=2 --tile-columns=4 --end-usage=q --cq-level=20 -o out.webm in.y4m

or this:
ffmpeg -y -i in.y4m -c:v libvpx-vp9 -b:v 0 -crf 20 -quality good -threads 8 -cpu-used 2 -tile-columns 4 -arnr-maxframes 7 -arnr-strength 5 -an -pass 1 -f null -
ffmpeg -y -i in.y4m -c:v libvpx-vp9 -b:v 0 -crf 20 -quality good -threads 8 -cpu-used 2 -tile-columns 4 -arnr-maxframes 7 -arnr-strength 5 -an -pass 2 out.webm

vpxenc --good -p 2 --threads=8 --cpu-used=2 --tile-columns=4 --end-usage=q --cq-level=20 -o out.webm in.y4m
I tried that but it doesn't account for the large difference in size.

[Neavrie]

@Clare
vpxenc uses 2-pass mode by default. That's why you get difference in file size.

[Clare]

@Clare
vpxenc uses 2-pass mode by default. That's why you get difference in file size.

Huh you're right! I didn't think a two-pass encode would bring such a huge difference in file size. Now I get the exact same metrics for both FFMPEG and vpxenc.

Thanks a lot.

[Jamaika]

@Clare
For pass one is 600kb difference in 100 frames 4K.
For pass two isn't difference between the file size(~100kb). This is puzzling, but it is worth knowing. So, I understand that the tests should only do to pass two, because otherwise there is no sense.
The correct command is:
ffmpeg.exe -loglevel trace -y -i iphone6s_4k.yuv -c:v libvpx-vp9 -b:v 0 -passlogfile vp9.log -crf 20 -quality good -threads 4 -cpu-used 2 -tile-columns 4 -arnr-maxframes 7 -arnr-type 3 -arnr-strength 5 -pass 1 -an out1a.ivf
ffmpeg.exe -loglevel trace -y -i iphone6s_4k.yuv -c:v libvpx-vp9 -b:v 0 -passlogfile vp9.log -crf 20 -quality good -threads 4 -cpu-used 2 -tile-columns 4 -arnr-maxframes 7 -arnr-type 3 -arnr-strength 5 -pass 2 -an out1b.ivf
vpxenc.exe -v --ivf --good --passes=2 --pass=1 --fpf=vp9.pass --threads=4 --target-bitrate=256 --cpu-used=2 --tile-columns=4 --end-usage=q --cq-level=20 --arnr-maxframes=7 --arnr-type=3 --arnr-strength=5 -o NUL iphone6s_4k.yuv
vpxenc.exe -v --ivf --good --passes=2 --pass=2 --fpf=vp9.pass --threads=4 --target-bitrate=256 --cpu-used=2 --tile-columns=4 --end-usage=q --cq-level=20 --arnr-maxframes=7 --arnr-type=3 --arnr-strength=5 -o out2.ivf iphone6s_4k.yuv

[sneaker_ger]

vpxenc has automated two-pass. You only run it once. If you run it twice you are basically doing 4 passes with the first 2 being wasted into NUL. Try running it only once and check if that is the case. It was for me the last time I tested. (Granted, long time ago and I don't remember the commands I used from the top of my head.)

[Jamaika]

OK. I accepted the news, but this command ffmpeg gave different results. Different (~300kb):(
ffmpeg.exe -loglevel trace -y -i iphone6s_4k.yuv -c:v libvpx-vp9 -b:v 0 -crf 20 -quality good -threads 4 -cpu-used 2 -tile-columns 4 -arnr-maxframes 7 -arnr-type 3 -arnr-strength 5 -pass 2 -an out1a.ivf

[easyfab]

Perhaps others settings have not the same defaults in FFmpeg and Vpxenc ?

[sneaker_ger]

Radeon Software Crimson ReLive Edition 16.12.2 Highlights
Fixed Issues
Chromium may fail to utilize hybrid decode for VP9 content.
http://support.amd.com/en-us/kb-articles/Pages/Radeon-Software-Crimson-ReLive-Edition-16.12.2-Release-Notes.aspx

[Selur]

According to https://en.wikipedia.org/wiki/VP9:
VP9 supports HDR video using Hybrid Log-Gamma (HLG) and Perceptual Quantizer (PQ)
but when I compile vp9 using (https://github.com/jb-alvarado/media-autobuild_suite) the vpxenc command line options don't mention it and I can't find any documentation about 'VP9-HLG' which describes how to create such content.
-> Does anyone here have a clue how to create VP9 with HLG ?

Cu Selur

[Jamaika]

VP9 supports HDR video using Hybrid Log-Gamma (HLG) and Perceptual Quantizer (PQ)
https://www.tapatalk.com/topic/47413-avsforum/2559657-hdr-android-7-0
I am writing as an amateur because he didn't compile VP9.
For me, reading google is an option only android 7.0. And I think this is April Fool's Day.
Then it was written that this isn't an option codec VP9 only container matroska. Not this topic.;)
Option container matroska can be found in the directory libwebm / mkvmuxer / mkvmuxerutil.cc
bool IsTransferCharacteristicsValueValid(uint64_t value) {
switch (value) {
case mkvmuxer::Colour::kIturBt709Tc:
case mkvmuxer::Colour::kUnspecifiedTc:
case mkvmuxer::Colour::kReservedTc:
case mkvmuxer::Colour::kGamma22Curve:
case mkvmuxer::Colour::kGamma28Curve:
case mkvmuxer::Colour::kSmpte170MTc:
case mkvmuxer::Colour::kSmpte240MTc:
case mkvmuxer::Colour::kLinear:
case mkvmuxer::Colour::kLog:
case mkvmuxer::Colour::kLogSqrt:
case mkvmuxer::Colour::kIec6196624:
case mkvmuxer::Colour::kIturBt1361ExtendedColourGamut:
case mkvmuxer::Colour::kIec6196621:
case mkvmuxer::Colour::kIturBt202010bit:
case mkvmuxer::Colour::kIturBt202012bit:
case mkvmuxer::Colour::kSmpteSt2084:
case mkvmuxer::Colour::kSmpteSt4281Tc:
case mkvmuxer::Colour::kAribStdB67Hlg:
return true;
}
return false;
}
Edit: Ok, In VP9 has inactive option HDR if we compile (hdr_util.cc, file_util.cc). In theory it should work. The strange thing is that I can't add a transfer option.
https://chromium-review.googlesource.com/#/c/400084/

PS The codec has damaged the function fps.;)

[nevcairiel]

-> Does anyone here have a clue how to create VP9 with HLG ?



VP9 itself doesn't signal any of the HDR metadata, neither in HDR10 (ie. PQ) mode or HLG mode, it needs to be signaled through the container (ie. webm/matroska).

So basically, just encode the content mastered in HLG, and then instruct the container to convey the HLG transfer information during muxing.
Luckily HLG doesn't need any fancy metadata, other then signaling that the content is in fact HLG.

[Selur]

So we will have to wait till muxers like mkvtoolnix, mp4box, ffmpeg,.. support signaling the content as HLG and there isn't anything needed for the encoders to support. :)
Thanks for the explanations.

Cu Selur

[sneaker_ger]

For mkvmerge see:
https://mkvtoolnix.download/doc/mkvmerge.html#d4e2614
https://www.matroska.org/technical/specs/index.html#Colour

[david.copperfield1337]

https://github.com/YouTubeHDR/hdr_metadata

Gesendet von meinem Nexus 7 mit Tapatalk

[Jamaika]

https://github.com/YouTubeHDR/hdr_metadata

Gesendet von meinem Nexus 7 mit Tapatalk
Sorry, something I don't understand. Youtube boasts movies VP9 HDR. Unfortunately, I don't see movies 10bit. Are they done with HDR at 8bit color range tv?
./src/mkvmerge \
-o output.mkv\
--colour-matrix 0:9 \
--colour-range 0:1 \
--colour-transfer-characteristics 0:16 \
--colour-primaries 0:9 \
--max-content-light 0:1000 \
--max-frame-light 0:300 \
--max-luminance 0:1000 \
--min-luminance 0:0.01 \
--chromaticity-coordinates 0:0.68,0.32,0.265,0.690,0.15,0.06 \
--white-colour-coordinates 0:0.3127,0.3290 \
input.mov --colour-bits-per-channel 0:10 ???
--chroma-subsample 0:1,0 ???
--colour-range 0:2 ???

https://www.youtube.com/watch?v=tO01J-M3g0U

[sneaker_ger]

10 bit VP9 is there. Doesn't have to be tagged as 10 bit on the container level to be 10 bit.
https://forum.doom9.org/showpost.php?p=1785300&postcount=543

Meaning of Matroska values are in the specs:
https://www.matroska.org/technical/specs/index.html#Colour

[Jamaika]

Doesn't have to be tagged as 10 bit on the container level to be 10 bit.

None of this don't understand.:(
"Free YouTube download" doesn't see the color matrix or HDR. For me fake.
Stream #0:1(eng): Video: vp9 (Profile 0), yuv420p(tv, bt709/unknown/unknown), 3840x2160, SAR 1:1 DAR 16:9, 59.94 fps, 59.94 tbr, 1k tbn, 1k tbc (default)

[sneaker_ger]

Read the post I linked. youtube-dl (https://rg3.github.io/youtube-dl/) can download the 10 bit (profile 2) streams.

[Selur]

@sneaker_ger: Thanks.

[Jamaika]

Read the post I linked. youtube-dl (https://rg3.github.io/youtube-dl/) can download the 10 bit (profile 2) streams.
Ah, I understood what was going on. I don't know why so many intricacies.
youtube-dl.exe -o "The World in HDR.webm" -v -f "(mp4)bestvideo/best[height=2160]+bestaudio/best[height=2160]" https://www.youtube.com/watch?v=tO01J-M3g0U
And even displays ffmpeg the message.
Stream #0:0(eng): Video: vp9 (Profile 2), yuv420p10le(tv, bt2020nc/bt2020/smpte2084), 3840x2160, SAR 1:1 DAR 16:9, 59.94 fps, 59.94 tbr, 1k tbn, 1k tbc (default)
Side data:
Mastering Display Metadata, has_primaries:1 has_luminance:1 r(0.6800,0.3200) g(0.2649,0.6900) b(0.1500 0.0600) wp(0.3127, 0.3290) min_luminance=0.001000, max_luminance=1000.000000
..., but where came it from? It's only change for file 10/12bit apart from the colour range and matix.

[Jamaika]

2017-01-09 v1.6.1 "Long Tailed Duck"
This release improves upon the VP9 encoder and speeds up the encoding and
decoding processes.

- Upgrading:
This release is ABI compatible with 1.6.0.

- Enhancements:
Faster VP9 encoding and decoding.
High bit depth builds now provide similar speed for 8 bit encode and decode
for x86 targets. Other platforms and higher bit depth improvements are in
progress.

- Bug Fixes:
A variety of fuzzing issues.
https://chromium.googlesource.com/webm/libvpx/+/v1.6.1
Patches are added to ffmpeg.;)

[Leeloo Minaï]

Can someone provide a vpxenc 1.6.1 build for Windows x64 ?

[LigH]

I can try to run jb-alvarado's media-autobuild_suite (https://github.com/jb-alvarado/media-autobuild_suite) tonight...

[wiak]

Can someone provide a vpxenc 1.6.1 build for Windows x64 ?
lets hope this works
https://awesome.nwgat.ninja/misc/libvpx-1.6.1-win64.7z

[LigH]

vpx v1.6.1-95-g9efc42f4f (https://www.mediafire.com/file/an4zqj21lfh4bp2/vpx_v1.6.1-95-g9efc42f4f.7z) (static builds, MSYS2, GCC 6.3.0, Win32+Win64)

[Leeloo Minaï]

Thanks for your help !
I asked for the latest standalone version in order to compare with older revisions and FFmpeg builds.

In fact, i get slower encoding for revision 1.6.0 and 1.6.1 than with previous revisions 1.4.0 and 1.5.0 using vpxenc at same parameters (the same appears with FFmpeg too) :

Video properties :
384x256 @ 55fps, 400 frames, YUV420 (game capture from the arcade game "R-Type" (1987) using MAME emulator)

Encoding parameters :
vpxenc test.y4m -o vp9.webm --codec=vp9 -p 2 --pass=1 --fpf=stats.log --good --cpu-used=1 --target-bitrate=400 --end-usage=vbr --auto-alt-ref=1 --minsection-pct=5 --maxsection-pct=800 --lag-in-frames=16 --kf-min-dist=0 --kf-max-dist=360 --static-thresh=0 --drop-frame=0 --min-q=0 --max-q=63
vpxenc test.y4m -o vp9.webm --codec=vp9 -p 2 --pass=2 --fpf=stats.log --good --cpu-used=1 --target-bitrate=400 --end-usage=vbr --auto-alt-ref=1 --minsection-pct=5 --maxsection-pct=800 --lag-in-frames=16 --kf-min-dist=0 --kf-max-dist=360 --static-thresh=0 --drop-frame=0 --min-q=0 --max-q=63


Encoding times :
- 1.4.0 : 1:38,56 (~4.06 fps)
- 1.5.0 : 1:26,25 (~4.64 fps)
- 1.6.1 : 1:38,78 (~4.04 fps)

My laptop is rather old and low-powered (Pentium Dual-Core @ 2.2Ghz under Windows 7 x64), but i still laugh when i read "Faster encoding" :rolleyes:

[Selur]

my guess is the 'faster encoding' would come from some assembler optimizations which might not help on older systems,...

[wiak]

my guess is the 'faster encoding' would come from some assembler optimizations which might not help on older systems,...

that was my first idea too, newer systems has alot of new instructions and moar cores!

[Nintendo Maniac 64]

moar cores!

You know it's becoming mainstream when even Intel is supposedly hopping on this bandwagon with Coffee Lake!

AMD was just too ahead of their time.

[wiak]

You know it's becoming mainstream when even Intel is supposedly hopping on this bandwagon with Coffee Lake!

AMD was just too ahead of their time.
they have been ahead of the time for a while, case in point gcn

[CruNcher]

Not only GCN remember Fusion/HSA as well which every console has become part of and where Mantle arose from :)

[LigH]

cries in German

My best available CPU is still a Phenom-II X6. No AVX to boost modern codecs. Part-time paid full-time jobs. Globalization FTW.

[mandarinka]

cries in German

My best available CPU is still a Phenom-II X6. No AVX to boost modern codecs. Part-time paid full-time jobs. Globalization FTW.

With Phenom, your main problem is lack of SSSE3 and SSE4, not lack of AVX. If you need more recent SIMD instructions tho, I see lots of refurbished office desktops with the likes of Core i5-2400 with prices starting at circa 175 € at the etailers that deal in such machines. Maybe that could be a viable way. Also Xeon E5-2600 chips of the "v1" (Sandy Bridge) category are cheap on ebay, but in this case it is hard to get the needed motherboard for cheap.

(And I bet we would all have even shittier computers and living standards were worse, if this "globalization" didn't occur. It's just scapegoat and/or red herring in somebody's agenda, as it happens.)

Sent from my second hand AMD A10-6800K...

[Nintendo Maniac 64]

I wouldn't be surprised if the Ryzen CPUs launching at the end of this month cause the prices on Sandy Bridge CPUs and LGA1155 systems to completely crater.

[mandarinka]

I wouldn't be surprised if the Ryzen CPUs launching at the end of this month cause the prices on Sandy Bridge CPUs and LGA1155 systems to completely crater.

That would be nice, but the prices of those second-hand office desktops are already low if you consider that they have 4 GB of DDR3 (which got more expensive since 2012 instead of getting cheaper) and W7 license sticker and you can use those serials to activate clean installs of W10 on the machine, AFAIK.

Ryzens will launch at prices of i7-7700K and higher, so they might lower prices of second hand gaming computers, motherboards and CPUs like i5-2500K, i7-2600K and other quadcores that gamers or "enthusiasts" have been buying.

[Nintendo Maniac 64]

Ryzens will launch at prices of i7-7700K and higher

You realize there will be fewer-core versions of Ryzen and not just the 8core/16thread variant...

[CruNcher]

Most probably though the most expensive ones 8 core will be launched first for pre adopters and then later the avg consumer 4 core ones following which would be the opposite from the GPU Strategy ;)

[mandarinka]

You realize there will be fewer-core versions of Ryzen and not just the 8core/16thread variant...

I know (https://s23.postimg.org/tqb06wo0p/9d7312ab91d7c638c9760dd5aec3f46ef3c2aa4c300b9ea9.jpg). My estimate is that they will get to about 150-175 USD price level ( 4C/4T). But currently it seems that such SKUs won't be part of the initial launch, it's unknown when will they start to sell.

[CruNcher]

Interesting now it seems we have a standard for Performance indexing with a Number from Intel and AMD :D

Ryzen 3,5,7
Core 3,5,7


which respectively points to 2,4,8 and 4,6,8 cores with a balancing in between thread executions and clock ;)

So in the middle Intel fights vs 6 cores now with 4 and its AVX2 :)

The most interesting compare point between both at those 65W :)

[nevcairiel]

which respectively points to 2,4,8 and 4,6,8 cores with a balancing in between thread executions ;)

But it doesn't, not for Intel anyway.
There is i7s with 4, 6, 8 and 10 cores.