Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > VP9 and AV1

Reply
 
Thread Tools Search this Thread Display Modes
Old 15th January 2013, 07:20   #41  |  Link
xooyoozoo
Registered User
 
Join Date: Dec 2012
Posts: 197
Played around with a checkout from master branch on Sat, and the decoder fails with --tune=ssim and --best during scene changes. I then just did a build with their current code, which merged in some goodies from their experimental branch. Now --tune=ssim breaks the encoder instead with "segmentation fault 11" during the first 16 frames of pass 2. Maybe it's auto-alt-ref related, but either way, looks like they definitely aren't developing for SSIM...

With the older build, I did quick 2pass runs of a dark, slightly grainy source and no tunings on either encoders. VP9 had a lot higher PSNR (maybe it defaults to this tuning) and slightly higher SSIM than x264 placebo (forgot to match keyint though whoops), but x264 fared better visually, especially in scenes of grain against a flat background. VP9 also had this weird thing where grain suddenly reappeared for 1 frame and then goes away again, and the contrast is extreme enough to be noticeable in normal playback.

I think there's definitely still some qp and rc tunings to be done. In terms of objective metrics, however, things are at least very promising.

Code:
vpxenc -p 2 -t 1 --codec=vp9 --best --cpu-used=0 \
  --target-bitrate=1500 --end-usage=vbr \
  --auto-alt-ref=1 -v \
  --minsection-pct=5 --maxsection-pct=800 \
  --lag-in-frames=16 --kf-min-dist=0 --kf-max-dist=360 \
  --token-parts=2 --static-thresh=0 --drop-frame=0 \
  --min-q=0 --max-q=60 \
  -o .webm .y4m
No hard numbers because I only wanted tune-ssim results and deleted everything after visual inspection.

Last edited by xooyoozoo; 15th January 2013 at 10:04. Reason: Clarity
xooyoozoo is offline   Reply With Quote
Old 17th January 2013, 09:58   #42  |  Link
xooyoozoo
Registered User
 
Join Date: Dec 2012
Posts: 197
Got around to actually testing the most recent build of libvpx, and it's quite something. Here's a chart of my results. If you're wondering about the abundance of metrics, it's because I was trying to see if anything could approximate x264's psy tweaks (nope).

There's no need to mince words: VP9 is impressive. Besides its apparent proficiency in objective metrics, it also *looks* good. I went through the 760-765 kbits clips and in terms of "pleasantness" I'd rank them as xs < xn < hevc =< vp9. The latter two are difficult to choose between because though HEVC makes the image look more 'clean', I think VP9 has some subtle psy tunings that gave more eye-pleasing bit allocations. Of course, this is only one sample, not representative, and so and so on, but we've certainly come a looonnng way since VP8's painfully underwhelming introduction.

Also, a confounding factor in the HEVC's test might be the changed intra period (normally 24 frames) and the 3 adaptive/psy QP switches I turned on. The changed intra period is to match with x264 and VP9, and I double checked the JCT-VC docs the adaptive-qp switches were based off of. The docs all showed increased obj+subj performance, but I guess you never can be sure.

Edit: Oh yea, VPX built on code from late Monday, HM from the weekend, and x264 from November.

Edit (2/2): should have done this little tidbit in the first place:

Last edited by xooyoozoo; 3rd February 2013 at 11:58.
xooyoozoo is offline   Reply With Quote
Old 3rd February 2013, 12:48   #43  |  Link
mandarinka
Registered User
 
mandarinka's Avatar
 
Join Date: Jan 2007
Posts: 729
I went back and made a sample encode with the decemberish build that Kurtnoise posted, since last time I deleted the samples/pictures.
This time I used a DVD source and encoded at 2500kbps. Basically, as you can see from the pictures, the problem is lack of texture/grain preservation, that leads to nasty banding/blocking in flat areas. At the same time, edges are kept dirty, which leaves the image sorta weird.
I didn't do a x264 sample, because with the utter lack of texture preservation in VP9, it would be meaningless to compare differences IMHO.

Here's a comparison with select pics, for illustration: http://check2pic.ru/compare/25338/

Code:
<BBB-work> we actually improved a lot since then, so if you want to play around, using a recent snapshot may be a good idea
I'm hoping that the final version of the encoder won't be so smooth-happy...

Settings:
Code:
vpxenc --codec=vp9 --width=704 --height=480 --fps=24000/1001 --i420 --target-bitrate=2500 --auto-alt-ref=1 --threads=1
--kf-max-dist=480 --passes=2 --end-usage=vbr --good --cpu-used=0 --min-q=0 --max-q=60 --drop-frame=0 --sharpness=0
mandarinka is offline   Reply With Quote
Old 3rd February 2013, 20:27   #44  |  Link
xooyoozoo
Registered User
 
Join Date: Dec 2012
Posts: 197
Quote:
Originally Posted by mandarinka View Post
I'm hoping that the final version of the encoder won't be so smooth-happy...
I did a test a while back and apparently tune=psnr is the default. As in, both encodes (no tunings and psnr tuned) produce exactly the same filesize/metrics, the doc website states something similarly, tune=ssim has been broken for several weeks, and oh, they don't even appear to be testing with ssim.

That said, at low bitrates where things get desperate, graceful quantization (a la x264) doesn't mean as much as simply having a larger mathematical buffer and getting as much into one bit as possible.
xooyoozoo is offline   Reply With Quote
Old 3rd February 2013, 20:45   #45  |  Link
Kurtnoise
Swallowed in the Sea
 
Kurtnoise's Avatar
 
Join Date: Oct 2002
Location: Aix-en-Provence, France
Posts: 5,191
See post #49

Last edited by Kurtnoise; 4th February 2013 at 20:01.
Kurtnoise is offline   Reply With Quote
Old 3rd February 2013, 21:25   #46  |  Link
mandarinka
Registered User
 
mandarinka's Avatar
 
Join Date: Jan 2007
Posts: 729
Oh, nice! I'll go and have a look later then.
mandarinka is offline   Reply With Quote
Old 3rd February 2013, 22:49   #47  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 7,255
thanks Kurtnoise
__________________
Hybrid here in the forum, homepage
Selur is offline   Reply With Quote
Old 4th February 2013, 08:06   #48  |  Link
m3sh
Registered User
 
Join Date: Apr 2012
Location: Edmonton
Posts: 9
Thanks Kurtnoise, been meaning to test this against HEVC.
m3sh is offline   Reply With Quote
Old 4th February 2013, 20:03   #49  |  Link
Kurtnoise
Swallowed in the Sea
 
Kurtnoise's Avatar
 
Join Date: Oct 2002
Location: Aix-en-Provence, France
Posts: 5,191
Any interests to play with lossless mode ?

new builds uploaded - added Lossless Mode to VP9 + WinXP compatibility + x86/x64 targets
Kurtnoise is offline   Reply With Quote
Old 5th February 2013, 00:39   #50  |  Link
xooyoozoo
Registered User
 
Join Date: Dec 2012
Posts: 197
I came across this discussion in vpx development google group. It produced this quote concerning GOP lengths:
Quote:
We used default for each (so 250 for x264, and 9999 for vpxenc).
We'll try to correct this for future comparisons.
which was troubling because, while they didn't specifically mention HEVC, it implies that in most of the tests encodes shown in VP9 slides, HEVC would have had 10+ intra frames while VP9 got away with 1! So I then decided to do my own tests with a few clips while matching GOP lengths. I used HEVC's official bitstream files and results, and matched VP9's GOP length to that. As all of VPx's promotional material emphasizes that it's meant to be a "Web codec", it (hopefully) shouldn't have any problems with 1 intraframe a second (Youtube uses 2 a sec).

Additionally, I wanted to see how VP9 performs with its bleeding edge code. I used the latest Jan-31 commit (c94e55...) and switched on all experiments except for CSM (broken?), 6tap and abovesprefmv; the latter two appear to exist mostly for simplification. A couple of sanity checks showed almost equal PSNR versus vanilla VP9.

VP9 encode paramaters were copied from WebM guide for best quality, but kf-max-dist was changed to match GOP and tune=psnr was added. The latter doesn't seem to do anything, but I might as well cover my bases. x264 tests were encoded with veryslow, psnr|ssim, matched keyint, open-gop, and crf.

Base spreadsheet and visual basic bd-rate/interpolation code copy-pasted from JCT-VC L0322.



(The data)

PSNR-HVS-M is included because I wanted something besides PSNR and single-scale SSIM. The downfalls of the first is obvious, and the second is getting deprecated as a perceptual metric. Some interesting tidbits pertaining to the second point: examples of relative ranking of subjectively vs objectively measured distortion (paper), SSIM's mediocre performance on MPEG distortion (paper), and statistical ranking of metrics on certain video distortion types (first value of each cell is for compression results, paper). I was going to include some other metrics (MSSSIM, NQM) but Matlab got the better of me, so next time.

As for subjective results? Well, it's what you'd expect. It's difficult to swing a 40+ percent bitrate disadvantage in VP9's favor.

Edit: (2/6: MOAR PIX) Well, here are some comparison pics for the 3 lower tiers of bitrates. A separate 2pass veryslow no tunings x264 was ran just for this to match bitrates.
Code:
Kimono (1920x1080p24)
0540: HEVC, VP9, x264
1050: HEVC, VP9, x264
2150: HEVC, VP9, x264
Code:
ParkScene (1920x1080p24)
0710: HEVC, VP9, x264
1520: HEVC, VP9, x264
3275: HEVC, VP9, x264
Code:
BasketballDrill (832x480p50)
0430: HEVC, VP9, x264
0815: HEVC, VP9, x264
1660: HEVC, VP9, x264

Last edited by xooyoozoo; 7th February 2013 at 01:13.
xooyoozoo is offline   Reply With Quote
Old 6th February 2013, 17:40   #51  |  Link
IgorC
Registered User
 
Join Date: Apr 2004
Posts: 1,315
Thank You for interesting information, xooyoozoo. And for screenshots.

Can I ask You to re-do the same test but on real scenario conditions? Like 1-2 Mbits 720p, 3-4 Mbits 1080p and 500-600 kbps for SD, 480-576p.

It's unlikely that someone encodes 1920x1080 at 535 kbps here.



P.S. The original.png will be good too.

Last edited by IgorC; 6th February 2013 at 19:00.
IgorC is offline   Reply With Quote
Old 7th February 2013, 00:36   #52  |  Link
xooyoozoo
Registered User
 
Join Date: Dec 2012
Posts: 197
Quote:
Originally Posted by IgorC View Post
It's unlikely that someone encodes 1920x1080 at 535 kbps here.
I wanted to see how and how much each encoder breaks under pressure, but I guess you make a pretty good point.

My post above is updated with two higher tiers of bitrates (which were all originally set by HEVC's constant quantization of 27, 32, and 37). I added reference shots too. Based off those, I think Netflix/iTunes/Hulu-level streaming bitrates are gonna produce significantly higher quality in the coming years.

VP9 and HEVC seem to be on par at about QP=27, with VP9 doing consistently better on the Basketball clip. Maybe VP9 does really well in areas with motion (some of this can be seen on the bicyclers* in the Park clip). I'd like to find out more, but some speed optimization commits are starting to come in the libvpx git, and for my sanity, I think I'll wait some weeks for more to land.

* Hmm after looking at the clips some more, I think that VP9 gives more bitrate than necessary to motion, as the bicyclers get attention even though the entire background suffers.

Last edited by xooyoozoo; 7th February 2013 at 01:23.
xooyoozoo is offline   Reply With Quote
Old 7th February 2013, 05:03   #53  |  Link
pieter3d
Registered User
 
Join Date: Jan 2013
Location: Santa Clara CA
Posts: 114
Quote:
Originally Posted by xooyoozoo View Post

* Hmm after looking at the clips some more, I think that VP9 gives more bitrate than necessary to motion, as the bicyclers get attention even though the entire background suffers.
That isn't really something you can attribute to VP9 being VP9, just this particular reference encoder.
pieter3d is offline   Reply With Quote
Old 7th February 2013, 05:03   #54  |  Link
pieter3d
Registered User
 
Join Date: Jan 2013
Location: Santa Clara CA
Posts: 114
By the way, where is the source code for VP9 ?
pieter3d is offline   Reply With Quote
Old 7th February 2013, 07:08   #55  |  Link
xooyoozoo
Registered User
 
Join Date: Dec 2012
Posts: 197
Here's their git. As VP9's bitstream is undergoing rapid change and is experimental in and of itself, one might as well hop on the experimental branch too.

If anyone's curious, here are the clips for 1660kbps BasketballDrill: HEVC, VP9, x264.

The first two are lossless mkvs at 60 MB each. The third is in its native 2MB (lol).
xooyoozoo is offline   Reply With Quote
Old 7th February 2013, 07:27   #56  |  Link
Kurtnoise
Swallowed in the Sea
 
Kurtnoise's Avatar
 
Join Date: Oct 2002
Location: Aix-en-Provence, France
Posts: 5,191
There is something wrong with your samples..these are all AVC streams.

Kurtnoise is offline   Reply With Quote
Old 7th February 2013, 08:03   #57  |  Link
xooyoozoo
Registered User
 
Join Date: Dec 2012
Posts: 197
They were reencoded losslessly (qp = 0) from decoded YUV files. "High 444 Predictive" shown above is the Predictive Lossless profile. (Edit: the snapshots taken above were taken via ffmpeg at time -ss 00:00:09.38 which I think makes it the 469th frame)

Last edited by xooyoozoo; 7th February 2013 at 08:45.
xooyoozoo is offline   Reply With Quote
Old 10th February 2013, 19:23   #58  |  Link
MasterNobody
Registered User
 
Join Date: Jul 2007
Posts: 552
My small H.265/HEVC vs VP9 vs x264 codec comparison
MasterNobody is offline   Reply With Quote
Old 11th February 2013, 17:44   #59  |  Link
MasterNobody
Registered User
 
Join Date: Jul 2007
Posts: 552
Quote:
Originally Posted by damn_gwwgle View Post
Comparison pictures for you:
...
Without encoded samples and used encoding params for both encoders that is only two blurry images nothing more. Also 1000 kbps is insane for 720p 50fps (if you really encoded it as 50fps) even if you take into account marketing 50% less bitrate than H.264 (except if we talk about anime or flash-animation).

Here is two more blurry images for you: x264_nopsy, x264_psy
Samples: x264_nopsy.mkv, x264_psy.mkv
Params:
Code:
x264 park_joy_420_720p50.y4m -o x264_nopsy.mkv --preset veryslow --keyint infinite --crf 36.2 --psy-rd 0 --threads 1
x264 park_joy_420_720p50.y4m -o x264_psy.mkv --preset veryslow --keyint infinite --crf 38.8 --threads 1
P.S. x264's psy at such bitrates looks psychedelic when watching video (not screenshots)

Last edited by MasterNobody; 11th February 2013 at 18:50.
MasterNobody is offline   Reply With Quote
Old 14th March 2013, 01:36   #60  |  Link
xooyoozoo
Registered User
 
Join Date: Dec 2012
Posts: 197
Not sure if this was posted elsewhere, but Google published a draft overview of VP9 on the IETF website last month.

This morning, they also made a blog post promising more details about VP9 over the next few weeks. In all likelihood, VP9 will be 'released' at Google I/O in May just like VP8 was back in 2010.

There's been a lot of code cleanups and speed improvements on libvpx's exp. branch over the past couple of weeks too. The encoder is a lot faster now and seems about as fast as the old JM, which sounds like a terrible endorsement, but it's certainly better than before . Half of the test batch I just made failed to decode though, so win some, lose some.
xooyoozoo is offline   Reply With Quote
Reply

Tags
google, ngov, vp8, vp9, vpx, webm

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 08:24.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.