AV1, VP9, x265, x264 1000kb/s comparison

[karasu]

Another round of visual comparison, this time at 1000kb/s

http://video.1ko.ch/codec-comparison/1000kbps.html

Note about AV1, I tried to enable some experiment when compiling, sadly after a lot of trial and error, I mostly got compiling error or runtime seg fault. The only working compilation did not show significant improvement, so I didn't include it.

[ChaosKing]

I would say VP9 > (AV1 and x265) > x264

[easyfab]

With my tests AV1 is always better the others codec but I don't use veryslow preset ( --cpu-used=2 ) .

Could you try with this --target-bitrate=1000 -t 8 --good --cpu-used=1 --tile-columns=4 --passes=2
or --cpu-used=0

[Sagittaire]

as always, screenshot comparison can't work for codec and for this reason:

Modern codec use I,P,B and bframes. These frames (for x264 and x265 for exemple) have really high quality difference in same GOP because default profil (for x264 and x265 at less) use really high ratio for quantisation, particullary at low bitrate. And moreover, codec like x264 and x265 don't use same GOP structure even for exactly same video part:

http://s009.radikal.ru/i310/1609/10/5f75086b2ae9.png

That mean that video frame can be Pframe for codec X and bframe for codec Y. Average quality can be higher for codec X but local quality can be higher for codec Y (in some frames).

Like this graph show very well, you can't simply use Screenshoot for compare codec quality, simply because conclusion for frame N will be totaly different for frame N+1.

If you want make serious screenshoot comparison, you must compare codec in constant quantisation encoding mode (for I, P, B and Bframes).

[sneaker_ger]

That's arguable. Frame-type decision and quantization / bit distribution are important quality characteristics of any given encoder. It's good to keep your argument in mind but it's not inherently unfair to choose random screenshots as long as they are random and in sufficient number. (Though I must admit I don't fully observe this either when I post things. Luckily x264 and 265 are very similar in this regard as your graph shows.)

[karasu]

If you want make serious screenshoot comparison, you must compare codec in constant quantisation encoding mode (for I, P, B and Bframes).

Thank you for your explanations Sagittaire, what kind of settings would you suggest in that regard ?

Ideally I'd like to test codecs in Constant Quality mode, as it's how I'll use them for work or myself. I also do this in my spare time, I may not have the time to produce a serious and absolutely fair comparison.

I'll probably add another frame where I believe x265 does a better job than VP9. Otherwise I choose the first and last frames as well as two random frames. Showing a particular codec in a better light is absolutely not my intent.

[Atak_Snajpera]

How come frame 200 (Source) is so blurry? Look at his eyes.

[Sagittaire]

Luckily x264 and 265 are very similar in this regard as your graph shows.)

Not really, simply because different max bframe number produce always completely different GOP structure. x264 and x265 use 1.40 for default ratio between PFrame and Bframe: that mean average quantisation can be for exemple q=22 and q=25 between consecutive PFrame and bframe. In this case you can have more than 1.5 dB for PSNR between 2 consecutives frames.

For exemple in the graph, frame 102 is at 6.6 dB for x264 and 8.1 dB for x265, but frame 103 is at 7.4 dB for x264 and 7.2 dB for x265. If you choose frame 102, x265 is clearly better than x264, but not for frame 103.

[Sagittaire]

Ideally I'd like to test codecs in Constant Quality mode, as it's how I'll use them for work or myself. I also do this in my spare time, I may not have the time to produce a serious and absolutely fair comparison.


use simply "--qp xx" mode for x264 and x265. It's by definition constant quality mode (best constant quality for mathematical sense)

AV1 and VP9 have certainely constant quantizer mode encoding too because it's really simple Rate Control mode.

[sneaker_ger]

Like you said they use q ratios of 1.4/1.3. You also have to set those to 1, not only use --qp.

[Sagittaire]

Like you said they use q ratios of 1.4/1.3. You also have to set those to 1, not only use --qp.

Yes. Anyway qcomp, aq-mode, cutree, RD, etc etc ... have big impact on quantizer curve. Better to use constant quantizer to have comparable result between codec.

[nevcairiel]

Results are not "comparable" if you disable half of the features they normally use to compress a video.

[Sagittaire]

Results are not "comparable" if you disable half of the features they normally use to compress a video.

well it's the only solution for make screenshoot comparison. And it's really not the half of the features to compress a video. It's just to have really constant local quality in the GOP and limitation will be the same for all codec.

[karasu]

How come frame 200 (Source) is so blurry? Look at his eyes.

the camera is moving up, vertical motion blur

[Atak_Snajpera]

the camera is moving up, vertical motion blur
How come encoded frames are without this motion blur?

[karasu]

How come encoded frames are without this motion blur?
Good question!

this is frame 13700 of ToS :
http://media.xiph.org/tearsofsteel/tearsofsteel-1080-png/graded_edit_final_13700.png

And frame 13701 :
http://media.xiph.org/tearsofsteel/tearsofsteel-1080-png/graded_edit_final_13701.png

the blur starts right at 13701 (corresponding frame 200 in my source)
So I guess the encoders are somewhat lagging on some details.

The sparks on the girl's wand are proof it's the same frame for all formats.

[Jamaika]

If you want make serious screenshoot comparison, you must compare codec in constant quantisation encoding mode (for I, P, B and Bframes).
What for? You pay for the analyser SSIM 1000$ or more and compare two or more videos with different parameters and the program will evaluate itself. This is important for movies with large motion dynasties in cages. This is a higher driving school.
http://streaminglearningcenter.com/articles/how-to-use-objective-quality-measurement-tools.html

[karasu]

With my tests AV1 is always better the others codec but I don't use veryslow preset ( --cpu-used=2 ) .

Could you try with this --target-bitrate=1000 -t 8 --good --cpu-used=1 --tile-columns=4 --passes=2
or --cpu-used=0

I tried with those parameters and the quality is not significantly better (a bit different but not better).

What are the purposes of -t and --cpu-used ?

[easyfab]

I tried with those parameters and the quality is not significantly better (a bit different but not better).

What are the purposes of -t and --cpu-used ?

-t is for threads number. As av1 is not multi-threaded for the moment it should not change much.
--cpu-used is an equivalent of --preset from x264/x265
--cpu-used 0 slowest / better quality
--cpu-used 8 fastest / less quality

I usualy choose --cpu-used 2 or 4