Benwaggoner HEVC encoding challenge

[sneaker_ger]

Originally Posted by SaurusX

Amazon is considered the gold standard for video encodes

You misspelled "Netflix".

Originally Posted by SaurusX

As it is, there is no animation tune given by the x265 developers and it appears it's because the combination of hard edges, flat cels, grain, and detailed backgrounds tends to throw HEVC for a loop.

When I tested x265 years ago it did very well on animation, except very grainy/old ones. Easily beat x264 and by a considerable margin. Much better edges than x264.
https://forum.doom9.org/showthread.p...98#post1693498
https://forum.doom9.org/showthread.p...81#post1693681

[benwaggoner]

Originally Posted by sneaker_ger

You misspelled "Netflix".


When I tested x265 years ago it did very well on animation, except very grainy/old ones. Easily beat x264 and by a considerable margin. Much better edges than x264.
https://forum.doom9.org/showthread.p...98#post1693498
https://forum.doom9.org/showthread.p...81#post1693681

Speaking just for myself, I consider HEVC as having some helpful features for cel animation that reduce the need to have a separate preset. More precise prediction modes are a huge help. Having the option of transform skip and (at high bitrates) lossless CUs can also be a big help.


A preset might be able to find a better speed/quality tradeoff, but I don't know that actual peak compression efficiency would improve THAT much. That said, I haven't taken a whack at doing a new cel animation setting since the 2.4 lambda tables were introduced.

Another complicating factor is that modern anime can have grain effects, CGI elements at 24p, and other things beyond the classic hand-drawn cels that traditional "--tune animation" was meant for. Particularly in title sequences. Encoding "Kabaneri of the Iron Fortress" is quite different than "Dumbo."

A good followup test to this one would be to use some anime! Anyone know of any good Creative Commons licensed 1080p24 anime/cel animation?

[Ma]

My core x265 options are:
--aq-mode 3 --qg-size 64 --rc-lookahead 120 --cbqpoffs -1 --crqpoffs -1 --subme 7 -F1

The main difference to benwaggoner options is '--qg-size 64' -- it really helps (so I've encoded at veryslow preset).

0.5M www.msystem.waw.pl/x265/tos_x265vs_500.mkv PSNR: 39.471, SSIM: 12.784 dB
1.0M www.msystem.waw.pl/x265/tos_x265vs_1000.mkv PSNR: 41.901, SSIM: 14.585 dB
1.5M www.msystem.waw.pl/x265/tos_x265vs_1500.mkv PSNR: 43.206, SSIM: 15.515 dB

In my opinion 0.5M version wins with MPEG2 1.5M version and is comparable to MPEG2 2M version.

I've attached encoding details (encode.txt) and proof that '--qg-size 64 --rc-lookahead 120' are coll and not time consuming (tests.txt).

[asarian]

Originally Posted by Ma

My core x265 options are:
--aq-mode 3 --qg-size 64 --rc-lookahead 120 --cbqpoffs -1 --crqpoffs -1 --subme 7 -F1

Are these what you'd call 'best' x265 settings?

I realize my question may be slightly off-topic, but since the OP kinda posted his challenge, with apparently super-x265 settings, this might be a good time to ask what ppl deem x265 settings for super-quality? Most of the x265 options (especially those that all affect more or less similar areas), are still somewhat confusing to me.

[Ma]

Originally Posted by asarian

Are these what you'd call 'best' x265 settings?

For low and medium bitrates -- yes (+ placebo or veryslow preset). If the source is anime I reduce '--psy-rd' and optionally '--aq-strength'.

For high bitrates I prefer 10-bit output + '--aq-mode 0'.

[asarian]

Originally Posted by Ma

For low and medium bitrates -- yes (+ placebo or veryslow preset). If the source is anime I reduce '--psy-rd' and optionally '--aq-strength'.

For high bitrates I prefer 10-bit output + '--aq-mode 0'.

Thanks. Interesting choice on '--aq-mode 0', btw; I had expected ppl would use --aq-mode 3 for best quality (although I never fully understood whether '--aq-mode 3' takes away from regular scenes towards darker ones, or whether it plain favors darker scenes).

[Ma]

The main idea about aq-mode is to more compress (worse quality) CU that are hard to compress and move the bitrate to CU that are easy to compress -- it degrades quality (a little bit) in complicated CU and boosts quality in not complicated CU. The quality degradation is percentage smaller than the boost (if you move 5 from 100 to 20 you will have 95 and 25 -- 5% less bitrate in one CU and 25% more bitrate in second CU).

First scene: launch of the rocket
aq-mode moves bitrate from fire and smoke to sky and rocket -- the scene looks better with aq-mode.

Second scene: two people talking on the bridge at blurry background
aq-mode moves bitrate from people to background (blurred leaves) -- the scene looks worse with aq-mode.

Whole movie (aq-mode 3 vs. aq-mode 0):
2.0M '--aq-mode 3' www.msystem.waw.pl/x265/tos_x265vs_2000.mkv PSNR: 44.081, SSIM: 16.075 dB
2.0M '--aq-mode 0' www.msystem.waw.pl/x265/tos_x265vs_2000aq0.mkv PSNR: 43.528, SSIM: 15.624 dB

For me both versions are watchable, the main difference is the noise -- in '--aq-mode 0' version is more live and annoying (aq-mode moves bitrate from noisy parts to clean parts of the scene). The noise is in source movie so it is OK.

For noise/details lover '--aq-mode 0' should be interesting for high bitrate encoding.

[asarian]

^^ Thanks, @Ma, for the great explanation, and the samples!

[TomV]

Here are my Beamr 5 encodes (the Google Drive includes copies of Ben's x265 UltraPlacebo encodes). I look forward to feedback.

[Blue_MiSfit]

The Beamr encodes look good - I spent some time this evening looking at the 1.5 Mbps encode. One area where I think x265 is still superior is in the scene with the robot legs around frame 1232 - the background has very distracting visual artifacts (mostly looking like mosquito noise) in the Beamr version that are not present in the x265 version.

I saw some similar issues in the scene around frame 4514, including a keyframe pop.

However, the tables turn in a dark scene around frame 3651 where x265 has similar artifacts in the background and on the sniper's face, but Beamr is quite good. Another example of this is in the (bright) scene around frame 7527 where Tom's face is more detailed and more stable in the Beamr version.

I need to spend more time reviewing these - but Beamr definitely makes a very good showing!

Thanks for doing this, Tom!

[TomV]

Originally Posted by Blue_MiSfit

The Beamr encodes look good - I spent some time this evening looking at the 1.5 Mbps encode. One area where I think x265 is still superior is in the scene with the robot legs around frame 1232 - the background has very distracting visual artifacts (mostly looking like mosquito noise) in the Beamr version that are not present in the x265 version.

I saw some similar issues in the scene around frame 4514, including a keyframe pop.

However, the tables turn in a dark scene around frame 3651 where x265 has similar artifacts in the background and on the sniper's face, but Beamr is quite good. Another example of this is in the (bright) scene around frame 7527 where Tom's face is more detailed and more stable in the Beamr version.

I need to spend more time reviewing these - but Beamr definitely makes a very good showing!

Thanks for doing this, Tom!

Thanks for the feedback Derek. These encodes were done with a pre-release (development) version of Beamr 5, v4.5. I'm in St. Petersburg Russia this week, working with our development team. We will take a look at your feedback and will produce a fresh set of encodes with the production release version, which is a few weeks away from being finished.

I can't find any robot legs or distracting artifacts around frame 1232... can you double-check the position of the issue you're talking about? I see the quality change in the wall behind the guy at frame 4519 (a keyframe), and some "live walls" behind the characters in the scene before that (frame 4136-4398).

By the way, the best way to compare video quality side by side is to use Beamr View, our frame-synchronized dual video player. I know Derek has a copy, but if any other video professionals need copies of Beamr View, ping me (tv at beamr dot com).

[Blue_MiSfit]

My bad - frame 11232

[Emulgator]

Beamr 4501: Impressive !
Until now I only looked at 4501 1Mbps, pixel peeping mode.
Remarkably kept/simulated texture, sometimes glued onto the underlying surface, but nicely done.
The expected artifacts at such low bitrate are unexpectedly small, and well shifted to domains where it doesn't hurt.
My favourite ToS warping artifact "robot arm moving before leaves" is very good suppressed,
as are all contour warping stuff that x265 still has, well, at these misery bitrates.
Nice research, never believed that would be possible !

[IgorC]

Originally Posted by TomV

Here are my Beamr 5 encodes (the Google Drive includes copies of Ben's x265 UltraPlacebo encodes). I look forward to feedback.

Beamr looks absolutely better than x265 ver 2.8. Not sure how x265 3.0 would perform.

Also, a relevant article here
https://www.streamingmedia.com/Artic...es-131923.aspx

[mandarinka]

Originally Posted by benwaggoner

I didn't do any content specific encoding in these. I just did the slowest, highest quality encode 2-pass I had the patience for. Basically

• --preset placebo
• --cu-lossless
• --tskip
• -F 1
• --ref 6
• --bframes 16
• --aq-mode 3
• --rd-refine

If I had infinite patience I'd add --me sea and --subme 7 for PlusUltraPlacebo.

Acording to x265 documentation, CU-lossless requires extra signaling so it adds some bitrate overhead whether it is useful or not. It is usually nigh impossible to check if it helps or not at higher bitrates, because due to ratecontrol variation, some frames will look worse, some better, and you will have no idea. But I tried to meassure SSIM at 10mbits and 18mbits with a bluray source (1440x1080p24) and despite this high bitrate, I got a SSIM drop from CU-lossless.

I observed SSIM dropping from 0.9872928 without cu-lossless to 0.9872879 with cu-lossless at 18mbits, and from 0.9820309 to 0.9820127 at 10mbits.

Are you sure using this tool on normal lossy encodes (as opposed on some extreme bitrate archival encodes or something, hybrid lossless?) really helps visually quality per unit of bitrate? Or perhaps it is different with clean uncompressed source but it harms on compressed sources?

[benwaggoner]

Originally Posted by mandarinka

Are you sure using this tool on normal lossy encodes (as opposed on some extreme bitrate archival encodes or something, hybrid lossless?) really helps visually quality per unit of bitrate? Or perhaps it is different with clean uncompressed source but it harms on compressed sources?

I am not sure if it does, no. It's more likely to help with zero-noise synthetic images, like the credits, but didn't do a lot of testing. Like I said, it was just an experiment to see what stuck after throwing at a wall.

I definitely want to try again with the new aq-mode and some other x265 improvements.

[mandarinka]

I see, thanks.

[Funky080900]

Hi there

here is my VVC encode using VTM Encoder Version 6.0. The original files are 37MB@403kbps. Encoding time ~ 1 week on i7-4720HQ.
I split the movie to encode in parallel and couldn't figure out how to merge the .bin files, so I recommend downloading the lossless h.264 video (5GB).
I don't know how representative my results are for VVC but since it took so long to encode I thought I'd share it anyway.

Files: https://drive.google.com/open?id=1Xs...fc_CZWXWtkLzJF

[benwaggoner]

Originally Posted by Funky080900

Hi there

here is my VVC encode using VTM Encoder Version 6.0. The original files are 37MB@403kbps. Encoding time ~ 1 week on i7-4720HQ.
I split the movie to encode in parallel and couldn't figure out how to merge the .bin files, so I recommend downloading the lossless h.264 video (5GB).
I don't know how representative my results are for VVC but since it took so long to encode I thought I'd share it anyway.

Files: https://drive.google.com/open?id=1Xs...fc_CZWXWtkLzJF

Awesome, thank you!

[Greenhorn]

So I'm not a VP9 expert by any stretch of the word, but I thought I'd take a stab at this as I'd been meaning to probe libvpx's options for a while anyway. Maybe someone'll find this useful.

https://mega.nz/#!RR9F0IYS!kAoSuKEHz...aO1eSa-0qgJ1rg

settings: --target-bitrate=2000 --buf-optimal-sz=2000 --kf-max-dist=120 --max-gf-interval=23 --tile-columns=0 --row-mt=1 --auto-alt-ref=6 --arnr-maxframes=15 --arnr-type=3 --frame-boost=1 --aq-mode=1. Overall encode speed was 1.56FPS (with ~20% utilization) on a Ryzen 7 3700X.
Average bitrate is 1998 kbps. I don't know how to measure the maximum local bitrate, sorry.

Notes/explanations/attempted justifications. Apologies in advance for the bad formatting and wordiness.
0) libvpx defaults to its highest-quality preset, and there doesn't seem to be a lot you can do to improve visuals sadly.
1) --threads appears to be the size of the worker pool, not the number of frame threads. I don't think there's any frame-level parallelism going on in the encoder, actually, as --frame-parallel enables/disables parallel decoding.
2) For a given --threads count, different --tile-columns counts will produce files that look more or less the same to my eyes and the basic ffmpeg metrics, but differ in slightly in filesize. I left --tile-rows at zero.
3) For a given --tile-columns count, altering --threads has the same effect.
4) Row-based multithreading (--row-mt) appears to be similar to WPP in HEVC; with tiles disabled, enabling it actually produced slightly smaller files with slightly higher visual metric scores, in addition to boosting encode speed from ~40FPM to ~1.5FPS.
5) Relative to the default settings (which appear to be --row-mt=0 --threads=8 --tile-rows=0 --tile-columns=6), --threads 1 --tile-columns 0 produced files that were on average 1% smaller for identical metric scores. (Caveat: this was done with several ~30second test clips extracted with FFMPEG.)
6) I really couldn't spot a visual difference between no-parallelism encodes and encodes with substantially more threads and tiles enabled than default, if I'm being honest.
7) There doesn't appear to be a strict analogue to --vbv-maxrate in libvpx, so I just set the "optimal buffer size" to 2000 milliseconds.
8) If I didn't set either a maximum or minimum quantizer value, bitrate would frequently rocket to absurdly high levels, then plummet down to equally absurd levels for an absurdly long amount of time. I believe this loose rate-control may be the cause of artifacting like you see in SmilingWolf's encodes from last year. There's still some in scenes like the sniper descending down his rope, but it does seem to be reigned in. Also: the overshoot-pct/undershoot-pct settings which you'd think might control or influence this appear to do precisely nothing.
9) Golden Frames and Alternate Reference Frames appear to be the same thing. By default they occur at an non-fixed interval of up to 16 frames (sort of a secondary GOP within the GOP . . .); I hope that increasing this to 23 (the maximum) was permissible, as it was one of the only tweaks I found that seemed to have a notable effect on the output files' visuals or metric scores.
10) I'm a bit fuzzy as to what it actually does in a technical sense, but setting auto-alt-ref to was the other really universally positive tweak I found.
11) AQ appears to be a little half-baked in libvpx (common theme . . .), but seemed to have an overall positive effect. Based on the code, aq-mode 1 is fairly similar to aq-mode 2 in the x26x encoders, while aq-mode 2 is similar to their aq-mode 1. AQ3 is silently disable if cpu-used is less than 5, and I really have no idea what AQ4 is trying to do but it didn't help anything for this clip.
12) None of the --tune-content settings had anything like a positive effect. "film" actually appears to be similar to "rc-grain" in x265, while "screen" just murdered quality.
13) The "--tune" option will always be set to "psnr"; setting it to "ssim" seemed like low-hanging fruit initially, but it appears that the actual code for the option was never ported from VP8 to VP9.
14) All of the "arnr-" settings had miniscule but actual effects on output. Setting arnr-type to three enables a bidirectional filter, and I assume that a larger sample size should be better.
15) --frame-boost seems like it might be silently ignored, but the intended effect (raise or lower QP of alt-ref frames based on 2-pass analysis) seemed good enough that I enabled it anyway.
16) "--sharpness" is actually an inverse to the strength of the loop filter (I think); reducing it increased filesize by about 10% and worsened artifacts.

I think the final encode is actually pretty OK, if not anything special.