HEVC killer sample (short movie)

[Leo 69]

Alright guys, here's one for your entertainment.

Despite our common belief that HEVC would be better (on average) than AVC whereas at the same quality the bitrate would be cut in half or at least 30-40%, there are samples, where H.265 is on par or even slightly inferior to H.264 (just a little bit, but still).

At https://media.xiph.org/ there is a 12 minute lossless 4k movie "Tears of Steel" weighing 67 Gb (compressed) and about 180 uncompressed.
https://media.xiph.org/tearsofsteel/tearsofsteel-4k.y4m.xz

So I downloaded and resized the movie to 1920x800, then saved it to lossless AVC.
Now it's 10.5 Gb and I uploaded it here:

https://drive.google.com/uc?id=0B_FlGgnVUBPgX2RMdnowRjAtekE&export=download

Here comes the most interesting part. I've been playing with this movie for several days and I cannot get HEVC better than AVC on it, whatever settings I threw on it.

I get it almost transparent at ~8900 kbit/s with HEVC and at the very same bitrate AVC looks the same or slightly better in scenes with high motion (please keep in mind that I assess the difference extremely scrupulously.. I don't advise to try finding it on a bad monitor or with frivolous attitude).

The difference is absolutely minuscule between the codecs (virtually none). You can download and play with the movie for yourself.

HEVC can't compress this movie better than AVC but is 4-5 times slower! Why so?

I would understand if this was a very grainy source, but it's not. This is a very high-quality, naturally sharp source with a bit of film noise.

I used only CRF for my tests. It would be interesting if others can also look into it and provide their opinion/conclusions.
Especially it would be cool to get some feedback from the development team.

Thanks

[x265_Project]

Alright guys, here's one for your entertainment.

Despite our common belief that HEVC would be better (on average) than AVC whereas at the same quality the bitrate would be cut in half or at least 30-40%, there are samples, where H.265 is on par or even slightly inferior to H.264 (just a little bit, but still).

At https://media.xiph.org/ there is a 12 minute lossless 4k movie "Tears of Steel" weighing 67 Gb (compressed) and about 180 uncompressed.
https://media.xiph.org/tearsofsteel/tearsofsteel-4k.y4m.xz

So I downloaded and resized the movie to 1920x800, then saved it to lossless AVC.
Now it's 10.5 Gb and I uploaded it here:

https://drive.google.com/uc?id=0B_FlGgnVUBPgX2RMdnowRjAtekE&export=download

Here comes the most interesting part. I've been playing with this movie for several days and I cannot get HEVC better than AVC on it, whatever settings I threw on it.

I get it almost transparent at ~8900 kbit/s with HEVC and at the very same bitrate AVC looks the same or slightly better in scenes with high motion (please keep in mind that I assess the difference extremely scrupulously.. I don't advise to try finding it on a bad monitor or with frivolous attitude).

The difference is absolutely minuscule between the codecs (virtually none). You can download and play with the movie for yourself.

HEVC can't compress this movie better than AVC but is 4-5 times slower! Why so?

I would understand if this was a very grainy source, but it's not. This is a very high-quality, naturally sharp source with a bit of film noise.

I used only CRF for my tests. It would be interesting if others can also look into it and provide their opinion/conclusions.
Especially it would be cool to get some feedback from the development team.

Thanks

If you throw enough bits at the problem, any video codec will be visually lossless. It will be extremely hard to notice any difference between the source and the encode at 8.9 Mbps (in other words, as you described, at this bit rate the encode is visually lossless to any normal person under normal circumstances). Try encoding with x264 and x265 at 400 kbps, and then try some encodes at increasingly higher bit rates. Keep in mind that to compare video, you need to watch video - not compare still frames. We have a special video player (UHDcode Pro Player) that we use to do these comparisons side by side, or over/under (playing both A and B videos in sync, while hiding or revealing more or less of A which is overlaid over B).

[HWK]

To further add to it, when there enough bits any encoder can do good job. Try with 500 to 2000 ABR ranage and see if you opinion changes. Basically with low bitrates difference should be visible.

[Leo 69]

x265_Project

Thanks for your reply! I agree with you on the absolute most part. My point, which I'm trying to get across is this:

There is something specific about this movie, that renders HEVC practically useless, since it doesn't offer any advantages but only disadvantages (in comparison to AVC) -- slow encoding speed (4-5 times slower), film grain in most cases looks less than original one or simply disappears in one scene (this one is very high-frequency...not easy to spot!), overall feeling that H.264-produced picture is more close to the original.

Compare the information I'm posting to that being posted at hydrogenaud.io, where audio-enthusiasts hold listening tests of different lossy codecs at various bitrates, certainly including those which suppose to bring transparency compared to the original WAV or FLAC. They seek for the artifacts to point them out in ABX test -- this gives some valuable food for thought to the developers, who continue to fine-tune and release new versions of the codecs.

While what you're saying is basically true => "at this bit rate the encode is visually lossless to any normal person under normal circumstances", it contradicts with the general idea that we may find here:
http://x265.org/hevc-h265/

"HEVC was developed with the goal of providing twice the compression efficiency of the previous standard, H.264 / AVC."

Which is not the case here. Why? If I was a developer, I would at least be intrigued as to why certain movie cannot be compressed more efficiently than with the former codec generation.

P.S. To most people with low-end or average audio equipment, so as to those who doesn't really care, 128 kbps MP3 is transparent. However, those, who strive for perfection, they point out the flaws of the lossy codec, proving that in the ABX test.
Hopefully, I got my point across.

[Leo 69]

To further add to it, when there enough bits any encoder can do good job. Try with 500 to 2000 ABR ranage and see if you opinion changes. Basically with low bitrates difference should be visible.

I'm talking about compressing the lossless source to the point where there is visual transparency. Your case is not in discussion.

[zub35]

Here is another example of such a sample, it is a slicing of three videos of 60 frames at a frame rate of 30.
Https://media.xiph.org/video/derf/
crowd_run + park_joy + in_to_tree (2160p)

ConvertToRGB ()
ResizeShader (1920,1080, "SSim")
ConvertToYV12 (chromaresample = "spline64")

raw_1080p30.y4m [534МБ] https://yadi.sk/d/O1bs1FTD3FPpgD

SSIM metric on the quality close to lossless is always in favor of the x265, but the picture is visually blurry.
To take into account this fact, I use MSU Blurring metric. First I measure the value of the original, taking it for 100%. After the value of the results of the encoding, and take a deviation from the original.

( SSIM(db) + MSU_Blurring(db*) ) / 2
* log(1-(encode_blur / source_blur))×(-10)

if encode_blur < source_blur then (encode_blur / source_blur) - [be blurred image]
if encode_blur > source_blur then (source_blur / encode_blur) - [be sharp image]
if encode_blur = source_blur then 0.9999..9

SSIM http://s009.radikal.ru/i310/1609/10/5f75086b2ae9.png
MSU Blurring http://s41.radikal.ru/i094/1609/4d/8b08d4ecb767.png
SSIM + MSU Blurring http://s017.radikal.ru/i402/1609/a3/32e7964dfc77.png

For example, if you select the bitrate of codecs, so that the value "SSIM + MSU Blurring" coincides, then the screenshots of the quality will also roughly coincide.

[CruNcher]

Please keep in mind you only asses 1 specific HEVC implementation in form of the X265 Encoder and Psy R&D though there are quiet a lot more ;)

You could and should also rate the result in Perceptual Scene stability

"An improved deblocking fillter"

Inloop filtering is always problematic and SAO is something really questionable hence metrics like it to much ;)

"HEVC was developed with the goal of providing twice the compression efficiency of the previous standard, H.264 / AVC."

This is very theoretical for complex grainy sources

Film Grain will never be really respectable without high bitrate overhead in motion the really first acceptable approach to it was Thomsons FGM/FGT, which though never really took off with it's database Modeling approach.

https://hopa.memberclicks.net/assets/documents/thomson_fgt_hpa2006.PDF


Though you should also understand that Film Grain as such will slowly anyways fade away in most content only very Hardcore Old Hollywood R&D folks still try to preserve it but it is already fading away grain as such is mostly artificially added already ;)

In case of Tears Of Steel it is extremely necessary to hide the artificial CGI results better, this is always the Perceptual Goal behind it in such content like it "Not let it look Artificial/Plastic" ;)

Though CGI becomes so extremely realistic that sooner or later this wont be necessary as well anymore ;)

[albt]

x265_Project

Thanks for your reply! I agree with you on the absolute most part. My point, which I'm trying to get across is this:

There is something specific about this movie, that renders HEVC practically useless, since it doesn't offer any advantages but only disadvantages (in comparison to AVC) -- slow encoding speed (4-5 times slower), film grain in most cases looks less than original one or simply disappears in one scene (this one is very high-frequency...not easy to spot!), overall feeling that H.264-produced picture is more close to the original.

Compare the information I'm posting to that being posted at hydrogenaud.io, where audio-enthusiasts hold listening tests of different lossy codecs at various bitrates, certainly including those which suppose to bring transparency compared to the original WAV or FLAC. They seek for the artifacts to point them out in ABX test -- this gives some valuable food for thought to the developers, who continue to fine-tune and release new versions of the codecs.

While what you're saying is basically true => "at this bit rate the encode is visually lossless to any normal person under normal circumstances", it contradicts with the general idea that we may find here:
http://x265.org/hevc-h265/

"HEVC was developed with the goal of providing twice the compression efficiency of the previous standard, H.264 / AVC."

Which is not the case here. Why? If I was a developer, I would at least be intrigued as to why certain movie cannot be compressed more efficiently than with the former codec generation.

P.S. To most people with low-end or average audio equipment, so as to those who doesn't really care, 128 kbps MP3 is transparent. However, those, who strive for perfection, they point out the flaws of the lossy codec, proving that in the ABX test.
Hopefully, I got my point across.

i also do think that hevc in their page, and also the media have made it look better than what it is, i also don't get why uhd discs have 50% less bitrate than what it would be with x264, from the tests i made it doesn't save 50% of file size.

i also think that word "efficient" shouldn't be used if you have to encode with 5X more time to save 25-30% of file size. it's not efficient, when it needs a 5X better & more expensive cpu/device for both encoding & playing.

[x265_Project]

x265_Project

Thanks for your reply! I agree with you on the absolute most part. My point, which I'm trying to get across is this:

There is something specific about this movie, that renders HEVC practically useless, since it doesn't offer any advantages but only disadvantages (in comparison to AVC) -- slow encoding speed (4-5 times slower), film grain in most cases looks less than original one or simply disappears in one scene (this one is very high-frequency...not easy to spot!), overall feeling that H.264-produced picture is more close to the original.

Compare the information I'm posting to that being posted at hydrogenaud.io, where audio-enthusiasts hold listening tests of different lossy codecs at various bitrates, certainly including those which suppose to bring transparency compared to the original WAV or FLAC. They seek for the artifacts to point them out in ABX test -- this gives some valuable food for thought to the developers, who continue to fine-tune and release new versions of the codecs.

While what you're saying is basically true => "at this bit rate the encode is visually lossless to any normal person under normal circumstances", it contradicts with the general idea that we may find here:
http://x265.org/hevc-h265/

"HEVC was developed with the goal of providing twice the compression efficiency of the previous standard, H.264 / AVC."

Which is not the case here. Why? If I was a developer, I would at least be intrigued as to why certain movie cannot be compressed more efficiently than with the former codec generation.

P.S. To most people with low-end or average audio equipment, so as to those who doesn't really care, 128 kbps MP3 is transparent. However, those, who strive for perfection, they point out the flaws of the lossy codec, proving that in the ABX test.
Hopefully, I got my point across.

The 2x efficiency goal of HEVC was for typical consumer video distribution bit rates. If you look at any rate-distortion curve (a plot of bit rate versus objective or subjective quality) comparing H.265 to H.264, you will see that as quality levels approach lossless, the bit rate savings of H.265 diminishes, as the quality difference diminishes and ultimately vanishes. So, we don't expect a full 50% reduction in bit rate at "visually lossless" levels of quality.

PS - You're right... if anyone believes that 128 kbps MP3 is audibly lossless, they need better audio gear, or better ears.

[nevcairiel]

i also think that word "efficient" shouldn't be used if you have to encode with 5X more time to save 25-30% of file size. it's not efficient, when it needs a 5X better & more expensive cpu/device for both encoding & playing.

It is space-efficient, which is the only thing that matters if you are constrained in bandwidth - which is the main concern we have these days on all distribution channels from web streaming, downloads, broadcasts and even optical discs.
Decoding is often done by hardware decoders, and encoding is a one-time job per stream, making a longer encode only a small problem.

You will not get any realistic size improvements without working harder for it.

[CruNcher]

Also the strength becomes really first visible if we talk about Spatial resolutions of above 1080p then HEVC becomes very interesting, all ready with it's much better internal banding handling results alone without even any 10 bit processing at all :)

This is one of the biggest visual hit you can see immediately no matter really which encoder you use when you have the right source @ hand ;)

It also showcases very well in the GPU (Asic) cores if they make efficient use of it.

What really becomes interesting though is trying to avoid most of the complexity decoding side wise but still maintaining most of the visual benefits without moving into above 4 core land and here also WPP becomes beneficial ;)

[HWK]

Also the strength becomes really first visible if we talk about Spatial resolutions of above 1080p then HEVC becomes very interesting, all ready with it's much better internal banding handling results alone without even any 10 bit processing at all :)


Personally I find comparing hevc on 1080 or lower sources is not ideal with AVC, since there will be little or no difference, unless bitrate is quite low. With that being said I did 6K source and thing gets quite interesting with regards to compression.

Next target 8K source.

[Jamaika]

In my opinion, quality HEVC and AVC movies can't be compared.
By setting a specific bitrate of 6000kbps, it turns out that for HEVC the bitrate is 4700, AVC has 8500 and VP9 12000.
My test: https://www.sendspace.com/filegroup/8whWtLvHnSTFZnKRa3RUa6g1XJ2REl2K
For vp9 codec I used aq-mode four panoramic. I don't know what it means and whether it is recommended.

[Leo 69]

In my opinion, quality HEVC and AVC movies can't be compared.
By setting a specific bitrate of 6000kbps, it turns out that for HEVC the bitrate is 4700, AVC has 8500 and VP9 12000.
My test: https://www.sendspace.com/filegroup/8whWtLvHnSTFZnKRa3RUa6g1XJ2REl2K
For vp9 codec I used aq-mode four panoramic. I don't know what it means and whether it is recommended.

I have absolutely no idea what was in your head while you were doing your comparison :)

Download the 1920x800 full version and try compressing it with HEVC and AVC with your favorite settings you're usually using to reach transparency. Then see if you get any reduction in bitrate with HEVC.

https://drive.google.com/uc?id=0B_FlGgnVUBPgX2RMdnowRjAtekE&export=download

[Leo 69]

Personally I find comparing hevc on 1080 or lower sources is not ideal with AVC, since there will be little or no difference, unless bitrate is quite low.

Wrong. I have encoded so many blu rays into HEVC and I know for sure that I'm going to get _at_least_ 30% reduction in bitrate compared to AVC with pretty much same quality. Usually it's about 40%.

It looks like many people don't understand the purpose of this thread. Also, overlooked the post by zub35.

Let me reiterate. The movies me and zub35 posted have specifics, where HEVC does not have advantage over AVC at all. Not even 20% reduction in bitrate, while keeping visual transparency.

However, the encoding process is 4-5 times slower.

Why?

[Jamaika]

Generally speaking. I don't know how to properly test animations. Apart from this fact. I don't know mean my "favorite settings". The veryslow preset isn't the same veryslow in x264. I can only state my setting and the opinion that x264 is no more than x265.
x265-10b.exe --y4m --input-csp i422 --input-depth 10 --output-depth 10 --preset medium --fps 29.970 --keyint 60 --info --no-open-gop --no-hrd --bitrate 6000 --colormatrix bt2020nc --range full
--output "yuv422p10le_bt2020nc_rangepc.h265" -
x264-10bit.exe --demuxer y4m --input-csp i422 --input-depth 10 --input-range pc --output-csp i422 --threads 4 --tune stillimage --preset veryslow --bitrate 6000 --fps 29.970 --keyint 60 --nal-hrd none
--colormatrix bt2020nc --range pc --output "yuv422p10le(x264)_bt2020nc_rengepc.h264" -
vpxenc.exe --bit-depth=10 --input-bit-depth=10 --i422 --codec=vp9 --best --threads=4 --cpu-used=0 --profile=3 --drop-frame=100 --end-usage=cbr --target-bitrate=6000 --kf-max-dist=60 --auto-alt-ref=1
--frame-boost=1 --aq-mode=4 --color-space=bt2020 --verbose --pass=1 --passes=1 --output=yuv422p10le_bt2020.webm -
I will add that the files should have the same size. Unfortunately they haven't.

[sneaker_ger]

Single-pass --bitrate isn't accurate enough on such short snippets. You need to use multi-pass or encode much longer samples.

[HWK]

Wrong. I have encoded so many blu rays into HEVC and I know for sure that I'm going to get _at_least_ 30% reduction in bitrate compared to AVC with pretty much same quality. Usually it's about 40%.

It looks like many people don't understand the purpose of this thread. Also, overlooked the post by zub35.

Let me reiterate. The movies me and zub35 posted have specifics, where HEVC does not have advantage over AVC at all. Not even 20% reduction in bitrate, while keeping visual transparency.

However, the encoding process is 4-5 times slower.

Why?


Oh okay, you have two issues going at same time.

1. bitrate reduction is not being achieved, even though HEVC standard says it will, while maintain quality.
2. Encode taking much longer for same source, but still no dice with regards to reduction.

I need to better study your output and zub35, I guess this is what happens when you half awake pass midnight :D

By any chance do you have setting which you used when encoding with x265, more so any specific setting used.

[Leo 69]

Oh okay, you have two issues going at same time.

1. bitrate reduction is not being achieved, even though HEVC standard says it will, while maintain quality.
2. Encode taking much longer for same source, but still no dice with regards to reduction.

I need to better study your output and zub35, I guess this is what happens when you half awake pass midnight :D

By any chance do you have setting which you used when encoding with x265, more so any specific setting used.

Yes.

AVC

x264.exe --preset placebo --tune film --crf 18 --ref 4 --no-mbtree --bframes 8 --deblock -4:-4 --subme 11 --psy-rd 1:0.05 --me umh --merange 32
------------------------------------
x264 [info]: frame I:145 Avg QP:16.96 size:217384
x264 [info]: frame P:3799 Avg QP:19.15 size: 88343
x264 [info]: frame B:13676 Avg QP:21.54 size: 32445
x264 [info]: consecutive B-frames: 2.5% 3.3% 5.1% 15.4% 30.2% 27.8% 8.6% 2.0% 5.1%
encoded 17620 frames, 5.16 fps, 8835.60 kb/s

HEVC

x265.exe --crf 19.5 --preset placebo --ctu 32 --limit-tu 3 --tu-inter-depth 3 --no-rect --no-amp --no-b-intra --qg-size 8 --qcomp 0.65 --cbqpoffs -3 --crqpoffs -3 --no-cutree --subme 4 --merange 57 --max-merge 3 --rc-lookahead 50 --no-strong-intra-smoothing --deblock -1:-1 --psy-rd 3.5 --psy-rdoq 8 --no-sao --no-rskip
------------------------------------
x265 [info]: frame I: 144, Avg QP:17.77 kb/s: 31575.81
x265 [info]: frame P: 3471, Avg QP:19.74 kb/s: 16834.43
x265 [info]: frame B: 14005, Avg QP:22.31 kb/s: 6578.72
x265 [info]: consecutive B-frames: 9.4% 7.5% 5.4% 22.0% 19.3% 15.0% 7.2% 5.6% 8.5%
encoded 17620 frames,1.60 fps, 8803.30 kb/s

http://screenshotcomparison.com/comparison/205248

HEVC smooths details here and there quite apparently -- especially where the flame bursts out of the nozzles.

Therefore, in this case AVC is somehow BETTER and FASTER than HEVC at the same bitrate.

[Leo 69]

Any comments?

[CruNcher]

1 spatial frame result tells nothing rely

But overall imho the wn or not win from this specific sample isn't worth the complexity increase perceptually, though you need to see it in it's motion result over time.

On the first look it looks x265 is to complex for the target @ 8 mbit gaining nothing except a complexity increase on the Encoding side which doesn't rely becomes perceptual noticeable at this bitrate.

But without seeing it longer time more scenes and the overall stability of those in motion it's really a judgement based on just 1 frame output result in your case.

And most of the complexity and it's advantages at a certain point become visible in Motion if you don't use very specific source which triggers a harder time for AVC at those levels where both perceptually overall meet each other.


Now do the same at 4 Mbit and see what you get that's the real promise that MPEG makes you get a better perceptual overall result saving money @ your customers end improving your costs (putting computation vs bandwith per frame) at least as good as the 8 mbit AVC result ;)

That these higher complexity vanishes at lower spatial resolution perceptually at some point every RD compare so far pretty much showed that.

Tears of Steel as source can't show every benefit of HEVC perceptually though, you need a broader range to judge it's overall Performance fairly.

What you do is cherry picking currently and this based on 1 HEVC implementation ;)


And with Tears of Steel results are mostly based on a Artficial Noise layer Compression tries to fight against.

You always can ask yourself where does a "Meaningfulll improvement" actually begins that as well can be very subjective for everyone not only Metrics evaluation ;)

[benwaggoner]

Any comments?

--crf 19.5 --preset placebo --ctu 32 --limit-tu 3 --tu-inter-depth 3 --no-rect --no-amp --no-b-intra --qg-size 8 --qcomp 0.65 --cbqpoffs -3 --crqpoffs -3 --no-cutree --subme 4 --merange 57 --max-merge 3 --rc-lookahead 50 --no-strong-intra-smoothing --deblock -1:-1 --psy-rd 3.5 --psy-rdoq 8 --no-sao --no-rskip

I'd suggest using
--limit-tu 4 instead (faster, better)
Sticking with placebo's --tu-inter-depth 4
Keep --rect and --amp
Keep b-intra
qg-size of 8 is probably worse than 16 for these frame sizes
Keep placebo's --subme 5
Your psy-rd and psy-rdoq values seem really high, and tuned more for apparent detail than preservation of detail.
Rskip is generally safe to use now
Try --tskip (with --tskip-fast if it's too slow)
And, maybe experiment with --cu-lossless, which might be useful at "perceptually transparent." But it is slow.
Why reduce --max-merge from the placebo default of 5 to 3?

If perf gets too bad with those changes, I'd instead add
--limit-modes (makes using --amp and --rect a lot less painful)
--limit-refs (1 is fine, could even do 4)
--rskip
And maybe --bframes 16? You were still getting 8.5% B-frames at 8, so there may be more to squeeze out there

You might start with just --preset placebo --tune grain and see how that looks. Or even --preset veryslow without all those features turned off.

Also, are you using a recent build with the new 8-bit lambda table? That can help quite a lot.

[Dope]

@benwaggoner

Thanks for the tips. Yeah, it's still me (Leo 69), I changed my nickname, which suits me better now :D

Absolutely, I used the latest build with new lambda table @8 bit.

The thing is, if I add these options:

"Keep --rect and --amp
Keep b-intra"

I will get something like 0.5 FPS, I know that. And in the end it will still don't do much in terms of compression. I can understand that those options are more suited for 4k content and upwards, but according to my tests they only vaguely influence compressibility and visual quality at 1080p, yet make the encoding process unbearably slow.

As for rskip - that's great that it doesn't have such an impact on quality I thought before, yet it does provide a solid speed boost. I'll certainly try that in my future encodes.

Many thanks.

[turab]

@CruNcher: The point of this thread isn't to assess x265 in general. OP already said it beats x264 in many examples. The point is to look at examples where it doesn't do well.

Looking at the screenshot comparison, x265 seems to struggle with fine-detailed textures. If you look at the bottom-left corner of the frame where there isn't much going on, I would say HEVC does a slightly better job actually, but I don't have the original source to compare. Anyway, have you tried increasing qcomp? In my experience, it can help with fine details, though I'm not an expert on settings.

[benwaggoner]

@benwaggonerThe thing is, if I add these options:

"Keep --rect and --amp
Keep b-intra"

I will get something like 0.5 FPS, I know that. And in the end it will still don't do much in terms of compression. I can understand that those options are more suited for 4k content and upwards, but according to my tests they only vaguely influence compressibility and visual quality at 1080p, yet make the encoding process unbearably slow.
That's why you want to use --limit-modes, which enables a early exit modes for --amp and --rect, and really reduces the speed hit while preserving almost all the quality gain.

I don't know that amp and rect are particularly tied to frame size, although other options are.

As for rskip - that's great that it doesn't have such an impact on quality I thought before, yet it does provide a solid speed boost. I'll certainly try that in my future encodes.
It used to be bad. But then it got good :).