Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > High Efficiency Video Coding (HEVC)

Reply
 
Thread Tools Search this Thread Display Modes
Old 10th October 2019, 17:52   #7101  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Hollola, Finland
Posts: 4,621
Have you compared the metadata between the two encodes?
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 10th October 2019, 20:32   #7102  |  Link
Barough
Registered User
 
Barough's Avatar
 
Join Date: Feb 2007
Location: Sweden
Posts: 337
x265 v3.2+6-f46aa2bc1c341 (32 & 64-bit 8/10/12bit Multilib Windows Binaries) (GCC 9.2.0)
Code:
https://bitbucket.org/multicoreware/x265/commits/branch/default
Barough is offline   Reply With Quote
Old 10th October 2019, 22:42   #7103  |  Link
MeteorRain
結城有紀
 
Join Date: Dec 2003
Location: NJ; OR; Shanghai
Posts: 617
Quote:
Originally Posted by excellentswordfight View Post
And tbh, I dont think x265 is better parallelized then x264, at default settings its actually worse for resolutions under 4k cause of the large CU size. And both x264 and x265 have a hard time to scale beyond 24'ish threads for 1080p at slower settings.

4770k have AVX2 to, if I'm not mistaken it was introduced with haswell. I own an 4790k and have some experience with 9900, I would say that there is about an 2,5x performance difference for x265.
x264 used to suffer from high thread count causing reduction of encoding quality. Later the threading is greatly improved and people are no longer limited to the (more optimized option of) 8 - 12 threads and can go beyond without much loss of quality.

Haswell is the first generation that introduced AVX2. However the performance of AVX2 is not consistent across multiple generations. The manufacturer is constantly improving instruction speed, and AVX2 on Haswell is slower than AVX2 on Skylake or later generations by some percent. That's why I said the performance difference on AVX2 needs to be taken into consideration.
MeteorRain is offline   Reply With Quote
Old 10th October 2019, 23:02   #7104  |  Link
MeteorRain
結城有紀
 
Join Date: Dec 2003
Location: NJ; OR; Shanghai
Posts: 617
Quote:
Originally Posted by aymanalz View Post
But HEVC is no longer for the future, is it? Successors to HEVC are in development, so HEVC is the present.

The part about how processors haven't improved much is absolutely right, and I had that in mind as well. From the year 2000 to 2010, processors (low end, mid grade, high end, everything) became several times faster, and sold for the same or even lower prices, probably due to the Intel-AMD competition. But incremental improvements have been a lot less in this decade, especially after Sandybridge or Haswell.

I could blame the CPU manufacturers, or blame the x265 developers for not anticipating that processing power per price will not keep increasing at the rate it used to, but I'm not really trying to assign blame; just making an observation that x265 is extremely slow on "normal" processors - by which I meant a reasonable home desktop without a gazillion cores and threads. (I'd say a quad core i7 or hexacore is the mainstream now.)

I wasn't assuming that code can be optimized further - I was wondering out loud whether it could. I was lamenting that perhaps they have reached a point where further optimizations for speed just isn't possible - in which case, only professional encoders or studios with 16+ core machines can use it at decent speeds. Not the casual home users.
From different point of view, you can say HEVC is the present, you can say it's not. For example, online streaming still uses AVC (some like Youtube may be using VP9 but still), we still buy Blu-rays instead of UHDs, majority of the content distributors are still using AVC. The use of HEVC would still be quite limited so far, even if assuming HEVC can be encoded at a faster speed. I admit there are other factors like royalty, but still.

For casual home users you can still use x265, just with a lower settings. On one of a workstation that I have, with only 6 cores Sandy Bridge, I can still get 4-5 FPS on 1080P with a reasonable settings (10-bit, medium preset, LP tuning). Today a $199 6 cores AMD is already twice the speed of that SNB, so 10 FPS is what you get if you max it out. I personally feel like this is an acceptable speed for casual home users. If someone needs to do extensive HEVC encoding at a decent speed, a higher performance CPU is inevitable. (Ryzen 3900X is only $499 which is still kinda in the affordable range.)

And I apologize for my incorrect assumption. Thanks for your input.

Last edited by MeteorRain; 10th October 2019 at 23:05.
MeteorRain is offline   Reply With Quote
Old 11th October 2019, 00:19   #7105  |  Link
RanmaCanada
Registered User
 
Join Date: May 2009
Posts: 105
And we already know that the current highest end Epyc 7742 (64 cores) can do 8k encoding in faster than real time (79fps). Now imagine what a 32 core Zen2 Threadripper will be able to do. Heck I wish someone here who has a 3900x would run Sagitare's benchmark and post their results. We haven't had any of the new Zen 2 chips benched. Though I think he would need to update all the binaries to reflect the changes that have been made in x265 as the benchmark is 2 years old.
RanmaCanada is offline   Reply With Quote
Old 11th October 2019, 00:38   #7106  |  Link
Natty
Noob
 
Join Date: Mar 2017
Posts: 219
Quote:
Originally Posted by Rousseau View Post
On UHD rips with 3.2 , the image looks darker when played in MPV than with rips made in 3.1 . They look the same in MPC with no tone mapping. I made no change other than the encoder.
i have observed the same issue with 3.2, but with HD. somehow managed to not make it darker or lighter through avisynth.
Natty is offline   Reply With Quote
Old 11th October 2019, 03:56   #7107  |  Link
Rousseau
Registered User
 
Join Date: Jun 2017
Posts: 3
Quote:
Originally Posted by Boulder View Post
Have you compared the metadata between the two encodes?
I'm getting some contradictory results now so this may just be some mpv wonkiness. I did notice that 3.1 stable (3.1.2+4-dc2dcb5) concatenates the cll settings with the master-display settings which might be an issue. Other versions have them separate.

3.1.2+4-dc2dcb5
master-display=G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(10000000,1)cll=0,0

3.2+5-354901970679
master-display=G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(10000000,1) / cll=0,0



I've also noticed AQ2 seems to have problems in HDR encodes (3.1 & 3.2 builds I've tested), leaving a lot of blotchy grain/puddling in flat areas of some scenes. AQ1 does much better. It's not that visible when you look at it without tone mapping, but in MPV it's very obvious. I tried AQ3 & 4 with no improvement.
Rousseau is offline   Reply With Quote
Old 11th October 2019, 07:08   #7108  |  Link
NikosD
Registered User
 
Join Date: Aug 2010
Location: Athens, Greece
Posts: 2,704
Oh guys, please.

How many times do I have to post it ?

x265 is not optimized for speed/multi-threading and AMD CPUs.

64C/128T EPYC processor managed to encode faster than real-time 8K H.265 stream, using Beamr H.265 encoding software, which is optimized for speed/multi-threading and EPYC.

H.265 is different than x265.

https://www.tomshardware.com/news/am...ing,40400.html
__________________
Win 10 x64 (18363.476) - Core i3-9100F - nVidia 1660 (436.15)
HEVC decoding benchmarks
H.264 DXVA Benchmarks for all
NikosD is offline   Reply With Quote
Old 11th October 2019, 07:52   #7109  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Hollola, Finland
Posts: 4,621
Quote:
Originally Posted by Rousseau View Post
I'm getting some contradictory results now so this may just be some mpv wonkiness. I did notice that 3.1 stable (3.1.2+4-dc2dcb5) concatenates the cll settings with the master-display settings which might be an issue. Other versions have them separate.

3.1.2+4-dc2dcb5
master-display=G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(10000000,1)cll=0,0

3.2+5-354901970679
master-display=G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(10000000,1) / cll=0,0



I've also noticed AQ2 seems to have problems in HDR encodes (3.1 & 3.2 builds I've tested), leaving a lot of blotchy grain/puddling in flat areas of some scenes. AQ1 does much better. It's not that visible when you look at it without tone mapping, but in MPV it's very obvious. I tried AQ3 & 4 with no improvement.
Did you try a higher aq-strength for mode 2? I think some people reported that HDR encoding requires that.
EDIT: I forgot that I was "there" as well : https://forum.doom9.org/showthread.php?t=175631 . I've not done any HDR encodes in a long time but I would probably go for aq-mode 1 at the default strength.

I think you should report an issue for both of these cases at https://bitbucket.org/multicoreware/x265/issues.

I have a feeling that HDR encoding is not optimal (and also opened an issue regarding this without a response), pointing to how much you need to change aq-strength and CRF compared to SDR encodes.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...

Last edited by Boulder; 11th October 2019 at 08:02.
Boulder is offline   Reply With Quote
Old 11th October 2019, 09:43   #7110  |  Link
excellentswordfight
Lost my old account :(
 
Join Date: Jul 2017
Posts: 115
Quote:
Originally Posted by NikosD View Post
x265 is not optimized for speed/multi-threading and AMD CPUs.
What are the basis for that statement? The lower clocked 3700X is on par or faster then 9900k at the same amount of threads. First and second gen wasnt that strong per thread, but that is much cause of avx speed and other cpu-design causes. And I have recieved several Epyc 7402p servers, no issue there either, blazing fast.

Last edited by excellentswordfight; 11th October 2019 at 09:52.
excellentswordfight is offline   Reply With Quote
Old 11th October 2019, 13:43   #7111  |  Link
redbtn
Registered User
 
redbtn's Avatar
 
Join Date: Jan 2019
Location: Russia
Posts: 87
Quote:
Originally Posted by Rousseau View Post

I've also noticed AQ2 seems to have problems in HDR encodes (3.1 & 3.2 builds I've tested), leaving a lot of blotchy grain/puddling in flat areas of some scenes. AQ1 does much better. It's not that visible when you look at it without tone mapping, but in MPV it's very obvious. I tried AQ3 & 4 with no improvement.
I always use AQ2 for HDR encodes, and I never thought about it, cuz somewhere here I read that AQ2 is optimal for HDR. But now, it seems like I should make some tests.
How much bigger bitrate you get with AQ1 vs AQ2?
redbtn is online now   Reply With Quote
Old 11th October 2019, 15:06   #7112  |  Link
mandarinka
Registered User
 
mandarinka's Avatar
 
Join Date: Jan 2007
Posts: 734
Quote:
Originally Posted by excellentswordfight View Post
What are the basis for that statement? The lower clocked 3700X is on par or faster then 9900k at the same amount of threads. First and second gen wasnt that strong per thread, but that is much cause of avx speed and other cpu-design causes. And I have recieved several Epyc 7402p servers, no issue there either, blazing fast.
I think what he meant is that results with Beamr can't be taken as something indicating how fast will x265 run.

Last edited by mandarinka; 11th October 2019 at 15:14.
mandarinka is offline   Reply With Quote
Old 12th October 2019, 10:49   #7113  |  Link
mini-moose
Registered User
 
Join Date: Oct 2007
Posts: 385
Quote:
Originally Posted by redbtn View Post
How much bigger bitrate you get with AQ1 vs AQ2?
I think that's because in AQ1 the aq-strength for all frames is the same, while AQ2 uses auto-variance, and x265 applies separate aq-strength for each frame (based on complexity).

At least on paper AQ2 sounds better to me and it is now the default.

One would hope the devs switched to that for a good reason (i.e it's not just slightly faster but also gives a better result).

Last edited by mini-moose; 12th October 2019 at 10:51.
mini-moose is offline   Reply With Quote
Old 12th October 2019, 11:52   #7114  |  Link
rco133
Registered User
 
Join Date: Mar 2006
Posts: 14
There seems to be a lot of discussion about AQ mode 1 vs 2.

When it comes to 4k HDR encodes what I found is that with mode 2 the Average QP result is in average 1 higher in mode 2.

For example mode 1 is 21.33 and mode 2 is 22.15.

I have measured the vmaf values of quite a few 4k HDR encode cuts, and not in one single instance has the vmaf score been worse on the aq mode 2 encode. Every single time the vmaf value is higher on the aq mode 2 encode.

Yes, the file size is smaller, and the average QP a bit higher.

The switch from aq mode 1 to aq mode 2 must have been made for a reason. If mode 2 gives worse quality than mode 1, why would the devs change the default.

Either the devs knows nothing about qhat they are doing, or the actually know what they are doing.

All I can say is that I am yet to find a mode 1 encode with a higher vmaf score than the identical mode 2 encode.

rco133
rco133 is offline   Reply With Quote
Old 13th October 2019, 12:09   #7115  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Hollola, Finland
Posts: 4,621
But have you made any visual comparisons? In my old tests, I found out that mode 2 causes problems in flat backgrounds.

Many of the changes in the default values have been made without any explanation, so we don't really know what the use case for each of them has been. I sometimes feel that many settings are based on squeezing everything out of the material with the least possible amount of bits, even if it means oversmoothing etc. which can then be irritating to others (like me).
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 13th October 2019, 13:52   #7116  |  Link
redbtn
Registered User
 
redbtn's Avatar
 
Join Date: Jan 2019
Location: Russia
Posts: 87
Quote:
Originally Posted by Boulder View Post
But have you made any visual comparisons? In my old tests, I found out that mode 2 causes problems in flat backgrounds.

Many of the changes in the default values have been made without any explanation, so we don't really know what the use case for each of them has been. I sometimes feel that many settings are based on squeezing everything out of the material with the least possible amount of bits, even if it means oversmoothing etc. which can then be irritating to others (like me).
Can I ask what settings do you use for 4k HDR encodes? I mean like --aq-strength --qcomp and CFR. I would like to know for example aq-strength is depends of bitrate or not. Cuz with low bitrate video may be very blocky and requires high aq-strength, but what about for bitrate ~ 20-22mb?
I usually use aq-strength 0.8 and I'm thinking should I raise it to 0.9 or even 1.0?

Last edited by redbtn; 13th October 2019 at 14:35.
redbtn is online now   Reply With Quote
Old 13th October 2019, 15:34   #7117  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Hollola, Finland
Posts: 4,621
Quote:
Originally Posted by redbtn View Post
Can I ask what settings do you use for 4k HDR encodes? I mean like --aq-strength --qcomp and CFR. I would like to know for example aq-strength is depends of bitrate or not. Cuz with low bitrate video may be very blocky and requires high aq-strength, but what about for bitrate ~ 20-22mb?
I usually use aq-strength 0.8 and I'm thinking should I raise it to 0.9 or even 1.0?
It's been quite a long time since I did any HDR encodes, but I would start with aq-mode 1 with the default strength 1.0. I use qcomp 0.7 and HDR requires a much lower CRF. My base for SDR HD is CRF 18 so I would use CRF 13 for HDR.

The encoder has probably gone through many changes so I would need to test things before doing any HDR encodes. But those would be a good starting point.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 13th October 2019, 16:40   #7118  |  Link
redbtn
Registered User
 
redbtn's Avatar
 
Join Date: Jan 2019
Location: Russia
Posts: 87
Quote:
Originally Posted by Boulder View Post
It's been quite a long time since I did any HDR encodes, but I would start with aq-mode 1 with the default strength 1.0. I use qcomp 0.7 and HDR requires a much lower CRF. My base for SDR HD is CRF 18 so I would use CRF 13 for HDR.

The encoder has probably gone through many changes so I would need to test things before doing any HDR encodes. But those would be a good starting point.
Thx for your reply! For 1080p HDR I use crf 10-13, but for 2160p HDR crf 13 is too low. With crf 20 I usually get 17-18mb and for grainy films I get 22-24mb.
I'll try to use strength 1.0, but I did some tests and it raise bitrate by 10-11%.
redbtn is online now   Reply With Quote
Old 13th October 2019, 17:27   #7119  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Hollola, Finland
Posts: 4,621
Yes, CRF 13 is probably quite low for UHD because any problems arising from too few bits will not be as apparent on a 4K TV as with 1080p and lower where they are enlarged during playback.
With aq-strength, you could probably go down to 0.8 or so with grainy sources as grain tends to attract bits anyway, so the flat parts of the image will not look bad.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 14th October 2019, 17:27   #7120  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 2,966
Quote:
Originally Posted by MeteorRain View Post
From different point of view, you can say HEVC is the present, you can say it's not. For example, online streaming still uses AVC (some like Youtube may be using VP9 but still), we still buy Blu-rays instead of UHDs, majority of the content distributors are still using AVC. The use of HEVC would still be quite limited so far, even if assuming HEVC can be encoded at a faster speed. I admit there are other factors like royalty, but still.
There are tons of devices where HEVC is the only supported 10-bit codec. I'd expect pretty much all streaming of 4K or HDR content to those devices to be in HEVC. I don't know of anyone doing HDR or >1080p in H.264.

It'd probably take something like Wireshark to figure out what is being used in other cases. In the adaptive streaming world, there can be dozens of encoded variants of a given title, in different frame sizes, bitrates, DRM, and codecs.

As a wild personal guess, I'd expect a lot more premium content is delivered in HEVC than in VP9+AV1. The On2 stuff has always been most focused on browser-based user generated content without DRM, and there aren't any AV1 HW DRM + decode products in the market, or even announced.

As for quality versus speed, At the same encoding speed, x265 beats x264 in the cases I've tested in the last couple of years. Sure, it doesn't take full advantage of what HEVC can do, but it's still better for most content (there are likely edge cases with a lot of grain). The fastest possible x264 will be faster than the fastest possible x265, of course. x265 and HEVC in general takes better advantage of AVX, multithreading, and 64-bit than x264, so the newer the processor, the better quality @ perf HEVC has.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 23:31.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.