Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > High Efficiency Video Coding (HEVC)

Reply
 
Thread Tools Search this Thread Display Modes
Old 17th January 2020, 00:59   #7361  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 3,070
Quote:
Originally Posted by jlpsvk View Post
Is there any way to fully utitlise x265 with single process (without distrubuted encoding) on Ryzen 9 3950X (16c/32t) without quality loss?
What resolution are you encoding to?

--preset slower --pmode --pme should saturate most systems. Although it's often faster to leave out --pmode and --pme if the cores/pixels ratio isn't that high. The goal is to maximize fps, not CPU load. PMode is typically a lot more useful than PME, which I've only seen to be helpful at SD resolutions and below.

PMode and PME don't have any negative quality impact, and pmode can actually help a trifle, potentially.

Doing 2160p on an 18/36 system encoding is ~30% faster without pmode than with it, even though CPU load is about 50% versus ~90%.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 17th January 2020, 05:54   #7362  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Hollola, Finland
Posts: 4,668
--pmode can have a negative impact in CRF mode, so be warned. I just experienced that.. it produced a lower bitrate than without --pmode at the same settings at CRF 18, and I happened to check a scene with a heavily noisy picture. The dancing of the noise was much uglier than in a test encode without --pmode.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 17th January 2020, 12:21   #7363  |  Link
Atak_Snajpera
RipBot264 author
 
Atak_Snajpera's Avatar
 
Join Date: May 2006
Location: Poland
Posts: 7,358
Quote:
Originally Posted by jlpsvk View Post
Is there any way to fully utitlise x265 with single process (without distrubuted encoding) on Ryzen 9 3950X (16c/32t) without quality loss?
--ctu 32
Atak_Snajpera is offline   Reply With Quote
Old 17th January 2020, 19:48   #7364  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 3,070
Quote:
Originally Posted by Atak_Snajpera View Post
--ctu 32
Right! There are other ways to increase parallelism, at the risk of some hits to compression efficiency. However, with a whole lot of cores, quality @ perf might have a different optimal configuration, like using more frame thread and a higher RD level. Other parameters in the same bucket:

-F >4 (this had quality hits in older versions; not sure of the impact in 3.x)
--lookahead slices >8 (It seems like this shouldn't have a quality hit, but it is 1 in the slowest presets)

Another thing that might help (haven't tested) without impacting quality is --selective-sao 2. SAO itself limits parallelism somewhat per https://x265.readthedocs.io/en/default/threading.html. So turning it off for B-frames could help. Although it's also possible that the latency hit is fixed whenever SAO is on at all, so --no-sao might help, although with efficiency hits, particularly at lower bitrates. Worth testing.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 17th January 2020, 19:49   #7365  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 3,070
Quote:
Originally Posted by Boulder View Post
--pmode can have a negative impact in CRF mode, so be warned. I just experienced that.. it produced a lower bitrate than without --pmode at the same settings at CRF 18, and I happened to check a scene with a heavily noisy picture. The dancing of the noise was much uglier than in a test encode without --pmode.
Oh, good tip! That isn't supposed to happen in theory. You should provide a bug report and repro to MultiCoreWare so they can look at this deeper.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 17th January 2020, 20:51   #7366  |  Link
jlpsvk
Registered User
 
Join Date: Dec 2014
Posts: 204
Quote:
Originally Posted by benwaggoner View Post
What resolution are you encoding to?

--preset slower --pmode --pme should saturate most systems. Although it's often faster to leave out --pmode and --pme if the cores/pixels ratio isn't that high. The goal is to maximize fps, not CPU load. PMode is typically a lot more useful than PME, which I've only seen to be helpful at SD resolutions and below.

PMode and PME don't have any negative quality impact, and pmode can actually help a trifle, potentially.

Doing 2160p on an 18/36 system encoding is ~30% faster without pmode than with it, even though CPU load is about 50% versus ~90%.
I am encoding 2160p, with CRF16.
__________________
AMD Ryzen 9 3950X, 32GB DDR4-3200 CL16, RTX 2070, 1TB NVMe SSD, 56TB NAS
jlpsvk is offline   Reply With Quote
Old 17th January 2020, 21:07   #7367  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Hollola, Finland
Posts: 4,668
Quote:
Originally Posted by benwaggoner View Post
--lookahead slices >8 (It seems like this shouldn't have a quality hit, but it is 1 in the slowest presets)
This probably affects frame type decision, at least that's what I observed when using four lookahead slices compared to just one.


Quote:
Quote:
--pmode can have a negative impact in CRF mode, so be warned. I just experienced that.. it produced a lower bitrate than without --pmode at the same settings at CRF 18, and I happened to check a scene with a heavily noisy picture. The dancing of the noise was much uglier than in a test encode without --pmode.
Oh, good tip! That isn't supposed to happen in theory. You should provide a bug report and repro to MultiCoreWare so they can look at this deeper.
It's probably due to the encoder doing deeper analysis as some performance related early skips are avoided. Nevertheless, I got a clip of the scene for my testing toolkit and I can definitely create a ticket for the issue. Maybe there is something there to fix.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 18th January 2020, 09:07   #7368  |  Link
jlpsvk
Registered User
 
Join Date: Dec 2014
Posts: 204
i tested, and without pme and pmode, it's faster.
__________________
AMD Ryzen 9 3950X, 32GB DDR4-3200 CL16, RTX 2070, 1TB NVMe SSD, 56TB NAS
jlpsvk is offline   Reply With Quote
Old 18th January 2020, 09:35   #7369  |  Link
froggy1
ffx264/ffhevc author
 
froggy1's Avatar
 
Join Date: May 2007
Location: Belgium
Posts: 1,605
Quote:
Originally Posted by benwaggoner View Post
Right! There are other ways to increase parallelism, at the risk of some hits to compression efficiency. However, with a whole lot of cores, quality @ perf might have a different optimal configuration, like using more frame thread and a higher RD level. Other parameters in the same bucket:

-F >4 (this had quality hits in older versions; not sure of the impact in 3.x)
--lookahead slices >8 (It seems like this shouldn't have a quality hit, but it is 1 in the slowest presets)

Another thing that might help (haven't tested) without impacting quality is --selective-sao 2. SAO itself limits parallelism somewhat per https://x265.readthedocs.io/en/default/threading.html. So turning it off for B-frames could help. Although it's also possible that the latency hit is fixed whenever SAO is on at all, so --no-sao might help, although with efficiency hits, particularly at lower bitrates. Worth testing.
I have tested a few samples with SAO and selective-sao set to 2. I find the result looking too soft. Without it, there's more detail present. But if others like it, who am I to stop them?
__________________
ffx264--ffhevc--ffxvid

AMD Ryzen 7 3700X
froggy1 is offline   Reply With Quote
Old 19th January 2020, 19:56   #7370  |  Link
jlpsvk
Registered User
 
Join Date: Dec 2014
Posts: 204
Quote:
Originally Posted by froggy1 View Post
I have tested a few samples with SAO and selective-sao set to 2. I find the result looking too soft. Without it, there's more detail present. But if others like it, who am I to stop them?
exactly... my 2160p HDR10 preset currently...any suggestions?

Code:
--preset slow --profile main10 --level-idc 5.1 --output-depth 10 --crf 16 --ctu 64 --aq-mode 4 --merange 57 --amp --no-rskip --qg-size 8 --vbv-bufsize 160000 --vbv-maxrate 160000 --bframes 8
--rc-lookahead 48 --gop-lookahead 30 --hdr10 --hdr10-opt --repeat-headers --no-info --no-deblock --no-sao --selective-sao 0 --allow-non-conformance --no-strong-intra-smoothing --high-tier --chromaloc 2
--fades --hme --hme-search umh,umh,star
__________________
AMD Ryzen 9 3950X, 32GB DDR4-3200 CL16, RTX 2070, 1TB NVMe SSD, 56TB NAS
jlpsvk is offline   Reply With Quote
Old 20th January 2020, 01:10   #7371  |  Link
froggy1
ffx264/ffhevc author
 
froggy1's Avatar
 
Join Date: May 2007
Location: Belgium
Posts: 1,605
Quote:
Originally Posted by jlpsvk View Post
exactly... my 2160p HDR10 preset currently...any suggestions?

Code:
--preset slow --profile main10 --level-idc 5.1 --output-depth 10 --crf 16 --ctu 64 --aq-mode 4 --merange 57 --amp --no-rskip --qg-size 8 --vbv-bufsize 160000 --vbv-maxrate 160000 --bframes 8
--rc-lookahead 48 --gop-lookahead 30 --hdr10 --hdr10-opt --repeat-headers --no-info --no-deblock --no-sao --selective-sao 0 --allow-non-conformance --no-strong-intra-smoothing --high-tier --chromaloc 2
--fades --hme --hme-search umh,umh,star
I'd set deblock to -1,-1 instead of completely disabling it - deblock has seen a major revision compared to x264.

Also strong-intra-smoothing seems to have a positive effect at keeping noise. Many people are scared by the name but I find it does improve things a bit. It will help reduce banding on very clean scenes too, like skies or other clean textures, by blending a bit instead of keeping the banding

merange has no effect when using HME since the latter has its own range parameters you can adjust with --hme-range
__________________
ffx264--ffhevc--ffxvid

AMD Ryzen 7 3700X

Last edited by froggy1; 20th January 2020 at 01:22.
froggy1 is offline   Reply With Quote
Old 20th January 2020, 12:19   #7372  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Hollola, Finland
Posts: 4,668
Ben Waggoner was doing some tests with --aq-mode 4, I don't think he got them finished though. He's probably the only one (apart from the devs) who has some kind of an insight to how it looks with real life examples.

For what it's worth, lately I've found --aq-mode 3 being troublesome. With normal sources originating from film, like the older seasons of X-Files which contain quite a lot of grain and noise, it works well at around ~0.6 for strength. Then there's something like Suits, or The Killing, which look terrible. The sources are terrible in quality -- lots of macroblocking which appears when the encoder starts removing the grain hiding some of that -- and aq-mode 3 produces a substantially lower average bitrate at the same CRF than aq-mode 1. I don't know why the rate control works this way, The Killing does contain quite a lot of dark scenes but I was unable to get a good result without switching to --aq-mode 1 at the default strength. I also raised psy-rd to 3.0 and deblock to 0:0 to try to keep the higher frequencies and avoid banding where the encoder removes too much from the original image.

I really scratched my head at the issue the whole weekend. Yes, I did test x264 at --preset veryslow, but it wasn't any better

Sometimes I have used a dirty trick and added a light amount of fake grain with GrainFactory3 before feeding the video to the encoder. I may need to test that too
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 20th January 2020, 16:37   #7373  |  Link
froggy1
ffx264/ffhevc author
 
froggy1's Avatar
 
Join Date: May 2007
Location: Belgium
Posts: 1,605
Quote:
Originally Posted by Boulder View Post
Ben Waggoner was doing some tests with --aq-mode 4, I don't think he got them finished though. He's probably the only one (apart from the devs) who has some kind of an insight to how it looks with real life examples.

For what it's worth, lately I've found --aq-mode 3 being troublesome. With normal sources originating from film, like the older seasons of X-Files which contain quite a lot of grain and noise, it works well at around ~0.6 for strength. Then there's something like Suits, or The Killing, which look terrible. The sources are terrible in quality -- lots of macroblocking which appears when the encoder starts removing the grain hiding some of that -- and aq-mode 3 produces a substantially lower average bitrate at the same CRF than aq-mode 1. I don't know why the rate control works this way, The Killing does contain quite a lot of dark scenes but I was unable to get a good result without switching to --aq-mode 1 at the default strength. I also raised psy-rd to 3.0 and deblock to 0:0 to try to keep the higher frequencies and avoid banding where the encoder removes too much from the original image.

I really scratched my head at the issue the whole weekend. Yes, I did test x264 at --preset veryslow, but it wasn't any better

Sometimes I have used a dirty trick and added a light amount of fake grain with GrainFactory3 before feeding the video to the encoder. I may need to test that too
I did my own testing, both with AQ mode 2 and 3. It did not make things beter here and I was hoping AQ mode 3 would work well on dark scenes, but it didn't. All I saw is increased bitrate for 2 and 3 with no meaningful improvements... so I went back and stick to my good ol' AQ mode 1

As for psy-rd/psy-rdoq, they are meant to keep the original energy of the source as much as possible at higher strength. So if you have a (very) noisy sample, the bitrate will shoot up in order to preserve the noise/grain. These two settings are set to high values when using --tune grain (psy-rd is set to 4 and psy-rdoq is set to 10). I personally now use psy-rd of 3.2 and psy-rdoq of 15. I do not mind the increase of bitrate as I aim to preserver the input as much as possible. These two especially have an effect on dark scenes and look better than using AQ mode 2 or 3. I haven't tested AQ mode 4, though
__________________
ffx264--ffhevc--ffxvid

AMD Ryzen 7 3700X

Last edited by froggy1; 20th January 2020 at 16:43.
froggy1 is offline   Reply With Quote
Old 20th January 2020, 16:45   #7374  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 3,070
Quote:
Originally Posted by Boulder View Post
Ben Waggoner was doing some tests with --aq-mode 4, I don't think he got them finished though. He's probably the only one (apart from the devs) who has some kind of an insight to how it looks with real life examples.
I finished making the content but not my comparisons. Thanks for the reminder to take a look again.

Quote:
For what it's worth, lately I've found --aq-mode 3 being troublesome. With normal sources originating from film, like the older seasons of X-Files which contain quite a lot of grain and noise, it works well at around ~0.6 for strength. Then there's something like Suits, or The Killing, which look terrible. The sources are terrible in quality -- lots of macroblocking which appears when the encoder starts removing the grain hiding some of that -- and aq-mode 3 produces a substantially lower average bitrate at the same CRF than aq-mode 1. I don't know why the rate control works this way, The Killing does contain quite a lot of dark scenes but I was unable to get a good result without switching to --aq-mode 1 at the default strength. I also raised psy-rd to 3.0 and deblock to 0:0 to try to keep the higher frequencies and avoid banding where the encoder removes too much from the original image.
Maybe throw in some --nr-inter? That tends to reduce grain swirling and lower QPs a bit at the same bitrate.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 20th January 2020, 16:47   #7375  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 3,070
Quote:
Originally Posted by froggy1 View Post
I did my own testing, both with AQ mode 2 and 3. It did not make things beter here and I was hoping AQ mode 3 would work well on dark scenes, but it didn't. All I saw is increased bitrate for 2 and 3 with no meaningful improvements... so I went back and stick to my good ol' AQ mode 1
The AQ comparisons really need to be made with 2-pass VBR to compare bitrates. Different AQ modes can require different CRF values in practice.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 20th January 2020, 17:01   #7376  |  Link
froggy1
ffx264/ffhevc author
 
froggy1's Avatar
 
Join Date: May 2007
Location: Belgium
Posts: 1,605
Quote:
Originally Posted by benwaggoner View Post
The AQ comparisons really need to be made with 2-pass VBR to compare bitrates. Different AQ modes can require different CRF values in practice.
I have no need for 2-pass VBR. I only do CRF encoding and that's what matters to me but also tested in 2-pass as you recommended previously in a post a week or so ago. Not much improvement. I find psy-rd/rdoq delivering better results than these AQ modes
__________________
ffx264--ffhevc--ffxvid

AMD Ryzen 7 3700X
froggy1 is offline   Reply With Quote
Old 20th January 2020, 19:25   #7377  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Hollola, Finland
Posts: 4,668
If you want to try things, I uploaded the sample clip I used to test: https://drive.google.com/open?id=1Ar...5cr6UNO1WjfV7- . Take a look at the flat background in the shots with the cop, it's ugly in the source and gets much worse after encoding.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 20th January 2020, 19:47   #7378  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 3,070
Quote:
Originally Posted by froggy1 View Post
I have no need for 2-pass VBR. I only do CRF encoding and that's what matters to me but also tested in 2-pass as you recommended previously in a post a week or so ago. Not much improvement. I find psy-rd/rdoq delivering better results than these AQ modes

Right. But if you want to figure out the optimum efficiency for a CRF encode, comparing the features in 2-pass VBR shows you how features compare at the same file size. Once you nail down the optimum parameters, then you figure out what CRF is optimal for you with those other parameters.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 20th January 2020, 19:51   #7379  |  Link
froggy1
ffx264/ffhevc author
 
froggy1's Avatar
 
Join Date: May 2007
Location: Belgium
Posts: 1,605
Quote:
Originally Posted by benwaggoner View Post
Right. But if you want to figure out the optimum efficiency for a CRF encode, comparing the features in 2-pass VBR shows you how features compare at the same file size. Once you nail down the optimum parameters, then you figure out what CRF is optimal for you with those other parameters.
I won't be bothered doing this. At the moment I'm quite happy with my own setting for CRF @ 21. I increased some of them by a small amount to give it a bit more headroom. If I can compress a source of say 22-25 GiB down to 5-6 or 7 GiB (with decent quality according to my eyes), I'm a happy camper
__________________
ffx264--ffhevc--ffxvid

AMD Ryzen 7 3700X
froggy1 is offline   Reply With Quote
Old 26th January 2020, 22:41   #7380  |  Link
zerowalker
Registered User
 
Join Date: Jul 2011
Posts: 1,116
Annoying question but might as well ask it.

It was a long time ago since i checked out x265, probably like 2 years ago, and back then x264 was better except for edge cases which was very low bitrate and very high resolutions mostly.
And then it looked crap anyway so even if x265 won it didn't really matter (for me).

How is it nowadays (know it's a vague question)?

If you have like a 1080p video and use crf=16 (which i would say results in high quality for x264 at least) would x265 achieve the "same quality" for less space?
Or is it only in very high resolutions like 4k and beyond that x265 can outdo it cause of how it works?

I know it sounds like i am saying x265 is crap compared to x264, but that's not what i am trying to say, but it's hard to ask the comparison question without making it sound that way.
I am not denying x265 nor the standard h265, it's obviously made for a good reason and the encoder is surely aiming to be as great as possible compared to what currently exists (either for all or some cases),
i just don't know that much about it except that the standard was made with very high resolutions in mind as h264 breaks down quickly there except with very high bitrates (which is the "throw money at the problem" solution usually;P).

Thanks
zerowalker is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 19:32.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, vBulletin Solutions Inc.