Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > High Efficiency Video Coding (HEVC)

Reply
 
Thread Tools Search this Thread Display Modes
Old 13th December 2022, 16:11   #61  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,869
Could you give me your command line instead? That's easier to parse.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 13th December 2022, 16:54   #62  |  Link
madey83
Guest
 
Posts: n/a
Quote:
Originally Posted by benwaggoner View Post
Could you give me your command line instead? That's easier to parse.

This is not mine. I found it on internet
  Reply With Quote
Old 30th December 2022, 18:37   #63  |  Link
HD MOVIE SOURCE
Registered User
 
HD MOVIE SOURCE's Avatar
 
Join Date: Mar 2021
Location: North Carolina
Posts: 138
Quote:
Originally Posted by benwaggoner View Post
--rc-lookahead is capped at --keyint in any case, so it doesn't really matter.


--aq-mode 3 is identical to --aq-mode 2 except that it lowers QP as luma values get nearer to black. This gets around Rec. 709's perceptual non-uniformity, where the difference between luma of 16 and 17 is much more visible than between, say 216 and 217. So the same compression artifacts can be much more visible near black than near white.

--aq-mode 3 can really help shadow detail and reduce banding near black. However, those lower QPs use more bits. For CBR and VBV-limited encodes, this increases QP of brighter frames, which can introduce new quality issues. Or in CRF mode, increase ABR quite a bit. It's a great tool when there's enough bandwidth, but needs to be used delicately at lower bitrates. It's not a safe feature to just leave always on when targeting best possible quality at low bitrates. It helps sometimes, and hurts others. Lower --aq-strength values are optimal with aq-mode 3 than 2 as well, as high values can really starve brighter macroblocks of bits.

It's also not appropriate for HDR-10, which is much more perceptually uniform, and where the visible difference between luma values is pretty much identical at any brightness.

10-bit encoding can make --aq-mode less needed as well, as the difference between 8-bit 16 and 17 now is the difference between 64 and 68, and those four extra steps make for more efficient encoding, reduce the need for dithering, and generally makes for much less visible banding and blocking.

I'm confident that --aq-mode 3 could be improved to be more content-adaptive. It'd prefer x264 to have aq-mode algorithm selection decouple from luma bias. So we could do stuff like

Code:
--aq-mode 4 --low-luma-bias 0.7
And make --aq-mode 3 essentially an alias to

Code:
--aq-mode 2 --low-luma-bias 1.0
Do you any other settings that could improve detail? How about rskip=0? Is this a setting that is good for grain retention? If it's good for grain retention, could it also be good for small details that are not grain?

Thanks.
HD MOVIE SOURCE is offline   Reply With Quote
Old 3rd January 2023, 18:56   #64  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,869
Quote:
Originally Posted by HD MOVIE SOURCE View Post
Do you any other settings that could improve detail? How about rskip=0? Is this a setting that is good for grain retention? If it's good for grain retention, could it also be good for small details that are not grain?
I've found --rskip 2 --rskip-threshold 2 or 3 works well with typical film grain, without as big a speed hit as --rskip 0.

Raising --psy-rd and --psy-rdoq some can help. Try 3 for each as a starting point.

--ctu 32 for sure

The built-in --nr-inter and --nr-intra aren't particularly advanced denoising filters, but as they operate at the quantization phase, can offer bigger compression efficiency benefits than other methods. Start with something like --nr-intra 100 --nr-inter 400. --nr-intra is a spatial denoising filter and --nr-inter is temporal, which is why we use a good multiple higher as film grain is all temporal.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 4th January 2023, 19:52   #65  |  Link
HD MOVIE SOURCE
Registered User
 
HD MOVIE SOURCE's Avatar
 
Join Date: Mar 2021
Location: North Carolina
Posts: 138
Quote:
Originally Posted by benwaggoner View Post
I've found --rskip 2 --rskip-threshold 2 or 3 works well with typical film grain, without as big a speed hit as --rskip 0.

Raising --psy-rd and --psy-rdoq some can help. Try 3 for each as a starting point.

--ctu 32 for sure

The built-in --nr-inter and --nr-intra aren't particularly advanced denoising filters, but as they operate at the quantization phase, can offer bigger compression efficiency benefits than other methods. Start with something like --nr-intra 100 --nr-inter 400. --nr-intra is a spatial denoising filter and --nr-inter is temporal, which is why we use a good multiple higher as film grain is all temporal.
Okay, thank you. Now, what is it about ctu=32 that has a benefit over ctu=64? This isn't something I know anything about. I just assumed it defaults to 64 and that's where it should be.

Does it just give more control over each block? Would this increase bit-rates?

Thanks.
HD MOVIE SOURCE is offline   Reply With Quote
Old 5th January 2023, 19:16   #66  |  Link
HD MOVIE SOURCE
Registered User
 
HD MOVIE SOURCE's Avatar
 
Join Date: Mar 2021
Location: North Carolina
Posts: 138
Quote:
Originally Posted by benwaggoner View Post
I've found --rskip 2 --rskip-threshold 2 or 3 works well with typical film grain, without as big a speed hit as --rskip 0.

Raising --psy-rd and --psy-rdoq some can help. Try 3 for each as a starting point.

--ctu 32 for sure

The built-in --nr-inter and --nr-intra aren't particularly advanced denoising filters, but as they operate at the quantization phase, can offer bigger compression efficiency benefits than other methods. Start with something like --nr-intra 100 --nr-inter 400. --nr-intra is a spatial denoising filter and --nr-inter is temporal, which is why we use a good multiple higher as film grain is all temporal.

As with tskip, it has a mode where it can be useful, but use less resources with --tskip-fast. If use rskip, are 0, 1, and 2 just less and less levels of rskip, or are they completely different rskip modes meant for different content? What really asking, is there an rskip-fast like the tskip-fast, because it's extremely useful.

I wanted to ask a bit more about aq-mode=3 and aq-strength. I believe aq-strength ranges from 0 (off) to 3 (full), But, when you set aq-strength to 3 what is that really doing? Is it trying to place 100% of it's bit-rate in darker areas? Or is there a cap even when set to 3 that only, say, 25% of it's bit-rate looks fo darker content?

----Lets just say it looked for 25% of it's bit-rate to be added in dark areas. If I lower aq-strength from 3 to 1.5, would that mean it it not distributes 12.5% of it's bit-rates into dark areas?

I'm just trying to understand how aq-strength works in relation to aq-mode 3.

Thanks.
HD MOVIE SOURCE is offline   Reply With Quote
Old 6th January 2023, 23:59   #67  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,869
All your questions and more are answered at: https://x265.readthedocs.io/en/master/cli.html

It specifies the data type of each parameter.

For --rskip, --rskip 2 is the mode you want to use (deprecating the older, poor at grain default --rskip 1). You control speed/quality tradeoff with --rskip 2 with the --rskip-edge-threshold parameter. Lower values are slower but more accurate. The default is 5, and I like to use 2-3 for high quality encoding.

I'm not aware of any methods to estimate the bitrate impact of different --aq-modes, as they are quite content dependent.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 10th January 2023, 23:06   #68  |  Link
HD MOVIE SOURCE
Registered User
 
HD MOVIE SOURCE's Avatar
 
Join Date: Mar 2021
Location: North Carolina
Posts: 138
Quote:
Originally Posted by benwaggoner View Post
All your questions and more are answered at: https://x265.readthedocs.io/en/master/cli.html

It specifies the data type of each parameter.

For --rskip, --rskip 2 is the mode you want to use (deprecating the older, poor at grain default --rskip 1). You control speed/quality tradeoff with --rskip 2 with the --rskip-edge-threshold parameter. Lower values are slower but more accurate. The default is 5, and I like to use 2-3 for high quality encoding.

I'm not aware of any methods to estimate the bitrate impact of different --aq-modes, as they are quite content dependent.
I appreciate it thank you.
HD MOVIE SOURCE is offline   Reply With Quote
Old 28th January 2023, 18:10   #69  |  Link
HD MOVIE SOURCE
Registered User
 
HD MOVIE SOURCE's Avatar
 
Join Date: Mar 2021
Location: North Carolina
Posts: 138
Quote:
Originally Posted by benwaggoner View Post
I've found --rskip 2 --rskip-threshold 2 or 3 works well with typical film grain, without as big a speed hit as --rskip 0.

Raising --psy-rd and --psy-rdoq some can help. Try 3 for each as a starting point.

--ctu 32 for sure

The built-in --nr-inter and --nr-intra aren't particularly advanced denoising filters, but as they operate at the quantization phase, can offer bigger compression efficiency benefits than other methods. Start with something like --nr-intra 100 --nr-inter 400. --nr-intra is a spatial denoising filter and --nr-inter is temporal, which is why we use a good multiple higher as film grain is all temporal.

With all the settings I'm using to pull out as much detail as possible using x265. One setting that Ive forgot about is cu-lossless. Can this help pull out more detail in grain and fine particles?
HD MOVIE SOURCE is offline   Reply With Quote
Old 1st February 2023, 00:39   #70  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,869
Quote:
Originally Posted by HD MOVIE SOURCE View Post
With all the settings I'm using to pull out as much detail as possible using x265. One setting that Ive forgot about is cu-lossless. Can this help pull out more detail in grain and fine particles?
No, --cu-lossless just tries mathematically lossless encoding for a given CU, and picks it if it is lower bitrate/distortion. In essence it's a really specialized and compute-intensive subset of --tskip. It would only be used with very high bitrates or some very specialized patterns, like a fully noise-free digitally rendered black/white checkerboard pattern, color bars, or something artificial like that. Grainy content is pretty much the inverse of when it might be helpful.

I've never found a real-world scenario where --cu-lossless would be useful for a distribution encode. Perhaps for a mezzanine file. --tskip should capture most or all of the value at a much lower performance hit. There's no --cu-lossless-fast equivalent to --tskip-fast.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 22nd February 2023, 05:19   #71  |  Link
HD MOVIE SOURCE
Registered User
 
HD MOVIE SOURCE's Avatar
 
Join Date: Mar 2021
Location: North Carolina
Posts: 138
Quote:
Originally Posted by benwaggoner View Post
No, --cu-lossless just tries mathematically lossless encoding for a given CU, and picks it if it is lower bitrate/distortion. In essence it's a really specialized and compute-intensive subset of --tskip. It would only be used with very high bitrates or some very specialized patterns, like a fully noise-free digitally rendered black/white checkerboard pattern, color bars, or something artificial like that. Grainy content is pretty much the inverse of when it might be helpful.

I've never found a real-world scenario where --cu-lossless would be useful for a distribution encode. Perhaps for a mezzanine file. --tskip should capture most or all of the value at a much lower performance hit. There's no --cu-lossless-fast equivalent to --tskip-fast.
Thanks, will run some more encodes to see what it's giving me. I'll also run some film grain tests, but as you said, it's very compute-intensive, and you need a monster setup to run this. It's okay for 4-minute content to test, but for full movies, it's a real struggle.

I appreciate the feedback.
HD MOVIE SOURCE is offline   Reply With Quote
Old 24th February 2023, 18:42   #72  |  Link
HD MOVIE SOURCE
Registered User
 
HD MOVIE SOURCE's Avatar
 
Join Date: Mar 2021
Location: North Carolina
Posts: 138
Quote:
Originally Posted by benwaggoner View Post
No, --cu-lossless just tries mathematically lossless encoding for a given CU, and picks it if it is lower bitrate/distortion. In essence it's a really specialized and compute-intensive subset of --tskip. It would only be used with very high bitrates or some very specialized patterns, like a fully noise-free digitally rendered black/white checkerboard pattern, color bars, or something artificial like that. Grainy content is pretty much the inverse of when it might be helpful.

I've never found a real-world scenario where --cu-lossless would be useful for a distribution encode. Perhaps for a mezzanine file. --tskip should capture most or all of the value at a much lower performance hit. There's no --cu-lossless-fast equivalent to --tskip-fast.
You're right, I tested this on grain content, and I got the exact same results, right down to the average QP. Hmm, interesting, good to know. I was also using tskip, and both encodes with cu-lossless and without have the exact same results. It may as well have been the exact same file. Very interesting.
HD MOVIE SOURCE is offline   Reply With Quote
Old 4th March 2023, 23:06   #73  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,869
Quote:
Originally Posted by HD MOVIE SOURCE View Post
You're right, I tested this on grain content, and I got the exact same results, right down to the average QP. Hmm, interesting, good to know. I was also using tskip, and both encodes with cu-lossless and without have the exact same results. It may as well have been the exact same file. Very interesting.
Yep, that's what will happen if x265 doesn't detect any CUs where lossless encoding has better rate/distortion. Which I would expect if the clip is all grainy frames.

You may not be getting any benefit from --tskip either. It can help with noise-free credits in grainy movies, but isn't going to be useful for any CU with significant random noise. It's pretty hard to simply spend more compute to make grainy content look better than --preset slower. What improves grain is using the --nr-i* features, lowering ipratio and pbratio, and other things that are more qualitative tuning, not just more expensive modes.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 21st March 2023, 04:22   #74  |  Link
HD MOVIE SOURCE
Registered User
 
HD MOVIE SOURCE's Avatar
 
Join Date: Mar 2021
Location: North Carolina
Posts: 138
Quote:
Originally Posted by benwaggoner View Post
Yep, that's what will happen if x265 doesn't detect any CUs where lossless encoding has better rate/distortion. Which I would expect if the clip is all grainy frames.

You may not be getting any benefit from --tskip either. It can help with noise-free credits in grainy movies, but isn't going to be useful for any CU with significant random noise. It's pretty hard to simply spend more compute to make grainy content look better than --preset slower. What improves grain is using the --nr-i* features, lowering ipratio and pbratio, and other things that are more qualitative tuning, not just more expensive modes.
Good to know, and thank you for the advice.
HD MOVIE SOURCE is offline   Reply With Quote
Old 4th May 2023, 16:28   #75  |  Link
HD MOVIE SOURCE
Registered User
 
HD MOVIE SOURCE's Avatar
 
Join Date: Mar 2021
Location: North Carolina
Posts: 138
Quote:
Originally Posted by benwaggoner View Post
Yep, that's what will happen if x265 doesn't detect any CUs where lossless encoding has better rate/distortion. Which I would expect if the clip is all grainy frames.

You may not be getting any benefit from --tskip either. It can help with noise-free credits in grainy movies, but isn't going to be useful for any CU with significant random noise. It's pretty hard to simply spend more compute to make grainy content look better than --preset slower. What improves grain is using the --nr-i* features, lowering ipratio and pbratio, and other things that are more qualitative tuning, not just more expensive modes.
Hey, was just running some tests for ctu=64 and ctu=32, and ctu=32 was minorly better. Is this a simple...test on the content first? To find out what's the best methodology? Are there rules like, animation is best on ctu=64 and film is best on ctu=32? Because I was testing film content and 32, again, was better, but by a hair.

Would going lower than this be viable? Have you ever done that?

Also, based on the cut size, does this effect tu-intra-depth and tu-inter-depth? From my understand, tu-intra-depth and tu-inter-depth
are how small into the details the encode goes, is that correct? So for fine details, film grain and things like that tu-intra-depth=4 and tu-inter-depth=4 would typically be better.
Does using ctu=32 effect the size of the intra depth and inter?
HD MOVIE SOURCE is offline   Reply With Quote
Old 6th May 2023, 07:51   #76  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,869
Quote:
Originally Posted by HD MOVIE SOURCE View Post
Hey, was just running some tests for ctu=64 and ctu=32, and ctu=32 was minorly better. Is this a simple...test on the content first? To find out what's the best methodology? Are there rules like, animation is best on ctu=64 and film is best on ctu=32? Because I was testing film content and 32, again, was better, but by a hair.

Would going lower than this be viable? Have you ever done that?
I've never seen any benefit to using ctu below 32 myself, other squeezing a last bit of extra encoder speed and parallelization.

Quote:
Also, based on the cut size, does this effect tu-intra-depth and tu-inter-depth? From my understand, tu-intra-depth and tu-inter-depth
are how small into the details the encode goes, is that correct? So for fine details, film grain and things like that tu-intra-depth=4 and tu-inter-depth=4 would typically be better.
Does using ctu=32 effect the size of the intra depth and inter?
They are relative to ctu. So going all 4 down is only possible with ctu 64. ctu 32 to 3, 16 to 2.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 6th May 2023, 18:24   #77  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Finland
Posts: 5,799
Quote:
Originally Posted by benwaggoner View Post
They are relative to ctu. So going all 4 down is only possible with ctu 64. ctu 32 to 3, 16 to 2.
They are relative to TU. The default TU should be the same as CTU as per the docs, but it actually is 32 unless CTU is lower. 32 is enough to use depth 4. I use CTU 32, TU 32, depth 4 in my encodes.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 8th May 2023, 04:04   #78  |  Link
HD MOVIE SOURCE
Registered User
 
HD MOVIE SOURCE's Avatar
 
Join Date: Mar 2021
Location: North Carolina
Posts: 138
Quote:
Originally Posted by Boulder View Post
They are relative to TU. The default TU should be the same as CTU as per the docs, but it actually is 32 unless CTU is lower. 32 is enough to use depth 4. I use CTU 32, TU 32, depth 4 in my encodes.
Interesting, are you seeing better encodes using CTU=32? I've seen one encode where the QP only dropped by 0.1 on average, so I didn't know whether to use it or not.

In theory can CTU=32 grab more detail than 64? I might just run some tests for even lower than 32 just to see it.
HD MOVIE SOURCE is offline   Reply With Quote
Old 8th May 2023, 05:03   #79  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Finland
Posts: 5,799
Quote:
Originally Posted by HD MOVIE SOURCE View Post
Interesting, are you seeing better encodes using CTU=32? I've seen one encode where the QP only dropped by 0.1 on average, so I didn't know whether to use it or not.

In theory can CTU=32 grab more detail than 64? I might just run some tests for even lower than 32 just to see it.
Let's say that I've not seen any quality loss because of it. I mostly do encodes between 720p and 1440p and the lower value improves multithreading a lot.

There is also something strange happening with CTU 64 + limit-tu 0 + rskip 2 and MultiCoreWare just ignored the report. I don't trust that it wouldn't appear also in other cases. CTU 32 doesn't have it.

https://forum.doom9.org/showthread.p...A0#post1919347
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 8th May 2023, 09:23   #80  |  Link
excellentswordfight
Lost my old account :(
 
Join Date: Jul 2017
Posts: 339
Quote:
Originally Posted by HD MOVIE SOURCE View Post
Interesting, are you seeing better encodes using CTU=32? I've seen one encode where the QP only dropped by 0.1 on average, so I didn't know whether to use it or not.
What resolution are you encoding at? There are benefits of using it at high resolutions, i.e. 4k/uhd. Many years ago I asked someone at multicoreware (when they were active here) why they didnt automatically lower it to 32 for lower resolutions as 64 seems a bit overkill there and since it improves multithreading so much by lowering it, and they replied with some numbers showing increased compression efficiency even at 720/1080p, it was minor mind you, and tbh I have never seen any kind noticeable improvement that was worth the speed-penalty on high thread count systems.
Quote:
In theory can CTU=32 grab more detail than 64? I might just run some tests for even lower than 32 just to see it.
The theory is that larger CU-sizes can compress the image more efficiently, and by that it should actually increase details as it can spend saved bits were more needed instead. But as with a lot of other parameters for improved compression, they might not yield "better" results when encoding for visually lossless, not sure why it would hurt quality in this case though. I've seen people mention that it would improve grain-encoding, but I have yet seen it demonstrated.

Last edited by excellentswordfight; 8th May 2023 at 14:39.
excellentswordfight is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 11:36.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.