Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > High Efficiency Video Coding (HEVC)

Reply
 
Thread Tools Search this Thread Display Modes
Old 12th December 2019, 18:42   #7261  |  Link
markiemarcus
Registered User
 
Join Date: May 2018
Posts: 28
Quote:
Originally Posted by redbtn View Post
I often see this also. But I don't understand why they use it. I did tests and noticed that CTU 32 and qg-size 8 is not better than CTU 64 qg-size 32 (and sometimes even worse). And also it seems to me that aq-mode 2 is better than 1. I'm not a professional, it's only my humble opinion.
Ps: I looked at your settings, do you sure that ctu 32 merange 40 is better for 1080p than ctu 64 merange 57?
Found the same thing. I even find CTU 32 worse for grain retention, though it does make it look less boxy if using RDOQ Level 2. No-cutree I've seen cause issues in animation.

Aq mode 1 I find more consistent than 2 if the source has intermittent low luma. Aq mode 3 obviously works well, though in my experience not always better than Aq-mode 1. Metrics don't tell the whole story.

Last edited by markiemarcus; 13th December 2019 at 19:27.
markiemarcus is offline   Reply With Quote
Old 12th December 2019, 18:48   #7262  |  Link
markiemarcus
Registered User
 
Join Date: May 2018
Posts: 28
Quote:
Originally Posted by RainyDog View Post
Sure, the below is always my starting point then I tweak from there if necessary with a dial or two down on --aq-strength and/or a notch or two up on --psy-rd. This is for a broad range of film sources at 1080p resolution.
I see I'm not the only one who disables rect! The difference is so fractional on most content I've tested.
markiemarcus is offline   Reply With Quote
Old 12th December 2019, 19:45   #7263  |  Link
redbtn
Registered User
 
redbtn's Avatar
 
Join Date: Jan 2019
Location: Russia
Posts: 136
Quote:
Originally Posted by markiemarcus View Post
Aq mode 1 I find more consistent than 2 if the source has intermittent low luma. Aq mode 3 obviously works well there, though in my experience not always better than Aq-mode 1. Metrics don't tell the whole story.
I use aq-mode 2 for HDR 10bit encodes. For SDR maybe better 1, I didn't much tests with SDR.

Any thoughts about using no-rskip? Does it make sense?

Quote:
Originally Posted by markiemarcus View Post
I see I'm not the only one who disables rect! The difference is so fractional on most content I've tested.
I use no-rect too.
redbtn is offline   Reply With Quote
Old 12th December 2019, 21:23   #7264  |  Link
markiemarcus
Registered User
 
Join Date: May 2018
Posts: 28
Quote:
Originally Posted by redbtn View Post
I use aq-mode 2 for HDR 10bit encodes. For SDR maybe better 1, I didn't much tests with SDR.

Any thoughts about using no-rskip? Does it make sense?



I use no-rect too.
I have zero experience with HDR I'm afraid, but at least a few hundred hours with SDR. I've heard a number of people say that too, though.

I never bother with no-rskip. Rskip works very well. Some other preferences:

I use rdLevel 4 and almost exclusively --limit-refs 1. If the performance impact weren't so dramatic, I'd use rdLevel 6. It's just better, there's no getting around that, but it is awfully slow.

Considering the performance boost, early-skip isn't half bad either. You seem to be able to overcome most of its shortcomings with a 0.1 to 0.2 drop in CRF, which is not the case with some of the other settings.

I've seen animated content that benefits from Sub-me 7 (yes, really), RC lookahead 80 and lookahead slices 1. It really doesn't hurt performance all that much either, but otherwise, the slower preset values for those are spot on.

I've seen reproducible grain freezing and other blocking craziness with tu-intra 4, tu-inter 4 so I'm weary of anything other than 1. Limit-tu I don't know what to make of. I don't fully understand it either, or its relation to intra/inter. If the content is super flat animation and clean as a whistle, values of 4, 4 and 4 seem to work well enough.

--cbqpoffs -1 --crqpoffs -2 works well for animation. x265 seems to struggle with deep reds without this.

-tskip is nice. t-skip-fast causes problems in animation.

--aq-motion is decent with Aq-mode 1 and generally allows a 0.5 drop in CRF for the same filesize, for which there is a noticeable benefit. I don't understand the dislike or distrust of the feature.

--weightb I think is a must, along with b-intra.

Both Rect and Amp are of extremely limited use IMO. Perhaps it's just the content I've been testing. I've seen comparable results with both disabled, to having both enabled along with limit-modes. With --no-limit-modes, yes, the difference I could spot, but it's dog slow. This was on animated content though; I haven't tested that on live action.

6 bframes I think is a good compromise for most content, though I've seen some small benefits to using double that on animated content.

--selective-sao 1 is nice and now my go-to.

qg-size 64 works well, even at 1080p. It's great on animated content.

I don't go higher than max-merge 3. I've seen 4 and 5 produce worse results than 3 during fast motion in animated content. I don't know why.

As I'd mentioned previously, I'm really not a fan of RDOQ Level 1 or 2. I much prefer Level 0 with much higher Psy-rd values instead. I understand why people like it, I just think it's ugly.

--deblock -4:-1 on grainy or dithered stuff, --deblock -3:0 or -2:0 otherwise.

--fades seems to work well.

Default Me range and Star. UMH is better, but it's fractional.

refs 4 is generally speaking adequate IMO.

I keep strong-intra-smoothing on.

Aq strength is difficult. Generally the default or 0.9 is fine, but if the content is super, super grainy, I've gone as low as 0.6 and put the difference into a lower CRF for better results.

So my settings are a bit weird. I've arrived at them from probably thousands of 2 pass tests over a few years, aiming for 2:1 over x264 and no more.
markiemarcus is offline   Reply With Quote
Old 12th December 2019, 22:27   #7265  |  Link
redbtn
Registered User
 
redbtn's Avatar
 
Join Date: Jan 2019
Location: Russia
Posts: 136
Quote:
Originally Posted by markiemarcus View Post
I have zero experience with HDR I'm afraid, but at least a few hundred hours with SDR. I've heard a number of people say that too, though.



I never bother with no-rskip. Rskip works very well. Some other preferences:



I use rdLevel 4 and almost exclusively --limit-refs 1. If the performance impact weren't so dramatic, I'd use rdLevel 6. It's just better, there's no getting around that, but it is awfully slow.



Considering the performance boost, early-skip isn't half bad either. You seem to be able to overcome most of its shortcomings with a 0.1 to 0.2 drop in CRF, which is not the case with some of the other settings.



I've seen animated content that benefits from Sub-me 7 (yes, really), RC lookahead 80 and lookahead slices 1. It really doesn't hurt performance all that much either, but otherwise, the slower preset values for those are spot on.



I've seen reproducible grain freezing and other blocking craziness with tu-intra 4, tu-inter 4 so I'm weary of anything other than 1. Limit-tu I don't know what to make of. I don't fully understand it either, or its relation to intra/inter. If the content is super flat animation and clean as a whistle, values of 4, 4 and 4 seem to work well enough.



--cbqpoffs -1 --crqpoffs -2 works well for animation. x265 seems to struggle with deep reds without this.



-tskip is nice. t-skip-fast causes problems in animation.



--aq-motion is decent with Aq-mode 1 and generally allows a 0.5 drop in CRF for the same filesize, for which there is a noticeable benefit. I don't understand the dislike or distrust of the feature.



--weightb I think is a must, along with b-intra.



Both Rect and Amp are of extremely limited use IMO. Perhaps it's just the content I've been testing. I've seen comparable results with both disabled, to having both enabled along with limit-modes. With --no-limit-modes, yes, the difference I could spot, but it's dog slow. This was on animated content though; I haven't tested that on live action.



6 bframes I think is a good compromise for most content, though I've seen some small benefits to using double that on animated content.



--selective-sao 1 is nice and now my go-to.



qg-size 64 works well, even at 1080p. It's great on animated content.



I don't go higher than max-merge 3. I've seen 4 and 5 produce worse results than 3 during fast motion in animated content. I don't know why.



As I'd mentioned previously, I'm really not a fan of RDOQ Level 1 or 2. I much prefer Level 0 with much higher Psy-rd values instead. I understand why people like it, I just think it's ugly.



--deblock -4:-1 on grainy or dithered stuff, --deblock -3:0 or -2:0 otherwise.



--fades seems to work well.



Default Me range and Star. UMH is better, but it's fractional.



refs 4 is generally speaking adequate IMO.



I keep strong-intra-smoothing on.



Aq strength is difficult. Generally the default or 0.9 is fine, but if the content is super, super grainy, I've gone as low as 0.6 and put the difference into a lower CRF for better results.



So my settings are a bit weird. I've arrived at them from probably thousands of 2 pass tests over a few years, aiming for 2:1 over x264 and no more.
Thank you for very useful information! I'll try selective-sao 1 and tu intra inter 1 (if I recall correctly, it's gives 8-10% more bitrate but 10% faster).
I didn't find on my tests that --strong-intra-smoothing is giving better results, but maybe I was wrong. My target bitrate is 16-18mbps for 1080p, so maybe it's a reason?

Last edited by redbtn; 12th December 2019 at 22:36.
redbtn is offline   Reply With Quote
Old 13th December 2019, 00:35   #7266  |  Link
markiemarcus
Registered User
 
Join Date: May 2018
Posts: 28
Quote:
Originally Posted by redbtn View Post
Thank you for very useful information! I'll try selective-sao 1 and tu intra inter 1 (if I recall correctly, it's gives 8-10% more bitrate but 10% faster).
I didn't find on my tests that --strong-intra-smoothing is giving better results, but maybe I was wrong. My target bitrate is 16-18mbps for 1080p, so maybe it's a reason?
No sweat. I'm only guessing the CRF you're going to need. The results are very sensitive to it; very fine noise requires a low CRF. I'd say that the useful range is CRF 16 to 19. Bear in mind that there's no RDOQ level 2, which immediately knocks around 0.6 off the CRF of any corresponding preset that uses it and another 0.6 from aq-motion. CRF 16 here is more like CRF 17.5 by default.

--crf 16 --preset slow --output-depth 10 --limit-refs 1 --limit-tu 4 --tu-intra-depth 3 --tu-inter-depth 3 --rdoq-level 0 --no-rect --tskip --early-skip --b-intra --qg-size 64 --aq-strength 0.7 --cbqpoffs -1 --crqpoffs -2 --aq-motion --subme 7 --weightb --bframes 6 --rc-lookahead 80 --lookahead-slices 1 --deblock -4:-1 --psy-rd 5 --fades --selective-sao 1 --aq-mode 1

Everybody seems to have their own way of doing these things, this just happens to be mine. It's fast also. There's no reason why Aq-mode 2 wouldn't work if that's what you want, but it is a little less detailed on SDR sources from what I've tested, especially with regards to dither preservation. --qg-size 32 also works, but I prefer the look of 64 and I'd swear it's more detailed in low luma and holds together the deep reds better. --aq-mode 3 also works well if you prefer.

It's worth tuning aq strength. The value depends on the source obviously, but 0.7 is a good a place to start.

RainyDog's suggestion looks really nice and is far better than the default IMO. CRF 19.5 there is about the same size as CRF 16 here. Most of my testing was done on grainy animation, or animation with a lot of dither. They're quite different looking.

Last edited by markiemarcus; 14th December 2019 at 00:50.
markiemarcus is offline   Reply With Quote
Old 13th December 2019, 08:08   #7267  |  Link
RainyDog
Registered User
 
Join Date: May 2009
Posts: 178
Quote:
Originally Posted by redbtn View Post
I often see this also. But I don't understand why they use it. I did tests and noticed that CTU 32 and qg-size 8 is not better than CTU 64 qg-size 32 (and sometimes even worse). And also it seems to me that aq-mode 2 is better than 1. I'm not a professional, it's only my humble opinion.
Ps: I looked at your settings, do you sure that ctu 32 merange 40 is better for 1080p than ctu 64 merange 57?
I've just done another test of CTU 32 vs 64 on my current encode and it's really just splitting hairs at the bitrates I aim for at least. Some parts of the frame might appear slightly better on extreme scrutiny with one then the other but it's generally a wash other than CTU 32 is slighty faster.

Me-range 40 is about the cut off point where you just get diminishing returns for 1080p content at least.

Qg-size is still a bit of ahead scratcher for me. In theory, 8 should yield bigger size encodes with the most detail retention but at the expense of more artifacting. Yet everytime I've tested at the same settings and CRF, qg-size 16 comes out the biggest, size 8 by far the smallest with 32 sat somewhere in between! So I usually just leave it at the default of 32.
RainyDog is offline   Reply With Quote
Old 13th December 2019, 08:14   #7268  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 6,036
@redbtn: Thank you for considering not to quote a several pages long post to reply only briefly. Often a @name is sufficient. Or trim the quote to a relevant part.
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Old 13th December 2019, 08:15   #7269  |  Link
RainyDog
Registered User
 
Join Date: May 2009
Posts: 178
Quote:
Originally Posted by markiemarcus View Post
--selective-sao 1 is nice and now my go-to.
You do realise that all selective sao 1 does is apply SAO to I-frames which are less than 1% of your encode, right?

SAO really needs a strength setting more than anything else. I feel it could be highly useful for all content if it could be tuned.
RainyDog is offline   Reply With Quote
Old 13th December 2019, 09:29   #7270  |  Link
markiemarcus
Registered User
 
Join Date: May 2018
Posts: 28
Quote:
Originally Posted by RainyDog View Post
You do realise that all selective sao 1 does is apply SAO to I-frames which are less than 1% of your encode, right?

SAO really needs a strength setting more than anything else. I feel it could be highly useful for all content if it could be tuned.
Yup! But I agree.
markiemarcus is offline   Reply With Quote
Old 13th December 2019, 09:34   #7271  |  Link
markiemarcus
Registered User
 
Join Date: May 2018
Posts: 28
Quote:
Originally Posted by RainyDog View Post
Qg-size is still a bit of ahead scratcher for me. In theory, 8 should yield bigger size encodes with the most detail retention but at the expense of more artifacting. Yet everytime I've tested at the same settings and CRF, qg-size 16 comes out the biggest, size 8 by far the smallest with 32 sat somewhere in between! So I usually just leave it at the default of 32.
If you restore limit-tu , tu-intra-depth and tu-inter-depth to the default, I suspect things will start behaving like you'd expect with the filesize.
markiemarcus is offline   Reply With Quote
Old 13th December 2019, 11:53   #7272  |  Link
_kermit
Registered User
 
Join Date: Apr 2017
Posts: 24
Quote:
Originally Posted by RainyDog View Post
Sure, the below is always my starting point then I tweak from there if necessary with a dial or two down on --aq-strength and/or a notch or two up on --psy-rd. This is for a broad range of film sources at 1080p resolution.
thanks, I'll give that a try.

Can anyone provide something like that for UHD/HDR?
_kermit is offline   Reply With Quote
Old 13th December 2019, 13:25   #7273  |  Link
RainyDog
Registered User
 
Join Date: May 2009
Posts: 178
Quote:
Originally Posted by _kermit View Post
thanks, I'll give that a try.

Can anyone provide something like that for UHD/HDR?
I use the same template for HDR encodes just with the additional required HDR knobs (hdr-opt etc.) turned on. CRF needs to be bumped down a couple of notches for HDR though ie. CRF 17 for SDR = CRF 15 for HDR.

Again, this is all at 1080p resolution. For UHD all I'd probably change is CTU 32 > 64, me-range 40 > 57 and possibly qg-size 32 > 64.
RainyDog is offline   Reply With Quote
Old 13th December 2019, 13:33   #7274  |  Link
RainyDog
Registered User
 
Join Date: May 2009
Posts: 178
Quote:
Originally Posted by markiemarcus View Post
If you restore limit-tu , tu-intra-depth and tu-inter-depth to the default, I suspect things will start behaving like you'd expect with the filesize.
I'll have a look at that then.

I've never really seen much difference in quality or size between tu-intra-depth inter-depth and limit-tu all set to 4 versus 1, 1 and 0 anyway. I only really use the former as it's a buffed setting with minor performance hit.

I think the gains from increasing tu-intra and inter-depth are probably only really seen without using limit-tu. But the speed decrease is pretty severe then.
RainyDog is offline   Reply With Quote
Old 13th December 2019, 19:04   #7275  |  Link
markiemarcus
Registered User
 
Join Date: May 2018
Posts: 28
Quote:
Originally Posted by RainyDog View Post
I'll have a look at that then.

I've never really seen much difference in quality or size between tu-intra-depth inter-depth and limit-tu all set to 4 versus 1, 1 and 0 anyway. I only really use the former as it's a buffed setting with minor performance hit.

I think the gains from increasing tu-intra and inter-depth are probably only really seen without using limit-tu. But the speed decrease is pretty severe then.
I don't really understand them to be perfectly honest lol. But I have found instances where tu-intra and tu-inter being set to 4, with limit-tu 0, causes grain freezing and huge blocks on flat textures. It spooked me so I've dodged them since.
markiemarcus is offline   Reply With Quote
Old 13th December 2019, 20:57   #7276  |  Link
fauxreaper
Registered User
 
Join Date: Oct 2014
Posts: 17
Without limit-tu, higher values of tu-intra and tu-inter only increase compression, not visual quality. limit-tu 1 + tu-inter 4 + tu-intra 4 should give better quality than using limit-tu 0.
fauxreaper is offline   Reply With Quote
Old 13th December 2019, 21:09   #7277  |  Link
markiemarcus
Registered User
 
Join Date: May 2018
Posts: 28
Quote:
Originally Posted by fauxreaper View Post
Without limit-tu, higher values of tu-intra and tu-inter only increase compression, not visual quality. limit-tu 1 + tu-inter 4 + tu-intra 4 should give better quality than using limit-tu 0.
That makes sense. I've also observed less severe versions of the issue with limit-tu 4 + tu-inter 4 + tu-intra 4. Would limit-tu 1 + tu-inter 4 + tu-intra 4 be higher quality?

Edit: Nope. After some more testing I prefer the results with limit-tu 4 + tu-inter 3 + tu-intra 3.

Last edited by markiemarcus; 13th December 2019 at 23:24.
markiemarcus is offline   Reply With Quote
Old 14th December 2019, 22:47   #7278  |  Link
vpupkind
Registered User
 
Join Date: Jul 2007
Posts: 25
Quote:
Originally Posted by markiemarcus View Post
...

I use rdLevel 4 and almost exclusively --limit-refs 1. If the performance impact weren't so dramatic, I'd use rdLevel 6. It's just better, there's no getting around that, but it is awfully slow.
Try --dynamic-rd, it typically provides a pretty decent quality boost over rd-level=4
vpupkind is offline   Reply With Quote
Old 16th December 2019, 12:31   #7279  |  Link
markiemarcus
Registered User
 
Join Date: May 2018
Posts: 28
Quote:
Originally Posted by vpupkind View Post
Try --dynamic-rd, it typically provides a pretty decent quality boost over rd-level=4
Cheers for the suggestion. How would one go about using this reliably in CRF mode? Seems to rely on VBV.
markiemarcus is offline   Reply With Quote
Old 16th December 2019, 13:23   #7280  |  Link
l00t
Where's my loot?
 
Join Date: May 2019
Posts: 22
Quote:
Originally Posted by vpupkind View Post
Try --dynamic-rd, it typically provides a pretty decent quality boost over rd-level=4
Over? According to the manual it should be helpful at 4 and below...

Quote:
Originally Posted by x265 read the docs
--dynamic-rd <0..4>
Increases the RD level at points where quality drops due to VBV rate control enforcement. The number of CUs for which the RD is reconfigured is determined based on the strength. Strength 1 gives the best FPS, strength 4 gives the best SSIM. Strength 0 switches this feature off. Default: 0.

Effective for RD levels 4 and below.
l00t is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 00:14.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, vBulletin Solutions Inc.