Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > High Efficiency Video Coding (HEVC)

Reply
 
Thread Tools Search this Thread Display Modes
Old 17th August 2018, 11:58   #6281  |  Link
zub35
Registered User
 
Join Date: Oct 2016
Posts: 56
x264 has a good optimization option
--tune film [--deblock -1:-1 --psy-rd <unset>:0.15]

why not have the same for x265 ?
--tune film [--no-sao --no-strong-intra-smoothing --psy-rd 4]

Last edited by zub35; 17th August 2018 at 12:00.
zub35 is offline   Reply With Quote
Old 17th August 2018, 14:09   #6282  |  Link
RieGo
Registered User
 
Join Date: Nov 2009
Posts: 59
Quote:
Originally Posted by zub35 View Post
x264 has a good optimization option
--tune film [--deblock -1:-1 --psy-rd <unset>:0.15]

why not have the same for x265 ?
--tune film [--no-sao --no-strong-intra-smoothing --psy-rd 4]
afaik a film preset is on the todo list... probably may take a while till they/we figure out all sane parameters.

now... why does everyone think it's a good idea to switch off Sample Adaptive Offset in-loop filter? i read about it and it sounds like a nice feature to improve efficiency - no matter what kind of video content is encoded.
i understand that there was supposely a little problem in the early stages of x265 with sao integration. but is this still a thing or is everybody just blindly turning off sao?
RieGo is offline   Reply With Quote
Old 17th August 2018, 14:36   #6283  |  Link
microchip8
ffx264/ffhevc author
 
microchip8's Avatar
 
Join Date: May 2007
Location: /dev/video0
Posts: 1,843
Quote:
Originally Posted by RieGo View Post
afaik a film preset is on the todo list... probably may take a while till they/we figure out all sane parameters.

now... why does everyone think it's a good idea to switch off Sample Adaptive Offset in-loop filter? i read about it and it sounds like a nice feature to improve efficiency - no matter what kind of video content is encoded.
i understand that there was supposely a little problem in the early stages of x265 with sao integration. but is this still a thing or is everybody just blindly turning off sao?
SAO still blurs too much so many people disable it if they want to retain as much details as possible. However, at very low bitrates where other artifacts are more visible/present, the blur of SAO produces "better looking" images than an encode without it
__________________
ffx264 || ffhevc || ffxvid || microenc
microchip8 is offline   Reply With Quote
Old 17th August 2018, 16:51   #6284  |  Link
RieGo
Registered User
 
Join Date: Nov 2009
Posts: 59
Quote:
Originally Posted by froggy1 View Post
SAO still blurs too much so many people disable it if they want to retain as much details as possible. However, at very low bitrates where other artifacts are more visible/present, the blur of SAO produces "better looking" images than an encode without it
thanks.
i did some visual comparisons lately but wasn't able to detect any kind of differences at high bitrate - i didn't look at still images, only at video scenes.
at very low bitrate (300kbit/s) there was a lot of quality differences with different parameters, but I didn't look at no-sao...

so probably i'm just a bad quality judge.
RieGo is offline   Reply With Quote
Old 17th August 2018, 17:58   #6285  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,750
Quote:
Originally Posted by RieGo View Post
thanks.
i did some visual comparisons lately but wasn't able to detect any kind of differences at high bitrate - i didn't look at still images, only at video scenes.
at very low bitrate (300kbit/s) there was a lot of quality differences with different parameters, but I didn't look at no-sao...

so probably i'm just a bad quality judge.
SAO should do less as QP goes down, so what you see is how it should work.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 18th August 2018, 16:56   #6286  |  Link
NikosD
Registered User
 
Join Date: Aug 2010
Location: Athens, Greece
Posts: 2,901
Quote:
Originally Posted by Atak_Snajpera View Post
Can somebody verify if those are correct switches?
Code:
Instance 1 = --numa-pools "+,-,-,-" 
Instance 2 = --numa-pools "-,+,-,-"
Instance 3 = --numa-pools "-,-,+,-"
Instance 4 = --numa-pools "-,-,-,+"
Quote:
Originally Posted by Atak_Snajpera View Post
Can somebody verify than I'm setting numa pools correctly in my previous post?
Please, don't expect answers regarding AMD optimizations in this thread.

They are all Intel fans or worse fanboys.

Even the developers.
__________________
Win 10 x64 (19042.572) - Core i5-2400 - Radeon RX 470 (20.10.1)
HEVC decoding benchmarks
H.264 DXVA Benchmarks for all
NikosD is offline   Reply With Quote
Old 18th August 2018, 17:28   #6287  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 6,753
In general, generalizations are wrong. If I could afford a new PC, I would buy a Ryzen. But I could still not buy the insight in its NUMA structure.
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Old 18th August 2018, 20:32   #6288  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 2,883
Quote:
Originally Posted by Sagittaire View Post
If you want really saturate 64 thread CPU, you must use at least 2 instance for 2160p source.
Not just 64 thread CPU, at work I have two Intel Xeon E5-2660V4 14c/28th for a total of 28c/56th and I can't still saturate both CPUs with a 2160p 10bit HDR10 content encoded with preset --medium and bluray compatible specs.


Quote:
Originally Posted by NikosD View Post
They are all Intel fans or worse fanboys.
Some consumers are moving to AMD, but the majority of businesses are using Intel Xeon CPUs (my company included), so that's what they ask for optimizations.
They are simply following the market needs, nothing more.
FranceBB is offline   Reply With Quote
Old 20th August 2018, 16:15   #6289  |  Link
Barough
Registered User
 
Barough's Avatar
 
Join Date: Feb 2007
Location: Sweden
Posts: 480
x265 v2.8+66-88ee12651e30 (32 & 64-bit 8/10/12bit Multilib Windows Binaries)

Code:
https://bitbucket.org/multicoreware/x265/commits/branch/default

Last edited by Barough; 20th August 2018 at 16:25.
Barough is offline   Reply With Quote
Old 21st August 2018, 00:49   #6290  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,750
Quote:
Originally Posted by FranceBB View Post
Not just 64 thread CPU, at work I have two Intel Xeon E5-2660V4 14c/28th for a total of 28c/56th and I can't still saturate both CPUs with a 2160p 10bit HDR10 content encoded with preset --medium and bluray compatible specs.
That's not surprising. Something like --preset slower would probably be better, but there's only so much threading that can be usefully done in a single instance, and Blu-ray restrictions reduce even that (b-frames can encode in parallel, but BD only allows 2 consecutive). Increasing -F will help, but high values can cause rate control issues.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 21st August 2018, 02:35   #6291  |  Link
jlpsvk
Registered User
 
Join Date: Dec 2014
Posts: 240
Quote:
Originally Posted by FranceBB View Post
Not just 64 thread CPU, at work I have two Intel Xeon E5-2660V4 14c/28th for a total of 28c/56th and I can't still saturate both CPUs with a 2160p 10bit HDR10 content encoded with preset --medium and bluray compatible specs.
could you post your uhd bd compatible command line?
__________________
AMD Ryzen 9 5950X, 32GB DDR4-3200 CL16, RTX 3060, 2TB NVMe PCIE4.0, NAS with 8x16TB HDD
jlpsvk is offline   Reply With Quote
Old 21st August 2018, 15:22   #6292  |  Link
excellentswordfight
Lost my old account :(
 
Join Date: Jul 2017
Posts: 322
Quote:
Originally Posted by RieGo View Post
afaik a film preset is on the todo list... probably may take a while till they/we figure out all sane parameters.

now... why does everyone think it's a good idea to switch off Sample Adaptive Offset in-loop filter? i read about it and it sounds like a nice feature to improve efficiency - no matter what kind of video content is encoded.
i understand that there was supposely a little problem in the early stages of x265 with sao integration. but is this still a thing or is everybody just blindly turning off sao?
Using --no-sao for a tune film is imo valid. In my experience no-sao does improve fine detail alot with almost no negative effects for general "film" content with lower crf values. Preset slow together with no-sao is imo enough for detail retention now days. Not sure what setting does it, but I find preset Medium to be way softer then preset slow (imo there should only be a bitrate difference between them when doing a CRF encode, but it doesnt work like that I guess).

I have found sao to be usefull for both animation and low bitrate content though (as expected).

Quote:
Originally Posted by FranceBB View Post
Not just 64 thread CPU, at work I have two Intel Xeon E5-2660V4 14c/28th for a total of 28c/56th and I can't still saturate both CPUs with a 2160p 10bit HDR10 content encoded with preset --medium and bluray compatible specs.
To add to this, I see around 70-80% utilization on dual Xeon E5-2680 v3 (48t) systems for 2160p content using preset slow. Imo that is a very reasonable ammount of multithread performance. For 1080p I wouldnt bother with anything more then 8-12C. Start using chunk-encoding if better multithread utilization is needed.

But I still think Atak question is valid, does 2990wx need any NUMA tweaking to perform correctly?

Last edited by excellentswordfight; 21st August 2018 at 15:42.
excellentswordfight is offline   Reply With Quote
Old 21st August 2018, 16:27   #6293  |  Link
RieGo
Registered User
 
Join Date: Nov 2009
Posts: 59
Quote:
Originally Posted by excellentswordfight View Post
Using --no-sao for a tune film is imo valid. In my experience no-sao does improve fine detail alot with almost no negative effects for general "film" content with lower crf values. Preset slow together with no-sao is imo enough for detail retention now days. Not sure what setting does it, but I find preset Medium to be way softer then preset slow (imo there should only be a bitrate difference between them when doing a CRF encode, but it doesnt work like that I guess).

I have found sao to be usefull for both animation and low bitrate content though (as expected).
thanks for your opinion
i will do some more visual tests with high and low bitrates and only film content. maybe i can finally understand all your motivation to turn off sao.

update: wow.
so I did a quick test @6000/1000/100 kbit/s with and without sao.
so you are 100% right. no-sao looks just much sharper and retains more details. even I can see it...
not just true on high bitrate but also on medium/low bitrate. but I can understand why it would make kinda sense to have a smooth low bitrate encoding.
so basically i'm sorry for not believing, need to change my presets now. lol

Last edited by RieGo; 21st August 2018 at 17:56.
RieGo is offline   Reply With Quote
Old 21st August 2018, 21:51   #6294  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,750
Quote:
Originally Posted by RieGo View Post
update: wow.
so I did a quick test @6000/1000/100 kbit/s with and without sao.
so you are 100% right. no-sao looks just much sharper and retains more details. even I can see it...
not just true on high bitrate but also on medium/low bitrate. but I can understand why it would make kinda sense to have a smooth low bitrate encoding.
so basically i'm sorry for not believing, need to change my presets now. lol
Can you share the bitrates and/or command lines you were using?
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 22nd August 2018, 03:01   #6295  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 2,883
Quote:
Originally Posted by benwaggoner View Post
That's not surprising. Something like --preset slower would probably be better, but there's only so much threading that can be usefully done in a single instance, and Blu-ray restrictions reduce even that (b-frames can encode in parallel, but BD only allows 2 consecutive). Increasing -F will help, but high values can cause rate control issues.
Yes... With slower I might get something more but still, it's still acceptable.

Quote:
Originally Posted by excellentswordfight View Post
I see around 70-80% utilization on dual Xeon E5-2680 v3 (48t) systems for 2160p content using preset slow. Imo that is a very reasonable ammount of multithread performance.
Yes, it kinda is.

Quote:
Originally Posted by jlpsvk View Post
could you post your uhd bd compatible command line?
Sure.

UHD HDR10 BD50:
Code:
x265.exe --y4m - --dither --preset medium --level 5.1 --tune fastdecode --no-high-tier --ref 4 --profile main10 --bitrate 75000 --deblock -1:-1 --hdr-opt --hrd --min-luma 64 --max-luma 940 --chromaloc 2 --range limited --videoformat component --colorprim bt2020 --transfer smpte2084 --colormatrix bt2020nc --master-display "G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(10000000,0.0050)" --max-cll 1000,400 --overscan show --no-open-gop --min-keyint 1 --keyint 24 --repeat-headers --rd 3 --vbv-maxrate 75000 --vbv-bufsize 75000 --asm=avx2  --wpp -o "H:\raw_video.hevc"
I know that some of you might be screaming "Ah!!" to that "brutal" clipping, but the uncompressed 16bit stream that x265 is gonna encode has already been brought in Tv Range with a proper LUT that tries to avoid to clip too much, so that's just for "safety reasons", especially 'cause otherwise QC refuses it, even if there's just a single scene in the video that is out of range. No, they don't stare at the video-scope all the time; such a process is automated by a machine that checks the file 1:1 and reports details about luma, chroma, whether there are freeze-frames, blocking of whatever type and so on. Sometimes it fails and it is spot-checked by a human, but still, they refuse the content if it's out of range.


Still, using --preset slow might help a bit, but I would have to specify parameters myself, especially 'cause I'm not using --uhd-bd and I would end up by limiting myself anyway.
Increasing --ref from 4 to 6 might also help.
As to the 75Mbit/s, the specs require the bitrate to stay below 82Mbit/s, but 75Mbit/s plus a bit of oscillation up and down and audio tracks is gonna be fine.

Last edited by FranceBB; 22nd August 2018 at 03:08.
FranceBB is offline   Reply With Quote
Old 22nd August 2018, 09:30   #6296  |  Link
RieGo
Registered User
 
Join Date: Nov 2009
Posts: 59
Quote:
Originally Posted by benwaggoner View Post
Can you share the bitrates and/or command lines you were using?
nothing fancy, just a simple "--pass x --bitrate 6000 --preset slow --pmode [--no-sao]"
RieGo is offline   Reply With Quote
Old 22nd August 2018, 09:49   #6297  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 6,753
Was --pmode useful in your case? It does not cause a speedup in general, it depends on the circumstances, I read...
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Old 22nd August 2018, 14:53   #6298  |  Link
RieGo
Registered User
 
Join Date: Nov 2009
Posts: 59
Quote:
Originally Posted by LigH View Post
Was --pmode useful in your case? It does not cause a speedup in general, it depends on the circumstances, I read...
i *think* it improves my cpu saturation. but only on 1080p or lower using 24 threads. wasn't able to get a constant saturation without it.
with 4k content everything is fine even without pmode.
i didn't really make any extensive speed tests though.

Last edited by RieGo; 22nd August 2018 at 15:00.
RieGo is offline   Reply With Quote
Old 22nd August 2018, 21:55   #6299  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,750
Quote:
Originally Posted by RieGo View Post
i *think* it improves my cpu saturation. but only on 1080p or lower using 24 threads. wasn't able to get a constant saturation without it.
with 4k content everything is fine even without pmode.
i didn't really make any extensive speed tests though.
Pmode can easily increase CPU utilization AND reduce encoding speed if you don’t have a whole lot of unused cores when running without it. I’ve seen it speed up encoding 400x224 on a 32 logical core system, but never 1080p or above. But I’ve not tried on anything with >36 logical cores.

Pmode can also theoretically increase quality a bit, since a lot of its parallel work is stuff that would normally have gotten skipped due to early exit. Occasionally it’ll find something better than what was found before the early exit. I’ve never seen it really make a material difference compared to veryslow or placebo.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 22nd August 2018, 22:32   #6300  |  Link
RieGo
Registered User
 
Join Date: Nov 2009
Posts: 59
Quote:
Originally Posted by benwaggoner View Post
Pmode can easily increase CPU utilization AND reduce encoding speed if you don’t have a whole lot of unused cores when running without it. I’ve seen it speed up encoding 400x224 on a 32 logical core system, but never 1080p or above. But I’ve not tried on anything with >36 logical cores.

Pmode can also theoretically increase quality a bit, since a lot of its parallel work is stuff that would normally have gotten skipped due to early exit. Occasionally it’ll find something better than what was found before the early exit. I’ve never seen it really make a material difference compared to veryslow or placebo.
yes my feeling was that it might be slower with pmode, but as i said i never actually did speed tests, just looked at cpu usage lol. my bad...
maybe it's a good idea for me to just remove it.
but going to slower is not an option (for me) - slow -> slower almost increases encoding time 100%

Last edited by RieGo; 22nd August 2018 at 22:34.
RieGo is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 18:15.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.