Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > High Efficiency Video Coding (HEVC)

Reply
 
Thread Tools Search this Thread Display Modes
Old 2nd August 2019, 11:54   #6981  |  Link
nevcairiel
Registered Developer
 
Join Date: Mar 2010
Location: Hamburg/Germany
Posts: 9,788
The concept of "tip" in mercurial is flawed anyway, would never rely on it. Someone pushed an experimental half-broken branch, and suddenly thats the tip? Clearly not what everyone wants to build off of. The "most recent change" seems like a particular worthless point of reference to use.
You should always be aware of what branch you build from.
__________________
LAV Filters - open source ffmpeg based media splitter and decoders
nevcairiel is offline   Reply With Quote
Old 3rd August 2019, 21:44   #6982  |  Link
mandarinka
Registered User
 
mandarinka's Avatar
 
Join Date: Jan 2007
Posts: 734
Quote:
Originally Posted by Boulder View Post
I tested HME some more, and got quite interesting results.

Source 1080p filtered with my standard methods and downsized to 720p, merange 26 and my standard settings in x265. 1000 frames.

umh,umh,umh 2.05 fps / 5313,76 kbps
umh,umh,star 2.24 fps / 5314,31 kbps
umh,star,star 2.00 fps / 5327,91 kbps
star,star,star 2.01 fps / 5316,15 kbps
hex,umh,umh 2.03 fps / 5299,69 kbps

no HME, umh 2.64 fps / 5323,86 kbps
no HME, star 2.79 fps / 5325,69 kbps

I'll make the same tests with some different clip to see if the difference simply occurs because of the source.
I tested HME (with umh/umh/umh, merange 92, against umh with same merange) on some cel anime bluray footage with motion (both full camera motion and objects in scene only motion).

I got notably decreased SSIM score with HME, so be careful, at these low resolutions it looks like it is possible that HME doesn't help but harms.
Why? The first ME pass with merange 92 is done on 4X downscale. So if you have 1080p resolution, you are down on 240p. It is likely that small objects won't get tracked properly at this stage. After that, the second (480p) and last (1080p) ME stage only only lower merange (search distance), I assume? So if the large downscaling fools the first pass, the second and third stage might not be able to correct it.

My footage was 1440x1080 and you would have even worse situation at 720p. Basically this factor may cause HME to be counter productive unless you use it on 4K for which ts was intended (IIRC).

Code:
"n:\x265-3.1+8-21db162c8622.exe" - --input-depth 8 --input-res 1440x1080 --fps 24000/1001 --preset slower --output-depth 10 --ctu 32 --max-tu-size 16 
--pass 2 --bitrate 10000 --tune ssim --ssim --tu-intra-depth 2 --tu-inter-depth 2 --rdpenalty 2 --me star --subme 7 --merange 92 --amp --rect --ref 6 
--weightb --weightp --keyint 300 --min-keyint 1 --bframes 8 --rd 5 --no-sao --no-open-gop --rc-lookahead 80 --max-merge 5 --qcomp 0.7 
--no-strong-intra-smoothing --no-limit-modes --limit-refs 0 --limit-tu 0 --frame-threads 2 --wpp --deblock -2:-2 --qg-size 8 --pbratio 1.2 --no-cutree --cu-lossless 
--lookahead-slices 2 --sar 1:1 --range limited --chromaloc 0 --colormatrix bt709 --no-rskip --rd-refine --cbqpoffs -2 --crqpoffs -2

encoded 853 frames in 5108.02s (0.17 fps), 10064.24 kb/s, Avg QP:19.49, SSIM Mean Y: 0.9820175 (17.451 dB)


"n:\x265-3.1+8-21db162c8622.exe" - --input-depth 8 --input-res 1440x1080 --fps 24000/1001 --preset slower --output-depth 10 --ctu 32 --max-tu-size 16 
--pass 2 --bitrate 10000 --tune ssim --ssim --tu-intra-depth 2 --tu-inter-depth 2 --rdpenalty 2 --me umh --subme 7 --merange 92 --amp --rect --ref 6 
--weightb --weightp --keyint 300 --min-keyint 1 --bframes 8 --rd 5 --no-sao --no-open-gop --rc-lookahead 80 --max-merge 5 --qcomp 0.7 
--no-strong-intra-smoothing --no-limit-modes --limit-refs 0 --limit-tu 0 --frame-threads 2 --wpp --deblock -2:-2 --qg-size 8 --pbratio 1.2 --no-cutree --cu-lossless 
--lookahead-slices 2 --sar 1:1 --range limited --chromaloc 0 --colormatrix bt709 --no-rskip --rd-refine --cbqpoffs -2 --crqpoffs -2

encoded 853 frames in 11135.11s (0.08 fps), 10064.75 kb/s, Avg QP:19.49, SSIM Mean Y: 0.9820200 (17.452 dB)

"n:\x265-3.1+8-21db162c8622.exe" - --input-depth 8 --input-res 1440x1080 --fps 24000/1001 --preset slower --output-depth 10 --ctu 32 --max-tu-size 16 
--pass 2 --hme --hme-search umh,umh,umh --bitrate 10000 --tune ssim --ssim --tu-intra-depth 2 --tu-inter-depth 2 --rdpenalty 2 --me umh --subme 7 
--merange 92 --amp --rect --ref 6 --weightb --weightp --keyint 300 --min-keyint 1 --bframes 8 --rd 5 --no-sao --no-open-gop --rc-lookahead 80 --max-merge 5 
--qcomp 0.7 --no-strong-intra-smoothing --no-limit-modes --limit-refs 0 --limit-tu 0 --frame-threads 2 --wpp --deblock -2:-2 --qg-size 8 --pbratio 1.2 --no-cutree 
--cu-lossless --lookahead-slices 2 --sar 1:1 --range limited --chromaloc 0 --colormatrix bt709 --no-rskip --rd-refine --cbqpoffs -2 --crqpoffs -2

encoded 853 frames in 7319.99s (0.12 fps), 10051.98 kb/s, Avg QP:19.52, SSIM Mean Y: 0.9819607 (17.438 dB)

p.s. Ignore the fps values, they are not indicative of the speed due to background cpu usage, system sleeping for a time and so on.
BTW, it would be interesting to determine what is the speed hit of a particular HME setting and then determine how much do you need to increase merange for the normal search mode to get at the same FPS - and then comapre the quality/SSIM. Because obvioously the added computation time could be used to improve quality by just increasing merange too, so that is your reference point for comparisons.

Edit: one other thing. Doing two-pass testing smaples, I found that just switching ME method between star and umh changes content of each frame and block quite a bit due to various indeterministic things going on. Individual frames would look worse and better based on how the rate control stars aligned.
So don't compare the results just by bitrate they give you!
It is quite possible the two files are not the exactly same quality, which would skew your judgement completely.

Last edited by mandarinka; 4th August 2019 at 02:36.
mandarinka is offline   Reply With Quote
Old 4th August 2019, 09:05   #6983  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Hollola, Finland
Posts: 4,589
Isn't it the other way around so that the first pass is made at full resolution and then the result is used as a predictor for the next pass which is done after downscaling. I agree that small details could well be lost when downscaling. The code seems to set the limit to use HME at 540 pixels of vertical resolution.

What I was wondering with those results of mine was that umh,umh,star was noticably faster than any other combination.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 8th August 2019, 21:46   #6984  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 2,951
Is there a build which includes the --aq-mode 4 patch? I've tried the latest ones here, and I keep getting:
Code:
C:\Users\benwagg\Desktop\x265>C:\Users\benwagg\Desktop\x265\x265.exe --input D:\Rights_Cleared_Sources\TearsofSteel_1920x800p24.y4m --preset slower --crf 28 --aq-mode 4 -F 1 --min-keyint 72 --keyint 120 --no-open-gop D:\TearsOfSteel\ToS_aqm4_crf28.hevc
y4m  [info]: 1920x800 fps 24/1 i420p8 frames 0 - 17619 of 17620
raw  [info]: output file: D:\TearsOfSteel\ToS_aqm4_crf28.hevc
x265 [info]: HEVC encoder version 3.1.2+1-76650bab70f9
x265 [info]: build info [Windows][GCC 9.1.0][64 bit] 8bit+10bit+12bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast LZCNT SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
x265 [error]: Aq-Mode is out of range
x265 [error]: failed to open encoder
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book

Last edited by benwaggoner; 8th August 2019 at 21:53. Reason: spelling
benwaggoner is offline   Reply With Quote
Old 8th August 2019, 21:55   #6985  |  Link
mandarinka
Registered User
 
mandarinka's Avatar
 
Join Date: Jan 2007
Posts: 734
I think https://forum.doom9.org/showpost.php...postcount=6935 worked for me.
mandarinka is offline   Reply With Quote
Old 9th August 2019, 11:27   #6986  |  Link
pradeeprama
Registered User
 
Join Date: Sep 2015
Posts: 48
Quote:
Originally Posted by Wolfberry View Post
The new aq-mode and HME is only present in the default branch (3.1)

3.1.1 and 3.1.2 from the Release_3.1 branch only include some bug fixes that are also on the default branch.

x265-3.1+11-de920e0-win64-static-multilib

Code:
x265 [info]: HEVC encoder version 3.1+11-de920e0a3183
x265 [info]: build info [Windows][GCC 9.1.1][64 bit] 8bit+10bit+12bit
x265 [info]: (libavcodec  58.55.100)
x265 [info]: (libavformat 58.30.100)
x265 [info]: (libavutil   56.33.100)
x265 [info]: (lsmash       2.16.1)
These features will be part of the next release, which will have its own branch. Only bug fixes for v3.1 will go into the Release_3.1 branch
pradeeprama is offline   Reply With Quote
Old 9th August 2019, 16:34   #6987  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 2,951
Quote:
Originally Posted by Wolfberry View Post
The new aq-mode and HME is only present in the default branch (3.1)

3.1.1 and 3.1.2 from the Release_3.1 branch only include some bug fixes that are also on the default branch.

x265-3.1+11-de920e0-win64-static-multilib
It worked! I'm going to knock out some aq-mode 2 versus 4 tests with Tears of Steel.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 14th August 2019, 09:20   #6988  |  Link
jlpsvk
Registered User
 
Join Date: Dec 2014
Posts: 190
@benwaggoner
any thought about aq-mode 2 vs aq-mode 4?
__________________
Core i9-7960X, 64GB DDR4, RTX 2070, 1TB NVMe SSD, 56TB NAS
jlpsvk is offline   Reply With Quote
Old 14th August 2019, 17:37   #6989  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 2,951
Quote:
Originally Posted by jlpsvk View Post
@benwaggoner
any thought about aq-mode 2 vs aq-mode 4?
CPUs have been too busy doing other things! But I'll kick them off now.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 14th August 2019, 20:53   #6990  |  Link
brumsky
Registered User
 
Join Date: Jun 2016
Posts: 112
Quote:
Originally Posted by benwaggoner View Post
CPUs have been too busy doing other things! But I'll kick them off now.
I'm looking forward to your results as well. I tested hevc-qa and it was bad... I had weird banding and everything was smoothed out really bad.
brumsky is offline   Reply With Quote
Old 14th August 2019, 23:53   #6991  |  Link
markiemarcus
Registered User
 
Join Date: May 2018
Posts: 4
Quote:
Originally Posted by brumsky View Post
I'm looking forward to your results as well. I tested hevc-qa and it was bad... I had weird banding and everything was smoothed out really bad.
I've had positive results with HEVC-AQ on animation; in part due to it taking AQ strength out of the equation.

Live action not so much.
markiemarcus is offline   Reply With Quote
Old 15th August 2019, 00:34   #6992  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 2,951
Quote:
Originally Posted by markiemarcus View Post
I've had positive results with HEVC-AQ on animation; in part due to it taking AQ strength out of the equation.

Live action not so much.
I'm running a 2-pass veryslow Tears of Steel encodes at 1 Mbps ABR comparing aq-mode 2, 3, 4 and aq-hevc. I should have some clips to evaluate tomorrow.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 15th August 2019, 01:23   #6993  |  Link
markiemarcus
Registered User
 
Join Date: May 2018
Posts: 4
Quote:
Originally Posted by benwaggoner View Post
I'm running a 2-pass veryslow Tears of Steel encodes at 1 Mbps ABR comparing aq-mode 2, 3, 4 and aq-hevc. I should have some clips to evaluate tomorrow.
It has actually surprised me how much variation there is between them. I've only really tested extensively on animation, often at 720p where the problems are more noticeable. The trouble really starts when you disable SAO, but you often have to in order to preserve high frequency detail. The artifacts resemble a mixture of ringing and mosquito noise around dark lines.

Under these circumstances Aq mode 1 is by far the most prone to distortion and though it's subjectively more detailed, I don't like it, unless the source has a lot of grain where it can be quite useful. Aq mode 2 is much less prone to distortion, but grain can look soft and it's noticeably poor in low luma. Aq mode 3 generally works well, but it's often a sub optimal usage of bits, especially if only fleeting scenes are low luma. That's usually where Hevc-aq does rather well (though Cbq and Crq need a -1 nudge down). I find it to be the most predictable and least troublesome in motion.

Aq mode 4 I don't really know what to make of. With grainy animation it seems a little more detailed than Aq mode 2. Just to clarify again that the above is all for animation; I haven't looked at the metrics and I don't have much experience with live action.

Looking forward to your results! I'm a long time lurker.
markiemarcus is offline   Reply With Quote
Old 15th August 2019, 10:11   #6994  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Hollola, Finland
Posts: 4,589
Quote:
Originally Posted by benwaggoner View Post
I'm running a 2-pass veryslow Tears of Steel encodes at 1 Mbps ABR comparing aq-mode 2, 3, 4 and aq-hevc. I should have some clips to evaluate tomorrow.
1 Mbps is quite low already, it would be nice to see some results with average bitrate around 7-8 Mbps or so.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 15th August 2019, 21:47   #6995  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 2,951
Quote:
Originally Posted by Boulder View Post
1 Mbps is quite low already, it would be nice to see some results with average bitrate around 7-8 Mbps or so.
I'd expect there's be a lot of convergence at high bitrates. I'm most interested in how the different modes do when they don't have enough bits to do it right.

I'm also replicating the procedure from my encoding challenge, for apples-to-apples

https://forum.doom9.org/showthread.php?t=175776
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 17th August 2019, 00:34   #6996  |  Link
tebugg
Registered User
 
Join Date: Jul 2010
Posts: 14
3900x cpu usage

hello everyone. i am seeking a little bit of help with cpu usage. i have an amd ryzen 3900x overclocked to 4200mhz. it is on an msi x570 godlike mobo. so the vrm's are the best. i currently can only get between 70-80% cpu usage on the first pass. on the second pass it drops down to 60% usage. this is encoding a 4K video file that was originally encoded with x265 from my cell phone. is there a way for me to squeeze out more usage from x265's settings?

edit: on crf the encoder maxes out all the cores. also on x264 with 2pass and crf both max out the cores.

Last edited by tebugg; 17th August 2019 at 01:23.
tebugg is offline   Reply With Quote
Old 18th August 2019, 16:04   #6997  |  Link
Forteen88
Herr
 
Join Date: Apr 2009
Location: North Europe
Posts: 366
Quote:
Originally Posted by tebugg View Post
i currently can only get between 70-80% cpu usage on the first pass. on the second pass it drops down to 60% usage..
If you can, then you should do 2 encodes at the same time.
Forteen88 is offline   Reply With Quote
Old 20th August 2019, 06:30   #6998  |  Link
LazyNcoder
Registered User
 
Join Date: Feb 2015
Posts: 16
Hello guys,
Any specific commands to encode HDR10+?
--transfer smpte2094 instead of --transfer smpte2084? is it enough? Does it work at all? because I couldn't find smpte2094 on x265 documentations ...

Edit:
Nope.
x265 [error]: invalid argument: transfer = smpte2094
Is there something wrong with my build?

Last edited by LazyNcoder; 20th August 2019 at 06:37.
LazyNcoder is offline   Reply With Quote
Old 20th August 2019, 07:46   #6999  |  Link
kabelbrand
Compression mode: Lousy
 
kabelbrand's Avatar
 
Join Date: Mar 2009
Location: Hamburg, Germany
Posts: 73
Quote:
Originally Posted by LazyNcoder View Post
Any specific commands to encode HDR10+?
There is a command line switch to insert HDR10+ metadata but you'll need a separate tool e.g. from Samsung to process your source file for metadata creation.
https://x265.readthedocs.io/en/defau...on-dhdr10-info
kabelbrand is offline   Reply With Quote
Old 20th August 2019, 08:15   #7000  |  Link
LazyNcoder
Registered User
 
Join Date: Feb 2015
Posts: 16
Quote:
Originally Posted by kabelbrand View Post
There is a command line switch to insert HDR10+ metadata but you'll need a separate tool e.g. from Samsung to process your source file for metadata creation.
https://x265.readthedocs.io/en/defau...on-dhdr10-info
Thank you.

1- What if the source we already have is HDR10+? Can we somehow extract or bypass the metadata into x265?

2- What tool from Samsung exactly? or any alternative
LazyNcoder is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 07:05.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.