Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > High Efficiency Video Coding (HEVC)

Reply
 
Thread Tools Search this Thread Display Modes
Old 18th April 2018, 19:48   #6041  |  Link
jlpsvk
Registered User
 
Join Date: Dec 2014
Posts: 240
Quote:
Originally Posted by Bhavnahari View Post
If the data that is being reused comes from encoding a downscaled video (scale-factor=2),
it does not make a lot of sense to use intra/inter refinement levels 0/1 as levels 0 and 1 reuse most of the information from
the previous pass with no or minimal re-evaluation of analysis information. This can have a huge impact on the quality of the encode.

We have observed that --refine-inter=3 and --refine-intra=4 gives the best quality, even better than standalone x265 encodes in some cases,
with a performance gain of up to 1.8X.
My settings then...

Code:
--crf 17 --profile main10 --level-idc 5.1 --output-depth 10 --ctu 32 --amp --vbv-bufsize 160000 --vbv-maxrate 160000 --me star
--max-merge 5 --rc-lookahead 40 --lookahead-slices 4 --gop-lookahead 34 --ref 5 --hdr --hdr-opt --repeat-headers --no-info --no-deblock
--no-sao --no-strong-intra-smoothing --high-tier --refine-inter 3 --refine-intra 4
Of course, display settings are entered too...

Warning:
Code:
x265 [warning]: Intra refinement requires analysis load, analysis-reuse-level 10, scale factor. Disabling intra refine.
x265 [warning]: Inter refinement requires analysis load, analysis-reuse-level 10, scale factor. Disabling inter refine.
Can it be used with CRF? Or? Some parameters missing?

Last edited by jlpsvk; 18th April 2018 at 20:10.
jlpsvk is offline   Reply With Quote
Old 18th April 2018, 21:47   #6042  |  Link
Loomes
Registered User
 
Join Date: Nov 2003
Location: Germany, Berlin
Posts: 48
So far no success for me running the avx512 option on my 7820x on Windows 10 64it. I tried
Code:
ffmpeg -i source.mkv -f yuv4mpegpipe - | x265.exe --preset slow --asm avx512 --crf 21 --y4m - -o dest.h265
which crashes immediately. Using --asm avx2 works fine.

I ran
Code:
x265.exe -V && x265.exe --asm avx2 -V && x265.exe --asm avx512 -V
and it showed
Code:
x265 [info]: HEVC encoder version 2.7+340-aa9102400f24
x265 [info]: build info [Windows][MSVC 1900][64 bit] 10bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast LZCNT SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
x265 [info]: HEVC encoder version 2.7+340-aa9102400f24
x265 [info]: build info [Windows][MSVC 1900][64 bit] 10bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast LZCNT SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
x265 [info]: HEVC encoder version 2.7+340-aa9102400f24
x265 [info]: build info [Windows][MSVC 1900][64 bit] 10bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast LZCNT SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2 AVX512
Quote:
Originally Posted by Ma View Post
You can test file avx512-patch.7z with x265 that check what is supported by CPU & OS up to avx512 if option '--asm avx512' is used.
I tried from that package x265.exe, x265-1.exe and x265-2.exe. All of them crashed immediately. I am using the latest nightly version of ffmpeg.
Am I missing something?

Last edited by Loomes; 18th April 2018 at 22:16.
Loomes is offline   Reply With Quote
Old 18th April 2018, 22:45   #6043  |  Link
Asmodian
Registered User
 
Join Date: Feb 2002
Location: San Jose, California
Posts: 4,406
Quote:
Originally Posted by Loomes View Post
I tried from that package x265.exe, x265-1.exe and x265-2.exe. All of them crashed immediately. I am using the latest nightly version of ffmpeg.
Am I missing something?
I had crashes using the builds foxyshadis posted but I was successful using LigH's build from this post. Win10 and an i9-7900X. Have you tried them?
__________________
madVR options explained
Asmodian is offline   Reply With Quote
Old 18th April 2018, 22:57   #6044  |  Link
Loomes
Registered User
 
Join Date: Nov 2003
Location: Germany, Berlin
Posts: 48
Quote:
Originally Posted by Asmodian View Post
I had crashes using the builds foxyshadis posted but I was successful using LigH's build from this post. Win10 and an i9-7900X. Have you tried them?
I did now but it also results in an instant crash. Are you using a ffmpeg pipe like me or what is your command line?
Loomes is offline   Reply With Quote
Old 18th April 2018, 23:22   #6045  |  Link
Ma
Registered User
 
Join Date: Feb 2015
Posts: 326
Quote:
Originally Posted by Loomes View Post
I tried from that package x265.exe, x265-1.exe and x265-2.exe. All of them crashed immediately. I am using the latest nightly version of ffmpeg.
Am I missing something?
x265-1.exe == x265-2.exe == x265.exe from package avx512-bisect1.7z (I commented out code for 8-bit encoding and compiled 10-bit version, sorry for that). So this package is to forget.

I want to precise: x265.exe from package avx512-patch.7z gives in your OS output:
x265 --asm avx512 -V
x265 [info]: build info [Windows][MSVC 1900][64 bit] 10bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast LZCNT SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2 AVX512
?
Ma is offline   Reply With Quote
Old 19th April 2018, 01:18   #6046  |  Link
Loomes
Registered User
 
Join Date: Nov 2003
Location: Germany, Berlin
Posts: 48
Quote:
Originally Posted by Ma View Post
I want to precise: x265.exe from package avx512-patch.7z gives in your OS output:
x265 --asm avx512 -V
x265 [info]: build info [Windows][MSVC 1900][64 bit] 10bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast LZCNT SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2 AVX512
?
Confirmed. I tried again on Windows 10 64bit with an 7820x using the command line shown above.

Last edited by Loomes; 19th April 2018 at 01:22.
Loomes is offline   Reply With Quote
Old 19th April 2018, 06:31   #6047  |  Link
Asmodian
Registered User
 
Join Date: Feb 2002
Location: San Jose, California
Posts: 4,406
Quote:
Originally Posted by Loomes View Post
I did now but it also results in an instant crash. Are you using a ffmpeg pipe like me or what is your command line?
No, actually I was using avs2pipemod.exe (ver 1.1.1 Aug 15, 2016) piping to x265.exe with y4m:
C:\Tools\avs2pipemod.exe -y4mp=1:1 %1 | C:\Tools\x265.exe --input - --y4m -o "D:\Temp\%~n1.mkv" --asm avx512 --preset veryslow --crf 18.5 --output-depth 10

edit:
"x265 --asm avx512 -V" gives:
x265 [info]: HEVC encoder version 2.7+340-aa9102400f24
x265 [info]: build info [Windows][GCC 7.3.0][64 bit] 8bit+10bit+12bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast LZCNT SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2 AVX512
__________________
madVR options explained

Last edited by Asmodian; 19th April 2018 at 06:34.
Asmodian is offline   Reply With Quote
Old 19th April 2018, 11:38   #6048  |  Link
Ma
Registered User
 
Join Date: Feb 2015
Posts: 326
Quote:
Originally Posted by Loomes View Post
Confirmed. I tried again on Windows 10 64bit with an 7820x using the command line shown above.
Thanks for confirmation -- it looks like your CPU & OS supporting avx512.

I've prepared some binaries to narrow this bug -- avx512-bisect2.7z.
x265-0.exe -- 10-bit encoder with commented out all avx512 function pointers (it works on my i7 8700 with --asm avx512 option). It is for check if the bug is before avx512 code is running but the execution path is exactly like in avx512 case.

x265-01.exe -- assigned only first 1/4 function pointers with avx512 code.
x265-02.exe -- assigned only second 1/4 function pointers with avx512 code.
x265-03.exe -- assigned only third 1/4 function pointers with avx512 code.
x265-04.exe -- assigned only last 1/4 function pointers with avx512 code.

Could you test these 5 exe's and report back which works and which not?
Ma is offline   Reply With Quote
Old 19th April 2018, 13:39   #6049  |  Link
Loomes
Registered User
 
Join Date: Nov 2003
Location: Germany, Berlin
Posts: 48
Quote:
Originally Posted by Ma View Post
Could you test these 5 exe's and report back which works and which not?
I'm home in about 6 hours and will do. By the way: Is it correct that --asm avx512 will not work on Windows 7/8 64bit, no matter what?
Loomes is offline   Reply With Quote
Old 19th April 2018, 13:56   #6050  |  Link
nevcairiel
Registered Developer
 
Join Date: Mar 2010
Location: Hamburg/Germany
Posts: 10,344
Quote:
Originally Posted by Loomes View Post
Is it correct that --asm avx512 will not work on Windows 7/8 64bit, no matter what?
That would appear to be the case. If you force avx512 usage through --asm avx512 on an other OS it'll just crash.

Thats really the problem with using the --asm option, it overrides the CPU/OS feature detection, a different option to enable it when present would be better.
__________________
LAV Filters - open source ffmpeg based media splitter and decoders
nevcairiel is offline   Reply With Quote
Old 19th April 2018, 22:42   #6051  |  Link
Loomes
Registered User
 
Join Date: Nov 2003
Location: Germany, Berlin
Posts: 48
Quote:
Originally Posted by Ma View Post
Could you test these 5 exe's and report back which works and which not?
I am afraid, none of the x265-xx.exe worked in my Windows 10 and crashed instantly. But I found something else: I tried to use directly the libx265 of ffmpeg by
Code:
ffmpeg-2018-04-18.exe -i source.mkv -c:v libx265 -x265-params crf=21 -preset slow -pix_fmt yuv420p10le -bsf hevc_mp4toannexb -f rawvideo target.h265
and it also crashed instantly. Then I changed this new ffmpeg-2018-04-18.exe for an older one ffmpeg-2018-03-24.exe and it worked. Then I ran again your x265-xx.exe files with the older ffmpeg-2018-03-24.exe but they all crashed again with
Code:
ffmpeg-2018-03-24.exe -i source.mkv -f yuv4mpegpipe - | x265-0.exe --preset slow --crf 21 --y4m - -o target.h265
Even more strange that ffmpeg-2018-04-18.exe works flawlessly on Windows 7 and so the x265-0.exe of your files does (it even claims to make use of AVX512 when it starts). All the other x265-0x.exe crashed immediately on Windows 7.

This is really weird. Let me know if I can provide you any more info of my system or do some more tests.

Last edited by Loomes; 19th April 2018 at 23:23.
Loomes is offline   Reply With Quote
Old 20th April 2018, 00:06   #6052  |  Link
Ma
Registered User
 
Join Date: Feb 2015
Posts: 326
Thanks for the details. It is quite a riddle.
On Win7 everything is OK, x265-0 should work (all avx512 code is not active), x265-0x should crash.
On Win10 it is interesting -- in your 2 examples it is no '--asm avx512' option. It crashes without avx512 too?
Is something special with your Win10, for example it works in VirtualBox?
Ma is offline   Reply With Quote
Old 20th April 2018, 00:15   #6053  |  Link
Loomes
Registered User
 
Join Date: Nov 2003
Location: Germany, Berlin
Posts: 48
Quote:
Originally Posted by Ma View Post
On Win10 it is interesting -- in your 2 examples it is no '--asm avx512' option. It crashes without avx512 too?
Yes, I had --asm avx512 included in the first run and all crashed. Then I had the idea that it's about ffmpeg and let it run without the asm option but well, did not work either.

Quote:
Originally Posted by Ma View Post
Is something special with your Win10, for example it works in VirtualBox?
No, not at all. It is Windows 10 Professional 64bit, nothing special about it as far as I know. The system drive has 14GB free space, the drive where x265, ffmpeg etc. is running with 80G free space. As I said, it works fine with the older ffmpeg version or older x265.exe.
Is it possible that there is something special about the 7820x CPU?
Loomes is offline   Reply With Quote
Old 20th April 2018, 08:41   #6054  |  Link
Ma
Registered User
 
Join Date: Feb 2015
Posts: 326
Quote:
Originally Posted by Loomes View Post
Is it possible that there is something special about the 7820x CPU?
It is possible that there are hardware problems -- easiest way to check this out is Prime95 https://www.mersenne.org/download/
If you run this app with button 'Just Stress Testing' it should find memory/CPU problems (in file 'results.txt' should be only [date] and XXX passed! entries).

There was some changes in ffmpeg, so you can test ffmpeg release 3.4.2 -- it should be different approach to thread synchronization.

There was some reports about x265 hangs with pipe input from ffmpeg but it is hard to reproduce -- maybe there is bug in x265 that is not fixed yet.
Ma is offline   Reply With Quote
Old 20th April 2018, 10:17   #6055  |  Link
Bhavnahari
Registered User
 
Join Date: Nov 2016
Posts: 4
Quote:
Originally Posted by jlpsvk View Post
My settings then...

Code:
--crf 17 --profile main10 --level-idc 5.1 --output-depth 10 --ctu 32 --amp --vbv-bufsize 160000 --vbv-maxrate 160000 --me star
--max-merge 5 --rc-lookahead 40 --lookahead-slices 4 --gop-lookahead 34 --ref 5 --hdr --hdr-opt --repeat-headers --no-info --no-deblock
--no-sao --no-strong-intra-smoothing --high-tier --refine-inter 3 --refine-intra 4
Of course, display settings are entered too...

Warning:
Code:
x265 [warning]: Intra refinement requires analysis load, analysis-reuse-level 10, scale factor. Disabling intra refine.
x265 [warning]: Inter refinement requires analysis load, analysis-reuse-level 10, scale factor. Disabling inter refine.
Can it be used with CRF? Or? Some parameters missing?

refine-inter and refine-intra features may be used with any rate control techniques including ABR, CRF and CQP. However, as indicated by the warnings, these features are enabled only for scaled save/load encodes. The "save" run will encode the content of resolution WxH (1920x1080, say) and generate an analysis.dat file. This analysis information is scaled up by the factor and used by the "load" run which works on the content of resolution 2Wx2H (3840x2160, say). Using the analysis information from a low-res encode to encode high-res content will impact the quality of the encode hence we try to minimize the repercussions by incorporating various intra and inter refinement strategies.

Your CLI should be:
HD encode : --crf 17 --profile main10 --level-idc 5.1 --output-depth 10 --ctu 32 --amp --vbv-bufsize 160000 --vbv-maxrate 160000 --me star
--max-merge 5 --rc-lookahead 40 --lookahead-slices 4 --gop-lookahead 34 --ref 5 --hdr --hdr-opt --repeat-headers --no-info --no-deblock --no-sao --no-strong-intra-smoothing --high-tier --analysis-save <filename> --analysis-reuse-level 10 --scale-factor 2

UHD encode: -crf 17 --profile main10 --level-idc 5.1 --output-depth 10 --ctu 32 --amp --vbv-bufsize 160000 --vbv-maxrate 160000 --me star
--max-merge 5 --rc-lookahead 40 --lookahead-slices 4 --gop-lookahead 34 --ref 5 --hdr --hdr-opt --repeat-headers --no-info --no-deblock --no-sao --no-strong-intra-smoothing --high-tier --analysis-load <filename> --analysis-reuse-level 10 --scale-factor 2 --refine-inter 3 --refine-intra 4
Bhavnahari is offline   Reply With Quote
Old 20th April 2018, 21:26   #6056  |  Link
jlpsvk
Registered User
 
Join Date: Dec 2014
Posts: 240
Quote:
Originally Posted by Bhavnahari View Post
refine-inter and refine-intra features may be used with any rate control techniques including ABR, CRF and CQP. However, as indicated by the warnings, these features are enabled only for scaled save/load encodes. The "save" run will encode the content of resolution WxH (1920x1080, say) and generate an analysis.dat file. This analysis information is scaled up by the factor and used by the "load" run which works on the content of resolution 2Wx2H (3840x2160, say). Using the analysis information from a low-res encode to encode high-res content will impact the quality of the encode hence we try to minimize the repercussions by incorporating various intra and inter refinement strategies.

Your CLI should be:
HD encode : --crf 17 --profile main10 --level-idc 5.1 --output-depth 10 --ctu 32 --amp --vbv-bufsize 160000 --vbv-maxrate 160000 --me star
--max-merge 5 --rc-lookahead 40 --lookahead-slices 4 --gop-lookahead 34 --ref 5 --hdr --hdr-opt --repeat-headers --no-info --no-deblock --no-sao --no-strong-intra-smoothing --high-tier --analysis-save <filename> --analysis-reuse-level 10 --scale-factor 2

UHD encode: -crf 17 --profile main10 --level-idc 5.1 --output-depth 10 --ctu 32 --amp --vbv-bufsize 160000 --vbv-maxrate 160000 --me star
--max-merge 5 --rc-lookahead 40 --lookahead-slices 4 --gop-lookahead 34 --ref 5 --hdr --hdr-opt --repeat-headers --no-info --no-deblock --no-sao --no-strong-intra-smoothing --high-tier --analysis-load <filename> --analysis-reuse-level 10 --scale-factor 2 --refine-inter 3 --refine-intra 4
Thanks!!! So it won't downscale and upscale video? Only stats? Right? So it's better for retain quality/details or not?

Last edited by jlpsvk; 20th April 2018 at 21:47.
jlpsvk is offline   Reply With Quote
Old 21st April 2018, 12:05   #6057  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Finland
Posts: 5,718
When watching some x265 encoded content on my TV, I can see very, very thin greyish vertical lines in scenes that are almost or completely black. Is this due to slice boundaries?
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 21st April 2018, 18:54   #6058  |  Link
foxyshadis
ангел смерти
 
foxyshadis's Avatar
 
Join Date: Nov 2004
Location: Lost
Posts: 9,558
Quote:
Originally Posted by Boulder View Post
When watching some x265 encoded content on my TV, I can see very, very thin greyish vertical lines in scenes that are almost or completely black. Is this due to slice boundaries?
That'd more likely be tile boundaries. Definitely sounds like a decoder bug.
foxyshadis is offline   Reply With Quote
Old 21st April 2018, 19:10   #6059  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Finland
Posts: 5,718
(I did some more testing and I cannot see the strips if lookahead-slices was disabled (vertical resolution less than 720).) EDIT: tested with existing encodes - couldn't replicate this with the latest x265 using the same source.

This is how it looks on the TV and on my display, a snippet from the upper left corner (seriously exaggerated by getting the brightness real high):

https://imgur.com/a/gO45OYB

I recall seeing this also with x264 encoded videos. Different decoders have been used, on the computer it's the NVIDIA GPU and on the TV it's whatever is on Apple TV 4K - VideoToolBox I think.

EDIT: this older clip shows the thing at the start, turn up the brightness to see it: https://drive.google.com/open?id=1ui...b7A0ob9nqtJKa2
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...

Last edited by Boulder; 21st April 2018 at 20:14.
Boulder is offline   Reply With Quote
Old 22nd April 2018, 19:33   #6060  |  Link
jlpsvk
Registered User
 
Join Date: Dec 2014
Posts: 240
just an addition to x265 AVX-512 problems...

ASRock X299 OC Formula + i9-7940X + Win10 x64 ... no crashing... AVX512 working flawlessly.

Binary from here (compiled in VS2017):
Code:
http://msystem.waw.pl/x265/x265-2.7+340-aa91024_vs2017.7z
jlpsvk is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 00:58.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.