Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > High Efficiency Video Coding (HEVC)

Reply
 
Thread Tools Search this Thread Display Modes
Old 26th April 2018, 07:44   #6081  |  Link
jlpsvk
Registered User
 
Join Date: Dec 2014
Posts: 191
Quote:
Originally Posted by RieGo View Post
so no AVX512 offset?

i got really weird results while using avx512 and x265:
A) temps don't really go much higher while using avx512, at least on my system
B) AVX512 offset almost doesn't make a difference. i had same performance on 12x 100MHz difference in offset
probably Intel is OC-ing itself as it has good thermal conditions. temps with AVX-512 are bit lower in fact on my system.
jlpsvk is offline   Reply With Quote
Old 26th April 2018, 19:48   #6082  |  Link
jlpsvk
Registered User
 
Join Date: Dec 2014
Posts: 191
x265 crashing on "2nd pass"... (--profile main10 --level-idc 5.1 --output-depth 10 --ctu 32 --amp --vbv-bufsize 160000 --vbv-maxrate 160000 --me star --max-merge 5 --rc-lookahead 40 --lookahead-slices 4 --gop-lookahead 34 --ref 5 --hdr --hdr-opt --repeat-headers --no-info --no-deblock --no-sao --no-strong-intra-smoothing --high-tier --analysis-load c:\stats.dat --analysis-reuse-level 10 --scale-factor 2 --refine-inter 3 --refine-intra 4 --asm avx512)

Code:
x265 [warning]: specified frame type (5) at 2091 is not compatible with keyframe interval
any suggestion?
jlpsvk is offline   Reply With Quote
Old 26th April 2018, 19:49   #6083  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 5,885
wild guess: still got free space on c: ? (iirc analysis files etc can get really large)
__________________
Hybrid here in the forum, homepage
Selur is offline   Reply With Quote
Old 26th April 2018, 20:18   #6084  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 5,943
And: Write access rights in the root of C:? Better use a secondary drive for video processing.
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Old 26th April 2018, 20:19   #6085  |  Link
jlpsvk
Registered User
 
Join Date: Dec 2014
Posts: 191
Quote:
Originally Posted by Selur View Post
wild guess: still got free space on c: ? (iirc analysis files etc can get really large)
yeah... 600GB free on C: SSD. It's happening in the second phase, READ, when the stats are reading... First, 1080p encode, with writing stats is OK.

Stats file 27GB big.
jlpsvk is offline   Reply With Quote
Old 26th April 2018, 20:25   #6086  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 5,943
Well, it's not completely impossible that encoding in AVX-512 mode might create a different output. But probably hard to confirm...
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Old 26th April 2018, 20:35   #6087  |  Link
jlpsvk
Registered User
 
Join Date: Dec 2014
Posts: 191
Quote:
Originally Posted by LigH View Post
Well, it's not completely impossible that encoding in AVX-512 mode might create a different output. But probably hard to confirm...
first phase was created using AVX-512 too... tried READ phase without it, result is the same. Probably will try to use first without also.
jlpsvk is offline   Reply With Quote
Old 27th April 2018, 01:35   #6088  |  Link
divxmaster
Registered User
 
Join Date: Mar 2015
Location: New Zealand
Posts: 45
As a follow up to my post a few months ago re spectre/meltdown patching and its influence on x265 encoding fps,
(where I found on my skylake quad it made virtually no difference),
I have now test my haswell system (4770k) with the microsoft microcode updates release two days ago (KB4091663 for 1703 I am running).

The latest KB patches the microcode for spectre variant two (branch target injection), at boot time only. GRC's Inspectre shows my system is now fully patched.
This latest KB DOES affect throughput fps. I have seen a drop of about 2.5%. Not a big deal luckily. I will just adjust my conservative OC
from 4.1ghz to 4.2ghz. (which is +2.5%).

Cheers,
Divxmaster
divxmaster is offline   Reply With Quote
Old 27th April 2018, 07:31   #6089  |  Link
jlpsvk
Registered User
 
Join Date: Dec 2014
Posts: 191
Quote:
Originally Posted by jlpsvk View Post
first phase was created using AVX-512 too... tried READ phase without it, result is the same. Probably will try to use first without also.
So... stats write encode (I am not writing here mastering display, options), spline36 downsized from 3840x1600 to 1920x800:
--crf 17 --keyint 240 --min-keyint 24 --profile main10 --level-idc 5.1 --output-depth 10 --ctu 32 --amp --vbv-bufsize 160000 --vbv-maxrate 160000 --me star --max-merge 5 --rc-lookahead 40 --lookahead-slices 4 --gop-lookahead 34 --ref 5 --hdr --hdr-opt --repeat-headers --no-info --no-deblock --no-sao --no-strong-intra-smoothing --high-tier --analysis-save c:\stats.dat --analysis-reuse-level 10 --scale-factor 2 --no-cutree

Stats read encode at full 3840x1600:
--crf 17 --keyint 240 --min-keyint 24 --profile main10 --level-idc 5.1 --output-depth 10 --ctu 32 --amp --vbv-bufsize 160000 --vbv-maxrate 160000 --me star --max-merge 5 --rc-lookahead 40 --lookahead-slices 4 --gop-lookahead 34 --ref 5 --hdr --hdr-opt --repeat-headers --no-info --no-deblock --no-sao --no-strong-intra-smoothing --high-tier --analysis-load c:\stats.dat --analysis-reuse-level 10 --scale-factor 2 --refine-inter 3 --refine-intra 4 --no-cutree

With or withou AVX-512, result is the same.
jlpsvk is offline   Reply With Quote
Old 3rd May 2018, 08:02   #6090  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 5,943
x265 2.7+346-69aafa6d70ad (stable merge)

AVX-512 cleanups and a few nits (getting a RC soon?)
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid

Last edited by LigH; 3rd May 2018 at 20:20.
LigH is offline   Reply With Quote
Old 4th May 2018, 06:06   #6091  |  Link
pradeeprama
Registered User
 
Join Date: Sep 2015
Posts: 48
Quote:
Originally Posted by divxmaster View Post
As a follow up to my post a few months ago re spectre/meltdown patching and its influence on x265 encoding fps,
(where I found on my skylake quad it made virtually no difference),
I have now test my haswell system (4770k) with the microsoft microcode updates release two days ago (KB4091663 for 1703 I am running).

The latest KB patches the microcode for spectre variant two (branch target injection), at boot time only. GRC's Inspectre shows my system is now fully patched.
This latest KB DOES affect throughput fps. I have seen a drop of about 2.5%. Not a big deal luckily. I will just adjust my conservative OC
from 4.1ghz to 4.2ghz. (which is +2.5%).

Cheers,
Divxmaster
Thanks for that. I typically classify any performance difference under 5% (in either direction) as noise when measured on a real system... so I suppose it is fair to say that Specrte and Meltdown don't affect us. This is not surprising as we barely make syscalls that require escalated privileges, which is where these bugs kick-in and safe guard!
pradeeprama is offline   Reply With Quote
Old 4th May 2018, 06:07   #6092  |  Link
pradeeprama
Registered User
 
Join Date: Sep 2015
Posts: 48
Quote:
Originally Posted by jlpsvk View Post
So... stats write encode (I am not writing here mastering display, options), spline36 downsized from 3840x1600 to 1920x800:
--crf 17 --keyint 240 --min-keyint 24 --profile main10 --level-idc 5.1 --output-depth 10 --ctu 32 --amp --vbv-bufsize 160000 --vbv-maxrate 160000 --me star --max-merge 5 --rc-lookahead 40 --lookahead-slices 4 --gop-lookahead 34 --ref 5 --hdr --hdr-opt --repeat-headers --no-info --no-deblock --no-sao --no-strong-intra-smoothing --high-tier --analysis-save c:\stats.dat --analysis-reuse-level 10 --scale-factor 2 --no-cutree

Stats read encode at full 3840x1600:
--crf 17 --keyint 240 --min-keyint 24 --profile main10 --level-idc 5.1 --output-depth 10 --ctu 32 --amp --vbv-bufsize 160000 --vbv-maxrate 160000 --me star --max-merge 5 --rc-lookahead 40 --lookahead-slices 4 --gop-lookahead 34 --ref 5 --hdr --hdr-opt --repeat-headers --no-info --no-deblock --no-sao --no-strong-intra-smoothing --high-tier --analysis-load c:\stats.dat --analysis-reuse-level 10 --scale-factor 2 --refine-inter 3 --refine-intra 4 --no-cutree

With or withou AVX-512, result is the same.
What CPU are you using?
pradeeprama is offline   Reply With Quote
Old 6th May 2018, 11:44   #6093  |  Link
jlpsvk
Registered User
 
Join Date: Dec 2014
Posts: 191
Quote:
Originally Posted by pradeeprama View Post
What CPU are you using?
Core i9-7940X
jlpsvk is offline   Reply With Quote
Old 9th May 2018, 16:40   #6094  |  Link
Barough
Registered User
 
Barough's Avatar
 
Join Date: Feb 2007
Location: Sweden
Posts: 337
x265 v2.7+348-0968a46d6ba4 (GCC 7.3.0, 32 & 64-bit 8/10/12bit Multilib Windows Binaries)

Code:
https://bitbucket.org/multicoreware/x265/commits/all
Barough is offline   Reply With Quote
Old 9th May 2018, 20:02   #6095  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 5,943
x265 2.7+348-0968a46d6ba4

fixes a build error on Mac and a calculation bug in scalefactor 0
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Old 12th May 2018, 00:48   #6096  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 5,885
Does anyone see where my mistake is?

I'm trying to convert a 8bit pc range clip to 10bit x265 using:
Code:
ffmpeg -y -threads 8 -i "C:\Users\Selur\Desktop\Color Range Test.MOV" -map 0:0 -an -sn -vsync 0 -strict -1 -pix_fmt yuv420p10le -f yuv4mpegpipe - | x265 --input - --output-depth 10 --y4m --profile main10 --no-open-gop --crf 18.00 --range full --colormatrix bt470bg --output "H:\Temp\x265.265"
Problem is the output histogram is compressed, see: https://s14.postimg.cc/pq865qnfj/compare.png
Uploaded the source to my google drive as Color Range Test.MOV.

for the comparision I used:
Code:
# Imports
import vapoursynth as vs
core = vs.get_core()
# Loading Plugins
core.std.LoadPlugin(path="G:/Hybrid/64bit/vsfilters/SourceFilter/LSmashSource/vslsmashsource.dll")


# Loading H:\Output\Color Range Test.MOV using LibavSMASHSource
clip = core.lsmas.LibavSMASHSource(source="H:/Output/Color Range Test.MOV")
# making sure input color matrix is set as 470bg
clip = core.resize.Point(clip, matrix_in_s="470bg")
# making sure frame rate is set to 30000/1001
clip = core.std.AssumeFPS(clip, fpsnum=30000, fpsden=1001)
# Making sure input color range is set to PC (full) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=0)
clip = core.text.Text(clip=clip, text="Original")
clip = core.hist.Levels(clip=clip)

clip2 = core.lsmas.LWLibavSource(source="H:/Temp/x265.265", format="YUV420P10", cache=0)
# making sure input color matrix is set as 470bg
clip2 = core.resize.Point(clip2, matrix_in_s="470bg")
# making sure frame rate is set to 30000/1001
clip2 = core.std.AssumeFPS(clip2, fpsnum=30000, fpsden=1001)
# Making sure input color range is set to PC (full) range.
clip2 = core.std.SetFrameProp(clip=clip2, prop="_ColorRange", intval=0)
clip2 = core.text.Text(clip=clip2, text="x265")
# adjusting output color from: YUV420P8 to YUV420P10
clip2 = core.resize.Bicubic(clip=clip2, format=vs.YUV420P8)
clip2 = core.hist.Levels(clip=clip2)


clip = core.std.StackHorizontal([clip, clip2])
# Output
clip.set_output()
My guess is that I need to change the ffmpeg call, but I don't see where.

Cu Selur

Ps.: Adding '-color_range 2' to the ffmpeg call doesn't help either.
__________________
Hybrid here in the forum, homepage

Last edited by Selur; 12th May 2018 at 01:18.
Selur is offline   Reply With Quote
Old 12th May 2018, 11:24   #6097  |  Link
sneaker_ger
Registered User
 
Join Date: Dec 2002
Posts: 5,495
As you say this has nothing to do with x265.

You could try to "trick" ffmpeg by pretending the input was limited range.
-vf zscale=rangein=limited:range=limited -pix_fmt yuv420p10le

Maybe someone with more ffmpeg knowledge knows a "proper" solution ...

Last edited by sneaker_ger; 12th May 2018 at 11:27.
sneaker_ger is offline   Reply With Quote
Old 12th May 2018, 12:20   #6098  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 5,885
Quote:
As you say this has nothing to do with x265.
That is what I guessed, not sure whether ffmpeg does something wrong or x265 has some problem with y4m output or ffmpeg.

Quote:
You could try to "trick" ffmpeg by pretending the input was limited range.
-vf zscale=rangein=limited:range=limited -pix_fmt yuv420p10le
That still changes the luma histogram. (https://s31.postimg.cc/tnpvy7cq3/compare2.png)
using:
Code:
-vf zscale=rangein=full:range=full
is better, but there are still some changes. (https://s31.postimg.cc/9hmdzaehn/compare3.png)

Quote:
Maybe someone with more ffmpeg knowledge knows a "proper" solution ...
I hope so.

Cu Selur
__________________
Hybrid here in the forum, homepage

Last edited by Selur; 12th May 2018 at 12:23.
Selur is offline   Reply With Quote
Old 12th May 2018, 18:18   #6099  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 3,976
maybe there is a bug with full range ?

waveform/histograms only match if you use lossless (even compared to low crf like --crf 8, where you 'd only expect tiny differences) , regardless if you use vapoursynth input or ffmpeg scaling with rawvideo pipe

crf8 - waveform is compressed
Code:
ffmpeg -i "Color Range Test.MOV" -vf scale=iw:ih:in_range=pc:out_range=pc,format=yuv420p10le -f rawvideo - | x265_10bit.exe --fps 30000/1001 --input-res 1280x720 --input-depth 10 --output-depth 10 --input-csp i420 --crf 8.00 --range full --colormatrix smpte170m -o rawpipe_x265crf8.265 -
--lossless works with ffmpeg rawvideo pipe
Code:
ffmpeg -i "Color Range Test.MOV" -vf scale=iw:ih:in_range=pc:out_range=pc,format=yuv420p10le -f rawvideo - | x265_10bit.exe --fps 30000/1001 --input-res 1280x720 --input-depth 10 --output-depth 10 --input-csp i420 --lossless --range full --colormatrix smpte170m -o rawpipe_x265lossless.265 -

vspipe y4m --crf 8 waveform is compressed
Code:
vspipe --y4m 1.vpy - | x265_10bit.exe --y4m --crf 8.00 --colormatrix smpte170m --range full -o vs_x265.265 -

vspipe y4m --lossless waveform is ok
Code:
vspipe --y4m 1.vpy - | x265_10bit.exe --y4m --lossless --colormatrix smpte170m --range full -o vs_x265_lossless.265 -
vpy input
Code:
clip = core.lsmas.LibavSMASHSource(r'F:\Color Range Test.MOV')
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P10)
clip.set_output()
poisondeathray is offline   Reply With Quote
Old 12th May 2018, 21:28   #6100  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 5,885
Quote:
waveform/histograms only match if you use lossless (even compared to low crf like --crf 8, where you 'd only expect tiny differences) , regardless if you use vapoursynth input or ffmpeg scaling with rawvideo pipe
I understand some small changes like when using '-vf zscale=rangein=full:range=full' and my lines from above might be a side effect of the video compression, but the compression of the histogram really looks like either a bug in x265 or all the other tools.

Cu Selur
__________________
Hybrid here in the forum, homepage
Selur is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 18:57.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.