x265 HEVC Encoder

[jpsdr]

Ok, thanks for the feedback. In that case, i'll change my encode scripts to use histogram mode.

[benwaggoner]

Wow, that's big news if the new histogram mode is good enough to be the default.

Anyone test --mctf or the new segment based rate control?

[vpupkind]

Originally Posted by benwaggoner

Wow, that's big news if the new histogram mode is good enough to be the default.

Anyone test --mctf or the new segment based rate control?

SBRC is a bit of a misnomer right now. It does some aq-related stuff, hopefully some actual rate control stuff will follow.

[Barough]

x265 v3.5+69
https://www.mediafire.com/file/kuh2ekbo2p6a7rf/

[ShortKatz]

Originally Posted by benwaggoner

Wow, that's big news if the new histogram mode is good enough to be the default.

Anyone test --mctf or the new segment based rate control?

Tested --mctf just once. It increased my encoding time 7-fold. So I did not do any further tests.

[LeXXuz]

Originally Posted by jpsdr

Ok, thanks for the feedback. In that case, i'll change my encode scripts to use histogram mode.

Could you possibly add 8-bit support to your version? I'd like to do some comparisons with aq5-mode between 8 and 10bit output.

[jpsdr]

.... There is not...? I ran cmake to create a VS projet and checked all possible options, i thought all the supports were enabled.
Setting --profile main, -D 8 (and all others related stuff saying that video is 8 bits) on the command line is not working ?

If not, for now, unfortunately i don't know what to do for...

[quietvoid]

If you enabled HIGH_BIT_DEPTH when configuring the build, it only builds 10 bit.
12 bit if you do MAIN12 too.

There's a multilib script to get a statically linked executable with all bit depths.
I can't really help for Windows/VS though.

[LeXXuz]

Originally Posted by jpsdr

.... There is not...? I ran cmake to create a VS projet and checked all possible options, i thought all the supports were enabled.
Setting --profile main, -D 8 (and all others related stuff saying that video is 8 bits) on the command line is not working ?

No there isn't. x265 exits with this error if forced to work in 8-bit:

x265 [FLAW]: main profile not supported, internal bit depth 10.

[jpsdr]

Ah.... Don't know how to do it for now, don't realy have time for now, sorry. Try to see if i can figure out something when there will be the need for a new build.
Otherwise, anyone can also make a build from my repo.

[quietvoid]

I just noticed that the new histogram scene cut commit removed the logging of consecutive B frames %, seemingly for no reason.
If anyone had a use for it, here's a diff: https://github.com/quietvoid/x265/co...a35be4377e390e

Should probably be renamed from m_histogram now but I just copied the old code back.

[Boulder]

Here's a very crude tool written in Python to get those average figures out of the csv log file written with csv-log-level 2 set. It expects that SSIM or PSNR is not enabled as those change the column structure. Only tested with two csv files I had on my HDD

Usage: "python logtool.py filename.csv". It will then write a .log file to the path where the csv file is and output the same information in the command prompt window.

import pandas as pd
import os
import sys

filename = os.path.abspath(sys.argv[1])

log_content = pd.read_csv(filename, sep=",", decimal=".", header=0)
log_content = log_content.iloc[:-4,11:-9]
log_content.columns = log_content.columns.str.replace('.1', '', regex=True)
log_content.columns = log_content.columns.str.replace('66', ' 16x16', regex=True)
log_content.columns = log_content.columns.str.replace('326', ' 32x16', regex=True)
log_content.columns = log_content.columns.str.replace('6x32', ' 16x32', regex=True)
log_content.columns = log_content.columns.str.replace('6x8', ' 16x8', regex=True)
log_content.columns = log_content.columns.str.replace('86', ' 8x16', regex=True)
log_content.columns = log_content.columns.str.replace('AMP6', 'AMP 16', regex=True)
log_content = log_content.replace('%', '', regex=True).astype(float)
average = log_content.describe().loc['mean'].to_string()

print (average)
with open(filename+str('.log'), mode='w') as outputfile:
    outputfile.write(str(average))
    outputfile.close()

[_kermit]

piping to x265 with powershell

sorry if that is too off-topic...

this works just fine in a command prompt:

ffmpeg.exe -i FILE -pix_fmt yuv420p10le -strict -1 -f yuv4mpegpipe - | x265.exe - --y4m .....etc.

Now I wanted to use Powershell for that and the piping isn't working, the x265 process doesn't even start, while ffpmpeg is running:

It seems that using the pipe character isn't working here.
Does anyone have an idea?

[excellentswordfight]

Originally Posted by _kermit

piping to x265 with powershell

sorry if that is too off-topic...

this works just fine in a command prompt:

ffmpeg.exe -i FILE -pix_fmt yuv420p10le -strict -1 -f yuv4mpegpipe - | x265.exe - --y4m .....etc.

Now I wanted to use Powershell for that and the piping isn't working, the x265 process doesn't even start, while ffpmpeg is running:

It seems that using the pipe character isn't working here.
Does anyone have an idea?

There is nothing "wrong" with the pipe command, but piping does not work in the same way in powershell as in cmd so you cannot pipe raw data in the same way.

There are some workaround and more info here https://stackoverflow.com/questions/...-in-powershell

[_kermit]

Originally Posted by excellentswordfight

There is nothing "wrong" with the pipe command, but piping does not work in the same way in powershell as in cmd so you cannot pipe raw data in the same way.

There are some workaround and more info here https://stackoverflow.com/questions/...-in-powershell

thanks for confirming, thought this might be the issue.
Looks rather messy doing it that way and the only reason I wanted to use PS was to retrieve values from the MKV and use them in the script.
Probably too much of a hassle now.

In case anyone is interested (pretty basic, with debug code...).
params: file, CRF, presetname

$file = $args[0]
$crf = $args[1]
$preset = $args[2]

$temp = $(&D:\temp\Mediainfo\MediaInfo.exe "--Inform=Video;%MaxCLL%" $file).tostring().split(" ")
$MaxCLL= $temp[0].trim()

$temp = $(&D:\temp\Mediainfo\MediaInfo.exe "--Inform=Video;%MaxFALL%" $file).tostring().split(" ")
$MaxFALL = $temp[0].trim()

$temp = $(&D:\temp\Mediainfo\MediaInfo.exe "--Inform=Video;%MasteringDisplay_Luminance%" $file).tostring().split(" ")
if ($temp) {
$LumMin = [int]([decimal]$temp[1].trim() * 10000)
$LumMax = [int]$temp[4].trim() * 10000}
else
{$LumMin = 0 ; $LumMax = 0}

$x265Params = "- --y4m --output-depth 10 --preset $preset --crf $crf --colorprim bt2020 --transfer smpte2084 --colormatrix bt2020nc --master-display ""G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L($LumMax,$LumMin)"" --vbv-bufsize 160000 --vbv-maxrate 160000 -o ""$file." + "$crf" + "_" + $preset + """ --rdoq-level 2 --cu-lossless --max-merge 3 --rc-lookahead 25 --lookahead-slices 4 --ref 4 --range limited --max-cll ""$maxCLL,$maxFALL"" --hdr --hdr-opt --repeat-headers --aud --deblock -3:-3 --no-strong-intra-smoothing --aq-mode 1 --rskip-edge-threshold 3 --psy-rd 4 --psy-rdoq 15 --ctu 32 --rskip 2 --deblock -3:-3"

"crf: $crf"
"preset: $preset"
"maxcll: $maxcll"
"maxfall: $maxfall"
"low: $LumMin"
"high: $LumMax"
"x265Params: $x265Params"

&D:\temp\ffmpeg-4.4-full_build\bin\ffmpeg.exe -i ""$file"" -pix_fmt yuv420p10le -strict -1 -f yuv4mpegpipe - | d:\temp\x265\x265.exe $x265Paramsffmpeg.exe -i %1 -pix_fmt yuv420p10le -strict -1 -f yuv4mpegpipe - | d:\temp\x265\x265.exe $x265Params

[quietvoid]

From a quick comparison of a full length encode in PQ/HDR, --hist-scenecut seems pretty good now.

Out of 628 expected shots..
Default: 180 false positives, 214 missed (or not exact frame), 414 frame perfect. 66% correct.
--hist-scenecut: 12 false positives, 121 missed (or not exact frame), 507 frame perfect. 81% correct.

YMMV.

[stax76]

@_kermit

This powershell code works:

$env:ffmpeg = 'D:\ffmpeg.exe'
$env:x265   = 'D:\x265.exe'
$env:avs    = 'D:\test.avs'
$env:output = 'D:\test.hevc'

$env:ffmpeg_args = '-f yuv4mpegpipe -strict -1 -loglevel fatal -hide_banner'

$env:x265_args = '--crf 18 --output-depth 10 --y4m'

cmd /s /c --% ""%ffmpeg%" -i "%avs%" %ffmpeg_args% - | "%x265%" %x265_args% --output "%output%" -"

In short:

cmd /s /c --% "<cmd style command>"

[benwaggoner]

Originally Posted by quietvoid

From a quick comparison of a full length encode in PQ/HDR, --hist-scenecut seems pretty good now.

Out of 628 expected shots..
Default: 180 false positives, 214 missed (or not exact frame), 414 frame perfect. 66% correct.
--hist-scenecut: 12 false positives, 121 missed (or not exact frame), 507 frame perfect. 81% correct.

YMMV.

Intriguing results, thanks! The drop in false positives could help compression efficiency quite a bit.

[quietvoid]

It seems there's a chance the current upstream x265 segfaults with --hist-scenecut, at the end of an encode.

#0 0x0000555555641be9 in x265::Lookahead::detectHistBasedSceneChange (this=this@entry=0x555555be6a70,
frames=frames@entry=0x7fff8521b870, p0=p0@entry=7, p1=p1@entry=8, p2=p2@entry=9)
at x265/source/encoder/slicetype.cpp:2848
2848 absIntDiffFuturePast = (uint8_t)X265_ABS((int16_t)futureFrame->averageIntensityPerSegment[segmentInFrameWidthIndex][segmentInFrameHeightIndex][0] - (int16_t)previousFrame->averageIntensityPerSegment[segmentInFrameWidthIndex][segmentInFrameHeightIndex][0]);

Seems futureFrame is a null pointer.

(gdb) p futureFrame
$1 = (x265::Lowres *) 0x0
(gdb) p previousFrame
$2 = (x265::Lowres *) 0x7ffef110f6c8

Should probably be reported.

[_kermit]

Originally Posted by stax76

@_kermit

This powershell code works:

$env:ffmpeg = 'D:\ffmpeg.exe'
$env:x265   = 'D:\x265.exe'
$env:avs    = 'D:\test.avs'
$env:output = 'D:\test.hevc'

$env:ffmpeg_args = '-f yuv4mpegpipe -strict -1 -loglevel fatal -hide_banner'

$env:x265_args = '--crf 18 --output-depth 10 --y4m'

cmd /s /c --% ""%ffmpeg%" -i "%avs%" %ffmpeg_args% - | "%x265%" %x265_args% --output "%output%" -"

In short:

cmd /s /c --% "<cmd style command>"

I had the same idea and it works now, thanks!