x265 HEVC Encoder [Archive] - Page 9

Pages : 1 2 3 4 5 6 7 8 [9] 10 11

Barough

6th February 2021, 19:14

x265 v3.5_RC1 (https://www.mediafire.com/file/q7kfnuum7vz2acs/x265-3.5_RC1_Win_GCC102.7z/file) (32 & 64-bit 8/10/12bit Multilib Windows Binaries) (GCC 10.2.0)

https://bitbucket.org/multicoreware/x265_git/commits/branch/Release_3.5

MeteorRain

6th February 2021, 23:11

Also you can ask Pat to make StaxRip edition, merging and unifying from both worlds :)

Second this, and I think it may be better to create an org to organize these projects. We are individuals and we may not always be available to fix issues or implement new features. If we can manage to get a small group of people collaborating on the stax project it may be easier.

stax76

7th February 2021, 12:45

The staxrip github org could be used too, no problem. The needed things are a project name, for instance community mod, people who think can work together, and there is the question if binaries should be released on github.

staxrip change I made yesterday is reading --version output to detect if it's a mod and what type of mod and then use different code paths depending on if it's aMod, Asuna or Vanilla, should have done this before.

nakTT

9th February 2021, 00:19

LigH

9th February 2021, 08:17

Note: Multicoreware asks to refresh x265 clone

We had to recently strip a commit that got pushed into Release_3.5 branch due to some unavoidable reasons. Hence, we request all those who auto-update x265 to refresh your git repos so that we'll be on the same page.

E.g. in media-autobuild suite, you would remove the build/x265_git-git directory before running the suite again.

PS: Well, m-ab-s does not aim for the Release_3.5 branch. Would have to be the HEAD if.

Barough

11th February 2021, 13:27

x265 v3.5_RC1+2-f3f27198 (http://www.mediafire.com/file/94n1ovortwx1vez/x265-3.5_RC1%252B2-f3f27198_Win_GCC102.7z/file) (32 & 64-bit 8/10/12bit Multilib Windows Binaries) (GCC 10.2.0)

https://bitbucket.org/multicoreware/x265_git/commits/branch/Release_3.5

PoeBear

12th February 2021, 15:23

Chroma bleeding is actually really weird, it shouldn't really happen and I think I know why it's happening to you!
Your input is an AVS Script from what I see, which handles 4:2:0 Type 1 but not 4:2:0 Type2!
So, the chroma location is wrong, hence the "bleeding".

Can you try with:

ffmpeg.exe -i "AVS Script.avs" -vf scale=out_color_matrix=bt2020nc:out_h_chr_pos=0:out_v_chr_pos=0 -pix_fmt yuv420p16le -strict -1 -an -f yuv4mpegpipe - | x265.exe --y4m - --dither

followed by the rest of your x265 command line? This should solve the chroma bleeding problem. ;)

Finally got around to trying this, after tracking down some 32-bit ffmpeg builds (the 64-bit builds try to load 64-bit AviSynth+ and 64-bit plugins, threw me for a loop momentarily)

But it's drastically altering the colors when running the same exact x265 commands

Here's a comparison between the normal avs2yuv pipe I was using, and then your ffmpeg pipe from above:
https://thumbs2.imagebam.com/d2/a8/80/6b51f91369858504.jpg (http://www.imagebam.com/image/6b51f91369858504) https://thumbs2.imagebam.com/ee/19/74/8322ec1369858473.jpg (http://www.imagebam.com/image/8322ec1369858473) https://thumbs2.imagebam.com/c1/c5/0d/e8b8791369858463.jpg (http://www.imagebam.com/image/e8b8791369858463)

Here they are tonemapped for better comparison of the changes:
https://thumbs2.imagebam.com/11/6c/d3/a31a8f1369859213.jpg (http://www.imagebam.com/image/a31a8f1369859213) https://thumbs2.imagebam.com/47/3c/bf/c346b01369859138.jpg (http://www.imagebam.com/image/c346b01369859138) https://thumbs2.imagebam.com/b5/42/8b/e430db1369859174.jpg (http://www.imagebam.com/image/e430db1369859174)

Here were the commands ran, I just used one of my recent tests so I could do a comparison

avs2yuv.exe "Script.avs" - | "x265.exe" -D 10 --crf 10.5 --profile main10 --level-idc 5.1 --high-tier --preset placebo --input-depth 16 --cu-lossless
--pmode --bframes 16 --qg-size 32 --frame-threads 4 --ref 6 --limit-refs 1 --merange 57 --no-amp --tskip --tskip-fast --limit-modes --no-open-gop --hrd
--cbqpoffs -0 --crqpoffs -1 --no-cutree --deblock -2:-2 --psy-rd 2.50 --psy-rdoq 1.00 --qcomp .6 --aq-mode 2 --aq-strength 1.0 --ipratio 1.3 --pbratio 1.2
--vbv-bufsize 160000 --vbv-maxrate 160000 --aud --hdr10 --hdr10-opt --range limited --colorprim bt2020 --transfer smpte2084 --colormatrix bt2020nc
--master-display G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(40000000,50) --max-cll 602,187 --chromaloc 2 --repeat-headers
--output "pmode-sao-tskip-r1-test.hevc" --frames 3000 --y4m -

ffmpeg.exe -i "Script.avs" -vf scale=out_color_matrix=bt2020nc:out_h_chr_pos=0:out_v_chr_pos=0 -pix_fmt yuv420p16le -strict -1 -an -f yuv4mpegpipe - | x265.exe
--y4m - --dither --output-depth 10 --crf 10.5 --profile main10 --level-idc 5.1 --high-tier --preset placebo --input-depth 16 --cu-lossless --pmode --bframes 16
--qg-size 32 --frame-threads 4 --ref 6 --limit-refs 1 --merange 57 --no-amp --tskip --tskip-fast --limit-modes --no-open-gop --hrd --cbqpoffs -0 --crqpoffs -1
--no-cutree --deblock -2:-2 --psy-rd 2.50 --psy-rdoq 1.00 --qcomp .6 --aq-mode 2 --aq-strength 1.0 --ipratio 1.3 --pbratio 1.2 --vbv-bufsize 160000
--vbv-maxrate 160000 --aud --hdr10 --hdr10-opt --range limited --colorprim bt2020 --transfer smpte2084 --colormatrix bt2020nc
--master-display G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(40000000,50) --max-cll 602,187 --chromaloc 2 --repeat-headers
--output "pmode-sao-tskip-r1-testFF.hevc" --frames 3000

True, but the thing is that he was using chromaloc 2 in x265 but his input (the AVS Script output) wasn't, so it was just wrong. Either he specifies the normal 4:2:0 or he converts it to type2 and specifies chromaloc 2. That's what I meant ;)

My input is just the raw video track from the UHD (which is chromaloc 2 per the UHD Blu-ray spec) indexed with DGDecNV. Is there a limitation with the way AviSynth+ outputs?

Script.avs:
DGSource("UHD.dgi")
Crop(0,276,-0,-276)
z_Spline36Resize(1920,804)

benwaggoner

12th February 2021, 21:40

I suspect you need to specify PQ as well as 2020 in ffmpeg somehow, if it supports it yet.

You want the whole SMPTE Rec.2100 package.

_kermit

22nd February 2021, 18:44

currently I use this to encode:

ffmpeg.exe -i abc.mkv -pix_fmt yuv420p -f yuv4mpegpipe -| x265.exe -o output

But this doesn't use all my new 12 cores :) to a 100%, maybe half.
How can I improve this?

microchip8

22nd February 2021, 19:32

stax76

22nd February 2021, 20:10

But this doesn't use all my new 12 cores to a 100%, maybe half.
How can I improve this?

Split into chunks maybe.

https://github.com/staxrip/staxrip/wiki/Usage#chunk-encoding

_kermit

22nd February 2021, 21:15

Have you tried passing --pmode to x265?

no, haven't tried anything.
The question came up when using fastflix, which utilized almost 100% with UHD. I don't think it's related to UHD, just that it does it differently.

like this (1080p):
"C:/Users/admin/AppData/Roaming/FFmpeg/bin/ffmpeg.exe" -y -to 6492.7 -i "abc.mkv" -metadata title="abc" -max_muxing_queue_size 1024 -filter_complex "[0:0]crop=1920:800:0:140[v]" -map "[v]" -c:v libx265 -pix_fmt yuv420p10le -x265-params "aq-mode=2:repeat-headers=0:strong-intra-smoothing=1:bframes=4:b-adapt=2:frame-threads=0:hdr10_opt=0:hdr10=0" -crf 20 -preset medium -map_metadata -1 -map_chapters 0 "abc.mkv"

it's not piping but using libx265 (?), but how do I use that?

benwaggoner

22nd February 2021, 21:18

no, haven't tried anything.
The question came up when using fastflix, which utilized almost 100% with UHD. I don't think it's related to UHD, just that it does it differently.
Being UHD is actually pretty substantial. There's a lot of frame-size proportional parallelism in x265, so bigger frames use more cores. Wavefront Parallel Processing is the most obvious one, where you extra parallelism out of each 64 pixels of frame height.

_kermit

22nd February 2021, 22:08

Being UHD is actually pretty substantial. There's a lot of frame-size proportional parallelism in x265, so bigger frames use more cores. Wavefront Parallel Processing is the most obvious one, where you extra parallelism out of each 64 pixels of frame height.

ok, I guess I have to check with 1080p and fastflix first, or vice versa.

EDIT:

Indeed, 1080p is also not using all CPU power as UHD does.
Adding --pmode to fastflix doesn't change that.
I'm actually not that much concerned. If it's easy to do fine, otherwise never mind :)

But, any suggestions for 1080p CPU usage?

thanks.

charliebaby

25th February 2021, 18:27

@_kermit ? video test 2pass =
https://i.goopics.net/bem88.png

MeteorRain

26th February 2021, 17:36

But, any suggestions for 1080p CPU usage?

If you don't mind slightly lower efficiency you can try --ctu 32. That essentially half the row size, doubling the number of rows.

_kermit

3rd March 2021, 08:06

@_kermit ? video test 2pass =
https://i.goopics.net/bem88.png

I always use CRF. Is 2pass doing something better in that regard?

_kermit

3rd March 2021, 08:08

If you don't mind slightly lower efficiency you can try --ctu 32. That essentially half the row size, doubling the number of rows.

makes no difference

LigH

3rd March 2021, 08:45

I always use CRF. Is 2pass doing something better in that regard?

2-pass encoding is only required if you aim for a specified target size. In fact, it will calculate the optimal CRF value for its 2nd pass from statistics gathered during a 1st pass.

tormento

3rd March 2021, 11:15

x265 v3.5_RC1+2-f3f27198
Thanks!

Would you please release processor builds? At least SandyBridge I still own :)

Barough

3rd March 2021, 11:34

Thanks!

Would you please release processor builds? At least SandyBridge I still own :)Sry, cant help ya with that. The suite im using to build with doesn't offer any easy way to do that.

Skickat från min SM-G988B via Tapatalk

LigH

3rd March 2021, 15:00

Is it even measurable when the core routines are all assembler optimized already?

tormento

3rd March 2021, 15:29

Is it even measurable when the core routines are all assembler optimized already?
Yes, it is. Only a few percentage points but there is a difference and, on a slow Sandy Bridge, it means some minutes less of encoding.

_kermit

3rd March 2021, 16:30

I always use CRF. Is 2pass doing something better in that regard?

right, thanks.
looks as if I can't get all cores utilized 100% then
I instead just process two files at once :)

charliebaby

3rd March 2021, 17:44

For Used max cpu add this --pmode --pme --frame-threads 16 --ctu 16 --qg-size 16 --min-cu-size 16 --max-tu-size 16

:-)

benwaggoner

4th March 2021, 02:35

For Used max cpu add this --pmode --pme --frame-threads 16 --ctu 16 --qg-size 16 --min-cu-size 16 --max-tu-size 16

:-)
And if it wasn't clear, this is actually not good advice :sly:!

And if we're really trying to encode 480p on 128 cores to heat the house in the winter, don't forget --slices!

_kermit

4th March 2021, 14:20

For Used max cpu add this --pmode --pme --frame-threads 16 --ctu 16 --qg-size 16 --min-cu-size 16 --max-tu-size 16

:-)

also made no difference, around 50% util, except that the file got around 20% larger

_kermit

4th March 2021, 14:22

And if it wasn't clear, this is actually not good advice :sly:!

And if we're really trying to encode 480p on 128 cores to heat the house in the winter, don't forget --slices!

with 50% util. it's fast enough anyway.
UHD does all, so I don't lose that much time there either.

benwaggoner

4th March 2021, 20:40

also made no difference, around 50% util, except that the file got around 20% larger
Those are all parameters that reduce compression efficiency to increase parallelism. And there's a reason they're not used in --preset veryfast, as lots of those tradoffs are Not Good Deals.

LigH

5th March 2021, 08:32

OT: We are on page 403 here with the 20 posts/page default ... I start to worry for a moment when I catch a glimpse of the tab title. :o

FranceBB

5th March 2021, 11:11

OT: We are on page 403 here with the 20 posts/page default ... I start to worry for a moment when I catch a glimpse of the tab title. :o

OT2: 2719 days passed ever since the topic was created on the 24th September 2013, with 403 pages and your average of 20 posts per page, not taking into account any outliers, it gives us a total of 8060 replies, which, divided by 2719 gives us an average of 3 posts per day about x265. That's quite alright, I guess. :D

And if we're really trying to encode 480p on 128 cores to heat the house in the winter, don't forget --slices!

ROTFL

Ok, back on topic:

Those are all parameters that reduce compression efficiency to increase parallelism.
[...] Lots of those tradoffs are Not Good Deals.

This is absolutely true, no wonder they're not used by default.

@charliebaby... x265 does a pretty good job in using up multicore/multithread CPUs and even multi-socket configurations thanks to the whole Pool/NUMA implementation, especially with UHD contents (a bit less for lower resolutions), so that's generally quite alright. Of course, there is always a trade-off between parallelism and quality and sometimes it's just not worth it, which is why, whenever I have some spare room in my 40c/80th Xeon Platinum, I run more encodes at once rather than being obsessed in getting CPU usage up in one single encode (also 'cause sometimes you see the CPU go up and up without a real improvement in speed or sometimes you do see an improvement in speed at the expense of quality or compression - i.e bigger file).

(for the records, that's one of the server I have a work, I wish I had such a monster at home hehehe... although I would be kinda happy with something like an old Z840 with a 28c/40th Xeon, to be fair, but that's not gonna happen in the near future... :( )

LigH

5th March 2021, 15:44

@FranceBB: I meant the association with HTTP errors 403 (denied) and 404 (not found), so I get worried about forum software errors...

benwaggoner

6th March 2021, 00:10

I generally run slower, higher quality encodes (--preset slower and up), and on my Xeon 2x18/36 workstation I rarely get much of a boost from using both sockets below 8K. I generally just use --pools "+,-" and "-,+" to run one encode per socket. That automatically reduces the available threads for calculating defaults that change based on available threads, like --frame-threads.

Latency versus throughput are very different tuning targets.

The newer --abr-ladder presumably improves throughput when encoding the same content multiple times, and thus latency when running them all in parallel.

ShortKatz

7th March 2021, 19:18

benwaggoner

8th March 2021, 20:29

Is there some need of macOS binaries? I could provide some. Or are there only Windows users here.
x265 is certainly integrated into Mac products like Handbreak, so it is of interest.

I'd be quite interested in seeing a native M1 ARM version for Mac as well. My vague understanding today is that x265 has better perf running in x64 emulation than native ARM on M1 due to how good Apple's emulator is, even with AVX2 code. But it'd be great to be able to test head-to-head, and see how perf evolves as the ARM (including M1 and AWS's Graviton) optimizations get added.

A good justification to expense a new MacBook Pro :sly:.

Ritsuka

9th March 2021, 17:27

If you build a macOS arm64 binary please include Apple's Neon patch, you can find it on HandBrake's git repository.

benwaggoner

9th March 2021, 20:52

If you build a macOS arm64 binary please include Apple's Neon patch, you can find it on HandBrake's git repository.
Any data on the perf delta from the patch? Does it include M1-specific optimizations?

Ritsuka

10th March 2021, 10:20

~2x than no-asm. There isn't anything specific to M1.

ShortKatz

10th March 2021, 19:33

x265 3.5_RC2 for macOS, Universal binary 2 (x86_64 / arm64)
https://polysom.verilite.de/tmp/x265_3.5_RC2_macOS.zip
sha256 checksum: 60c93733f5d3703f8d7d9e1a9152fc5a84f7d35f4ccb9a1ca55c855c3b9f32a7

x265 3.5_RC2 for macOS, Universal binary 2 (x86_64 / arm64e)
https://polysom.verilite.de/tmp/x265_3.5_RC2_macOS2.zip
sha256 checksum: d46ac744a683db0453f77b856c286a789530ef47caf1f3470fa08279c4d2d956

To install, extract x265_3.5_RC2_macOS.tar.gz. With Terminal cd into the folder "x265_3.5_RC2_macOS" and drag&drop install.command into the Terminal window. Hit Enter.
To check if x265 is properly installed, open Terminal and type:
x265 --version

It should respond "x265 [info]: HEVC encoder version 3.5_RC2".

I had to include Apple's Neon patch, otherwise arm64(e) did not build properly. arm64e adds Pointer authentication, Nested virtualization and Advanced SIMD complex number support. I don't have a M1, so I can't test if the arm64e binary will work.

ghostshadow

17th March 2021, 08:39

Hello, I have a problem with my settings since version 3.4 + 26 of x265 (I am currently in 3.5_RC2, windows 10 64bit, PC Ryzen 3900x).
My concern is in increasing my Avg QP, I am almost always above 24, whereas before with version 3.4 + 20 I managed to have my Avg QP below 20.
here are the parameters used for the3.5_RC2 : --scenecut-aware-qp 3 --masking-strength 500,2,0,200,-1,-1 **** I also tested: --scenecut-aware-qp 1 --masking-strength 300,2,2 *** --scenecut-aware-qp 3 --masking-strength 500,5,6.5,100,-1,-1
for the rest of the parameters : --rd 6 --rskip 2 --deblock -3:-3 --cutree --ctu 64 --min-cu-size 8 --bframes 4 --b-adapt 2 --rc-lookahead 60 --ref 4 --limit-refs 2 --merange 57 --subme 7 --aq-mode 4 --aq-strength 0.70 --me 3 --psy-rd 2.10 --psy-rdoq 1.35 --gop-lookahead 54
I tested by increasing my --bframes and reducing my --gop-lookahead, it is a little better but nothing more.
In 3.4 + 20 I had this in parameters: --scenecut-aware-qp --scenecut-window 500 --qp-delta-ref 6 --qp-delta-nonref 7 and my QP was impeccable
Thank you for your help

Barough

17th March 2021, 13:20

x265 v3.5+8-57e817329 (https://www.mediafire.com/file/1co8t8zsvfvkbe9/x265-3.5%252B8-57e817329_Win_GCC102.7z/file) (32 & 64-bit 8/10/12bit Multilib Windows Binaries) (GCC 10.2.0)

https://bitbucket.org/multicoreware/x265_git/commits/branch/master

benwaggoner

17th March 2021, 18:20

And the 3.5 release notes (https://bitbucket.org/multicoreware/x265_git/commits/f0c1022b6be121a753ff02853fbe33da71988656) are in github. With some minor formatting fixes:

Version 3.5
===========

Release date - 16th March, 2021.

New feature
-----------
1. Real-time VBV for ABR (Average BitRate) encodes in --pass 2 using option '--vbv-live-multi-pass': Improves VBV compliance with no significant impact on coding efficiency.

Enhancements to existing features
---------------------------------
1. Improved hist-based scene cut algorithm: Reduces false positives by leveraging motion and scene transition info.
2. Support for RADL pictures at IDR scene cuts: Improves coding efficiency with no significant impact on performance.
3. Bidirectional scene cut aware Frame Quantizer Selection: Saves bits than forward masking with no noticeable perceptual quality difference.

API changes
-----------
1. Additions to x265_param structure to support the newly added features and encoder enhancements.
2. New x265_param options option '--min-vbv-fullness' and option '--max-vbv-fullness' to control min and max VBV fullness.

Bug fixes
---------
1. Incorrect VBV lookahead in option '--analysis-load' option '--scale-factor'.
2. Encoder hang when VBV is used with slices.
3. QP spikes in the row-level VBV rate-control when WPP enabled.
4. Encoder crash in option '--abr-ladder'.

benwaggoner

17th March 2021, 18:28

I'm curious about "2. Support for RADL pictures at IDR scene cuts: Improves coding efficiency with no significant impact on performance."

Has anyone tested with this? I found some value in --radl with fixed-GOP CBR encoding some years back, but haven't played with it much since. The description doesn't sound any different than what it was originally released in 2.7.

Although I see in ReadtheDocs (https://x265.readthedocs.io/en/master/cli.html#slice-decision-options)that its use in variable GOP encoding has been added.

--radl <integer>
Number of RADL pictures allowed infront of IDR. Requires closed gop interval. If enabled for fixed keyframe interval, inserts RADL at every IDR. If enabled for closed gop interval, in case of --hist-scenecut inserts RADL at every hard scenecut whereas for the --scenecut, inserts RADL at every scenecut. Recommended value is 2-3. Default 0 (disabled).

**Range of values: Between 0 and –bframes

I had not theorized that RADL would be of general benefit at hard scene cuts before. Anyone have any experience/data on this front?

P.S. MultiCoreWare, remember to update the Release Notes on ReadTheDocs! It looks like all the new parameters have already been added.

vpupkind

17th March 2021, 18:30

ghostshadow

17th March 2021, 21:12

Have you tried using just --scenecut-aware-qp 3 --masking-strength 500,-1,-1 ?

Also, what happens to quality?

The quality is very good on my zidoo. But also before.
I don't think I tested the --scenecut-aware-qp 3 with only fwdWindow, fwdRefQPDelta and fwdNonRefQPDelta.
Because in this case the much to be in --scenecut-aware-qp 1? No ?

LigH

18th March 2021, 12:43

New upload: x265 3.5+8-57e817329 (https://www.mediafire.com/file/fd739fjr6fnufxb/x265_3.5+8-57e817329.7z/file)

Bugs not yet fixed:

NASM 2.15.05 multi-line macro warnings (reported August 2020)

filler56789

18th March 2021, 22:01

New upload: x265 3.5+8-57e817329 (https://www.mediafire.com/file/fd739fjr6fnufxb/x265_3.5+8-57e817329.7z/file)

:thanks:

Bugs not yet fixed:

NASM 2.15.05 multi-line macro warnings (reported August 2020)

Can't you somehow force MABS to use an older version of nasm? :confused:

LigH

19th March 2021, 08:19

I see no reason to downgrade. Despite 9 MB of warnings scrolling through, it still works correctly. MABS wants to keep MSYS2 up-to-date, that's sensible. A fix similar to the one in x264 has been proposed to the x265 team but got denied. As a workaround, I wonder if there is an easy way to disable legacy warnings for NASM by adding the CLI parameter mentioned in the warning.

tormento

19th March 2021, 16:33

x265 3.5+8-57e817329
The number after "+", i.e. +8, is the number of applied patches?

Boulder

19th March 2021, 17:18

The number after "+", i.e. +8, is the number of applied patches?

I think it's the number of commits in that particular release branch since the initial release.

LigH

22nd March 2021, 09:34

What Boulder said.

IIRC, a specific commit gets the tag "version 3.5" assigned, which may happen "belated" (does not need to be the tip of the branch while assigning).

ShortKatz

22nd March 2021, 22:37

Bugs not yet fixed:

NASM 2.15.05 multi-line macro warnings (reported August 2020)

Yes, this is a bit annoying. I submitted a patch for this. But they didn't like, even though it was exactly the same which was done for x264.

Barough

28th March 2021, 17:49

x265 v3.5+9-bf91444e0 (https://www.mediafire.com/file/dlimoxr9z9lc6n1/x265-3.5%252B9-bf91444e0_Win_GCC102.7z/file) (32 & 64-bit 8/10/12bit Multilib Windows Binaries) (GCC 10.2.0)

https://bitbucket.org/multicoreware/x265_git/commits/branch/master

tormento

30th March 2021, 15:17

Is it even measurable when the core routines are all assembler optimized already?
In the spare time, I did some tests.

HEVC encoder version 3.5+20-4c4aee0bc [DJATOM's Mod] - SandyBridge build: 4.15 fps

HEVC encoder version 3.5+20-4c4aee0bc [DJATOM's Mod] - generic build: 3.51 fps

HEVC encoder version 3.5+9+14-6c69ed37d [Mod by Patman] - generic build: 3.76 fps

As you can see, there is some gain.

------------------------- System Environment -------------------------

StaxRip : 2.3.0
Windows : Windows 10 Enterprise 2009
Language : Italian (Italy)
CPU : Intel(R) Core(TM) i7-2600K CPU @ 3.40GHz
GPU : NVIDIA GeForce GTX 1060 3GB, Intel(R) HD Graphics 3000

------------------------- Source Script Info -------------------------

Width : 1440
Height : 1080
Frames : 6984
Time : 04:51.291
Framerate : 23.976023 (24000/1001)
Format : YUV420P8

--------------------------- Video encoding ---------------------------

D:\Eseguibili\Media\StaxRip\Apps\Encoders\x265\x265.exe --crf 22 --preset slow --output-depth 10 --colorprim bt709 --colormatrix bt709 --transfer bt709 --range limited --input-depth 16 --dither --output "F:\In\4_02 Zack Snyder's Justice League\Justice_0_temp\Justice_0_out.hevc" "F:\In\4_02 Zack Snyder's Justice League\Justice_0_temp\Justice_0.avs"

LeXXuz

31st March 2021, 10:37

In the spare time, I did some tests.

HEVC encoder version 3.5+20-4c4aee0bc [DJATOM's Mod] - SandyBridge build: 4.15 fps

HEVC encoder version 3.5+20-4c4aee0bc [DJATOM's Mod] - generic build: 3.51 fps

HEVC encoder version 3.5+9+14-6c69ed37d [Mod by Patman] - generic build: 3.76 fps

Btw. are there any Zen+/Zen2 optimized builds, if any at all?

DJATOM

31st March 2021, 11:12

Yes, I'm also using zen2 build.

benwaggoner

31st March 2021, 16:45

Yes, I'm also using zen2 build.
Someone who is good at such things could do us all a big favor by having a page of x265 binaries optimized for common cpu architectures.

That big a perf gain as reported above is surprising. And suggests that profile guided optimization may also yield some further perf benefits.

LeXXuz

31st March 2021, 19:06

Someone who is good at such things could do us all a big favor by having a page of x265 binaries optimized for common cpu architectures.

That big a perf gain as reported above is surprising. And suggests that profile guided optimization may also yield some further perf benefits.

Second that. Even a small percentage in performance increase would be great and really pay off for bigger projects.

I'm a complete noob when it comes to compiling and would love to have an optimized build for my 39x0 Ryzen machines. :o

DJATOM

31st March 2021, 20:07

You can grab those optimized builds from my repo: https://github.com/DJATOM/x265-aMod/releases/tag/3.5%2B20
They are all in the 7z archive, since it's easier to upload stuff to github with single file.
Zen3 flags aren't present in gcc10 (but that available with gcc11), so no such build for now. I'll consider doing it when will buy 5950x.

PS: cpu-specific builds are 10-bits only, while generic is multilib.

tormento

31st March 2021, 20:27

You can grab those optimized builds from my repo
Thanks for your work, that I appreciate with x264 too.

I am always wondering why, on modern computers, you and others builders keep on crunching x265.exe with upx :p

benwaggoner

31st March 2021, 20:32

You can grab those optimized builds from my repo: https://github.com/DJATOM/x265-aMod/releases/tag/3.5%2B20
They are all in the 7z archive, since it's easier to upload stuff to github with single file.
Zen3 flags isn't present in gcc10 (but that available with gcc11), so no such build for now. I'll consider doing it when will buy 5950x.

PS: cpu-specific builds are 10-bits only, while generic is multilib.
Excellent!

For a Cascade Lake system, I would use the Skylake version?

DJATOM

31st March 2021, 21:40

Yes, from 6xxx to 10xxx Intel CPUs they are using the same microarchitecture.
11xxx however uses Cypress Cove, which near 19% better than Skylake on the same clocks.

Asmodian

31st March 2021, 22:32

11xxx however uses Cypress Cove, which near 19% better than Skylake on the same clocks.

At what?

None of the standard benchmarks hold that up, so I am curious in what context that +19% claim is true. The i9-11900K seems to be slower than the i7-10700K for x265.

https://www.techpowerup.com/review/intel-core-i9-11900k/15.html

Edit: I cannot read graphs, apparently. :o
+14% for the i9-11900K v.s. the i7-10700K at only 4% higher clocks, for x265 encoding. It really is the loss of two cores that hurts the i9 11 series this generation. At 10nm and with a 12 core option Cypress Cove would probably be amazing.

LeXXuz

1st April 2021, 08:04

DJATOM

1st April 2021, 08:33

excellentswordfight

1st April 2021, 08:49

At what?

None of the standard benchmarks hold that up, so I am curious in what context that +19% claim is true. The i9-11900K seems to be slower than the i7-10700K for x265.

https://www.techpowerup.com/review/intel-core-i9-11900k/15.html

Edit: I cannot read graphs, apparently. :o
+14% for the i9-11900K v.s. the i7-10700K at only 4% higher clocks, for x265 encoding. It really is the loss of two cores that hurts the i9 11 series this generation. At 10nm and with a 12 core option Cypress Cove would probably be amazing.
I just meant that if you lock both CPUs at the same clock speed, you will have 19% uplift due to IPC improvements. That's all. Overall improvement might differ due to termals and other factors.
Swedish site sweclockers does benchmarks at locked frequency (2.8Ghz)

Cinebench R20 (floating point) 19%
https://cdn.sweclockers.com/artikel/diagram/23153?key=5f49827dc3c9a52beaf553bfe7a4aac8

Geekbench 5 (floating point) 16%
https://cdn.sweclockers.com/artikel/diagram/23189?key=d6ab89c5b375c73a183b5a04d052042e

Geekbench 5 (interger) 14%, matches zen3
https://cdn.sweclockers.com/artikel/diagram/23190?key=ca599e8494a02de1ecf6b86cf624109c

Also note that performance claims could be at stock RAM speed (2933 & 2666 for comet lake vs 3200) and most sites does tests at the same speed across platforms. And that IPC is kind of a bad metric to use in generic terms as it will very a lot between architectures depending at what kind of load we are talking about (Floating Point? AVX? Interger? Does it benefit from large cache? RAM bandwidth? etc.).

And yeah, especially for the money the i9 is disappointing at just 8c, but the i7 compared to the more expensive 5800X seems to be a rather good product.

DJATOM

1st April 2021, 09:24

I'd wait for adequate pricing on 5950x, no point for me to buy now for overprice.

tormento

1st April 2021, 15:00

I'd wait for adequate pricing on 5950x, no point for me to buy now for overprice.
Waiting for waiting, I will jump to DDR5, much better.

DJATOM

1st April 2021, 15:03

Waiting for waiting, I will jump to DDR5, much better.

I already have AM4 platform with 3900x and decent cooler, it will be a pretty cheap upgrade for me.

tormento

1st April 2021, 15:04

I already have AM4 platform with 3900x and decent cooler, it will be a pretty cheap upgrade for me.
I have a Sandy Bridge @ 4.6 stable. Still the best for the bucks. :p

Boulder

1st April 2021, 15:32

Memory speed has very little effect on the encoding speed. It's the pure calculation power that rocks.

benwaggoner

1st April 2021, 16:39

I have a Sandy Bridge @ 4.6 stable. Still the best for the bucks. :p
The lack of AVX2 is pretty painful for x265. A Skylake offers quite a bit better fps per clock.

My Sandy Bridge to Skylake upgrade gave me a >>2x fps improvement, although that included an increase in clock speed and core count.

tormento

2nd April 2021, 17:17

The lack of AVX2 is pretty painful for x265. A Skylake offers quite a bit better fps per clock.
I know. I have a Haswell Xeon in my server but can't overclock it.

benwaggoner

2nd April 2021, 18:24

I just meant that if you lock both CPUs at the same clock speed, you will have 19% uplift due to IPC improvements. That's all. Overall improvement might differ due to termals and other factors.
Thermals can be a big problem here. Often the first version of a processor supporting a new AVX version has some severe thermal throttling when heavily using those instructions which get addressed in a later revision. I recall AVX2 got much better for that reason at some point. Similar thermal issues limited the utility of AVX-512. Does anyone know if the latest Intel CPUs address that? Not like we have any with enough cores for optimal x265 encoding yet.

DJATOM

2nd April 2021, 19:18

Yet 4670K was hot enough for AVX workloads. IIRC 60 degrees at 4.3 GHz with SSE2 and 90 with AVX. I had it delidded btw.

Boulder

2nd April 2021, 19:44

benwaggoner

2nd April 2021, 20:57

If you use the system mainly for x265 encodes, a new Zen build is simply the best bang for buck. 3900X is already quite fast and Zen3 CPUs even faster. With new Zens the temps still should be quite low, my Zen2 3900X runs ~65C with full load (fixed clocks at 3.8GHz, no PBO, undervolted). Overclocking only causes a lot more heat and use of electricity will smallish benefits.
Yeah, I'm going to look at that. Got some compute-heavy stuff coming up in the next few months.

My current workstation is configured to match an AWS EC c5.18xlarge instance, so I can extrapolate performance for those. But there are now the c5a AMD instances. Up to, wow, 96 vCPU.

I wonder if I'll be looking at a Graviton2 equivalent workstation in a couple of years. There's not a lot of info about it for video encoding outside of this oddly specific post comparing H.264 encoding speeds between Graviton2 and Xeon 8-core instances (https://community.arm.com/developer/tools-software/tools/b/tools-software-ides-blog/posts/thirty-six-percent-better-video-encoding-with-aws-graviton2_2d00_based-c6g). A 36% improvement in min/dollar isn't that impressive yet, and a comparison with an AMD instance might have gone the other direction.

Curious to see what x265 can do on ARM with full SIMD optimization. The ARM SIMD world seems to becoming a lot more heterogenous than with x86-64, though.

LeXXuz

2nd April 2021, 21:42

Overclocking only causes a lot more heat and use of electricity will smallish benefits.

Completely agree. But that was always the case. Both my Zen2 CPUs are watercooled, but still, I leave them running with default PBO settings. The 3950x gets up to 180W TDP with maximum stable OC. That's about 45% more from its default limit. At the same time x265 speed only increases by less than 10%. Absolutely not worth it.

DJATOM

2nd April 2021, 22:00

I'm also fixing my cpu multiplier to 38 while voltage is 1.086. The temps are near range 58-62C, I'm with air cooling (Dark rock pro 4).

abban270

3rd April 2021, 18:48

hi every one
am asking for good preset to encode small videos size of home videos without losing most of quality
am using Staxrip
and this this an sample
https://www.mediafire.com/file/vp1x7sh0qe0zmze/test.mp4/file
thanks in advance

LigH

4th April 2021, 08:49

martinpickett

4th April 2021, 11:38

I wonder if I'll be looking at a Graviton2 equivalent workstation in a couple of years. There's not a lot of info about it for video encoding outside of this oddly specific post comparing H.264 encoding speeds between Graviton2 and Xeon 8-core instances (https://community.arm.com/developer/tools-software/tools/b/tools-software-ides-blog/posts/thirty-six-percent-better-video-encoding-with-aws-graviton2_2d00_based-c6g). A 36% improvement in min/dollar isn't that impressive yet, and a comparison with an AMD instance might have gone the other direction.

I found this article (https://www.anandtech.com/show/15578/cloud-clash-amazon-graviton2-arm-against-intel-and-amd) from AnandTech illuminating in general. For specific video encoding data you have to look at the 525.x264_r benchmark in SPECint2017 covered on pages seven (https://www.anandtech.com/show/15578/cloud-clash-amazon-graviton2-arm-against-intel-and-amd/7) and eight (https://www.anandtech.com/show/15578/cloud-clash-amazon-graviton2-arm-against-intel-and-amd/8).

imhh11

4th April 2021, 14:56

microchip8

4th April 2021, 15:43

Hi, thank you so much for doing this.
Which one should I get for Kaby Lake cpu ?

Skylake build

abban270

5th April 2021, 11:17

If you need StaxRip specific help, better ask in the StaxRip thread...

In general: There is no magic. Preserving enough quality needs enough bitrate (1-pass CRF encoding will help here). The rest is avoiding more loss with wrong options... But this clip seems to be rather generic and without interlacing.

ok i will post the same thread in staxRip section
and if you have try to re-encode this clip to x265 with good bitrate and small size dont forget to post the result
maybe i will take it
thanks

benwaggoner

5th April 2021, 21:20

I found this article (https://www.anandtech.com/show/15578/cloud-clash-amazon-graviton2-arm-against-intel-and-amd) from AnandTech illuminating in general. For specific video encoding data you have to look at the 525.x264_r benchmark in SPECint2017 covered on pages seven (https://www.anandtech.com/show/15578/cloud-clash-amazon-graviton2-arm-against-intel-and-amd/7) and eight (https://www.anandtech.com/show/15578/cloud-clash-amazon-graviton2-arm-against-intel-and-amd/8).
Thanks. And wow, Graviton2 is the winner for 16-core H.264 reference encoder perf! And also for x264 at 64vcpu. Of course, a Graviton2 vCPU is a full core and a Xeon/AMD is half of a SMT core. Still, very impressive!

IIRC, x264 has more even amounts of ARM and x86-64 SIMD optimization than x265, so x265 might not be as competitive yet. I'll try to get some tests run when I've got some time.

LigH

7th April 2021, 15:48

Fun with Yuuki-Asuna again:

Because version 3.5 is official, and I saw that the Yuuki branch had a lot of accepted commits 3 days ago, I decided to try to build a fresh clone in MSYS2/MinGW64 (MABS). But the first surprise I saw was the version number being calculated, so I stopped here...

-- x265 Release Version 2.3M+850-g2b25c9ba0+47

Please give some advice if/when/how there will be another chance to get it running again, with your help, and where to discuss that (probably better not here; possibly in msg7086's Issue tracker?)...

aegisofrime

9th April 2021, 10:19

Hey,

Could anyone point me to a "for dummies" guide to compiling x265 for Windows with gcc? I'm aware there's instructions on the x265 wiki, but... It looks like it's written for someone who is already very experienced with cross compilation, which unfortunately I'm not. I do have experience with general Linux use, so hopefully I would not need my hand held too much.

As for why I'm planning to cross compile... My objective is just to try out the new znver3 march option on my 5950X.

Thanks!

LigH

9th April 2021, 10:41

DJATOM

9th April 2021, 12:14

I just built znver3 optimized binary using gcc10.3, you can get it here (https://github.com/DJATOM/x265-aMod/releases/download/3.5%2B20/x265-x64-v3.5+20-aMod-gcc10.3.0-opt-znver3.7z).

aegisofrime

9th April 2021, 14:11

My binaries were built on an MSYS2/MinGW basis of the "media-autobuild suite" under Windows, but with a small set of manually executed shell scripts in an interactive shell, derived from the "build/msys" templates provided by MultiCoreWare, which I may share, so you could adapt them with your specific compiler optimizations as env-vars.

But if you do so, don't be afraid of a plethora of NASM warnings not yet fixed...

Adapting these scripts for MinGW cross-compilation under a real Linux is also possible, I sometimes tried that in an Ubuntu MATE VM...

If you could share them that would be great! :thanks:

I just built znver3 optimized binary using gcc10.3, you can get it here (https://github.com/DJATOM/x265-aMod/releases/download/3.5%2B20/x265-x64-v3.5+20-aMod-gcc10.3.0-opt-znver3.7z).

If it's not too much trouble, could I request for a 12-bit build as well? :)

DJATOM

9th April 2021, 14:37

Sure, link (https://github.com/DJATOM/x265-aMod/releases/download/3.5%2B20/x265-x64-v3.5+20-aMod-gcc10.3.0-opt-znver3-12b.7z).

aegisofrime

9th April 2021, 14:43

Much obliged, cheers!

LigH

9th April 2021, 17:55

x265 multilib build script for x86-64 Windows EXE in media-autobuild suite's MSYS2/MinGW64 mintty shell (https://gist.github.com/LigH-de/30dcb3d8434b29752041107482c77f55)

Configured for media-autobuild suite with ccache enabled. Naming is a bit historical, I have a few similar scripts with different content and architectures (also W32 and W32XP).

Results will remain in /build/x265_git-git/build/msys64_hdr10_ml/8bit in the MABS directory structure. This is for pure git sources from MultiCoreWare, without modifications.

eclipse98

10th April 2021, 01:18

Hi All,

Please help me understand what's going on here, because it does not quite compute.

Using a higher level (and slower) preset should, theoretically, result in:

1. Lower QP for the same bitrate.
2. Lower bitrate for the same QP.

Yet, results below on the same 2 min clip seem to contradict this assumption:

slow preset: encoded 7194 frames in 0:09:57.54 (12.04 fps), 5112.86 kb/s, Avg QP:33.05
slower preset: encoded 7194 frames in 0:37:57.12 (3.16 fps), 5515.60 kb/s, Avg QP:33.24
very slow preset: encoded 7194 frames in 1:20:43.82 (1.49 fps), 5440.53 kb/s, Avg QP:33.33

Compared to Slow preset, Slower and 'Very Slow' presets have both produced higher bitrate and higher QP.
Using default preset settings with '--crf 25 --no-sao', x265 3.4+70-aMod-gcc10.2.1 DJATOM Mod.

The only conclusion I can up with is that 'Avg QP' value is not to be trusted.

Cheers and thanks for your help!

LeXXuz

10th April 2021, 12:15

LigH

10th April 2021, 12:43

Higher presets preserving more quality is only "true with constraints", more among the faster presets than among the slower ones ... a slower preset enables more elaborate features to search for "redundancies" (similarities useful to save space). If more can be found, chances are that the difference left is smaller, so the precision of difference encoding in P and B frames can be better. But doesn't need to, certainly; it is also possible that saving some additional data for these more elaborate features requires more space than without. At least it's quite probable that there need to be fewer intra encoded blocks, so the encoding can be more efficient, in general. Yet, in practice, most users won't need any slower preset than "slower". And "placebo" is certainly a waste of time and energy.

In addition, the target for the CRF mode is the "rate factor", a metric for the loss of quality. It is possible that the features enabled in a faster preset may return more efficient results for a specific video material than the enabled features of a slower preset. There are no guarantees.

eclipse98

10th April 2021, 20:25

LigH

12th April 2021, 15:19

New upload: x265 (Yuuki-Asuna mod) 3.5+2-g2b25c9ba0+45 (Yuuki branch) (https://www.mediafire.com/file/qh6tfc5ji32trs2/x265_3.5+2-g2b25c9ba0+45_Yuuki.7z/file)

MSYS2/MinGW (Win32, Win32XP, Win64)

x265 [info]: HEVC encoder version 3.5+2-g2b25c9ba0+45
x265 [info]: build info [Windows][GCC 10.2.0][64 bit] Yuuki 8bit+10bit+12bit
x265 [info]: (lsmash 2.16.1)
x265 [info]: (libavformat 58.78.100)
x265 [info]: (libavcodec 58.136.101)
x265 [info]: (libavutil 56.72.100)

MeteorRain

13th April 2021, 04:50

Fun with Yuuki-Asuna again:

-- x265 Release Version 2.3M+850-g2b25c9ba0+47

I always forget to push new tag (3.5) from upstream to my repo, so it slips all the way to 2.3. Simply let me know next time if I forget again.

Also I've rewritten ruby code with native cmake code. It should now work without ruby package.

LigH

13th April 2021, 07:10

Thanks @MeteorRain. As you can see, it worked well; the only change missing, to my knowledge, is adapting FindFF.cmake which had an exception for Win32 which may have meant to handle MSVC only but is not suitable for GCC in MinGW.

aegisofrime

13th April 2021, 10:33

Stacey Spears

21st April 2021, 02:46

Been trying to encode HD HDR at 90 Mbps for my upcoming Ultra HD calibration disc. x265 seems to do worse the higher the bitrate you go. I get lots of keyframe pulsing and the artifact shown below, which seems to occur on I-frames. The same content at UHD resolution has much less keyframe pulsing. I have submitted a few reports on bitbucket and the only person to respond is Ben, who does not work for them. Anyone have an ear at Multicoreware that can take a look? Happy to provide the YUV source to them. I am using the latest code (3.5) on Windows.

The I-frames seem to spike while P and B encode at much lower QPs. In fact, the HD encode at 90 Mbps has higher QP spikes than the same content at UHD resolution, which is also using 90 Mbps average and 98 Mbps peak. The bitrate for the HD encode ends up ~58 Mbps after when the 2nd pass finishes.

For some reason this shot has a serious issue. It occurs every time I encode it. One other shot has the same problem. Here is an iPhone video (https://www.icloud.com/attachment/?u=https%3A%2F%2Fcvws.icloud-content.com%2FB%2FAaYbW-Pijo8mNKQTskk4M0niNXLnAUOdNdEU5m6wCW1RbcEbyBh8jAWh%2F%24%7Bf%7D%3Fo%3DAuMkslLqROuxhBikC38zLTUQpXyG2WrgjGPkzDVWCBn2%26v%3D1%26x%3D3%26a%3DCAoguDs44HiZC6zGQe_fwMhkTXIrRq8j2OqKC9jYnXyzDKMSdBDBpMCQjy8YwbS75JgvIgEAKgkC6AMA_w-vxNhSBOI1cudaBHyMBaFqJCbwSMo8Z0pDAX5e-03G6cC-B7jYWyhTdpct6SoPunVlS1K2r3Ik3INNZJuT3kPscEG3zorBhCMDv9qo9FuQWjLaE_RKbATNsgq4%26e%3D1621560842%26fl%3D%26r%3DB8C29C73-9A6C-462D-AC96-381022726046-1%26k%3D%24%7Buk%7D%26ckc%3Dcom.apple.largeattachment%26ckz%3D110FD85C-44BB-49C0-AF32-3CDA3135F8AC%26p%3D33%26s%3DuZcBjSctm_EdHuhYeaIo80C3uQ0&uk=uMwS7lcY8qN_lHE0CZzRSg&f=IMG_8636.mov&sz=21418228) of the artifact in question. If the link does not work, let me know. Not sure what it is about this shot that causes the ugly blocking near the bottom of the frame. This artifact does not show in the UHD version, just HD. Ignore the blown out look, that is Windows tone mapping.

https://www.dropbox.com/s/zlzxvwel4f9zp2x/01_Deer_Artifact.PNG?dl=1

Shot list (https://www.dropbox.com/scl/fi/z9y0h9fp1a9ae7t3gwnzg/HDR_Montage_ShotLis_21_04_19_HD_Artifacts.xlsx?dl=0&rlkey=a11q8xpd6q576yh7w9kjfmv1o) with notes on which shots have keyframe pulsing. Of these, only shot 47 has keyframe pulsing at UHD resolution. UHD has much more keyframe pulsing if I use the default qpstep value. I have to lower it to 1 to reduce the pulsing to shot 47 only.

This is a cost no object encode where cost is time. Just want the highest quality possible. It is short at ~8-minutes in length. A friend is trying the Sony encoder to see how it turns out. The UHD encode is good overall, except for one shot (Shot 47) with keyframe pulsing. The full enhancement layer of Dolby Vision almost fixes up the pulsing at UHD resolution.

Thank you in advance for any help you can provide.

LigH

21st April 2021, 07:23

If it is urgent for you, the best way to contact the developers is via their mailing list; they may read here occasionally, but not guaranteed. Point them to the Link (https://forum.doom9.org/showthread.php?p=1941161#post1941161) in the top right corner of your post. In any case, a complete command line set will be helpful.

It looks indeed like decoding issues as if the data is corrupt from a specific distance from the start of the frame bitcode, or at least the bitrate control may fail, like spending excess in the beginning and not having any reserve in the end. Here it will be important to know if you set any limiting parameters like VBV.

Stacey Spears

21st April 2021, 16:36

It looks indeed like decoding issues as if the data is corrupt from a specific distance from the start of the frame bitcode, or at least the bitrate control may fail, like spending excess in the beginning and not having any reserve in the end. Here it will be important to know if you set any limiting parameters like VBV.

Here are the majority of settings:

--frames 11235 --deblock=-2:-2 --frame-threads 1 --no-sao --qpstep 1 --rskip 0 --no-tskip --subme 7 --rd-refine --bframes 3 --b-adapt 0 --aq-mode 2 --no-cutree --input-res 1920x1080 --input-csp i420 --preset placebo --fps 24000/1001 --profile main10 --level-idc 51 --high-tier --no-open-gop --keyint 24 --min-keyint 1 --bitrate 90000 --vbv-maxrate 98000 --vbv-bufsize 99000 --colorprim bt2020 --transfer smpte2084 --colormatrix bt2020nc --chromaloc 2 --hrd --aud --sar 1:1 --input-depth 10 --output-depth 10 --uhd-bd --repeat-headers --rc-lookahead 24 --range limited --slices 1

In theory, BD can have vbv-bufsize set to 100000 but x265 does not seem that strict at enforcing and can underflow, so I set to 99000. Same for vbv-maxrate. 100000 is the max, but I find going over 98000 can underflow when trying to mux.

These same settings are used for HD and UHD encodes. Here is the stats file and csv (https://www.dropbox.com/s/32fsukqtx9989es/HDR_Montage_HD_Logs.zip?dl=1) from the 2nd pass. The deer shot starts at frame 877.

Boulder

21st April 2021, 18:49

HD MOVIE SOURCE

21st April 2021, 19:18

LigH

21st April 2021, 21:07

The mailing list address is x265-devel@videolan.org (register at mailman (https://mailman.videolan.org/listinfo/x265-devel) before writing there) ... and they will assign a matching developer internally, I believe.

Stacey Spears

22nd April 2021, 01:10

Have you tried running a CRF encode to see if it's the 2-pass rate control acting up? I'm not the expert on VBV related matters, but having the average bitrate so close to the limits could work against you. Or run a third pass and see if it changes anything.

For HDR encodes, you should also set --hdr-opt.

I am signaling HDR, I left it off the post to reduce all the settings. :)

I tried multipass and since I am using the same settings for pass 1 and 2, the extra passes made no difference. e.g. The 1st and middle pass were virtually identical. Based on my testing, the multipass helps if you use faster settings for pass 1.

I tried a lower average and it made no meaningful difference. e.g. I tried making the peak 1.5x and 2x the average.

Have not tired CRF, not sure it will mux on BD.

Thank you for your feedback.

Stacey Spears

22nd April 2021, 03:39

Boulder

22nd April 2021, 05:34

I am signaling HDR, I left it off the post to reduce all the settings. :)

I tried multipass and since I am using the same settings for pass 1 and 2, the extra passes made no difference. e.g. The 1st and middle pass were virtually identical. Based on my testing, the multipass helps if you use faster settings for pass 1.

I tried a lower average and it made no meaningful difference. e.g. I tried making the peak 1.5x and 2x the average.

Have not tired CRF, not sure it will mux on BD.

Thank you for your feedback.
--hdr-opt is actually not for signalling HDR, it contains some specific HDR-related optimizations :)

Your results are very interesting, especially that passes 1 and 2 are identical. There definitely should be a somewhat noticable difference, at least I've always seen the average bitrate of the first pass be off from the target. CRF should not cause any muxing problems if you use VBV, it does use the same rate control as a multipass but you just cannot control the final bitrate.

Does that issue occur also with standard settings from a profile, say --slower (with the HDR stuff added)? I know personally that rskip is one weird thing, it can mess things up.

PS. Aq-mode 2 is still crap for high bitrate :D

Stacey Spears

22nd April 2021, 14:02

--hdr-opt is actually not for signalling HDR, it contains some specific HDR-related optimizations :)

Your results are very interesting, especially that passes 1 and 2 are identical. There definitely should be a somewhat noticable difference, at least I've always seen the average bitrate of the first pass be off from the target. CRF should not cause any muxing problems if you use VBV, it does use the same rate control as a multipass but you just cannot control the final bitrate.

Does that issue occur also with standard settings from a profile, say --slower (with the HDR stuff added)? I know personally that rskip is one weird thing, it can mess things up.

PS. Aq-mode 2 is still crap for high bitrate :D

This is what I removed from the command line to reduce clutter:
--master-display G(8500,39850)B(6550,2300)R(35400,14600)WP(15635,16450)L(%peak%0000,50) --max-cll "%maxcll%,%maxfall%" --hdr10 --hdr10-opt

hdr10-opt lowers the QP of chroma relative to luma for perceptual reasons.

If I use the slow first pass with identical settings, then pass 1 and the middle pass are virtually identical. The final pass does not change. If I use faster settings for pass 1, then the middle pass does make a difference. If I use the same settings for both passes, the final pass is much better at converging the final bitrate with UHD. If I use a faster first pass, then the final pass comes in 2-3 Mbps lower than the target bitrate. Hope that makes sense. I have tested these more than once as I thought something went wrong.

With CRF, can you still set a peak? I do have the hard limit of 100 Mbps, though if I use > 98, it often won't mux. x265 seems a little slapdash on enforcing VBV constraints, so I always have to leave room for a little slop.

I have used aqmode of 0 and 1 as well. I used the 0 since the QP reported in the CSV is not the QP I was interested in, which is shown in the stats file. I wish I could dump a stats file of the final pass since it contains several different QP values.

I have tried faster, slow and veryslow in addition to placebo.

When I worked on VC1, we always honored the P and B frame delta the users set. e.g. if P delta was set to 1.5 and I was 5, then P would be 6.5. x265 does not seem to work this way. I have I frames with QPs over 20 and then the P and B frames with QP under 10 in the same GOP. It is this situation where the keyframe pulsing rears its ugly head.

I also tried the rc-grain rate control, which was interesting, but did not solve the pulsing issue. It looked promising at first.

I may try breaking the montage into shorter clips and encoding. I do worry about the concat points. I think if I encode the one shot with pulsing, by itself, it encodes fine. Breaking things up does make the Dolby Vision version a bit more complicated. On a side note, the Dolby Vision Full Enhancement layer actually cleans up the keyframe pulsing a lot. However, I am not using Dolby Vision for the HD version. Here is the menu mock-up (https://www.dropbox.com/scl/fi/mzj8b3nfb9y9753vpn7i4/Ultra_HD_Menu_Layout_HDR_Disc_03_HDR_Montage.pptx?dl=0&rlkey=otzmm1tlrntvopudw8pfsipln) for the Montage disc. (it is a 3-disc set) We support all HDR formats on Blu-ray and several different nit levels.

Thank you again, I do appreciate all of the feedback.

Boulder

22nd April 2021, 18:41

With CRF, can you still set a peak? I do have the hard limit of 100 Mbps, though if I use > 98, it often won't mux. x265 seems a little slapdash on enforcing VBV constraints, so I always have to leave room for a little slop.
Yes, VBV can be applied also to CRF encodes.

However, looking at the frame shows something more wrong than any of the usual suspects because the blocking is just at the bottom and not in the whole frame.

When I worked on VC1, we always honored the P and B frame delta the users set. e.g. if P delta was set to 1.5 and I was 5, then P would be 6.5. x265 does not seem to work this way. I have I frames with QPs over 20 and then the P and B frames with QP under 10 in the same GOP. It is this situation where the keyframe pulsing rears its ugly head.
Keyframe pulsing might be fought by lowering the --ipratio and --pbratio values. --tune grain sets them low for this reason I think.

Thank you again, I do appreciate all of the feedback.
No worries :) Problems like these tickle my brain a bit which is always a plus.

EDIT: and if you like, I can try encoding the source on my system to verify the issue. The whole file is probably way too big, but maybe a scene or two before and after the problemous one could do.

Blue_MiSfit

22nd April 2021, 21:58

I too am eagerly monitoring this thread :) Quite fascinating as I've never seen behavior like this with x265 before.

excellentswordfight

23rd April 2021, 08:22

With CRF, can you still set a peak? I do have the hard limit of 100 Mbps, though if I use > 98, it often won't mux. x265 seems a little slapdash on enforcing VBV constraints, so I always have to leave room for a little slop.

Yes, you can set vbv with crf. Although you might still want to use 2ass abr for your use case, it would still be interesting to see if the issue persists.

Also note that different bd disks support different muxrates, not all will support a 100Mbps video track, and altough I havnt done any Dolby vision authoring from what i can tell from the spec is that the 100Mbps video track limit includes the enhancement layer.

I have tried faster, slow and veryslow in addition to placebo.

I guess that you have also tried more default like settings?

Does the issue persist with something simple like this muxed to an mp4?

--preset slower --hdr10-opt --aq-mode 1 --vbv-maxrate 98000 --vbv-bufsize 98000 --uhd-bd --bitrate 50000 + hdr/color flags

Stacey Spears

23rd April 2021, 16:28

EDIT: and if you like, I can try encoding the source on my system to verify the issue. The whole file is probably way too big, but maybe a scene or two before and after the problemous one could do.

I will go ahead and share everything so you can test encoding. It gives everyone some more content to test encoders. The moose in shot 23 has been used for 8K vs 4K blind test hosted at WB Labs and presented at HPA/SMPTE. Scott Wilkinson had written about that blind test in the past.

The content was shot on RED Dragon VistaVision and Monstro VistaVision. The camera and lens used are in the shot list. Was graded by Dolby on a Pulsar set to BT.2020 (not P3) mode. Resolve was used for the actual grading. Dolby exported 16-bit EXRs, which preserved the out of gamut colors. Resolve does not color manage wide gamut content very well and the source was far greater than 2020. I used Transkoder to render the various Dolby trim pass versions and to fix the gamut issues from Resolve. I believe Transkoder is 2nd to none in gamut and tone mapping. This was the 1000 nit trim pass render from the 10,000 nit grade. Since the Pulsar is a 4000 nit display, we used scopes to place stuff > 4000 nits and > P3.

For anyone that might have the current disc, this has been re-edited with four shots replaced and others deleted to increase the length of shots. Trying to get a lot of different content into a single demo clip. It was also re-graded to fix some of the gamut and luminance clipping, but still needed Transkoder to fully address the gamut issues.

We also have new Atmos audio, but that is not included since the rights I paid for are on-disc only.

Here are all the files (https://www.dropbox.com/sh/mundy1o2kpdtyfc/AACd4PsLBnByG85VGWTm9hJZa?dl=0) you need.
1. YUV (inside a 7z to reduce the size to 14 GB. When you decompress, it will be ~65 GB or 11235 frames)

2. The bat files. I use two bat files. 01_Encode_HDR_Montage_HD_01000.bat is the file that feeds the main bat file Encode_HDR_Montage_HD_2pass_90.bat. Obviously you will need to tweak the paths. And yes, the MaxCLL is higher than Mastering Display Luminance. Those are the odd specular highlights that go above 1000. Lots of them.

3. The QPFile is set so that I can set chapter points on any shot. For authoring, we have a marker every second that allows the user to switch between the different montage flavors and jump to roughly the same point, which makes it easier to compare.

4. There is some moiré in various shots due to the scaling from UHD2 (7680x4320) to HD. I will re-visit before the final-final encoding. The peacock feather and parrots feathers show the moiré. Looks much better at UHD. In fact, one of the montages is a UHD vs. HD. In that case, I had to scale the HD to UHD and then I do the circular wipe.

5. AnalyzeIt is my YUV viewer. Lots of little tools built-in. (WFM, Vector, Histogram, Pixel Values, etc...) The raw_data is a text file with basic image info so you don't have to manually type in the resolution, color space and FOURCC codes when opening a YUV file. This also opens AVIs and BMPs. I wrote this back in 2004, so please forgive the Win32 UI. :) It is also a viewer, not a player. It does not scale the image, so you can drag it around inside the viewer. There is a zoom window you can bring up. (alt + z) The highlight feature will show all pixels <16 (or 64 in 10-bit) as green and > 235 (940) as hot pink. It highlights the pixels outside the range AFTER conversion to RGB, not in YUV. You can view any channel, R, G, B, Y, U and V. The histogram can show both YUV and RGB histograms. It is the tool we wrote so we can verify our test patterns. Alt + D will bring up the macroblock widow. (16x16 grid of pixel values) Use the arrow keys to move the cursor around.

6. The shot list with the shots in question called out in column P. (Called HD Encode for some odd reason)

7. The hevc encoded file from the bat file.

Look forward to seeing what people come up with.

We use HTR mode, which allows ~126 Mbps for combined audio and video. Video is still limited to 100 Mbps. When doing Dolby Vision, both layers must fit within the 100 Mbps. This clip is HDR10 only. HTR mode is limited to a % of the disc. e.g. On a BD100, you can use HTR for ~87 GBs of the disc.

Normally I would not share the YUV source, but this is the HD version and it would be nice to get this resolved.

Previously I said cost-no-object, where cost is time. I did try and disable WPP, but that is tad slow!!!! :p So really not cost-no-object in that regard.

The overall APL of this clip is high, which trips up some tone mapping algorithms. e.g. Windows tone mapping fails miserably trying to tone map. MadVR, on the other hand, handles it like a hot knife through butter. Normally I QC on an LG OLED. Dynamic tone mapping enabled will reduce clipping, but it changes stuff it should not.

You may notice the dropbox folder is named Ben. I originally shared this with Ben Waggoner.

benwaggoner

23rd April 2021, 19:00

Here are the majority of settings:

--frames 11235 --deblock=-2:-2 --frame-threads 1 --no-sao --qpstep 1 --rskip 0 --no-tskip --subme 7 --rd-refine --bframes 3 --b-adapt 0 --aq-mode 2 --no-cutree --input-res 1920x1080 --input-csp i420 --preset placebo --fps 24000/1001 --profile main10 --level-idc 51 --high-tier --no-open-gop --keyint 24 --min-keyint 1 --bitrate 90000 --vbv-maxrate 98000 --vbv-bufsize 99000 --colorprim bt2020 --transfer smpte2084 --colormatrix bt2020nc --chromaloc 2 --hrd --aud --sar 1:1 --input-depth 10 --output-depth 10 --uhd-bd --repeat-headers --rc-lookahead 24 --range limited --slices 1

In theory, BD can have vbv-bufsize set to 100000 but x265 does not seem that strict at enforcing and can underflow, so I set to 99000. Same for vbv-maxrate. 100000 is the max, but I find going over 98000 can underflow when trying to mux.

These same settings are used for HD and UHD encodes. Here is the stats file and csv (https://www.dropbox.com/s/32fsukqtx9989es/HDR_Montage_HD_Logs.zip?dl=1) from the 2nd pass. The deer shot starts at frame 877.
Have you found any value from --subme 7? It's beyond even --preset placebo and I suspect hasn't gotten any real tuning in years.

Stacey Spears

23rd April 2021, 20:27

Have you found any value from --subme 7? It's beyond even --preset placebo and I suspect hasn't gotten any real tuning in years.

On test patterns using a fixed QP, which is what I do for patterns, yes. On motion content with rate control, hard to say.

benwaggoner

23rd April 2021, 22:19

On test patterns using a fixed QP, which is what I do for patterns, yes. On motion content with rate control, hard to say.
That would make sense.

Just curious - have you tried just --qp to what you want, keeping --vbv-maxrate and --vbv-bufsize without setting bitrate, and changing --aq-mode 0? That should result in a truly fixed QP except for when it would exceed the peak bitrate. With --csv-log-level 2, you can think look at the log file to see if the QP was exceeded at any point.

With test content where encoding time isn't material, you might also try --tskip and perhaps even --cu-lossless. At very high quality/low qp, they can save some bits for very sharp-edged blocks. In special cases, --fades and --frame-dup might help a bit as well.

Stacey Spears

24th April 2021, 11:38

That would make sense.

Just curious - have you tried just --qp to what you want, keeping --vbv-maxrate and --vbv-bufsize without setting bitrate, and changing --aq-mode 0? That should result in a truly fixed QP except for when it would exceed the peak bitrate. With --csv-log-level 2, you can think look at the log file to see if the QP was exceeded at any point.

With test content where encoding time isn't material, you might also try --tskip and perhaps even --cu-lossless. At very high quality/low qp, they can save some bits for very sharp-edged blocks. In special cases, --fades and --frame-dup might help a bit as well.

tksip made things worse on some synthetic test patterns, which surprised me given that it should help them. To qualify worse, I mean I am encoding all test patterns at QP 0 and they were further away from mathematically lossless than w/o. The pixel values matter on a test pattern, so visually lossless is not enough. tu-inter of 4 also had a negative impact more often than not and that is the default for placebo. I do use cu-lossless for all test patterns.

Boulder

25th April 2021, 09:40

Stacey Spears

25th April 2021, 16:13

I was able to reproduce the bad frame in the deer scene with a 2-pass encode and a mix of your settings and --preset slower, but it did not happen with identical settings and CRF 1. I still don't know how you got pretty much identical results out of both passes as the avg bitrate for pass 1 was 37581 kbps and pass 2 ended up at 60238 kbps. I just tuned the settings slightly for some faster encoding, --limit-refs 3, --no-amp and --rskip 2 were the biggest changes.

By the way, if you want to get a stats file of the second and final pass, I think you can use --pass 3 instead of --pass 2. That will overwrite the stats file with the results from the current encode.

Glad you were able to download everything. Lately dropbox seems to error out near the end of large files when using a browser. The app seems to always finish as it is more resilient. Would you mind trying a pass 3 and see if the pass 3 stats and pass 1 stats are identical or at least close to each other? Assuming you used the same settings for pass 1, 2 and 3.

It is interesting how pass 1 has such a low bitrate, I get the same results. Then the estimate for pass 2 is about 10 Mbps (going from memory) higher than where it actually ends up.

Does your CRF encode have the same keyframe pulsing as 2pass? The snow shot (Shot 5) with the fence is one, just stare at the top of the fence where the snow sits on it. The grass area on the bison shot. (Shot 16) Or the dirt/rocks, near the tree stump. (shot 47) What bitrate did CRF end up at?

Would you mind posting the changes you made to setup CRF vs. 2pass? It is not something I have ever used with x264 or x265.

Stacey Spears

25th April 2021, 16:35

This is a bit off topic, but these steps were used to create the montage, so I though it was worth sharing. Images 7 and 8 are the end results, prior to scaling down to UHD / HD resolutions of course. If this is not appropriate here, please let me know and I will remove it.

I thought some might be interested in this. I put together some TIFF images at each stage of processing that we do on our content so you can see how it evolved. The first six images are in the REDWideGamutRGB / Log3G10 space. The images are 7680x4320. One of the montage videos is Graded vs. Ungraded, which I hope some will find interesting as you rarely get to see how much work is done to make the images / videos look the way they do.

If you are not used to looking at a log image, or wide gamut image, then you might find them odd looking. They will look flat and milky when looking at a log mage in gamma. Same when looking at an HDR image in gamma instead of PQ.

https://www.dropbox.com/s/iemk0ukdil961b5/SM_Processing_Example.7z?dl=1

1. 00_Legacy.tif - This is the legacy demosaic algorithm. It was used 100% from 2007 - 2017, which is when IPP2 was released. Some still choose Legacy today.
2. 01_IPP2.tif - The order of operations changed to improve quality and it has a new demosaic algorithm that retains more detail than legacy.
3. 03_IPP2_TNR.tif - This is the IPP2 image with temporal noise reduction applied.
4. 04_IPP2_TNR_Decon_Mask_C00.tif - This mask is showing the pixels that deconvolution will touch, which is every pixel in this example.
5. 05_IPP2_TNR_Decon_Mask_C05.tif - This mask shows the settings I used. This way I avoid the noise in the sky getting sharper.
6. 06_IPP2_TNR_Decon.tif - This is IPP2 w/ temporal noise reduction and deconvolution.
7. 07_IPP2_TNR_Decon_Grade_HDR.tif - This is image number 6 graded for HDR (New disc)
8. 08_IPP2_TNR_Decon_Grade_SDR.tif - This is image number 6 graded for SDR (New disc)

Not sure why I skipped over number 2.

Majorlag

25th April 2021, 17:05

Stacy,
The deer scene error is caused from --no-cutree in your Encode_HDR_Montage_HD_2pass_90.bat file. Remove that and it cleans up a lot of the areas that have weird pixel issues.

Some things I would suggest, lower subme to at least 4 or default of 2, raise both keyint, and min-keyint (lots of bits wasted on I frames that don't compress as much), --b-adapt 0 with to low of keyint's can cause a pulsating issue as well, since it is fixed length GOP, random size GOP are harder to see clean crisp I frames because you are not seeing them at the same intervals

Hope that helps.

Boulder

25th April 2021, 17:53

raise both keyint, and min-keyint (lots of bits wasted on I frames that don't compress as much), --b-adapt 0 with to low of keyint's can cause a pulsating issue as well, since it is fixed length GOP, random size GOP are harder to see clean crisp I frames because you are not seeing them at the same intervals
I think that the problem is the --uhd-bd parameter, which fixes those keyframe values.

Would you mind trying a pass 3 and see if the pass 3 stats and pass 1 stats are identical or at least close to each other? Assuming you used the same settings for pass 1, 2 and 3.
They are quite different.
1st pass:
in:877 out:877 type:I q:12.00 q-aq:8.56 q-noVbv:12.00 q-Rceq:312.57 tex:1854107 mv:81619 misc:8710 icu:8160.00 pcu:0.00 scu:0.00 sc:1 ;
in:881 out:878 type:P q:12.00 q-aq:8.39 q-noVbv:12.00 q-Rceq:268.43 tex:979467 mv:33511 misc:11910 icu:99.48 pcu:6890.41 scu:1170.10 sc:0 ;
in:879 out:879 type:B q:13.00 q-aq:9.89 q-noVbv:13.14 q-Rceq:268.43 tex:453163 mv:23512 misc:10933 icu:5.54 pcu:4624.25 scu:3530.21 sc:0 ;

The 3rd (actually second but with --pass 3 to update the stats file) pass:
in:877 out:877 type:I q:14.86 q-aq:10.36 q-noVbv:-8.54 q-Rceq:222.02 tex:3080701 mv:198493 misc:7562 icu:8160.00 pcu:0.00 scu:0.00 sc:0 ;
in:881 out:878 type:P q:0.00 q-aq:0.13 q-noVbv:-8.62 q-Rceq:222.02 tex:3427499 mv:52581 misc:3872 icu:1937.24 pcu:6173.14 scu:49.61 sc:0 ;
in:879 out:879 type:B q:1.14 q-aq:0.30 q-noVbv:1.14 q-Rceq:222.02 tex:2602273 mv:36488 misc:5751 icu:116.61 pcu:7713.47 scu:329.93 sc:0 ;

Does your CRF encode have the same keyframe pulsing as 2pass?
There definitely is pulsating in the CRF encode with --no-cutree but it's not really visible if cutree is enabled. --tune grain would probably make it even less visible (and spend more bits in the P- and B-frames).

Would you mind posting the changes you made to setup CRF vs. 2pass?
It's very simple - you can use the same settings as with multipass, just remove the --pass and --bitrate parameters and put --crf X in. I used --crf 1 in my tests. The lower the value, the higher expected quality. Basically it's a one-pass encode closest to constant quality that x265 can give you.

Or if you want to run a CRF encode and get the stats file, use --pass 1 but don't give the bitrate.

EDIT: just tested a multipass encode of 1000 frames without the VBV settings and no issues with the deer frame, it's clean. So it definitely is the VBV which causes the issue.

Stacey Spears

25th April 2021, 18:30

Boulder

25th April 2021, 19:40

I can't raise keyint. BD has a GOP limit of 1-second, so 24.

Should I try --b-adapt 1 or 2?

I will:
1. Remove --no-cutree
2. Set --b-adapt 2
3. Set --subme 4
4. Add pass 3

And then kick off a new encode.

Thank you.
Those look good. You can replace --pass 2 directly by --pass 3, the only difference is that --pass 3 will update the stats file. So set a new name for it or copy the original if you want to compare.

Stacey Spears

25th April 2021, 19:51

Those look good. You can replace --pass 2 directly by --pass 3, the only difference is that --pass 3 will update the stats file. So set a new name for it or copy the original if you want to compare.

Any thoughts on ipratio or pbratio? Or aq-mode?

Luckily each pass takes long enough that it is easy to copy the stats file before its overwritten. :)

Boulder

25th April 2021, 20:45

I'd try maybe --ipratio 1.2 --pbratio 1.1 since you seem to have a good headroom. Aq-mode 0 or 1 would be my choice.

rwill

25th April 2021, 22:45

@Stacey Spears: Have you tried a different encoder yet ?

The keyframe pulsing at frame ~600 is caused because x265 is too afraid to pump bits into the I-Frames, so they are encoded with a really high quantizer in comparison to the surrounding frames. Likely related to VBV settings.

The broken lower part at the deer at frame ~920 are caused because otherwise the frame would exceed the bits allocated for the frame. The encoder entered the "emergency mode". x265 can only spend so much bits on a frame when frame threading is enabled and video buffer limits have to be honored.

Stacey Spears

25th April 2021, 23:52

@Stacey Spears: Have you tried a different encoder yet ?.

It is being encoded with Sony's at the moment.

Boulder

26th April 2021, 05:26

The broken lower part at the deer at frame ~920 are caused because otherwise the frame would exceed the bits allocated for the frame. The encoder entered the "emergency mode". x265 can only spend so much bits on a frame when frame threading is enabled and video buffer limits have to be honored.

The interesting thing is that CRF mode seems to work much better, even though it should use the same rate control method than a multipass.

Sharc

26th April 2021, 08:53

Interesting findings.
It reminds me somehow on the x264 discussion here:
https://forum.doom9.org/showthread.php?p=1848989#post1848989

Boulder

26th April 2021, 10:08

Interesting findings.
It reminds me somehow on the x264 discussion here:
https://forum.doom9.org/showthread.php?p=1848989#post1848989
I wouldn't be surprised if the rate control code was ported from x264 as-is.

Stacey Spears

26th April 2021, 14:22

Interesting findings.
It reminds me somehow on the x264 discussion here:
https://forum.doom9.org/showthread.php?p=1848989#post1848989

Thank you for sharing that post. I liked Ben's comment about % of users using VBV vs. % of content actually using it.

When I have the Sony, and possible Ateme, encodes I will share them so everyone can compare the results to x265.

If the x265 devs take some time to investigate and address the issue(s), I will share the UHD source as well so that it can be used for future CODEC comparisons, if there is interest of course. Its ~260 GBs in 10-bit 4:2:0. Same for the full resolution (7680x3420).

rwill

27th April 2021, 12:54

When I have the Sony, and possible Ateme, encodes I will share them so everyone can compare the results to x265.

I gave the sequence a shot using some other encoder. Sadly I used a Dolby Pulsar P3D65 as Mastering Display and left the CLL SEI at 0,0.

It is here: https://drive.google.com/file/d/1-Uz8jcFZYJD-t-NRQ6cPvRoZvht8kU7L/view?usp=sharing

I set the average rate to 80M and video buffer rate to 100M. GOP is closed and keyframe interval is fixed.

The encoder has problems reaching the average rate because of the low resolution of the sequence. I did a lossless HEVC encode and it ended up being ~9.2GB, this encode is ~4.5GB. If someone wants to see a lower average rate encode - I am staying around here for a while, just request.

I don't know if the encode is really HEVC BluRay compliant, please advise, this currently interests me the most.

Maybe we should start a new thread if we diverge from x265 to general encoding of high rate HEVC using different encoders....

Stacey Spears

27th April 2021, 15:35

Those look good. You can replace --pass 2 directly by --pass 3, the only difference is that --pass 3 will update the stats file. So set a new name for it or copy the original if you want to compare.

With the settings mentioned, I still had the deer issue and the pass 3 stats was 100% identical to pass 1.

I have since made some changes, which you posted about using. (preset slower, limit refs to 3, etc...) In that case the stats file is a lot different between pass 1 and pass 3. I want to encode the first 1000 frames, as you did, and track down what is making pass 3 the same vs. different.

While I wait for that 3 pass encode to finish, I wanted to share some stats file differences between the HD and UHD encodes, both targeting 90 Mbps. This is frame 590, the shot with the fence in the snow.

I do need to break into three posts due to length.

HD Stats Pass 1
in:590 out:590 type:I q:32.65 q-aq:24.73 q-noVbv:9.56 q-Rceq:0.98 tex:2362738 mv:322911 misc:8316 icu:8160.00 pcu:0.00 scu:0.00 sc:0 ;
in:594 out:591 type:P q:24.13 q-aq:16.24 q-noVbv:24.45 q-Rceq:0.98 tex:2802507 mv:65130 misc:17084 icu:251.10 pcu:6673.32 scu:1235.59 sc:0 ;
in:592 out:592 type:B q:24.00 q-aq:22.54 q-noVbv:24.45 q-Rceq:0.98 tex:2135 mv:2263 misc:2657 icu:4.03 pcu:93.19 scu:8062.79 sc:0 ;
in:591 out:593 type:b q:24.00 q-aq:23.66 q-noVbv:24.45 q-Rceq:0.98 tex:1212 mv:1070 misc:1244 icu:0.00 pcu:170.00 scu:7990.00 sc:0 ;
in:593 out:594 type:b q:24.00 q-aq:23.79 q-noVbv:24.45 q-Rceq:0.98 tex:1497 mv:613 misc:952 icu:0.00 pcu:284.84 scu:7875.16 sc:0 ;
in:598 out:595 type:P q:22.00 q-aq:14.14 q-noVbv:22.45 q-Rceq:0.98 tex:724517 mv:20146 misc:29920 icu:31.23 pcu:3160.99 scu:4967.78 sc:0 ;
in:596 out:596 type:B q:22.00 q-aq:20.38 q-noVbv:22.00 q-Rceq:0.98 tex:2700 mv:5692 misc:4673 icu:0.25 pcu:287.87 scu:7871.88 sc:0 ;
in:595 out:597 type:b q:22.00 q-aq:21.67 q-noVbv:22.00 q-Rceq:0.98 tex:1306 mv:2392 misc:2169 icu:0.00 pcu:341.51 scu:7818.49 sc:0 ;
in:597 out:598 type:b q:22.00 q-aq:21.71 q-noVbv:22.00 q-Rceq:0.98 tex:1656 mv:1427 misc:1580 icu:0.00 pcu:398.93 scu:7761.07 sc:0 ;
in:602 out:599 type:P q:20.00 q-aq:12.14 q-noVbv:20.45 q-Rceq:0.98 tex:1260706 mv:26360 misc:28813 icu:14.10 pcu:4541.39 scu:3604.50 sc:0 ;
in:600 out:600 type:B q:20.00 q-aq:18.31 q-noVbv:20.00 q-Rceq:0.98 tex:5446 mv:11262 misc:8326 icu:0.00 pcu:455.35 scu:7704.65 sc:0 ;
in:599 out:601 type:b q:20.00 q-aq:19.67 q-noVbv:20.00 q-Rceq:0.98 tex:2422 mv:4193 misc:3666 icu:0.00 pcu:295.93 scu:7864.07 sc:0 ;
in:601 out:602 type:b q:20.00 q-aq:19.58 q-noVbv:20.00 q-Rceq:0.98 tex:2857 mv:2804 misc:2831 icu:0.00 pcu:355.11 scu:7804.89 sc:0 ;
in:606 out:603 type:P q:18.00 q-aq:9.99 q-noVbv:18.45 q-Rceq:0.98 tex:1771367 mv:42536 misc:31538 icu:14.36 pcu:5179.59 scu:2966.06 sc:0 ;
in:604 out:604 type:B q:18.00 q-aq:16.09 q-noVbv:18.00 q-Rceq:0.98 tex:7529 mv:16065 misc:10904 icu:0.25 pcu:637.94 scu:7521.81 sc:0 ;
in:603 out:605 type:b q:18.00 q-aq:17.47 q-noVbv:18.00 q-Rceq:0.98 tex:4214 mv:6237 misc:5051 icu:0.00 pcu:272.76 scu:7887.24 sc:0 ;
in:605 out:606 type:b q:18.00 q-aq:17.52 q-noVbv:18.00 q-Rceq:0.98 tex:4142 mv:3512 misc:3854 icu:4.03 pcu:450.31 scu:7705.66 sc:0 ;
in:610 out:607 type:P q:16.00 q-aq:8.07 q-noVbv:16.45 q-Rceq:0.98 tex:2048147 mv:40697 misc:28198 icu:8.56 pcu:5704.70 scu:2446.74 sc:0 ;
in:608 out:608 type:B q:16.00 q-aq:14.10 q-noVbv:16.00 q-Rceq:0.98 tex:16180 mv:20156 misc:13966 icu:0.00 pcu:835.90 scu:7324.10 sc:0 ;
in:607 out:609 type:b q:16.00 q-aq:15.54 q-noVbv:16.00 q-Rceq:0.98 tex:6289 mv:9631 misc:7120 icu:4.03 pcu:327.66 scu:7828.31 sc:0 ;
in:609 out:610 type:b q:16.00 q-aq:15.34 q-noVbv:16.00 q-Rceq:0.98 tex:7507 mv:6508 misc:5773 icu:0.00 pcu:400.44 scu:7759.56 sc:0 ;
in:613 out:611 type:P q:14.00 q-aq:10.87 q-noVbv:14.45 q-Rceq:0.98 tex:149569 mv:27015 misc:20631 icu:4.28 pcu:1307.61 scu:6848.10 sc:0 ;
in:612 out:612 type:B q:14.00 q-aq:12.12 q-noVbv:14.00 q-Rceq:0.98 tex:45918 mv:20063 misc:13717 icu:0.00 pcu:1103.87 scu:7056.13 sc:0 ;
in:611 out:613 type:b q:14.00 q-aq:13.77 q-noVbv:14.00 q-Rceq:0.98 tex:13451 mv:12476 misc:8502 icu:0.00 pcu:1041.91 scu:7118.09 sc:0 ;
in:614 out:614 type:I q:32.74 q-aq:24.80 q-noVbv:15.17 q-Rceq:0.98 tex:2319615 mv:326495 misc:8189 icu:8160.00 pcu:0.00 scu:0.00 sc:0 ;
in:618 out:615 type:P q:28.00 q-aq:20.28 q-noVbv:27.76 q-Rceq:0.98 tex:1172141 mv:38582 misc:18528 icu:112.83 pcu:4053.30 scu:3993.87 sc:0 ;
in:616 out:616 type:B q:27.00 q-aq:25.85 q-noVbv:26.76 q-Rceq:0.98 tex:1288 mv:1832 misc:1712 icu:0.50 pcu:110.06 scu:8049.44 sc:0 ;
in:615 out:617 type:b q:28.00 q-aq:28.00 q-noVbv:27.76 q-Rceq:0.98 tex:47 mv:290 misc:202 icu:0.00 pcu:216.09 scu:7943.91 sc:0 ;
in:617 out:618 type:b q:28.00 q-aq:28.00 q-noVbv:27.76 q-Rceq:0.98 tex:79 mv:256 misc:149 icu:0.00 pcu:215.08 scu:7944.92 sc:0 ;
in:622 out:619 type:P q:26.00 q-aq:18.25 q-noVbv:25.76 q-Rceq:0.98 tex:541182 mv:21561 misc:28746 icu:28.21 pcu:2836.86 scu:5294.93 sc:0 ;
in:620 out:620 type:B q:26.00 q-aq:25.46 q-noVbv:26.00 q-Rceq:0.98 tex:799 mv:2535 misc:2298 icu:0.00 pcu:105.02 scu:8054.98 sc:0 ;
in:619 out:621 type:b q:26.00 q-aq:26.05 q-noVbv:26.00 q-Rceq:0.98 tex:158 mv:951 misc:909 icu:0.00 pcu:251.85 scu:7908.15 sc:0 ;
in:621 out:622 type:b q:26.00 q-aq:25.96 q-noVbv:26.00 q-Rceq:0.98 tex:315 mv:641 misc:590 icu:0.00 pcu:298.95 scu:7861.05 sc:0 ;
in:626 out:623 type:P q:24.00 q-aq:16.08 q-noVbv:23.76 q-Rceq:0.98 tex:1059164 mv:29050 misc:31322 icu:39.79 pcu:4044.24 scu:4075.97 sc:0 ;
in:624 out:624 type:B q:24.00 q-aq:23.09 q-noVbv:24.00 q-Rceq:0.98 tex:1299 mv:5739 misc:4323 icu:0.00 pcu:206.27 scu:7953.73 sc:0 ;
in:623 out:625 type:b q:24.00 q-aq:23.68 q-noVbv:24.00 q-Rceq:0.98 tex:888 mv:2411 misc:2154 icu:0.00 pcu:227.93 scu:7932.07 sc:0 ;
in:625 out:626 type:b q:24.00 q-aq:23.81 q-noVbv:24.00 q-Rceq:0.98 tex:602 mv:1005 misc:1017 icu:0.00 pcu:288.37 scu:7871.63 sc:0 ;
in:630 out:627 type:P q:22.00 q-aq:14.11 q-noVbv:21.76 q-Rceq:0.98 tex:1390404 mv:35178 misc:31387 icu:30.73 pcu:4608.13 scu:3521.14 sc:0 ;
in:628 out:628 type:B q:22.00 q-aq:20.82 q-noVbv:22.00 q-Rceq:0.98 tex:2155 mv:9697 misc:6795 icu:0.25 pcu:252.10 scu:7907.64 sc:0 ;
in:627 out:629 type:b q:22.00 q-aq:21.43 q-noVbv:22.00 q-Rceq:0.98 tex:1404 mv:3607 misc:3162 icu:0.00 pcu:205.01 scu:7954.99 sc:0 ;
in:629 out:630 type:b q:22.00 q-aq:21.94 q-noVbv:22.00 q-Rceq:0.98 tex:984 mv:1500 misc:1561 icu:0.00 pcu:332.95 scu:7827.05 sc:0 ;
in:634 out:631 type:P q:20.00 q-aq:12.20 q-noVbv:19.76 q-Rceq:0.98 tex:1371847 mv:33025 misc:30889 icu:18.39 pcu:4678.15 scu:3463.47 sc:0 ;
in:632 out:632 type:B q:20.00 q-aq:18.47 q-noVbv:20.00 q-Rceq:0.98 tex:4457 mv:11976 misc:8300 icu:0.00 pcu:451.57 scu:7708.43 sc:0 ;
in:631 out:633 type:b q:20.00 q-aq:19.63 q-noVbv:20.00 q-Rceq:0.98 tex:2124 mv:4908 misc:4063 icu:0.00 pcu:375.26 scu:7784.74 sc:0 ;
in:633 out:634 type:b q:20.00 q-aq:19.50 q-noVbv:20.00 q-Rceq:0.98 tex:2748 mv:2582 misc:2707 icu:0.00 pcu:399.94 scu:7760.06 sc:0 ;
in:637 out:635 type:P q:18.00 q-aq:14.70 q-noVbv:17.76 q-Rceq:0.98 tex:54523 mv:11585 misc:13551 icu:0.25 pcu:524.86 scu:7634.89 sc:0 ;
in:636 out:636 type:B q:18.00 q-aq:16.06 q-noVbv:18.00 q-Rceq:0.98 tex:11752 mv:7230 misc:6632 icu:0.00 pcu:807.69 scu:7352.31 sc:0 ;
in:635 out:637 type:b q:18.00 q-aq:17.58 q-noVbv:18.00 q-Rceq:0.98 tex:3613 mv:4417 misc:3808 icu:0.00 pcu:890.55 scu:7269.45 sc:0 ;
in:638 out:638 type:I q:32.65 q-aq:24.73 q-noVbv:19.61 q-Rceq:0.98 tex:2318699 mv:330385 misc:8411 icu:8160.00 pcu:0.00 scu:0.00 sc:0 ;
in:642 out:639 type:P q:29.00 q-aq:21.42 q-noVbv:29.01 q-Rceq:0.98 tex:758451 mv:28317 misc:20004 icu:51.38 pcu:3276.59 scu:4832.03 sc:0 ;
in:640 out:640 type:B q:29.00 q-aq:28.93 q-noVbv:29.01 q-Rceq:0.98 tex:199 mv:2403 misc:1725 icu:0.00 pcu:184.86 scu:7975.14 sc:0 ;
in:639 out:641 type:b q:29.00 q-aq:28.91 q-noVbv:29.01 q-Rceq:0.98 tex:140 mv:394 misc:409 icu:0.00 pcu:60.95 scu:8099.05 sc:0 ;
in:641 out:642 type:b q:28.00 q-aq:27.84 q-noVbv:28.00 q-Rceq:0.98 tex:201 mv:362 misc:262 icu:0.00 pcu:281.82 scu:7878.18 sc:0 ;
in:646 out:643 type:P q:27.00 q-aq:19.17 q-noVbv:26.50 q-Rceq:0.98 tex:614185 mv:25269 misc:29464 icu:34.76 pcu:3058.24 scu:5067.01 sc:0 ;
in:644 out:644 type:B q:27.00 q-aq:26.40 q-noVbv:27.00 q-Rceq:0.98 tex:593 mv:2142 misc:1956 icu:0.25 pcu:59.69 scu:8100.06 sc:0 ;
in:643 out:645 type:b q:27.00 q-aq:26.96 q-noVbv:27.00 q-Rceq:0.98 tex:256 mv:875 misc:840 icu:0.00 pcu:158.41 scu:8001.59 sc:0 ;
in:645 out:646 type:b q:27.00 q-aq:26.75 q-noVbv:27.00 q-Rceq:0.98 tex:415 mv:606 misc:593 icu:0.00 pcu:350.58 scu:7809.42 sc:0 ;
in:650 out:647 type:P q:25.00 q-aq:17.16 q-noVbv:24.50 q-Rceq:0.98 tex:809770 mv:25665 misc:30624 icu:35.26 pcu:3429.72 scu:4695.02 sc:0 ;
in:648 out:648 type:B q:25.00 q-aq:24.00 q-noVbv:25.00 q-Rceq:0.98 tex:1057 mv:3914 misc:3173 icu:0.00 pcu:113.33 scu:8046.67 sc:0 ;
in:647 out:649 type:b q:25.00 q-aq:24.88 q-noVbv:25.00 q-Rceq:0.98 tex:450 mv:1238 misc:1206 icu:0.00 pcu:187.63 scu:7972.37 sc:0 ;
in:649 out:650 type:b q:25.00 q-aq:24.99 q-noVbv:25.00 q-Rceq:0.98 tex:384 mv:741 misc:814 icu:0.00 pcu:250.34 scu:7909.66 sc:0 ;
in:654 out:651 type:P q:23.00 q-aq:15.18 q-noVbv:22.50 q-Rceq:0.98 tex:1166046 mv:28097 misc:30656 icu:29.72 pcu:4218.01 scu:3912.27 sc:0 ;
in:652 out:652 type:B q:23.00 q-aq:22.01 q-noVbv:23.00 q-Rceq:0.98 tex:1979 mv:6842 misc:5082 icu:0.00 pcu:215.84 scu:7944.16 sc:0 ;
in:651 out:653 type:b q:23.00 q-aq:22.60 q-noVbv:23.00 q-Rceq:0.98 tex:1057 mv:2257 misc:2084 icu:0.00 pcu:281.57 scu:7878.43 sc:0 ;
in:653 out:654 type:b q:23.00 q-aq:22.69 q-noVbv:23.00 q-Rceq:0.98 tex:940 mv:1441 misc:1369 icu:0.00 pcu:436.46 scu:7723.54 sc:0 ;
in:658 out:655 type:P q:21.00 q-aq:13.19 q-noVbv:20.50 q-Rceq:0.98 tex:1112119 mv:28565 misc:31763 icu:17.63 pcu:4007.47 scu:4134.90 sc:0 ;
in:656 out:656 type:B q:21.00 q-aq:19.60 q-noVbv:21.00 q-Rceq:0.98 tex:2863 mv:8182 misc:6199 icu:0.00 pcu:362.41 scu:7797.59 sc:0 ;
in:655 out:657 type:b q:21.00 q-aq:20.56 q-noVbv:21.00 q-Rceq:0.98 tex:1383 mv:3331 misc:3065 icu:0.00 pcu:206.52 scu:7953.48 sc:0 ;
in:657 out:658 type:b q:21.00 q-aq:20.69 q-noVbv:21.00 q-Rceq:0.98 tex:1504 mv:2113 misc:2176 icu:0.00 pcu:369.72 scu:7790.28 sc:0 ;
in:661 out:659 type:P q:19.00 q-aq:15.81 q-noVbv:18.50 q-Rceq:0.98 tex:37120 mv:6955 misc:10331 icu:0.25 pcu:329.42 scu:7830.33 sc:0 ;
in:660 out:660 type:B q:19.00 q-aq:17.41 q-noVbv:19.00 q-Rceq:0.98 tex:7122 mv:5023 misc:4883 icu:0.25 pcu:802.40 scu:7357.35 sc:0 ;
in:659 out:661 type:b q:19.00 q-aq:18.50 q-noVbv:19.00 q-Rceq:0.98 tex:2784 mv:3355 misc:2847 icu:0.25 pcu:861.84 scu:7297.91 sc:0 ;
in:662 out:662 type:I q:32.60 q-aq:24.68 q-noVbv:21.61 q-Rceq:0.98 tex:2307803 mv:332618 misc:8220 icu:8160.00 pcu:0.00 scu:0.00 sc:0 ;
in:666 out:663 type:P q:30.00 q-aq:22.09 q-noVbv:29.53 q-Rceq:0.98 tex:391205 mv:16887 misc:19470 icu:32.24 pcu:2264.40 scu:5863.36 sc:0 ;
in:664 out:664 type:B q:30.00 q-aq:29.92 q-noVbv:29.53 q-Rceq:0.98 tex:118 mv:1302 misc:875 icu:0.00 pcu:148.34 scu:8011.66 sc:0 ;
in:663 out:665 type:b q:30.00 q-aq:30.00 q-noVbv:29.53 q-Rceq:0.98 tex:38 mv:313 misc:343 icu:0.00 pcu:133.48 scu:8026.52 sc:0 ;
in:665 out:666 type:b q:29.00 q-aq:29.00 q-noVbv:29.00 q-Rceq:0.98 tex:58 mv:237 misc:215 icu:0.00 pcu:145.82 scu:8014.18 sc:0 ;
in:670 out:667 type:P q:27.00 q-aq:19.15 q-noVbv:27.26 q-Rceq:0.98 tex:1013220 mv:32938 misc:30300 icu:47.85 pcu:4291.30 scu:3820.84 sc:0 ;
in:668 out:668 type:B q:27.00 q-aq:26.42 q-noVbv:27.00 q-Rceq:0.98 tex:753 mv:4591 misc:3304 icu:0.00 pcu:137.51 scu:8022.49 sc:0 ;
in:667 out:669 type:b q:27.00 q-aq:26.98 q-noVbv:27.00 q-Rceq:0.98 tex:180 mv:1008 misc:959 icu:0.00 pcu:124.67 scu:8035.33 sc:0 ;
in:669 out:670 type:b q:27.00 q-aq:26.90 q-noVbv:27.00 q-Rceq:0.98 tex:229 mv:566 misc:524 icu:0.00 pcu:311.54 scu:7848.46 sc:0 ;
in:674 out:671 type:P q:25.00 q-aq:17.21 q-noVbv:25.26 q-Rceq:0.98 tex:1083700 mv:38463 misc:31783 icu:39.29 pcu:4039.70 scu:4081.01 sc:0 ;
in:672 out:672 type:B q:25.00 q-aq:23.75 q-noVbv:25.00 q-Rceq:0.98 tex:1421 mv:6754 misc:4925 icu:0.50 pcu:188.64 scu:7970.86 sc:0 ;
in:671 out:673 type:b q:25.00 q-aq:24.58 q-noVbv:25.00 q-Rceq:0.98 tex:628 mv:2760 misc:2379 icu:0.25 pcu:155.39 scu:8004.36 sc:0 ;
in:673 out:674 type:b q:25.00 q-aq:24.60 q-noVbv:25.00 q-Rceq:0.98 tex:516 mv:1083 misc:1035 icu:0.00 pcu:250.34 scu:7909.66 sc:0 ;
in:678 out:675 type:P q:23.00 q-aq:15.17 q-noVbv:23.26 q-Rceq:0.98 tex:1465435 mv:43623 misc:30519 icu:38.79 pcu:5045.60 scu:3075.61 sc:0 ;
in:676 out:676 type:B q:23.00 q-aq:22.07 q-noVbv:23.00 q-Rceq:0.98 tex:2102 mv:11615 misc:7575 icu:4.28 pcu:242.79 scu:7912.93 sc:0 ;
in:675 out:677 type:b q:23.00 q-aq:22.83 q-noVbv:23.00 q-Rceq:0.98 tex:868 mv:4363 misc:3469 icu:0.00 pcu:125.93 scu:8034.07 sc:0 ;
in:677 out:678 type:b q:23.00 q-aq:22.56 q-noVbv:23.00 q-Rceq:0.98 tex:1040 mv:1578 misc:1682 icu:0.00 pcu:400.70 scu:7759.30 sc:0 ;
in:682 out:679 type:P q:21.00 q-aq:13.17 q-noVbv:21.26 q-Rceq:0.98 tex:1354912 mv:39755 misc:31668 icu:17.88 pcu:4662.28 scu:3479.84 sc:0 ;
in:680 out:680 type:B q:21.00 q-aq:19.13 q-noVbv:21.00 q-Rceq:0.98 tex:3682 mv:11364 misc:8085 icu:0.00 pcu:352.34 scu:7807.66 sc:0 ;
in:679 out:681 type:b q:21.00 q-aq:20.94 q-noVbv:21.00 q-Rceq:0.98 tex:1951 mv:4446 misc:3804 icu:0.00 pcu:218.36 scu:7941.64 sc:0 ;
in:681 out:682 type:b q:21.00 q-aq:20.67 q-noVbv:21.00 q-Rceq:0.98 tex:2231 mv:2217 misc:2349 icu:0.00 pcu:346.80 scu:7813.20 sc:0 ;
in:685 out:683 type:P q:19.00 q-aq:15.90 q-noVbv:19.26 q-Rceq:0.98 tex:64531 mv:11918 misc:15140 icu:1.01 pcu:551.56 scu:7607.44 sc:0 ;
in:684 out:684 type:B q:19.00 q-aq:17.35 q-noVbv:19.00 q-Rceq:0.98 tex:8333 mv:6071 misc:5632 icu:0.00 pcu:836.40 scu:7323.60 sc:0 ;
in:683 out:685 type:b q:19.00 q-aq:18.66 q-noVbv:19.00 q-Rceq:0.98 tex:2378 mv:3890 misc:3307 icu:0.00 pcu:918.76 scu:7241.24 sc:0 ;
in:686 out:686 type:I q:32.73 q-aq:24.82 q-noVbv:22.28 q-Rceq:0.98 tex:2272721 mv:336478 misc:8154 icu:8160.00 pcu:0.00 scu:0.00 sc:0 ;

Stacey Spears

27th April 2021, 15:36

HD Stats Pass 3
in:590 out:590 type:I q:35.85 q-aq:27.95 q-noVbv:6.74 q-Rceq:0.98 tex:2434054 mv:227251 misc:9511 icu:8160.00 pcu:0.00 scu:0.00 sc:0 ;
in:594 out:591 type:P q:7.00 q-aq:0.01 q-noVbv:6.80 q-Rceq:0.98 tex:11112740 mv:299729 misc:4935 icu:2949.44 pcu:5178.07 scu:32.49 sc:0 ;
in:592 out:592 type:B q:6.00 q-aq:4.03 q-noVbv:5.80 q-Rceq:0.98 tex:1464416 mv:18144 misc:25872 icu:0.25 pcu:3824.37 scu:4335.38 sc:0 ;
in:591 out:593 type:b q:6.00 q-aq:5.96 q-noVbv:5.80 q-Rceq:0.98 tex:677596 mv:37133 misc:25141 icu:0.00 pcu:2686.50 scu:5473.50 sc:0 ;
in:593 out:594 type:b q:6.00 q-aq:5.98 q-noVbv:5.80 q-Rceq:0.98 tex:423269 mv:34377 misc:22158 icu:0.00 pcu:1960.67 scu:6199.33 sc:0 ;
in:598 out:595 type:P q:7.00 q-aq:0.01 q-noVbv:6.79 q-Rceq:0.98 tex:4184604 mv:58795 misc:15859 icu:3.53 pcu:7266.68 scu:889.79 sc:0 ;
in:596 out:596 type:B q:7.00 q-aq:5.13 q-noVbv:7.00 q-Rceq:0.98 tex:648103 mv:42639 misc:25616 icu:0.25 pcu:2409.72 scu:5750.03 sc:0 ;
in:595 out:597 type:b q:7.00 q-aq:6.94 q-noVbv:7.00 q-Rceq:0.98 tex:267850 mv:57541 misc:23942 icu:0.00 pcu:1925.91 scu:6234.09 sc:0 ;
in:597 out:598 type:b q:7.00 q-aq:6.91 q-noVbv:7.00 q-Rceq:0.98 tex:263074 mv:56154 misc:24962 icu:0.00 pcu:2021.87 scu:6138.13 sc:0 ;
in:602 out:599 type:P q:7.00 q-aq:0.02 q-noVbv:6.79 q-Rceq:0.98 tex:3672075 mv:87155 misc:16717 icu:2.27 pcu:6921.64 scu:1236.09 sc:0 ;
in:600 out:600 type:B q:7.00 q-aq:5.10 q-noVbv:7.00 q-Rceq:0.98 tex:635851 mv:60861 misc:27041 icu:0.00 pcu:2761.05 scu:5398.95 sc:0 ;
in:599 out:601 type:b q:7.00 q-aq:6.98 q-noVbv:7.00 q-Rceq:0.98 tex:261604 mv:64684 misc:25420 icu:0.00 pcu:2226.12 scu:5933.88 sc:0 ;
in:601 out:602 type:b q:7.00 q-aq:6.96 q-noVbv:7.00 q-Rceq:0.98 tex:254308 mv:58216 misc:24680 icu:0.00 pcu:2023.63 scu:6136.37 sc:0 ;
in:606 out:603 type:P q:7.00 q-aq:0.01 q-noVbv:6.79 q-Rceq:0.98 tex:3655254 mv:99916 misc:16120 icu:2.52 pcu:6946.33 scu:1211.16 sc:0 ;
in:604 out:604 type:B q:7.00 q-aq:5.10 q-noVbv:7.00 q-Rceq:0.98 tex:625295 mv:63970 misc:28045 icu:0.25 pcu:2839.63 scu:5320.12 sc:0 ;
in:603 out:605 type:b q:7.00 q-aq:6.98 q-noVbv:7.00 q-Rceq:0.98 tex:266139 mv:67373 misc:25801 icu:0.00 pcu:2352.04 scu:5807.96 sc:0 ;
in:605 out:606 type:b q:7.00 q-aq:6.97 q-noVbv:7.00 q-Rceq:0.98 tex:252535 mv:58687 misc:24555 icu:0.00 pcu:2038.74 scu:6121.26 sc:0 ;
in:610 out:607 type:P q:7.00 q-aq:0.01 q-noVbv:6.79 q-Rceq:0.98 tex:3597650 mv:98632 misc:16446 icu:1.26 pcu:6876.56 scu:1282.18 sc:0 ;
in:608 out:608 type:B q:7.00 q-aq:5.14 q-noVbv:7.00 q-Rceq:0.98 tex:612758 mv:59891 misc:27674 icu:0.25 pcu:2759.54 scu:5400.21 sc:0 ;
in:607 out:609 type:b q:7.00 q-aq:6.98 q-noVbv:7.00 q-Rceq:0.98 tex:258194 mv:65971 misc:25936 icu:0.00 pcu:2335.67 scu:5824.33 sc:0 ;
in:609 out:610 type:b q:7.00 q-aq:6.93 q-noVbv:7.00 q-Rceq:0.98 tex:262086 mv:56838 misc:24438 icu:0.00 pcu:1966.71 scu:6193.29 sc:0 ;
in:613 out:611 type:P q:7.00 q-aq:4.03 q-noVbv:6.78 q-Rceq:0.98 tex:1440796 mv:89741 misc:26375 icu:0.25 pcu:4472.13 scu:3687.61 sc:0 ;
in:612 out:612 type:B q:7.00 q-aq:5.09 q-noVbv:7.00 q-Rceq:0.98 tex:629560 mv:69409 misc:27596 icu:0.00 pcu:2987.47 scu:5172.53 sc:0 ;
in:611 out:613 type:b q:7.00 q-aq:6.96 q-noVbv:7.00 q-Rceq:0.98 tex:242681 mv:60975 misc:24814 icu:0.00 pcu:2256.34 scu:5903.66 sc:0 ;
in:614 out:614 type:I q:33.29 q-aq:25.42 q-noVbv:6.66 q-Rceq:0.98 tex:2337982 mv:302928 misc:8834 icu:8160.00 pcu:0.00 scu:0.00 sc:0 ;
in:618 out:615 type:P q:7.00 q-aq:0.01 q-noVbv:6.72 q-Rceq:0.98 tex:10973153 mv:183249 misc:6015 icu:1365.79 pcu:6774.31 scu:19.90 sc:0 ;
in:616 out:616 type:B q:6.00 q-aq:3.88 q-noVbv:5.72 q-Rceq:0.98 tex:1454460 mv:16242 misc:25956 icu:0.50 pcu:3835.70 scu:4323.79 sc:0 ;
in:615 out:617 type:b q:6.00 q-aq:5.98 q-noVbv:5.72 q-Rceq:0.98 tex:708985 mv:35368 misc:25141 icu:0.00 pcu:2751.99 scu:5408.01 sc:0 ;
in:617 out:618 type:b q:6.00 q-aq:5.98 q-noVbv:5.72 q-Rceq:0.98 tex:420046 mv:34759 misc:22474 icu:0.00 pcu:1905.01 scu:6254.99 sc:0 ;
in:622 out:619 type:P q:7.00 q-aq:0.02 q-noVbv:6.72 q-Rceq:0.98 tex:4258091 mv:56643 misc:15464 icu:3.78 pcu:7342.99 scu:813.23 sc:0 ;
in:620 out:620 type:B q:7.00 q-aq:5.06 q-noVbv:7.00 q-Rceq:0.98 tex:655097 mv:42611 misc:25300 icu:0.25 pcu:2409.72 scu:5750.03 sc:0 ;
in:619 out:621 type:b q:7.00 q-aq:7.00 q-noVbv:7.00 q-Rceq:0.98 tex:259786 mv:56489 misc:23649 icu:0.00 pcu:1872.77 scu:6287.23 sc:0 ;
in:621 out:622 type:b q:7.00 q-aq:6.95 q-noVbv:7.00 q-Rceq:0.98 tex:264567 mv:53038 misc:24476 icu:0.00 pcu:1861.44 scu:6298.56 sc:0 ;
in:626 out:623 type:P q:7.00 q-aq:0.01 q-noVbv:6.72 q-Rceq:0.98 tex:4230694 mv:85242 misc:14242 icu:3.78 pcu:7271.97 scu:884.25 sc:0 ;
in:624 out:624 type:B q:7.00 q-aq:5.12 q-noVbv:7.00 q-Rceq:0.98 tex:624785 mv:53503 misc:26579 icu:0.00 pcu:2593.57 scu:5566.43 sc:0 ;
in:623 out:625 type:b q:7.00 q-aq:6.98 q-noVbv:7.00 q-Rceq:0.98 tex:264471 mv:63642 misc:25794 icu:0.00 pcu:2161.64 scu:5998.36 sc:0 ;
in:625 out:626 type:b q:7.00 q-aq:6.97 q-noVbv:7.00 q-Rceq:0.98 tex:266207 mv:56854 misc:24536 icu:0.00 pcu:1954.87 scu:6205.13 sc:0 ;
in:630 out:627 type:P q:7.00 q-aq:0.02 q-noVbv:6.71 q-Rceq:0.98 tex:4266794 mv:85812 misc:13966 icu:1.51 pcu:7304.21 scu:854.28 sc:0 ;
in:628 out:628 type:B q:7.00 q-aq:5.07 q-noVbv:7.00 q-Rceq:0.98 tex:647859 mv:51065 misc:26650 icu:0.00 pcu:2549.24 scu:5610.76 sc:0 ;
in:627 out:629 type:b q:7.00 q-aq:6.99 q-noVbv:7.00 q-Rceq:0.98 tex:266340 mv:61590 misc:24803 icu:0.00 pcu:1992.40 scu:6167.60 sc:0 ;
in:629 out:630 type:b q:7.00 q-aq:6.99 q-noVbv:7.00 q-Rceq:0.98 tex:264022 mv:55224 misc:24703 icu:0.00 pcu:1944.55 scu:6215.45 sc:0 ;
in:634 out:631 type:P q:7.00 q-aq:0.01 q-noVbv:6.71 q-Rceq:0.98 tex:4091313 mv:86549 misc:14374 icu:1.76 pcu:7304.71 scu:853.53 sc:0 ;
in:632 out:632 type:B q:7.00 q-aq:5.09 q-noVbv:7.00 q-Rceq:0.98 tex:641508 mv:51583 misc:27686 icu:0.00 pcu:2698.59 scu:5461.41 sc:0 ;
in:631 out:633 type:b q:7.00 q-aq:6.99 q-noVbv:7.00 q-Rceq:0.98 tex:264289 mv:60926 misc:24755 icu:0.00 pcu:2079.54 scu:6080.46 sc:0 ;
in:633 out:634 type:b q:7.00 q-aq:6.94 q-noVbv:7.00 q-Rceq:0.98 tex:270107 mv:55517 misc:24628 icu:0.00 pcu:1952.36 scu:6207.64 sc:0 ;
in:637 out:635 type:P q:7.00 q-aq:4.02 q-noVbv:6.71 q-Rceq:0.98 tex:1568588 mv:83218 misc:26104 icu:0.00 pcu:4621.23 scu:3538.77 sc:0 ;
in:636 out:636 type:B q:7.00 q-aq:5.11 q-noVbv:7.00 q-Rceq:0.98 tex:624232 mv:66453 misc:27631 icu:0.00 pcu:2853.48 scu:5306.52 sc:0 ;
in:635 out:637 type:b q:7.00 q-aq:6.95 q-noVbv:7.00 q-Rceq:0.98 tex:246549 mv:59410 misc:24570 icu:0.00 pcu:2140.49 scu:6019.51 sc:0 ;
in:638 out:638 type:I q:35.06 q-aq:27.19 q-noVbv:6.58 q-Rceq:0.98 tex:2511473 mv:246752 misc:9377 icu:8160.00 pcu:0.00 scu:0.00 sc:0 ;
in:642 out:639 type:P q:7.00 q-aq:0.01 q-noVbv:6.65 q-Rceq:0.98 tex:11128541 mv:277687 misc:4540 icu:2624.30 pcu:5503.47 scu:32.24 sc:0 ;
in:640 out:640 type:B q:6.00 q-aq:3.95 q-noVbv:5.65 q-Rceq:0.98 tex:1509456 mv:17386 misc:26144 icu:1.26 pcu:3955.08 scu:4203.66 sc:0 ;
in:639 out:641 type:b q:6.00 q-aq:5.94 q-noVbv:5.65 q-Rceq:0.98 tex:765024 mv:35466 misc:26224 icu:0.00 pcu:2927.02 scu:5232.98 sc:0 ;
in:641 out:642 type:b q:6.00 q-aq:5.81 q-noVbv:5.65 q-Rceq:0.98 tex:418326 mv:37153 misc:22470 icu:0.00 pcu:1906.01 scu:6253.99 sc:0 ;
in:646 out:643 type:P q:7.00 q-aq:0.02 q-noVbv:6.65 q-Rceq:0.98 tex:4279066 mv:57053 misc:15481 icu:4.28 pcu:7304.21 scu:851.51 sc:0 ;
in:644 out:644 type:B q:7.00 q-aq:5.02 q-noVbv:7.00 q-Rceq:0.98 tex:674350 mv:41790 misc:25534 icu:0.00 pcu:2507.44 scu:5652.56 sc:0 ;
in:643 out:645 type:b q:7.00 q-aq:6.90 q-noVbv:7.00 q-Rceq:0.98 tex:271810 mv:55567 misc:23759 icu:0.00 pcu:1879.07 scu:6280.93 sc:0 ;
in:645 out:646 type:b q:7.00 q-aq:6.99 q-noVbv:7.00 q-Rceq:0.98 tex:266929 mv:56058 misc:24502 icu:0.00 pcu:2010.53 scu:6149.47 sc:0 ;
in:650 out:647 type:P q:7.00 q-aq:0.02 q-noVbv:6.65 q-Rceq:0.98 tex:3662313 mv:82857 misc:17295 icu:2.01 pcu:6911.82 scu:1246.16 sc:0 ;
in:648 out:648 type:B q:7.00 q-aq:5.11 q-noVbv:7.00 q-Rceq:0.98 tex:622850 mv:61315 misc:27513 icu:0.25 pcu:2756.52 scu:5403.23 sc:0 ;
in:647 out:649 type:b q:7.00 q-aq:6.95 q-noVbv:7.00 q-Rceq:0.98 tex:256161 mv:63066 misc:25146 icu:0.00 pcu:2096.41 scu:6063.59 sc:0 ;
in:649 out:650 type:b q:7.00 q-aq:6.98 q-noVbv:7.00 q-Rceq:0.98 tex:257582 mv:56258 misc:23998 icu:0.00 pcu:1922.39 scu:6237.61 sc:0 ;
in:654 out:651 type:P q:7.00 q-aq:0.02 q-noVbv:6.64 q-Rceq:0.98 tex:3472724 mv:98754 misc:17313 icu:3.02 pcu:6805.04 scu:1351.94 sc:0 ;
in:652 out:652 type:B q:7.00 q-aq:5.03 q-noVbv:7.00 q-Rceq:0.98 tex:622978 mv:65414 misc:27858 icu:0.00 pcu:2861.29 scu:5298.71 sc:0 ;
in:651 out:653 type:b q:7.00 q-aq:6.92 q-noVbv:7.00 q-Rceq:0.98 tex:251307 mv:66363 misc:25536 icu:0.00 pcu:2243.75 scu:5916.25 sc:0 ;
in:653 out:654 type:b q:7.00 q-aq:6.94 q-noVbv:7.00 q-Rceq:0.98 tex:252957 mv:58093 misc:24344 icu:0.00 pcu:1931.45 scu:6228.55 sc:0 ;
in:658 out:655 type:P q:7.00 q-aq:0.01 q-noVbv:6.64 q-Rceq:0.98 tex:3360027 mv:103990 misc:17556 icu:0.76 pcu:6733.76 scu:1425.48 sc:0 ;
in:656 out:656 type:B q:7.00 q-aq:5.06 q-noVbv:7.00 q-Rceq:0.98 tex:613138 mv:67150 misc:28021 icu:0.00 pcu:2912.41 scu:5247.59 sc:0 ;
in:655 out:657 type:b q:7.00 q-aq:6.97 q-noVbv:7.00 q-Rceq:0.98 tex:249854 mv:66469 misc:25763 icu:0.00 pcu:2316.53 scu:5843.47 sc:0 ;
in:657 out:658 type:b q:7.00 q-aq:6.98 q-noVbv:7.00 q-Rceq:0.98 tex:243932 mv:58848 misc:24637 icu:0.00 pcu:2106.99 scu:6053.01 sc:0 ;
in:661 out:659 type:P q:7.00 q-aq:4.06 q-noVbv:6.64 q-Rceq:0.98 tex:1322302 mv:96329 misc:26845 icu:0.00 pcu:4411.69 scu:3748.31 sc:0 ;
in:660 out:660 type:B q:7.00 q-aq:5.00 q-noVbv:7.00 q-Rceq:0.98 tex:619953 mv:72925 misc:27948 icu:0.00 pcu:3057.99 scu:5102.01 sc:0 ;
in:659 out:661 type:b q:7.00 q-aq:6.97 q-noVbv:7.00 q-Rceq:0.98 tex:243788 mv:62459 misc:24797 icu:0.00 pcu:2346.76 scu:5813.24 sc:0 ;
in:662 out:662 type:I q:34.83 q-aq:26.95 q-noVbv:6.51 q-Rceq:0.98 tex:2538831 mv:246661 misc:9399 icu:8160.00 pcu:0.00 scu:0.00 sc:0 ;
in:666 out:663 type:P q:7.00 q-aq:0.01 q-noVbv:6.58 q-Rceq:0.98 tex:11076213 mv:268956 misc:4843 icu:2488.80 pcu:5639.97 scu:31.23 sc:0 ;
in:664 out:664 type:B q:6.00 q-aq:3.97 q-noVbv:5.58 q-Rceq:0.98 tex:1486925 mv:18242 misc:26471 icu:0.50 pcu:3878.01 scu:4281.48 sc:0 ;
in:663 out:665 type:b q:6.00 q-aq:5.99 q-noVbv:5.58 q-Rceq:0.98 tex:678492 mv:36952 misc:25070 icu:0.00 pcu:2709.67 scu:5450.33 sc:0 ;
in:665 out:666 type:b q:6.00 q-aq:5.81 q-noVbv:5.58 q-Rceq:0.98 tex:433970 mv:35594 misc:22401 icu:0.00 pcu:2029.17 scu:6130.83 sc:0 ;
in:670 out:667 type:P q:7.00 q-aq:0.01 q-noVbv:6.58 q-Rceq:0.98 tex:4694160 mv:56490 misc:14094 icu:1.76 pcu:7423.84 scu:734.40 sc:0 ;
in:668 out:668 type:B q:7.00 q-aq:5.04 q-noVbv:7.00 q-Rceq:0.98 tex:663964 mv:34842 misc:24178 icu:0.00 pcu:2353.56 scu:5806.44 sc:0 ;
in:667 out:669 type:b q:7.00 q-aq:6.88 q-noVbv:7.00 q-Rceq:0.98 tex:277659 mv:59940 misc:24996 icu:0.00 pcu:2020.36 scu:6139.64 sc:0 ;
in:669 out:670 type:b q:7.00 q-aq:6.93 q-noVbv:7.00 q-Rceq:0.98 tex:296364 mv:55286 misc:24639 icu:0.00 pcu:1960.16 scu:6199.84 sc:0 ;
in:674 out:671 type:P q:7.00 q-aq:0.01 q-noVbv:6.58 q-Rceq:0.98 tex:4979860 mv:76713 misc:12252 icu:1.51 pcu:7559.84 scu:598.65 sc:0 ;
in:672 out:672 type:B q:7.00 q-aq:5.16 q-noVbv:7.00 q-Rceq:0.98 tex:676086 mv:41193 misc:25366 icu:0.00 pcu:2358.09 scu:5801.91 sc:0 ;
in:671 out:673 type:b q:7.00 q-aq:6.79 q-noVbv:7.00 q-Rceq:0.98 tex:283987 mv:60124 misc:25784 icu:0.00 pcu:2041.76 scu:6118.24 sc:0 ;
in:673 out:674 type:b q:7.00 q-aq:6.99 q-noVbv:7.00 q-Rceq:0.98 tex:308569 mv:53876 misc:25427 icu:0.00 pcu:1961.17 scu:6198.83 sc:0 ;
in:678 out:675 type:P q:7.00 q-aq:0.01 q-noVbv:6.57 q-Rceq:0.98 tex:5046505 mv:72877 misc:12368 icu:0.50 pcu:7624.56 scu:534.93 sc:0 ;
in:676 out:676 type:B q:7.00 q-aq:5.01 q-noVbv:7.00 q-Rceq:0.98 tex:720323 mv:34567 misc:26228 icu:0.00 pcu:2760.55 scu:5399.45 sc:0 ;
in:675 out:677 type:b q:7.00 q-aq:6.86 q-noVbv:7.00 q-Rceq:0.98 tex:298688 mv:58927 misc:25199 icu:0.00 pcu:1991.90 scu:6168.10 sc:0 ;
in:677 out:678 type:b q:7.00 q-aq:6.97 q-noVbv:7.00 q-Rceq:0.98 tex:310062 mv:53343 misc:26226 icu:0.00 pcu:2082.56 scu:6077.44 sc:0 ;
in:682 out:679 type:P q:7.00 q-aq:0.01 q-noVbv:6.57 q-Rceq:0.98 tex:4448168 mv:75434 misc:13841 icu:2.27 pcu:7473.20 scu:684.53 sc:0 ;
in:680 out:680 type:B q:7.00 q-aq:4.99 q-noVbv:7.00 q-Rceq:0.98 tex:670567 mv:40258 misc:26180 icu:0.00 pcu:2569.39 scu:5590.61 sc:0 ;
in:679 out:681 type:b q:7.00 q-aq:6.95 q-noVbv:7.00 q-Rceq:0.98 tex:281306 mv:57083 misc:23788 icu:0.00 pcu:1883.60 scu:6276.40 sc:0 ;
in:681 out:682 type:b q:7.00 q-aq:6.93 q-noVbv:7.00 q-Rceq:0.98 tex:281914 mv:56638 misc:25673 icu:0.00 pcu:2072.24 scu:6087.76 sc:0 ;
in:685 out:683 type:P q:7.00 q-aq:4.00 q-noVbv:6.56 q-Rceq:0.98 tex:1760105 mv:83266 misc:26082 icu:0.00 pcu:5016.39 scu:3143.61 sc:0 ;
in:684 out:684 type:B q:7.00 q-aq:5.04 q-noVbv:7.00 q-Rceq:0.98 tex:645621 mv:58941 misc:27825 icu:0.00 pcu:2866.58 scu:5293.42 sc:0 ;
in:683 out:685 type:b q:7.00 q-aq:6.96 q-noVbv:7.00 q-Rceq:0.98 tex:256500 mv:56784 misc:24274 icu:0.00 pcu:2091.63 scu:6068.37 sc:0 ;
in:686 out:686 type:I q:34.96 q-aq:27.18 q-noVbv:6.44 q-Rceq:0.98 tex:2511364 mv:247085 misc:9487 icu:8160.00 pcu:0.00 scu:0.00 sc:0 ;

Stacey Spears

27th April 2021, 15:37

UHD Stats Pass 1
in:590 out:590 type:I q:20.74 q-aq:19.07 q-noVbv:7.65 q-Rceq:721.59 tex:9417908 mv:980056 misc:41732 icu:32400.00 pcu:0.00 scu:0.00 sc:0 ;
in:594 out:591 type:P q:16.05 q-aq:13.66 q-noVbv:16.05 q-Rceq:571.97 tex:7245125 mv:98876 misc:59443 icu:145.00 pcu:26658.50 scu:5596.50 sc:0 ;
in:592 out:592 type:B q:17.19 q-aq:15.26 q-noVbv:17.19 q-Rceq:571.97 tex:218923 mv:65960 misc:45517 icu:17.75 pcu:3284.00 scu:29098.25 sc:0 ;
in:591 out:593 type:b q:18.32 q-aq:16.73 q-noVbv:18.32 q-Rceq:571.97 tex:52628 mv:41290 misc:30810 icu:9.25 pcu:2257.75 scu:30133.00 sc:0 ;
in:593 out:594 type:b q:18.32 q-aq:16.43 q-noVbv:18.32 q-Rceq:571.97 tex:49388 mv:14207 misc:18359 icu:8.50 pcu:3079.25 scu:29312.25 sc:0 ;
in:598 out:595 type:P q:15.05 q-aq:13.03 q-noVbv:15.05 q-Rceq:469.28 tex:2348076 mv:78526 misc:84169 icu:22.25 pcu:13824.00 scu:18553.75 sc:0 ;
in:596 out:596 type:B q:16.69 q-aq:14.90 q-noVbv:16.69 q-Rceq:469.28 tex:134754 mv:50241 misc:46115 icu:8.00 pcu:5064.25 scu:27327.75 sc:0 ;
in:595 out:597 type:b q:18.07 q-aq:16.93 q-noVbv:18.07 q-Rceq:469.28 tex:27584 mv:28668 misc:20765 icu:3.50 pcu:2482.25 scu:29914.25 sc:0 ;
in:597 out:598 type:b q:17.57 q-aq:16.44 q-noVbv:17.57 q-Rceq:469.28 tex:48859 mv:23761 misc:22307 icu:2.00 pcu:3999.25 scu:28398.75 sc:0 ;
in:602 out:599 type:P q:14.05 q-aq:12.03 q-noVbv:14.05 q-Rceq:402.28 tex:2928217 mv:100708 misc:92580 icu:15.75 pcu:16145.75 scu:16238.50 sc:0 ;
in:600 out:600 type:B q:15.69 q-aq:13.89 q-noVbv:15.69 q-Rceq:402.28 tex:319405 mv:78139 misc:67356 icu:2.50 pcu:7348.50 scu:25049.00 sc:0 ;
in:599 out:601 type:b q:17.07 q-aq:15.81 q-noVbv:17.07 q-Rceq:402.28 tex:47050 mv:46821 misc:30085 icu:0.25 pcu:3696.25 scu:28703.50 sc:0 ;
in:601 out:602 type:b q:16.57 q-aq:15.07 q-noVbv:16.57 q-Rceq:402.28 tex:72563 mv:39870 misc:32645 icu:0.75 pcu:4540.25 scu:27859.00 sc:0 ;
in:606 out:603 type:P q:13.05 q-aq:10.98 q-noVbv:13.05 q-Rceq:361.88 tex:4218902 mv:144499 misc:99008 icu:12.50 pcu:19832.25 scu:12555.25 sc:0 ;
in:604 out:604 type:B q:14.69 q-aq:13.00 q-noVbv:14.69 q-Rceq:361.88 tex:380283 mv:110437 misc:79189 icu:7.00 pcu:7821.00 scu:24572.00 sc:0 ;
in:603 out:605 type:b q:16.07 q-aq:14.57 q-noVbv:16.07 q-Rceq:361.88 tex:69764 mv:67593 misc:40990 icu:0.50 pcu:3846.25 scu:28553.25 sc:0 ;
in:605 out:606 type:b q:15.57 q-aq:13.89 q-noVbv:15.57 q-Rceq:361.88 tex:154199 mv:53354 misc:50439 icu:1.00 pcu:8745.00 scu:23654.00 sc:0 ;
in:610 out:607 type:P q:12.05 q-aq:9.98 q-noVbv:12.05 q-Rceq:338.10 tex:5194839 mv:165803 misc:98955 icu:14.75 pcu:22139.75 scu:10245.50 sc:0 ;
in:608 out:608 type:B q:13.69 q-aq:11.96 q-noVbv:13.69 q-Rceq:338.10 tex:577864 mv:134501 misc:87394 icu:1.00 pcu:8756.75 scu:23642.25 sc:0 ;
in:607 out:609 type:b q:15.07 q-aq:13.37 q-noVbv:15.07 q-Rceq:338.10 tex:129067 mv:96798 misc:53364 icu:1.25 pcu:4710.75 scu:27688.00 sc:0 ;
in:609 out:610 type:b q:14.57 q-aq:12.87 q-noVbv:14.57 q-Rceq:338.10 tex:230880 mv:83463 misc:58083 icu:0.75 pcu:6478.75 scu:25920.50 sc:0 ;
in:613 out:611 type:P q:11.05 q-aq:8.99 q-noVbv:11.05 q-Rceq:322.71 tex:5693012 mv:188790 misc:101052 icu:18.25 pcu:23061.75 scu:9320.00 sc:0 ;
in:612 out:612 type:B q:12.52 q-aq:10.77 q-noVbv:12.52 q-Rceq:322.71 tex:905538 mv:173938 misc:99593 icu:5.25 pcu:10436.25 scu:21958.50 sc:0 ;
in:611 out:613 type:b q:13.99 q-aq:12.45 q-noVbv:13.99 q-Rceq:322.71 tex:234576 mv:135219 misc:70400 icu:1.75 pcu:6403.00 scu:25995.25 sc:0 ;
in:614 out:614 type:I q:21.02 q-aq:19.36 q-noVbv:10.81 q-Rceq:711.56 tex:8919685 mv:968621 misc:41386 icu:32400.00 pcu:0.00 scu:0.00 sc:0 ;
in:618 out:615 type:P q:18.88 q-aq:16.46 q-noVbv:18.88 q-Rceq:565.49 tex:2799880 mv:56916 misc:71148 icu:68.75 pcu:15558.00 scu:16773.25 sc:0 ;
in:616 out:616 type:B q:20.02 q-aq:17.83 q-noVbv:20.02 q-Rceq:565.49 tex:110495 mv:50303 misc:37207 icu:26.50 pcu:2554.00 scu:29819.50 sc:0 ;
in:615 out:617 type:b q:21.15 q-aq:19.43 q-noVbv:21.15 q-Rceq:565.49 tex:19512 mv:16766 misc:15041 icu:7.50 pcu:1472.50 scu:30920.00 sc:0 ;
in:617 out:618 type:b q:21.15 q-aq:19.60 q-noVbv:21.15 q-Rceq:565.49 tex:22118 mv:9902 misc:11770 icu:12.50 pcu:2035.50 scu:30352.00 sc:0 ;
in:622 out:619 type:P q:17.88 q-aq:15.86 q-noVbv:17.88 q-Rceq:465.10 tex:1473509 mv:69570 misc:75329 icu:28.00 pcu:10446.50 scu:21925.50 sc:0 ;
in:620 out:620 type:B q:19.52 q-aq:17.40 q-noVbv:19.52 q-Rceq:465.10 tex:60857 mv:36698 misc:30066 icu:34.00 pcu:3284.50 scu:29081.50 sc:0 ;
in:619 out:621 type:b q:20.90 q-aq:20.03 q-noVbv:20.90 q-Rceq:465.10 tex:10112 mv:15468 misc:11904 icu:0.50 pcu:1651.00 scu:30748.50 sc:0 ;
in:621 out:622 type:b q:20.40 q-aq:18.89 q-noVbv:20.40 q-Rceq:465.10 tex:18531 mv:11189 misc:11477 icu:1.50 pcu:2858.75 scu:29539.75 sc:0 ;
in:626 out:623 type:P q:16.88 q-aq:14.98 q-noVbv:16.88 q-Rceq:403.25 tex:2501031 mv:114114 misc:91022 icu:25.50 pcu:15447.00 scu:16927.50 sc:0 ;
in:624 out:624 type:B q:18.52 q-aq:16.65 q-noVbv:18.52 q-Rceq:403.25 tex:137489 mv:51871 misc:46946 icu:5.50 pcu:5322.25 scu:27072.25 sc:0 ;
in:623 out:625 type:b q:19.90 q-aq:19.07 q-noVbv:19.90 q-Rceq:403.25 tex:10638 mv:22959 misc:16154 icu:1.25 pcu:1832.25 scu:30566.50 sc:0 ;
in:625 out:626 type:b q:19.40 q-aq:17.95 q-noVbv:19.40 q-Rceq:403.25 tex:25665 mv:16503 misc:15633 icu:1.50 pcu:3283.25 scu:29115.25 sc:0 ;
in:630 out:627 type:P q:15.88 q-aq:13.95 q-noVbv:15.88 q-Rceq:366.03 tex:3424538 mv:177159 misc:100818 icu:39.00 pcu:18913.00 scu:13448.00 sc:0 ;
in:628 out:628 type:B q:17.52 q-aq:16.05 q-noVbv:17.52 q-Rceq:366.03 tex:120182 mv:76582 misc:56582 icu:9.50 pcu:5767.50 scu:26623.00 sc:0 ;
in:627 out:629 type:b q:18.90 q-aq:18.08 q-noVbv:18.90 q-Rceq:366.03 tex:21301 mv:33974 misc:21742 icu:0.25 pcu:2607.25 scu:29792.50 sc:0 ;
in:629 out:630 type:b q:18.40 q-aq:16.88 q-noVbv:18.40 q-Rceq:366.03 tex:39023 mv:22268 misc:19983 icu:0.50 pcu:3603.75 scu:28795.75 sc:0 ;
in:634 out:631 type:P q:14.88 q-aq:12.98 q-noVbv:14.88 q-Rceq:342.51 tex:3609715 mv:185166 misc:103209 icu:22.50 pcu:19499.25 scu:12878.25 sc:0 ;
in:632 out:632 type:B q:16.52 q-aq:14.81 q-noVbv:16.52 q-Rceq:342.51 tex:238353 mv:98798 misc:73415 icu:3.75 pcu:6427.50 scu:25968.75 sc:0 ;
in:631 out:633 type:b q:17.90 q-aq:16.68 q-noVbv:17.90 q-Rceq:342.51 tex:29740 mv:46310 misc:27812 icu:0.25 pcu:3039.75 scu:29360.00 sc:0 ;
in:633 out:634 type:b q:17.40 q-aq:15.69 q-noVbv:17.40 q-Rceq:342.51 tex:57690 mv:32825 misc:27877 icu:0.25 pcu:4382.00 scu:28017.75 sc:0 ;
in:637 out:635 type:P q:13.88 q-aq:11.91 q-noVbv:13.88 q-Rceq:325.14 tex:3460038 mv:158479 misc:101680 icu:20.50 pcu:18189.25 scu:14190.25 sc:0 ;
in:636 out:636 type:B q:15.35 q-aq:13.63 q-noVbv:15.35 q-Rceq:325.14 tex:379193 mv:113269 misc:80118 icu:5.75 pcu:7556.50 scu:24837.75 sc:0 ;
in:635 out:637 type:b q:16.82 q-aq:15.46 q-noVbv:16.82 q-Rceq:325.14 tex:46405 mv:63600 misc:38532 icu:0.25 pcu:3900.00 scu:28499.75 sc:0 ;
in:638 out:638 type:I q:20.99 q-aq:19.32 q-noVbv:13.67 q-Rceq:711.92 tex:8896344 mv:970390 misc:41414 icu:32400.00 pcu:0.00 scu:0.00 sc:0 ;
in:642 out:639 type:P q:18.93 q-aq:16.59 q-noVbv:18.93 q-Rceq:567.78 tex:2552005 mv:69386 misc:76288 icu:55.50 pcu:15176.00 scu:17168.50 sc:0 ;
in:640 out:640 type:B q:20.06 q-aq:18.22 q-noVbv:20.06 q-Rceq:567.78 tex:112905 mv:52989 misc:38762 icu:21.50 pcu:2946.00 scu:29432.50 sc:0 ;
in:639 out:641 type:b q:21.20 q-aq:19.76 q-noVbv:21.20 q-Rceq:567.78 tex:16728 mv:18405 misc:15537 icu:2.50 pcu:1139.25 scu:31258.25 sc:0 ;
in:641 out:642 type:b q:21.20 q-aq:19.98 q-noVbv:21.20 q-Rceq:567.78 tex:20026 mv:10150 misc:11915 icu:1.75 pcu:2091.75 scu:30306.50 sc:0 ;
in:646 out:643 type:P q:16.93 q-aq:14.90 q-noVbv:16.93 q-Rceq:466.88 tex:3453938 mv:113190 misc:91953 icu:57.25 pcu:18479.00 scu:13863.75 sc:0 ;
in:644 out:644 type:B q:19.06 q-aq:17.43 q-noVbv:19.06 q-Rceq:466.88 tex:55483 mv:58182 misc:40500 icu:10.00 pcu:3085.25 scu:29304.75 sc:0 ;
in:643 out:645 type:b q:20.70 q-aq:19.80 q-noVbv:20.70 q-Rceq:466.88 tex:9654 mv:20986 misc:14965 icu:6.00 pcu:1345.50 scu:31048.50 sc:0 ;
in:645 out:646 type:b q:19.70 q-aq:18.68 q-noVbv:19.70 q-Rceq:466.88 tex:25414 mv:13221 misc:13880 icu:10.50 pcu:2711.00 scu:29678.50 sc:0 ;
in:650 out:647 type:P q:14.93 q-aq:12.75 q-noVbv:14.93 q-Rceq:400.07 tex:4860434 mv:112723 misc:92018 icu:38.25 pcu:21269.25 scu:11092.50 sc:0 ;
in:648 out:648 type:B q:17.06 q-aq:15.36 q-noVbv:17.06 q-Rceq:400.07 tex:181971 mv:88697 misc:66339 icu:12.25 pcu:4751.25 scu:27636.50 sc:0 ;
in:647 out:649 type:b q:18.70 q-aq:18.05 q-noVbv:18.70 q-Rceq:400.07 tex:17448 mv:35552 misc:23648 icu:0.50 pcu:1938.25 scu:30461.25 sc:0 ;
in:649 out:650 type:b q:17.70 q-aq:16.18 q-noVbv:17.70 q-Rceq:400.07 tex:49749 mv:23784 misc:23905 icu:7.25 pcu:4414.00 scu:27978.75 sc:0 ;
in:654 out:651 type:P q:12.93 q-aq:10.74 q-noVbv:12.93 q-Rceq:359.37 tex:6346580 mv:139910 misc:90402 icu:41.00 pcu:24633.75 scu:7725.25 sc:0 ;
in:652 out:652 type:B q:15.06 q-aq:13.47 q-noVbv:15.06 q-Rceq:359.37 tex:328093 mv:117717 misc:82173 icu:1.25 pcu:6665.50 scu:25733.25 sc:0 ;
in:651 out:653 type:b q:16.70 q-aq:15.41 q-noVbv:16.70 q-Rceq:359.37 tex:47873 mv:65283 misc:38875 icu:5.50 pcu:2853.50 scu:29541.00 sc:0 ;
in:653 out:654 type:b q:15.70 q-aq:14.17 q-noVbv:15.70 q-Rceq:359.37 tex:122081 mv:53270 misc:43604 icu:0.25 pcu:5083.00 scu:27316.75 sc:0 ;
in:658 out:655 type:P q:10.93 q-aq:8.72 q-noVbv:10.93 q-Rceq:335.85 tex:8136039 mv:171716 misc:87596 icu:28.25 pcu:27253.25 scu:5118.50 sc:0 ;
in:656 out:656 type:B q:13.06 q-aq:11.38 q-noVbv:13.06 q-Rceq:335.85 tex:702323 mv:150229 misc:94949 icu:2.25 pcu:8872.75 scu:23525.00 sc:0 ;
in:655 out:657 type:b q:14.70 q-aq:13.03 q-noVbv:14.70 q-Rceq:335.85 tex:159634 mv:116822 misc:61641 icu:0.50 pcu:4930.50 scu:27469.00 sc:0 ;
in:657 out:658 type:b q:13.70 q-aq:12.15 q-noVbv:13.70 q-Rceq:335.85 tex:353740 mv:105045 misc:71443 icu:1.00 pcu:7244.00 scu:25155.00 sc:0 ;
in:661 out:659 type:P q:8.93 q-aq:6.77 q-noVbv:8.93 q-Rceq:321.52 tex:10085655 mv:212847 misc:81959 icu:33.75 pcu:28963.00 scu:3403.25 sc:0 ;
in:660 out:660 type:B q:10.73 q-aq:8.96 q-noVbv:10.73 q-Rceq:321.52 tex:1701220 mv:210087 misc:112924 icu:3.00 pcu:12724.00 scu:19673.00 sc:0 ;
in:659 out:661 type:b q:12.53 q-aq:10.95 q-noVbv:12.53 q-Rceq:321.52 tex:494534 mv:186503 misc:91622 icu:0.25 pcu:7750.25 scu:24649.50 sc:0 ;
in:662 out:662 type:I q:20.91 q-aq:19.25 q-noVbv:12.84 q-Rceq:711.40 tex:8915234 mv:970048 misc:41228 icu:32400.00 pcu:0.00 scu:0.00 sc:0 ;
in:666 out:663 type:P q:19.38 q-aq:16.96 q-noVbv:19.38 q-Rceq:565.91 tex:1524170 mv:44877 misc:68638 icu:39.50 pcu:11065.25 scu:21295.25 sc:0 ;
in:664 out:664 type:B q:20.51 q-aq:18.63 q-noVbv:20.51 q-Rceq:565.91 tex:87690 mv:33189 misc:28763 icu:17.00 pcu:2707.00 scu:29676.00 sc:0 ;
in:663 out:665 type:b q:21.65 q-aq:20.91 q-noVbv:21.65 q-Rceq:565.91 tex:11465 mv:11524 misc:10883 icu:1.00 pcu:991.00 scu:31408.00 sc:0 ;
in:665 out:666 type:b q:21.65 q-aq:20.31 q-noVbv:21.65 q-Rceq:565.91 tex:19446 mv:7853 misc:9471 icu:6.50 pcu:1917.50 scu:30476.00 sc:0 ;
in:670 out:667 type:P q:17.38 q-aq:15.30 q-noVbv:17.38 q-Rceq:469.86 tex:3976949 mv:121189 misc:90081 icu:59.50 pcu:20252.75 scu:12087.75 sc:0 ;
in:668 out:668 type:B q:19.51 q-aq:17.88 q-noVbv:19.51 q-Rceq:469.86 tex:47541 mv:61272 misc:40779 icu:20.50 pcu:2869.00 scu:29510.50 sc:0 ;
in:667 out:669 type:b q:21.15 q-aq:20.17 q-noVbv:21.15 q-Rceq:469.86 tex:9628 mv:22542 misc:15445 icu:0.25 pcu:1248.25 scu:31151.50 sc:0 ;
in:669 out:670 type:b q:20.15 q-aq:18.67 q-noVbv:20.15 q-Rceq:469.86 tex:21903 mv:11465 misc:12803 icu:4.75 pcu:2190.50 scu:30204.75 sc:0 ;
in:674 out:671 type:P q:15.38 q-aq:13.37 q-noVbv:15.38 q-Rceq:413.48 tex:5801742 mv:234587 misc:96440 icu:40.25 pcu:24936.25 scu:7423.50 sc:0 ;
in:672 out:672 type:B q:17.51 q-aq:15.80 q-noVbv:17.51 q-Rceq:413.48 tex:145165 mv:92370 misc:66726 icu:8.50 pcu:5054.50 scu:27337.00 sc:0 ;
in:671 out:673 type:b q:19.15 q-aq:18.09 q-noVbv:19.15 q-Rceq:413.48 tex:20184 mv:41919 misc:25512 icu:1.75 pcu:2380.00 scu:30018.25 sc:0 ;
in:673 out:674 type:b q:18.15 q-aq:16.41 q-noVbv:18.15 q-Rceq:413.48 tex:69539 mv:24097 misc:26707 icu:3.50 pcu:4566.75 scu:27829.75 sc:0 ;
in:678 out:675 type:P q:13.38 q-aq:11.38 q-noVbv:13.38 q-Rceq:381.19 tex:7447218 mv:281964 misc:92415 icu:42.75 pcu:26994.75 scu:5362.50 sc:0 ;
in:676 out:676 type:B q:15.51 q-aq:13.74 q-noVbv:15.51 q-Rceq:381.19 tex:296535 mv:119988 misc:81663 icu:3.50 pcu:5912.00 scu:26484.50 sc:0 ;
in:675 out:677 type:b q:17.15 q-aq:15.73 q-noVbv:17.15 q-Rceq:381.19 tex:56922 mv:80495 misc:42571 icu:4.00 pcu:3064.50 scu:29331.50 sc:0 ;
in:677 out:678 type:b q:16.15 q-aq:14.56 q-noVbv:16.15 q-Rceq:381.19 tex:137139 mv:43862 misc:40403 icu:0.50 pcu:4287.50 scu:28112.00 sc:0 ;
in:682 out:679 type:P q:11.38 q-aq:9.29 q-noVbv:11.38 q-Rceq:353.80 tex:8839319 mv:244640 misc:87670 icu:32.50 pcu:28147.25 scu:4220.25 sc:0 ;
in:680 out:680 type:B q:13.51 q-aq:11.86 q-noVbv:13.51 q-Rceq:353.80 tex:616643 mv:150017 misc:96701 icu:2.00 pcu:7981.50 scu:24416.50 sc:0 ;
in:679 out:681 type:b q:15.15 q-aq:13.41 q-noVbv:15.15 q-Rceq:353.80 tex:148321 mv:110886 misc:58298 icu:2.00 pcu:4329.50 scu:28068.50 sc:0 ;
in:681 out:682 type:b q:14.15 q-aq:12.35 q-noVbv:14.15 q-Rceq:353.80 tex:364847 mv:86474 misc:65944 icu:1.25 pcu:6469.00 scu:25929.75 sc:0 ;
in:685 out:683 type:P q:9.38 q-aq:7.32 q-noVbv:9.38 q-Rceq:332.22 tex:10016105 mv:232399 misc:82305 icu:41.00 pcu:29078.00 scu:3281.00 sc:0 ;
in:684 out:684 type:B q:11.18 q-aq:9.47 q-noVbv:11.18 q-Rceq:332.22 tex:1575095 mv:199183 misc:109967 icu:1.75 pcu:12011.75 scu:20386.50 sc:0 ;
in:683 out:685 type:b q:12.98 q-aq:11.32 q-noVbv:12.98 q-Rceq:332.22 tex:442900 mv:175938 misc:87557 icu:1.00 pcu:7063.25 scu:25335.75 sc:0 ;
in:686 out:686 type:I q:20.91 q-aq:19.25 q-noVbv:12.77 q-Rceq:713.22 tex:8935011 mv:977406 misc:41075 icu:32400.00 pcu:0.00 scu:0.00 sc:0 ;

HD has the I-frames QP between 32 and 35. The UHDs I-frame QP is ~20. Much different! In this case, the settings are not the same, which I plan to rectify, but thought the difference was enough to call out for now.

The question I have is, why are the HD encoded I frames at QP 35 while the UHD are at QP 20. Since the bitrates are the same, I would hope that HD would at least be the same as UHD.

I will redo all of this with the same exact settings for HD and UHD for a more apples to apples comparison. It will take a few days to complete. I first want to figure out what setting is making pass 3 identical to pass 1 in terms of stats file. The other setting I changed was to remove --slow-firstpasss from pass 1 AND to also remove --multi-pass-opt-analysis from pass 3. It throws a warning that you can't use this and reports that it disabled it.

Dumb question, can pass 3 use --slow-firstpass? It does not throw a warning if I include it. Just not sure if it actually does anything for pass 3.

Stacey Spears

27th April 2021, 15:38

I gave the sequence a shot using some other encoder. Sadly I used a Dolby Pulsar P3D65 as Mastering Display and left the CLL SEI at 0,0.

It is here: https://drive.google.com/file/d/1-Uz8jcFZYJD-t-NRQ6cPvRoZvht8kU7L/view?usp=sharing

I set the average rate to 80M and video buffer rate to 100M. GOP is closed and keyframe interval is fixed.

The encoder has problems reaching the average rate because of the low resolution of the sequence. I did a lossless HEVC encode and it ended up being ~9.2GB, this encode is ~4.5GB. If someone wants to see a lower average rate encode - I am staying around here for a while, just request.

I don't know if the encode is really HEVC BluRay compliant, please advise, this currently interests me the most.

Maybe we should start a new thread if we diverge from x265 to general encoding of high rate HEVC using different encoders....

If we want to start a dedicated thread for this issue, I am good with that. I do feel bad about filling this thread with something that might not interest everyone.

benwaggoner

27th April 2021, 17:36

The keyframe pulsing at frame ~600 is caused because x265 is too afraid to pump bits into the I-Frames, so they are encoded with a really high quantizer in comparison to the surrounding frames. Likely related to VBV settings.
If you're having IDR/i QP issues, you should try:
--hist-scenecut
--scenecut-aware-qp

Both have parameters for fine tuning

And using a qpfile to specify a low QP for that frame should work as well.

And, possibly, since you are using Closed GOP
--radl 2

Boulder

27th April 2021, 18:57

Dumb question, can pass 3 use --slow-firstpass? It does not throw a warning if I include it. Just not sure if it actually does anything for pass 3.
I think it does nothing because it's not pass number 1.

If you're having IDR/i QP issues, you should try:
--hist-scenecut
--scenecut-aware-qp

Unfortunately at least --hist-scenecut is not good at all. They just left it half finished.

benwaggoner

28th April 2021, 00:26

I think it does nothing because it's not pass number 1.

Unfortunately at least --hist-scenecut is not good at all. They just left it half finished.
I thought it was improved in 4.5 (https://bitbucket.org/multicoreware/x265_git/commits/f0c1022b6be121a753ff02853fbe33da71988656).
+1. Improved hist-based scene cut algorithm: Reduces false positives by leveraging motion and scene transition info.
+2. Support for RADL pictures at IDR scene cuts: Improves coding efficiency with no significant impact on performance.
+3. Bidirectional scene cut aware Frame Quantizer Selection: Saves bits than forward masking with no noticeable perceptual quality difference.
I've not thoroughly tested the 4.5 version, though.

quietvoid

28th April 2021, 01:05

No, the "improvements" they did (back in July 2020) made it much worse and the threshold is practically useless now.

Stacey Spears

28th April 2021, 01:28

I think it does nothing because it's not pass number 1.

Unfortunately at least --hist-scenecut is not good at all. They just left it half finished.

I tracked it down. If you call pass 3 with --multi-pass-opt-analysis then the pass 3 stats is 100% identical to pass 1 stats. I guess I misunderstood the warning in the log file.

x265 [warning]: --multi-pass-opt-analysis doesn't support refining analysis through multiple-passes; it only reuses analysis from the second-to-last pass to the last pass. Disabling reading

I had assumed the warning meant it was not going to use "multi-pass-opt-analysis", but the "disabling reading" comment might mean that it is not reading the pass 1 file, which is why pass 3 file is just do over.

Stacey Spears

28th April 2021, 01:54

And using a qpfile to specify a low QP for that frame should work as well.

This is one of the first things I tried. It basically ignored the QPs I specified for the I frames. You can see in the stats I set the value, but it still coded with high QPs. I don't know if I tried to raise the P and B frames QP. I may give that a shot.

What does RADL do? The docs don't say much.

I have kicked off a 3pass with the UHD 1000 nit tonight. Here is the bat file (https://www.dropbox.com/s/muiffpjoir8921t/Encode_HDR_Montage_UHD_2pass_90.bat?dl=0) I am using for this encode test.

Boulder

28th April 2021, 05:17

No, the "improvements" they did (back in July 2020) made it much worse and the threshold is practically useless now.

Yes, I think we both did test the improvements and found out that it's basically not working at all. And then it was just left as it is.

Stacey Spears

28th April 2021, 16:04

Here is the Sony encode (https://www.dropbox.com/s/cmdu735z2qwbilo/HDR-Montage-V2-01000-HD_Start0_Frames11235_HDR10.mp4?dl=1). It does not have the keyframe pulsing artifacts or the blocking on the deer shot. It ended up ~29 Mbps.

Stacey Spears

28th April 2021, 18:13

And using a qpfile to specify a low QP for that frame should work as well.

And, possibly, since you are using Closed GOP
--radl 2

I tried RADL2, did not help. It did raise the QP of two B frames at the start of each GOP.

I was able to raise the P and B frame QPs, which in turn lowered the I frame QPs using the QP file.

I am testing to see if open-gop can be used on BD. The encode from Sony looks like it was open gop or at least the MUX said it was. I made a change to the --uhd-bd flag to force open gop and will test mux it shortly.

Edit: Open GOP does mux. I guess the question is, why did they disable Open GOP when you enable --ud-bd? Sony seems to be using Open GOP for their UHD BD encodes. I am going to re-encode the UHD version with Open GOP enabled.

excellentswordfight

28th April 2021, 19:36

I tried RADL2, did not help. It did raise the QP of two B frames at the start of each GOP.

I was able to raise the P and B frame QPs, which in turn lowered the I frame QPs using the QP file.

I am testing to see if open-gop can be used on BD. The encode from Sony looks like it was open gop or at least the MUX said it was. I made a change to the --uhd-bd flag to force open gop and will test mux it shortly.

Edit: Open GOP does mux. I guess the question is, why did they disable Open GOP when you enable --ud-bd? Sony seems to be using Open GOP for their UHD BD encodes. I am going to re-encode the UHD version with Open GOP enabled.
Looks like the spec doesnt mandate closed or open gop, both should be viable.

Section 3.2

https://web.archive.org/web/20160605030640/http://www.blu-raydisc.com/assets/Downloadablefile/BD-ROM_Part3_V3.0_WhitePaper_150724.pdf

edit. Getting the same issue here with your source, and the issue persist even with open-gop (skipping uhd-bd preset and setting the other settings manually except for closed gop). And the issue is gone using crf mode...

Stacey Spears

29th April 2021, 01:51

Looks like the spec doesnt mandate closed or open gop, both should be viable.

Section 3.2

https://web.archive.org/web/20160605030640/http://www.blu-raydisc.com/assets/Downloadablefile/BD-ROM_Part3_V3.0_WhitePaper_150724.pdf

edit. Getting the same issue here with your source, and the issue persist even with open-gop (skipping uhd-bd preset and setting the other settings manually except for closed gop). And the issue is gone using crf mode...

I have gone ahead and modified x265 to honor the open-gop settings for BD. I did a test mux and it worked. I also confirmed that another facility encodes all of their UHD BDs using open gop with the Sony encoder.

The Sony encode I provided earlier used 8 slices, which is what they use for UHD. They re-encoded with 4 slices and also adjusted the quant table to preserve more high frequency detail. It bumped the bitrate 10 Mbps. I will share it after I QC.

I am doing a 3-pass right now. Once pass 3 (2nd pass) is complete, I will probably create a QP file, from the stats file, to push up the P and B frame QPs for the shots with keyframe pulsing and then re-start pass 1 using the new QP file.

In a short test earlier, it looked like the open gop helped with the keyframe pulsing in the fence shot.

It does seem like x265 over spends on P and B frames, at least on this content. The shots are mostly static in terms of motion.

I wish someone would implement segment re-encoding, then I could fix the shots a little more easily.

benwaggoner

29th April 2021, 23:11

I have gone ahead and modified x265 to honor the open-gop settings for BD. I did a test mux and it worked. I also confirmed that another facility encodes all of their UHD BDs using open gop with the Sony encoder.

The Sony encode I provided earlier used 8 slices, which is what they use for UHD. They re-encoded with 4 slices and also adjusted the quant table to preserve more high frequency detail. It bumped the bitrate 10 Mbps. I will share it after I QC.

I am doing a 3-pass right now. Once pass 3 (2nd pass) is complete, I will probably create a QP file, from the stats file, to push up the P and B frame QPs for the shots with keyframe pulsing and then re-start pass 1 using the new QP file.
Wow, doesn't this increasingly feel like authoring VC-1 HD-DVD/BD titles circa 2006 :sly:?

In a short test earlier, it looked like the open gop helped with the keyframe pulsing in the fence shot.
I hope your open GOP fix makes it into the main branch! This is exactly the sort of thing Open GOP is supposed to help. And it matters all the more with short GOPs like this (1 sec) since most IDRs won't align with cuts. Open GOP is better than RADL.

It does seem like x265 over spends on P and B frames, at least on this content. The shots are mostly static in terms of motion.
You could potentially play with --ipratio and --pbratio there.

I wish someone would implement segment re-encoding, then I could fix the shots a little more easily.
I think it is actually possible with x265 if you don't mind appending in mp4box or something.
You'd use --seek and --frames to set in/out points, and set --chunk-start and --chunk-end to expand that range by a few GOPs to get VBV state to match.

If it's a --pass 2 or 3, I'm not sure if the --chunk parameters are needed; maybe VBV state of the initial state works.

It seems sufficiently advanced encoding often winds up replicating PEP, eh?

Stacey Spears

30th April 2021, 00:46

Wow, doesn't this increasingly feel like authoring VC-1 HD-DVD/BD titles circa 2006 :sly:?

I hope your open GOP fix makes it into the main branch! This is exactly the sort of thing Open GOP is supposed to help. And it matters all the more with short GOPs like this (1 sec) since most IDRs won't align with cuts. Open GOP is better than RADL.

You could potentially play with --ipratio and --pbratio there.

I think it is actually possible with x265 if you don't mind appending in mp4box or something.
You'd use --seek and --frames to set in/out points, and set --chunk-start and --chunk-end to expand that range by a few GOPs to get VBV state to match.

If it's a --pass 2 or 3, I'm not sure if the --chunk parameters are needed; maybe VBV state of the initial state works.

It seems sufficiently advanced encoding often winds up replicating PEP, eh?

I tried setting ipratio and pbratio to higher values like 2.0, as a test, but it still ends up lowering P and B while maintaining the high I frame QPs on the same shots.

For the short comings of VC1, PEP had some great segment re-encoding capabilities.

You can use the concat flag (don't recall the exact name off hand) in x265 to split the clip up. I know someone who does this to break the movie into chunks for parallel encoding ala PEP. But for an 8-minute video, seems odd to break-up even further. :)

I need to do some testing with the encode on a few BD players to make sure there is not a playback issue. I wish I knew why they disabled it. Could have been confusion about it being supported OR to work around a player issue.

I made a change where I could use the zones file to set --qpmax, but when I used the feature, no I frames were inserted for the rest of the encode, and I have not gotten around to debugging it. :) I was going to set a qpmax on the GOPs where the I frames went higher. Nothing like brute force!

benwaggoner

30th April 2021, 02:31

I tried setting ipratio and pbratio to higher values like 2.0, as a test, but it still ends up lowering P and B while maintaining the high I frame QPs on the same shots.

For the short comings of VC1, PEP had some great segment re-encoding capabilities.

You can use the concat flag (don't recall the exact name off hand) in x265 to split the clip up. I know someone who does this to break the movie into chunks for parallel encoding ala PEP. But for an 8-minute video, seems odd to break-up even further. :)

I need to do some testing with the encode on a few BD players to make sure there is not a playback issue. I wish I knew why they disabled it. Could have been confusion about it being supported OR to work around a player issue.

I made a change where I could use the zones file to set --qpmax, but when I used the feature, no I frames were inserted for the rest of the encode, and I have not gotten around to debugging it. :) I was going to set a qpmax on the GOPs where the I frames went higher. Nothing like brute force!
I used to have an Excel script that would take x264 per-frame console output (like the x265 .csv files) and turn that into a qpfile. The main use was to determine optimal IDR placement in a first pass and synchronize that across different ABR bitrates.

It'd be easier to implement that with .csv, and then you could tweak each frame's type and qp for each frame.

Stacey Spears

30th April 2021, 13:31

I used to have an Excel script that would take x264 per-frame console output (like the x265 .csv files) and turn that into a qpfile. The main use was to determine optimal IDR placement in a first pass and synchronize that across different ABR bitrates.

It'd be easier to implement that with .csv, and then you could tweak each frame's type and qp for each frame.

I have a python script that you run against the stats file to dump out the frame types and QPs if you like. This is used to encode the Dolby Vision enhancement layer since they need the same frame types across the two layers.

I need to write a tool that will dump the QPs on any hevc file. We wrote a tool recently that allows us to move the SEI from one hevc file to another. I use this to move the SL-HDR2 SEI messages from the file Philips modified to a re-encoded file. This way Philips does not have to reprocess the clip for me. They have done it twice already, now they don't have to. :)

Selur

2nd May 2021, 14:25

using:

x265 --input - --output-depth 10 --y4m --profile main10 --hme --hme-search star,hex,full --limit-modes --no-early-skip --no-open-gop --opt-ref-list-length-pps --crf 18.00 --opt-qp-pps --cbqpoffs -2 --crqpoffs -2 --limit-refs 0 --ssim-rd --psy-rd 2.50 --rdoq-level 2 --psy-rdoq 10.00 --aq-mode 0 --deblock=-1:-1 --limit-sao --no-repeat-headers --range limited --colormatrix bt470bg --output "E:\Temp\2021-05-02@15_19_28_0110_01.265"

resulted in sei:

cpuid=1111039 / frame-threads=4 / numa-pools=16 / wpp / no-pmode / no-pme / no-psnr / no-ssim / log-level=2 / input-csp=1 / input-res=640x352 / interlace=0 / total-frames=0 / level-idc=0 / high-tier=1 / uhd-bd=0 / ref=3 / no-allow-non-conformance / no-repeat-headers / annexb / no-aud / no-hrd / info / hash=0 / no-temporal-layers / no-open-gop / min-keyint=25 / keyint=250 / gop-lookahead=0 / bframes=4 / b-adapt=2 / b-pyramid / bframe-bias=0 / rc-lookahead=20 / lookahead-slices=0 / scenecut=40 / hist-scenecut=0 / radl=0 / no-splice / no-intra-refresh / ctu=64 / min-cu-size=8 / no-rect / no-amp / max-tu-size=32 / tu-inter-depth=1 / tu-intra-depth=1 / limit-tu=0 / rdoq-level=2 / dynamic-rd=0.00 / ssim-rd / signhide / no-tskip / nr-intra=0 / nr-inter=0 / no-constrained-intra / strong-intra-smoothing / max-merge=3 / limit-refs=0 / limit-modes / me=1 / subme=2 / merange=57 / temporal-mvp / no-frame-dup / no-hme / weightp / no-weightb / no-analyze-src-pics / deblock=-1:-1 / sao / no-sao-non-deblock / rd=6 / selective-sao=4 / no-early-skip / rskip / no-fast-intra / no-tskip-fast / no-cu-lossless / b-intra / no-splitrd-skip / rdpenalty=0 / psy-rd=2.50 / psy-rdoq=10.00 / rd-refine / no-lossless / cbqpoffs=-2 / crqpoffs=-2 / rc=crf / crf=18.0 / qcomp=0.60 / qpstep=4 / stats-write=0 / stats-read=0 / ipratio=1.40 / pbratio=1.30 / aq-mode=1 / aq-strength=0.00 / cutree / zone-count=0 / no-strict-cbr / qg-size=32 / no-rc-grain / qpmax=69 / qpmin=0 / no-const-vbv / sar=1 / overscan=0 / videoformat=5 / range=0 / colorprim=2 / transfer=2 / colormatrix=5 / chromaloc=0 / display-window=0 / cll=0,0 / min-luma=0 / max-luma=1023 / log2-max-poc-lsb=8 / vui-timing-info / vui-hrd-info / slices=1 / opt-qp-pps / opt-ref-list-length-pps / no-multi-pass-opt-rps / scenecut-bias=0.05 / hist-threshold=0.03 / opt-cu-delta-qp / no-aq-motion / no-hdr10 / no-hdr10-opt / no-dhdr10-opt / no-idr-recovery-sei / analysis-reuse-level=0 / analysis-save-reuse-level=0 / analysis-load-reuse-level=0 / scale-factor=0 / refine-intra=0 / refine-inter=0 / refine-mv=1 / refine-ctu-distortion=0 / limit-sao / ctu-info=0 / no-lowpass-dct / refine-analysis-type=0 / copy-pic=1 / max-ausize-factor=1.0 / no-dynamic-refine / no-single-sei / no-hevc-aq / no-svt / no-field / qp-adaptation-range=1.00 / no-scenecut-aware-qpconformance-window-offsets / right=0 / bottom=0 / decoder-max-rate=0 / no-vbv-live-multi-pass

-> is it a bug that "no-hme" is shown or am I missing something ?

Cu Selur

benwaggoner

3rd May 2021, 17:29

It does seem like a bug, but you can check by looking at the output of hme on/off and verify that there are differences in output and encoding time. Visual differences with HME are most likely encoding very noisy content at 4K, as the first 960x540 analysis will have averaged out lots of the noise and have picked out the underlying features a lot better.

Stacey Spears

8th May 2021, 13:38

And using a qpfile to specify a low QP for that frame should work as well.

So far using a QPFile to raise the QP of the P and B frames seems to be working well. For the UHD version, one shot needed it and I am also using it for two other shots where the I frame QP value is 3-6 higher than P and B. These additional shots had pulsing in the HD version.

The end result is running 4.75 passes for each encode. I have 12 versions to encode this way. :)

1. Encode to passes and stop the 2nd pass (really pass 3 for the stats file) after it finishes the last shot in question.
2. create the QP File. I use the pass 3 stats file since its QPs are closer to final than the pass 1 stats file.
3. Encode three passes using the QP File.

Will be running a mux test today with the Atmos audio just to confirm this will actually work in the end.

For the shot with visible pulsing, I am rounding the pass 3 QP and then adding 3 to it. For the other two shots, I am rounding and then adding 1.

At this point I can only assume the x265 devs are busy on x266.

Edit: It muxed.

benwaggoner

10th May 2021, 21:38

So far using a QPFile to raise the QP of the P and B frames seems to be working well. For the UHD version, one shot needed it and I am also using it for two other shots where the I frame QP value is 3-6 higher than P and B. These additional shots had pulsing in the HD version.

The end result is running 4.75 passes for each encode. I have 12 versions to encode this way. :)

1. Encode to passes and stop the 2nd pass (really pass 3 for the stats file) after it finishes the last shot in question.
2. create the QP File. I use the pass 3 stats file since its QPs are closer to final than the pass 1 stats file.
3. Encode three passes using the QP File.

Will be running a mux test today with the Atmos audio just to confirm this will actually work in the end.

For the shot with visible pulsing, I am rounding the pass 3 QP and then adding 3 to it. For the other two shots, I am rounding and then adding 1.

At this point I can only assume the x265 devs are busy on x266.

Edit: It muxed.
I'm glad it muxed! But yikes, that sounds painful.

If you're having to do so many versions, perhaps --abr-ladder could be helpful?

Stacey Spears

12th May 2021, 22:28

I'm glad it muxed! But yikes, that sounds painful.

If you're having to do so many versions, perhaps --abr-ladder could be helpful?

Isn't --abr-ladder more for layered encodes in the streaming scenario?

3 of 12 are done so far. :) I currently have the next three running their initial two pass to figure out what to use for the qp file. This is for a single project, so not the end of the world. :p If I were a streaming service, this would not be practical. The real solution is for someone to debug the issue and figure out why the rate control is opting for low P and B QPs with high I QPs. Perhaps I should see if I can get MSU to use the content for future CODEC shootouts.

BTW, off topic, is it true that Blue Star has closed many locations in Portland? So sad if true! :p

benwaggoner

13th May 2021, 03:19

Isn't --abr-ladder more for layered encodes in the streaming scenario?
I thought you were encoding multiple variants of the same source. If not applicable, than it is not applicable ;).

3 of 12 are done so far. :) I currently have the next three running their initial two pass to figure out what to use for the qp file. This is for a single project, so not the end of the world. :p If I were a streaming service, this would not be practical. The real solution is for someone to debug the issue and figure out why the rate control is opting for low P and B QPs with high I QPs. Perhaps I should see if I can get MSU to use the content for future CODEC shootouts.
Good stress test content is always a delight. My new favorite stress test clip is Netflix's Sol Volante. http://download.opencontent.netflix.com/?prefix=SolLevante/hdr10/

BTW, off topic, is it true that Blue Star has closed many locations in Portland? So sad if true! :p
I see four still open, but I don't recall how many there used to be. I've barely been to restaurants the last 14 months. But my fiancé is getting her second shot on Friday, so hopefully venturing into the world more soon.

Marsu42

13th May 2021, 09:25

Since then I've just given up on hist-scenecut.

Too bad they don't seem to care for the original method anymore

Since Steve Borho and chen moved on, the main quality check is "does it compile". Almost every submitted patch is immediately committed without discussion now, and reverted later if need be.

Unfortunately at least --hist-scenecut is not good at all. They just left it half finished.

So the expert consensus is that --hist-scenecut is broken atm, and users should simply use the traditional method - correct?

If so, it's unfortunate this isn't marked as "Experimental feature" in the command line options :-\ https://x265.readthedocs.io/en/master/cli.html

ShortKatz

13th May 2021, 16:53

Did x265 development kind of stoped? It is not much going on currently.

benwaggoner

13th May 2021, 18:35

So the expert consensus is that --hist-scenecut is broken atm, and users should simply use the traditional method - correct?

If so, it's unfortunate this isn't marked as "Experimental feature" in the command line options :-\ https://x265.readthedocs.io/en/master/cli.html
I haven't seen much feedback about it 3.5, where it was improved. From the release notes:
+1. Improved hist-based scene cut algorithm: Reduces false positives by leveraging motion and scene transition info.

Anyone retried --hist-scencut again in 3.5?

Boulder

13th May 2021, 18:43

Yes, we have tested it and it's worse than before. Mind you, it didn't work properly even before v3.5..

It is very much a reality that they don't seem to have many hardcore devs left. Of course, the encoder is very stable and works well, but there are these reported cases which are never investigated.

benwaggoner

14th May 2021, 01:09

Yes, we have tested it and it's worse than before. Mind you, it didn't work properly even before v3.5.
Worse results, but works more reliably?

It is very much a reality that they don't seem to have many hardcore devs left. Of course, the encoder is very stable and works well, but there are these reported cases which are never investigated.
I am aware that some work is going on with x265, but yeah, pretty much nothing in the public repository.

Boulder

14th May 2021, 09:45

Worse results, but works more reliably?

Unfortunately just worse results. It's very flaky and detects scene changes in motion too easily (transition from a slightly motion blurred frame to a crisper one triggers it). The threshold seems to affect it in the wrong way.

https://forum.doom9.org/showthread.php?p=1919698#post1919698

What comes to development, I think most of the recent (since ~year or so) stuff has been towards streaming and (very) low bitrate encodes. For regular encodes, much less so. So there's probably someone paying for those things and that's why they get done.

nakTT

1st June 2021, 20:28

After many month of no update, now they have made a new commit.

https://bitbucket.org/multicoreware/x265_git/commits/

benwaggoner

3rd June 2021, 21:07

After many month of no update, now they have made a new commit.

https://bitbucket.org/multicoreware/x265_git/commits/
Documentation only, though, and a somewhat confusing one.

If the encoder is unable to reach the level it issues a warning and aborts the encode. The requested level will be signaled in the bitstream even if it is higher than the actual level.

If it is aborted, how is it signaled? If the parameters are wrong, the first frame wouldn't have been even written.

Or is it saying that if a stream is started but has a VBV violation during the encode, it'll abort, but the stream that had been written up to that point would still have the originally specified level?

If so, that would be the expected and appropriate behavior, since the frame that violated the spec never was written and so the file up to that point should still be conformant.

filler56789

3rd June 2021, 22:13

OUCH! It does appear that, since Steve Borho and Min Chen left the x265 project, the remaining devs DON'T KNOW very-well what they're doing :-/

So... to all the regular/frequent users of x265, ¿could you please say which old version of x265 you recommend?

DJATOM

3rd June 2021, 23:34

benwaggoner

4th June 2021, 00:32

Definitely before ABR ladder introduced. It added complexity to codebase and it was a source of bugs for certain amount of time (maybe some bugs still not discovered). So I recommend to use 3.3 if you want less buggy encoder.
Yeah. While there are promising improvements in 3.4 & 3.5, none that I've really seen tested out enough to judge, and how much of the VBV fixes were for things that got broken post 3.3.

https://x265.readthedocs.io/en/master/releasenotes.html

No one claims even no regressions with the improved 3.5 --hist-scenecut
I've not seen test results for --rskip 2, although it is conceptually sound, and seems likely to help motion graphics and cell animation.
Have heard nothing about using RADL for variable duration GOPs yet. I've not grokked the theory for why it would be better.
Nor any results from the now bidirectional --scenecut-aware-qp, or its tuning with --scenecut-window and --max-qp delta. Using it seems to give me a syntax error some of the time (which is weird). All are conceptually sound, though.
--frame-dup seems to work fine, but those are frames that were small and early-exited already, and I've not noticed significant net improvements yet. The new Sol Levante source for my Encoding Challenge (https://forum.doom9.org/showthread.php?t=175776) should be great to test it due to all the static credits and titles.

Anyone have anything to share on those?

filler56789

4th June 2021, 18:18

LeXXuz

5th June 2021, 07:19

Definitely before ABR ladder introduced. It added complexity to codebase and it was a source of bugs for certain amount of time (maybe some bugs still not discovered). So I recommend to use 3.3 if you want less buggy encoder.

Would you mind compiling an 8-bit & 10-bit version of 3.3 with optimizations for Zen2 and Zen3? :o

I'd like to have an optimized and stable version for every day use.

masterkivat

5th June 2021, 22:25

Would you mind compiling an 8-bit & 10-bit version of 3.3 with optimizations for Zen2 and Zen3? :o

I'd like to have an optimized and stable version for every day use.
...I second that! :thanks:

Boulder

6th June 2021, 10:50

Yeah. While there are promising improvements in 3.4 & 3.5, none that I've really seen tested out enough to judge, and how much of the VBV fixes were for things that got broken post 3.3.

https://x265.readthedocs.io/en/master/releasenotes.html

No one claims even no regressions with the improved 3.5 --hist-scenecut
I've not seen test results for --rskip 2, although it is conceptually sound, and seems likely to help motion graphics and cell animation.
Have heard nothing about using RADL for variable duration GOPs yet. I've not grokked the theory for why it would be better.
Nor any results from the now bidirectional --scenecut-aware-qp, or its tuning with --scenecut-window and --max-qp delta. Using it seems to give me a syntax error some of the time (which is weird). All are conceptually sound, though.
--frame-dup seems to work fine, but those are frames that were small and early-exited already, and I've not noticed significant net improvements yet. The new Sol Levante source for my Encoding Challenge (https://forum.doom9.org/showthread.php?t=175776) should be great to test it due to all the static credits and titles.

Anyone have anything to share on those?
My views on some of those:

--hist-scenecut has been proven to not work properly, it cannot be used in any productive encodes. It went worse with the bugfixes they made and then they left it as it is.

--rskip 2 seemed to help with some onion artifacts I was seeing in flat backgrounds with some noise in my Star Trek TNG 720p encodes. I didn't find any problems with it, and shouldn't more computation and analysis be better than skipping anyway? I use --rskip 2 --rskip-edge-threshold 3 as a base setting in my encoding commandline.

Just note that --rskip 2 --limit-tu 0 --ctu 64 is asking for trouble. That combination is broken, but my bug report has not been processed at all.

Scenecut-aware-qp is a bit silly implementation because it only works in 2-pass encodes. I did some tests on a modded version that allows using it in CRF mode but didn't find it useful.

charliebaby

7th June 2021, 17:50

3.5+10-82786fc GCC 11.1

https://www.mediafire.com/file/gsyvua78dui1zut/x265-3.5+10-82786fc_gcc111-AVX2.7z/file

benwaggoner

7th June 2021, 19:57

3.5+10-82786fc GCC 11.1

https://www.mediafire.com/file/gsyvua78dui1zut/x265-3.5+10-82786fc_gcc111-AVX2.7z/file
Why is your build labeled "AVX2?" Does it exclude AVX-512 or something?

benwaggoner

7th June 2021, 20:23

I'm starting tests for the new anime version of my Encoder Challenge (https://forum.doom9.org/showthread.php?p=1944475#post1944475).

Despite content that seems ripe for --frame-dup helping static frames of credits, it doesn't seem to be working. Looking at the docs for --dup-threshold (https://x265.readthedocs.io/en/master/cli.html#input-output-file-options), it's unclear what the scale is, or if higher values make it more or less strict. Anyone have any insights?

fauxreaper

7th June 2021, 21:04

benwaggoner

8th June 2021, 18:41

Scale is PSNR between two frames. Higher values make it more strict. Default is PSNR 70.
Awesome, thank you. I'm trying another pass of my encode with --dup-threshold of 50.

Gravitator

8th June 2021, 21:15

v3.5+10 - defective image output (v3.0+1 is okay).
ffmpeg -i z:\space_6s.mkv -f yuv4mpegpipe - | x265 --y4m - --preset medium --bitrate 1000 --max-merge 1 z:\test.mkv

8th June 2021, 22:42

v3.5+10 - defective image output (v3.0+1 is okay).

It is caused by
https://bitbucket.org/multicoreware/x265_git/commits/f280829062a61792333405dbb62adf4ed7299d82

To go back to old behavior please add
--no-early-skip --no-b-intra --limit-refs 3
options to new version of x265.

benwaggoner

9th June 2021, 19:39

I've yet to get --frame-dup to actually do anything, even with --dup-threshold at 20. And with source where there are lots of identical consecutive frames.

x265.exe --input SolLevante_SDRv2_1080p24_8bit.y4m --level-idc 4.0 --preset slower --tune animation --frame-dup --dup-threshold 20 --pools "-,+" --pass 1 --keyint 120 --bitrate 1000 --vbv-maxrate 12000 --vbv-bufsize 12000 --hrd --aud --colorprim bt709 --transfer bt709 --colormatrix bt709 -o SolLevante_SDR-1080p_1000_slower-dup20_p1.hevc --psnr --ssim --csv-log-level 1 --stats SolLevante_SDR-1080p_1000_slower-dup20.stats --csv SolLevante_SDR-1080p_1000_slower-dup20_p1.csv

Every frame is at least 600 bytes. They're mostly Merge or Skip, but with very low percentages of Intra and Inter blocks of various sizes in the b-frames.

Same in first and second passes.

Anyone have any insights/suggestions?

I'm trying a single-pass test.

benwaggoner

9th June 2021, 21:18

And still no dup frames even with --dup-threshold 10, even!

Boulder

10th June 2021, 05:04

And still no dup frames even with --dup-threshold 10, even!

Works as well as the threshold in --hist-scenecut then :D

benwaggoner

10th June 2021, 19:14

Works as well as the threshold in --hist-scenecut then :D
Yeah, it is frustrating. I'm still getting around 200 Kbps for what's a single B&W frame that's static for seconds. x265 without using --zones spends way too much on static title card and scrolling credits.

My guess is adaptive quantization isn't accounting for the benefit of SAO (ringing reduction) and tskip (efficient encoding of sharp edges) for discreet-tone content. The sequence is going down to QP 20 despite getting PSNR averaging >50. I'd think cutree would be providing more benefit as well. This kind of content looks pretty perfect at QP 40 with SAO and tskip.

_kermit

11th June 2021, 12:09

I'm using ffmpeg 4.4 and this command line:

ffmpeg.exe -i %1 -pix_fmt yuv420p -f yuv4mpegpipe -| x265.exe ....

sometimes I get this and then it stops:

av_interleaved_write_frame(): Broken pipe
Error writing trailer of pipe:: Broken pipe

How can this be solved?

Gravitator

11th June 2021, 12:19

Yeah, it is frustrating. I'm still getting around 200 Kbps for what's a single B&W frame that's static for seconds. x265 without using --zones spends way too much on static title card and scrolling credits.
Another very useful for video surveillance cameras.

nakTT

13th June 2021, 00:20

Documentation only, though, and a somewhat confusing one.

If it is aborted, how is it signaled? If the parameters are wrong, the first frame wouldn't have been even written.

Or is it saying that if a stream is started but has a VBV violation during the encode, it'll abort, but the stream that had been written up to that point would still have the originally specified level?

If so, that would be the expected and appropriate behavior, since the frame that violated the spec never was written and so the file up to that point should still be conformant.
Yes. Seems like the development of x265 is basically ended already.

_kermit

13th June 2021, 09:02

rwill

13th June 2021, 09:54

wrong forum?

Pretty much wrong thread but

av_interleaved_write_frame(): Broken pipe
Error writing trailer of pipe:: Broken pipe

most likely means that the program behind ffmpeg, in your case x265, stopped accepting input because it closed its standard input. So probably x265 exited early or crashed.

Gravitator

13th June 2021, 19:09

v3.5+10 - I get a refusal of encoding.

ffmpeg -i z:\space_6s.mkv -f yuv4mpegpipe - | x265 --y4m - --preset veryslow --hme --hme-search star,sea,full --bitrate 1000 z:\test.mkv

ffmpeg version 4.3.2-2021-02-20-full_build-www.gyan.dev Copyright (c) 2000-2021 the FFmpeg developers
built with gcc 10.2.0 (Rev6, Built by MSYS2 project)
Input #0, matroska,webm, from 'z:\space_6s.mkv':
Metadata:
creation_time : 2019-08-01T04:39:18.000000Z
ENCODER : Lavf58.2.103
Duration: 00:00:06.22, start: 0.000000, bitrate: 7719 kb/s
Stream #0:0(eng): Video: h264 (High), yuv420p(progressive), 1920x1080 [SAR 1:1 DAR 16:9], 23.98 fps, 23.98 tbr, 1k tbn, 47.95 tbc (default)
Metadata:
DURATION : 00:00:06.215000000
Stream mapping:
Stream #0:0 -> #0:0 (h264 (native) -> wrapped_avframe (native))
Press [q] to stop, [?] for help
Output #0, yuv4mpegpipe, to 'pipe:':
Metadata:
encoder : Lavf58.45.100
Stream #0:0(eng): Video: wrapped_avframe, yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 23.98 fps, 23.98 tbn, 23.98 tbc (default)
Metadata:
DURATION : 00:00:06.215000000
encoder : Lavc58.91.100 wrapped_avframe
y4m [info]: 1920x1080 fps 13978/583 i420p8 sar 1:1 unknown frame count
raw [info]: output file: z:\test.mkv
x265 [info]: HEVC encoder version 3.5+10-82786fccc
x265 [info]: build info [Windows][MSVC 1929][64 bit] 8bit+10bit+12bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.1 Cache64
x265 [info]: Main profile, Level-4 (Main tier)
x265 [info]: Thread pool created using 4 threads
x265 [info]: Slices : 1
x265 [info]: frame threads / pool features : 2 / wpp(17 rows)
x265 [info]: Coding QT: max CU size, min CU size : 64 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 3 inter / 3 intra
x265 [info]: HME L0,1,2 / range / subpel / merge : star, sea, full / 48 / 4 / 5
x265 [info]: Keyframe min / max / scenecut / bias : 23 / 250 / 40 / 5.00
x265 [info]: Lookahead / bframes / badapt : 40 / 8 / 2
x265 [info]: b-pyramid / weightp / weightb : 1 / 1 / 1
x265 [info]: References / ref-limit cu / depth : 5 / off / off
x265 [info]: AQ: mode / str / qg-size / cu-tree : 2 / 1.0 / 32 / 1
x265 [info]: Rate Control / qCompress : ABR-1000 kbps / 0.60
x265 [info]: tools: rect amp rd=6 psy-rd=2.00 rdoq=2 psy-rdoq=1.00 rskip mode=1
x265 [info]: tools: signhide tmvp b-intra strong-intra-smoothing deblock sao
av_interleaved_write_frame(): Broken pipeB time=00:00:01.41 bitrate=596600.6kbits/s speed=2.81x
Error writing trailer of pipe:: Broken pipe
frame= 56 fps= 35 q=-0.0 Lsize= 167063kB time=00:00:02.33 bitrate=585947.1kbits/s speed=1.45x
video:29kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 569837.375000%
Conversion failed!

benwaggoner

14th June 2021, 02:51

v3.5+10 - I get a refusal of encoding.

ffmpeg -i z:\space_6s.mkv -f yuv4mpegpipe - | x265 --y4m - --preset veryslow --hme --hme-search star,sea,full --bitrate 1000 z:\test.mkv
Have you tried using just ffplay with the source to confirm it decodes without any errors?

Also, can you use names instead of numbers for --hme-search? I would do it as "--hme-search 2,3,4"

FranceBB

14th June 2021, 17:16

My new baby arrived today.
I look forward to take it for a spin with x265 (*_*)

https://i.imgur.com/CGORtyy.png

Intel Xeon Gold, 56c/112th with AVX-512 and this time is in a server room with freezing cold temperatures, so I dare AVX512 to make it hot! xD

_kermit

14th June 2021, 17:28

Pretty much wrong thread but

av_interleaved_write_frame(): Broken pipe
Error writing trailer of pipe:: Broken pipe

most likely means that the program behind ffmpeg, in your case x265, stopped accepting input because it closed its standard input. So probably x265 exited early or crashed.

so, when x265 crashed, it's the right thread after all? :)

I'll check a newer version then.

Gravitator

14th June 2021, 17:31

He doesn't like high resolution :confused:

ffmpeg -i z:\Park.mkv -f yuv4mpegpipe - | x265 --y4m - --preset veryslow --hme --hme-search star,sea,full --bitrate 1000 z:\test.mkv
ffmpeg version 2021-06-13-git-3ce272a9da-full_build-www.gyan.dev Copyright (c) 2000-2021 the FFmpeg developers
built with gcc 10.3.0 (Rev2, Built by MSYS2 project)
Input #0, matroska,webm, from 'z:\Park.mkv':
Metadata:
creation_time : 2020-07-09T11:13:13.000000Z
ENCODER : Lavf58.48.100
Duration: 00:00:01.97, start: 0.000000, bitrate: 64630 kb/s
Stream #0:0: Video: h264 (Main), yuv420p(tv, bt470bg/unknown/unknown, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 30 fps, 30 tbr, 1k tbn (default)
Metadata:
ENCODER : Lavc58.94.100 nvenc_h264
DURATION : 00:00:01.966000000
Stream mapping:
Stream #0:0 -> #0:0 (h264 (native) -> wrapped_avframe (native))
Press [q] to stop, [?] for help
Output #0, yuv4mpegpipe, to 'pipe:':
y4m [info]: 1920x1080 fps 30/1 i420p8 sar 1:1 unknown frame count
Metadata:
encoder : raw [info]: output file: z:\test.mkv
Lavf59.3.100x265 [info]: HEVC encoder version 3.5+10-82786fccc
x265 [info]: build info [Windows][MSVC 1929][64 bit] 8bit+10bit+12bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.1 Cache64

x265 [info]: Main profile, Level-4 (Main tier)
Stream #0:0x265 [info]: Thread pool created using 4 threads
: Video: wrapped_avframe, yuv420p(tv, bt470bg/unknown/unknown, progressive), 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/sx265 [info]: Slices : 1
, x265 [info]: frame threads / pool features : 2 / wpp(17 rows)
30 fps, 30 tbn (default)
Metadata:
DURATION : 00:00:01.966000000
encoder : Lavc59.1.101 wrapped_avframe
x265 [info]: Coding QT: max CU size, min CU size : 64 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 3 inter / 3 intra
x265 [info]: HME L0,1,2 / range / subpel / merge : star, sea, full / 48 / 4 / 5ts/s speed=0.522x
x265 [info]: Keyframe min / max / scenecut / bias : 25 / 250 / 40 / 5.00
x265 [info]: Lookahead / bframes / badapt : 40 / 8 / 2
x265 [info]: b-pyramid / weightp / weightb : 1 / 1 / 1
x265 [info]: References / ref-limit cu / depth : 5 / off / off
x265 [info]: AQ: mode / str / qg-size / cu-tree : 2 / 1.0 / 32 / 1
x265 [info]: Rate Control / qCompress : ABR-1000 kbps / 0.60
x265 [info]: tools: rect amp rd=6 psy-rd=2.00 rdoq=2 psy-rdoq=1.00 rskip mode=1
x265 [info]: tools: signhide tmvp b-intra strong-intra-smoothing deblock sao
av_interleaved_write_frame(): Broken pipeB time=00:00:01.16 bitrate=746497.8kbits/s speed=1.96x
Error writing trailer of pipe:: Broken pipe
frame= 56 fps= 31 q=-0.0 Lsize= 167063kB time=00:00:01.86 bitrate=733167.3kbits/s speed=1.02x
video:24kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 694187.375000%
Conversion failed!

ffmpeg -i z:\Park.mkv -vf scale="720:480" -f yuv4mpegpipe - | x265 --y4m - --preset veryslow --hme --hme-search star,sea,full --bitrate 1000 z:\test.mkv

ffmpeg version 2021-06-13-git-3ce272a9da-full_build-www.gyan.dev Copyright (c) 2000-2021 the FFmpeg developers
built with gcc 10.3.0 (Rev2, Built by MSYS2 project)
Input #0, matroska,webm, from 'z:\Park.mkv':
Metadata:
creation_time : 2020-07-09T11:13:13.000000Z
ENCODER : Lavf58.48.100
Duration: 00:00:01.97, start: 0.000000, bitrate: 64630 kb/s
Stream #0:0: Video: h264 (Main), yuv420p(tv, bt470bg/unknown/unknown, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 30 fps, 30 tbr, 1k tbn (default)
Metadata:
ENCODER : Lavc58.94.100 nvenc_h264
DURATION : 00:00:01.966000000
Stream mapping:
Stream #0:0 -> #0:0 (h264 (native) -> wrapped_avframe (native))
Press [q] to stop, [?] for help
Output #0, yuv4mpegpipe, to 'pipe:':
y4m [info]: 720x480 fps 30/1 i420p8 sar 32:27 unknown frame count
Metadata:
encoder : raw [info]: output file: z:\test.mkv
Lavf59.3.100x265 [info]: HEVC encoder version 3.5+10-82786fccc

x265 [info]: build info [Windows][MSVC 1929][64 bit] 8bit+10bit+12bit
Stream #0:0x265 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.1 Cache64
: Video: wrapped_avframe, yuv420p(tv, bt470bg/unknown/unknown, progressive), 720x480 [SAR 32:27 DAR 16:9], q=2-31, 200 kb/sx265 [warning]: Source height < 540p is too low for HME. Disabling HME.
x265 [info]: Main profile, Level-3 (Main tier)
x265 [info]: Thread pool created using 4 threads
, x265 [info]: Slices : 1
30 fps, x265 [info]: frame threads / pool features : 2 / wpp(8 rows)
30 tbn (default)
Metadata:
DURATION : 00:00:01.966000000
encoder : Lavc59.1.101 wrapped_avframe
x265 [info]: Coding QT: max CU size, min CU size : 64 / 803 bitrate=124438.4kbits/s speed=0.473x
x265 [info]: Residual QT: max TU size, max depth : 32 / 3 inter / 3 intra
x265 [info]: ME / range / subpel / merge : star / 57 / 4 / 5
x265 [info]: Keyframe min / max / scenecut / bias : 25 / 250 / 40 / 5.00
x265 [info]: Lookahead / bframes / badapt : 40 / 8 / 2
x265 [info]: b-pyramid / weightp / weightb : 1 / 1 / 1
x265 [info]: References / ref-limit cu / depth : 5 / off / off
x265 [info]: AQ: mode / str / qg-size / cu-tree : 2 / 1.0 / 32 / 1
x265 [info]: Rate Control / qCompress : ABR-1000 kbps / 0.60
x265 [info]: tools: rect amp rd=6 psy-rd=2.00 rdoq=2 psy-rdoq=1.00 rskip mode=1
x265 [info]: tools: signhide tmvp b-intra strong-intra-smoothing deblock sao
6frame= 59 fps= 19 q=-0.0 Lsize= 29869kB time=00:00:01.96 bitrate=124417.8kbits/s speed=0.63x
video:25kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 117719.859375%
x265 [info]: frame I: 1, Avg QP:27.58 kb/s: 8555.76
x265 [info]: frame P: 13, Avg QP:34.91 kb/s: 3119.93
x265 [info]: frame B: 45, Avg QP:39.58 kb/s: 314.22
x265 [info]: Weighted P-Frames: Y:0.0% UV:0.0%
x265 [info]: Weighted B-Frames: Y:4.4% UV:2.2%
x265 [info]: consecutive B-frames: 7.1% 0.0% 0.0% 64.3% 14.3% 14.3% 0.0% 0.0% 0.0%

encoded 59 frames in 50.33s (1.17 fps), 1072.11 kb/s, Avg QP:38.35

Gravitator

14th June 2021, 18:16

Have you tried using just ffplay with the source to confirm it decodes without any errors?
No.

Also, can you use names instead of numbers for --hme-search? I would do it as "--hme-search 2,3,4"
The problem is in the configuration --hme-search star,sea,full

ffmpeg -i z:\Park.mkv -f yuv4mpegpipe - | x265 --y4m - --preset veryslow --hme --hme-search umh,star,sea --bitrate 1000 z:\test.mkv
x265 [info]: Coding QT: max CU size, min CU size : 64 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 3 inter / 3 intra4.6kbits/s speed=0.47x
x265 [info]: HME L0,1,2 / range / subpel / merge : umh, star, sea / 48 / 4 / 5
x265 [info]: Keyframe min / max / scenecut / bias : 25 / 250 / 40 / 5.00
x265 [info]: Lookahead / bframes / badapt : 40 / 8 / 2
x265 [info]: b-pyramid / weightp / weightb : 1 / 1 / 1
x265 [info]: References / ref-limit cu / depth : 5 / off / off
x265 [info]: AQ: mode / str / qg-size / cu-tree : 2 / 1.0 / 32 / 1
x265 [info]: Rate Control / qCompress : ABR-1000 kbps / 0.60
x265 [info]: tools: rect amp rd=6 psy-rd=2.00 rdoq=2 psy-rdoq=1.00 rskip mode=1
x265 [info]: tools: signhide tmvp b-intra strong-intra-smoothing deblock sao
frame= 59 fps=1.0 q=-0.0 Lsize= 179213kB time=00:00:01.96 bitrate=746497.6kbits/s speed=0.0327x
video:25kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 706810.750000%
x265 [info]: frame I: 1, Avg QP:33.19 kb/s: 18844.56
x265 [info]: frame P: 12, Avg QP:44.86 kb/s: 2957.10
x265 [info]: frame B: 46, Avg QP:49.15 kb/s: 288.71
x265 [info]: Weighted P-Frames: Y:0.0% UV:0.0%
x265 [info]: Weighted B-Frames: Y:2.2% UV:2.2%
x265 [info]: consecutive B-frames: 7.7% 0.0% 0.0% 53.8% 0.0% 38.5% 0.0% 0.0% 0.0%

encoded 59 frames in 634.11s (0.09 fps), 1145.94 kb/s, Avg QP:48.00

Gravitator

14th June 2021, 18:45

My new baby arrived today.
I look forward to take it for a spin with x265 (*_*)

VVC :cool:

rwill

14th June 2021, 19:15

so, when x265 crashed, it's the right thread after all? :)

I'll check a newer version then.

What you can try is to pipe ffmpeg output into a file like so:

ffmpeg.exe -i %1 -pix_fmt yuv420p -f yuv4mpegpipe - > test.y4m

and then use test.y4m as an input file to a standalone x265.exe call to see what x265 does and debug that.

benwaggoner

14th June 2021, 20:25

What you can try is to pipe ffmpeg output into a file like so:

ffmpeg.exe -i %1 -pix_fmt yuv420p -f yuv4mpegpipe - > test.y4m

and then use test.y4m as an input file to a standalone x265.exe call to see what x265 does and debug that.
Yeah, I either output to .y4m or pipe into x265. It really simplifies debugging, and allows for specific x265 builds to be used.

Note that if you are doing 10-bit, you need to use:

ffmpeg.exe -i %1 -pix_fmt yuv420p10le -strict -1 -f yuv4mpegpipe - > test.y4m

And for piping:

ffmpeg.exe -i foo.mov -pix_fmt yuv420p10le -strict -1 -f yuv4mpegpipe - | x265.exe - --y4m

asarian

15th June 2021, 06:41

After painstakingly long denoising, I am finally ready to run the final x265-only process on The Fith Element (combined from 4x seperately denoised parts). I used the following command line:

VSPipe --y4m "f:\jobs\fifthfin.vpy" - | x265 --y4m --input - --preset medium --input-depth 10 --output-depth 10 --crf 10 --colorprim 9 --transfer 16 --colormatrix 9 --master-display "G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(40000000,50)" --max-cll "10000,724" --frames 181104 --chromaloc 2 --output "g:\video\fifthfin.hevc"

x265 outputs:

[0.8%] 1483/181104 frames, 4.180 fps, 18984 kb/s, 145.92 MB, eta 11:56:15, est.size 17819 MB

Says around 19 Mb/s, and estimated file-size of ca. 18 G. Those values seem extremely low, especially considering the source was 45 G. I even set CRF to 10. But it still seems too low.

Am I doing anything wrong?

Boulder

15th June 2021, 07:16

asarian

15th June 2021, 07:30

Looks ok to me. The CRF value's behaviour in HDR and SDR differs quite a lot as the HDR input is very flat and the encoder does not do any internal grading when analysing the video. I usually lower the CRF value by 4 with HDR encodes to get to the similar quality level.

Thx. So, should I even go to CRF = 6? Or should 10 suffice? Overall, I'm very impressed with the compression ratio of x265. :)

rwill

15th June 2021, 07:36

My new baby arrived today.
I look forward to take it for a spin with x265 (*_*)

Intel Xeon Gold, 56c/112th with AVX-512 and this time is in a server room with freezing cold temperatures, so I dare AVX512 to make it hot! xD

Too low base frequency, too many cores.

_kermit

15th June 2021, 08:07

Yeah, I either output to .y4m or pipe into x265. It really simplifies debugging, and allows for specific x265 builds to be used.

Note that if you are doing 10-bit, you need to use:

ffmpeg.exe -i %1 -pix_fmt yuv420p10le -strict -1 -f yuv4mpegpipe - > test.y4m

And for piping:

ffmpeg.exe -i foo.mov -pix_fmt yuv420p10le -strict -1 -f yuv4mpegpipe - | x265.exe - --y4m

thanks to both.

so, this would have been wrong?

ffmpeg.exe -i %1 -pix_fmt yuv420p -f yuv4mpegpipe -| x265.exe --input - --y4m ....

so I tried this:

ffmpeg.exe -i %1 -pix_fmt yuv420p10le -strict -1 -f yuv4mpegpipe - > %1.y4m
x265.exe --input %1.y4m --y4m ....

what I noticed:
- ffmpeg runs async in the background, continuously creating the file while x265 already starts processing.
- The file is huge, already 90GB out of a 20GB mkv

Is both expected and ok?

Boulder

15th June 2021, 08:43

Thx. So, should I even go to CRF = 6? Or should 10 suffice? Overall, I'm very impressed with the compression ratio of x265. :)

I kind of would expect CRF 10 to look very good. Of course, it depends on your requirements and viewing distance so testing the results with different CRF in your real life viewing conditions is always recommended.

In addition to your settings, I'd set --no-sao to disable the infamous Smooth All Objects functionality. I'd probably also use --deblock -1:-1.

FranceBB

15th June 2021, 08:51

Too low base frequency, too many cores.

Not really, I'm not an home user, I'm gonna be encoding professional UHD 12bit masterfiles coming from IMF Packages with MJPEG2000 RGB 12bit sources and I expect x265 to scale quite efficiently on all the cores. We'll see.

rwill

15th June 2021, 12:41

what I noticed:
- ffmpeg runs async in the background, continuously creating the file while x265 already starts processing.
- The file is huge, already 90GB out of a 20GB mkv

Is both expected and ok?

My intention was to check whether x265 would crash on the .y4m or complete. Because the input is the same as piped but pipe failed.

Regarding the size, you get ~6Mb per HD frame or ~24Mb per UHD frame. So at 24fps thats ~150 or ~600 Megabytes per second of video. So its not practical.

Gravitator

15th June 2021, 12:42

--ssim-rd - results in terribly poor image quality at low bitrate.
ffmpeg -i z:\space_6s.mkv -f yuv4mpegpipe - | x265 --y4m --input - --output-depth 10 --profile main10 --max-merge 1 --no-early-skip --rskip 0 --bframes 1 --ref 1 --bitrate 1000 --tune ssim --ssim-rd z:\ssim-rd.mkv
ffmpeg -i z:\space_6s.mkv -f yuv4mpegpipe - | x265 --y4m --input - --output-depth 10 --profile main10 --max-merge 1 --no-early-skip --rskip 0 --bframes 1 --ref 1 --bitrate 1000 --tune ssim z:\ssim-rd-off.mkv
Download an example > ssim-rd vs ssim-rd-off (https://files.videohelp.com/u/227452/ssim-rd%20vs%20ssim-rd-off.rar)

Boulder

15th June 2021, 13:34

Gravitator

15th June 2021, 13:44

Which are not a measure for visual quality I'm afraid.
The problem is in the combination of settings.

excellentswordfight

15th June 2021, 14:35

Not really, I'm not an home user, I'm gonna be encoding professional UHD 12bit masterfiles coming from IMF Packages with MJPEG2000 RGB 12bit sources and I expect x265 to scale quite efficiently on all the cores. We'll see.
I get decent utilization up to about 24C/48T for UHD, we ofc have servers/clusters that exceeds that as we can push multiple encodes through them at the same time.

Could you please report back what frequency you are getting under high utilization with and without AVX512? We are moving to Epyc on all our encoding hardware, the frequency under load are just much higher compared to Intel in simulair price bracket were 32/64 CPU:s can push over 3Ghz under load.

Boulder

15th June 2021, 15:16

The problem is in the combination of settings.

Have you found out which particular one triggers it?

Not that I expect them to fix it even if you reported the issue at the bug tracker :p

_kermit

15th June 2021, 16:48

My intention was to check whether x265 would crash on the .y4m or complete. Because the input is the same as piped but pipe failed.

Regarding the size, you get ~6Mb per HD frame or ~24Mb per UHD frame. So at 24fps thats ~150 or ~600 Megabytes per second of video. So its not practical.

that's fine, I use pipeline again and with that command line the error is gone.

thanks!

benwaggoner

15th June 2021, 17:46

I kind of would expect CRF 10 to look very good. Of course, it depends on your requirements and viewing distance so testing the results with different CRF in your real life viewing conditions is always recommended.

In addition to your settings, I'd set --no-sao to disable the infamous Smooth All Objects functionality. I'd probably also use --deblock -1:-1.
I've seen quality improvements with some unusual sources down to CRF 12, certainly. But never anything below 10.

benwaggoner

15th June 2021, 18:02

Thank you, I've lowered it to 16 and will report back when its finished.

I can live with 'slight' bitrate increases (which is perfectly understandable).
Also, are you looking at still frames of moving video? x265 and other encoders are tuned to optimize quality in-motion. Lots of things that look not-great when paused will go unnoticed when playing at full speed.

Reducing --ipratio and --pbratio can help with this as well, also increasing bitrate. There's also the experimental --aq-motion which might be worth poking out if you have idle encoding time.

I'm not sure why reducing --qg-size would help. That specifies the block size in which QP be can be varied, and is more useful for improving quality of content of highly variable detail.

asarian

15th June 2021, 18:06

I kind of would expect CRF 10 to look very good. Of course, it depends on your requirements and viewing distance so testing the results with different CRF in your real life viewing conditions is always recommended.

In addition to your settings, I'd set --no-sao to disable the infamous Smooth All Objects functionality. I'd probably also use --deblock -1:-1.

Thx, I will add your parameters for the next encoding. :)

asarian

15th June 2021, 18:10

I've seen quality improvements with some unusual sources down to CRF 12, certainly. But never anything below 10.

I never went below 14 with x264 (and even then friends declared me insane). Since I read the quality of x265 can often be lower, esecially in dark spots, I went as low as 10. With 10 more hours to gp on the final run, file size is estimated at 44G now, which I still deem acceptable. Shouldn't become much larger, though.

StormMeows

23rd June 2021, 21:54

Hey guys new here and really looking for some solid advice. I would greatly appreciate it.

I am in the process of backing up my Blu-Ray Collection that has a ton of movies in it. My CPU is pretty good, but for x265 especially, one movie would take a crazy amount of time. I have a lot of space on my NAS but not enough for full REMUXES of every movie so I want to encode the video ONLY. I want to passthrough the original Blu-Ray and 4K UHD Audio (DTS-MA/Dolby Atmos/True HD, etc). Can you guys please offer advice on the best program and settings to use? I know I want to use a really low CRF, and will keep the settings constant for every movie. In the event that the file ends up being just as big or bigger than the original, I can keep the original file instead of the encoded one. I prefer to keep the video as untouched as possible so no cropping or killing the grain, etc. I will watch the movies on something like Plex or KODI (direct play).

*Also, I should mentioned I tried Handbrake, but it seems limited on settings. For instance, you can only choose up to "Slow" as the slowest setting (no slower or placebo option). I would like a program easy to use that I can just run and not have to worry about checking screenshots for each movie, especially since I have so many movies.

Thank you guys so much!

RanmaCanada

24th June 2021, 07:17

benwaggoner

24th June 2021, 17:27

I never went below 14 with x264 (and even then friends declared me insane). Since I read the quality of x265 can often be lower, esecially in dark spots, I went as low as 10. With 10 more hours to gp on the final run, file size is estimated at 44G now, which I still deem acceptable. Shouldn't become much larger, though.
If you're having trouble with dark areas, --aq-mode 3 can also be worth trying. It lowers the QP in those dark areas, and thus allows --crf to be higher without getting blocking and banding in black. --aq-mode 3 does increase bitrate by itself, but using it with a higher --crf that suits the non-dark parts of the image can be smaller on net than having to use a ---crf low enough to make everything look good.

LeXXuz

25th June 2021, 08:50

If you're having trouble with dark areas, --aq-mode 3 can also be worth trying. It lowers the QP in those dark areas, and thus allows --crf to be higher without getting blocking and banding in black. --aq-mode 3 does increase bitrate by itself, but using it with a higher --crf that suits the non-dark parts of the image can be smaller on net than having to use a ---crf low enough to make everything look good.

Dark areas are always troubling. Aq3 never really convinced me, as it raises the overall bitrate of crf-encodes quite a lot - not only on dark scenes.

I never go below crf16. To tackle dark scenes I prefer to play with --psy-rd and --psy-rdoq and use 10-bit encodes, as all my playback devices can handle Main 10 profiles for HEVC.

And I think it's always good to have a little bit of noise left in the video as it helps many TVs/monitors to display solid backgrounds and dark areas much better. Backgrounds of totally 'clean' videos can look quite awful on many displays.

microchip8

25th June 2021, 09:01

If you're having trouble with dark areas, --aq-mode 3 can also be worth trying. It lowers the QP in those dark areas, and thus allows --crf to be higher without getting blocking and banding in black. --aq-mode 3 does increase bitrate by itself, but using it with a higher --crf that suits the non-dark parts of the image can be smaller on net than having to use a ---crf low enough to make everything look good.

This is bullshit. No matter how many times I tried aq-mode 3, it never got rid of banding in dark areas. In theory it may work but in practive it does not. psy-rd and psy-rdoq are FAR more effective in getting rid of banding and halo effects in darker scenes. Try it yourself if you don't believe it (I use psy-rd 4 and psy-rdoq 15)

Boulder

25th June 2021, 09:10

Boulder

25th June 2021, 09:12

And I think it's always good to have a little bit of noise left in the video as it helps many TVs/monitors to display solid backgrounds and dark areas much better. Backgrounds of a totally 'clean' videos can look quite awful on many displays.

This is true. Almost or completely flat areas often have issues with the default settings since the encoder doesn't spend enough bits on them.

asarian

25th June 2021, 10:42

Dark areas are always troubling. Aq3 never really convinced me, as it raises the overall bitrate of crf-encodes quite a lot - not only on dark scenes.

I never go below crf16. To tackle dark scenes I prefer to play with --psy-rd and --psy-rdoq and use 10-bit encodes, as all my playback devices can handle Main 10 profiles for HEVC.

And I think it's always good to have a little bit of noise left in the video as it helps many TVs/monitors to display solid backgrounds and dark areas much better. Backgrounds of totally 'clean' videos can look quite awful on many displays.

Recently re-encoded (denoised) Alien UHD blu-ray. With a CRF of 10, I did not get any noticable banding in dark scenes, but overall the movie appeared too dark for my taste. Way too dark. I read about aq-mode 3, of course, but what does 'with bias to[wards] dark scenes' mean exactly? Does it mean dark scenes get better bitrate, at the expense of other scenes? Or just extra bitrate? And will those scenes get brighter? I have a Q90T Samsung TV, capable of outputting 2,000 nits, so it ain't my TV.

Boulder

25th June 2021, 10:55

Is your TV calibrated? I think Alien is quite dark on purpose, it's more or less for the suspense and stuff. Aq-mode won't affect the brightness at all, mode 3 just does less quantization in dark areas of a single frame. If you use CRF and VBV is not activated or does not restrict the bitrate, it means that more bits will be spent on average.

Boulder

25th June 2021, 10:55

psy-rd and psy-rdoq are FAR more effective in getting rid of banding and halo effects in darker scenes. Try it yourself if you don't believe it (I use psy-rd 4 and psy-rdoq 15)

Out of interest, do you use the same CRF level compared to stock settings or did you raise CRF to compensate?

microchip8

25th June 2021, 10:59

Out of interest, do you use the same CRF level compared to stock settings or did you raise CRF to compensate?

I've standardized on CRF 21. Cannot tell the difference with psy-rd/psy-rdoq between it and lower ones.

LeXXuz

25th June 2021, 11:01

Recently re-encoded (denoised) Alien UHD blu-ray. With a CRF of 10, I did not get any noticable banding in dark scenes, but overall the movie appeared too dark for my taste. Way too dark. I read about aq-mode 3, of course, but what does 'with bias to[wards] dark scenes' mean exactly? Does it mean dark scenes get better bitrate, at the expense of other scenes? Or just extra bitrate? And will those scenes get brighter? I have a Q90T Samsung TV, capable of outputting 2,000 nits, so it ain't my TV.

aq3 is supposed to add more bits to darker areas to prevent banding/blocking. Overall filesize increases, so I don't think these bits are spend at the expense of other scenes.

So much for the theory. In daily use I never experienced much improvement in troubling dark areas by aq3.

If your encode is noticeable darker than the original, it sounds more like a colourspace or HDR metadata problem to me.

asarian

25th June 2021, 16:05

aq3 is supposed to add more bits to darker areas to prevent banding/blocking. Overall filesize increases, so I don't think these bits are spend at the expense of other scenes.

So much for the theory. In daily use I never experienced much improvement in troubling dark areas by aq3.

If your encode is noticeable darker than the original, it sounds more like a colourspace or HDR metadata problem to me.

Thanks for the explanation on aq-mode 3, guys.

I had taken the base HDR command line options directly from what DGindexNV appended to the .dgi file. That has worked 100% well for other HDR material (The Fifth Element, Total Recall, etc), so I assume those parameters are correct.

Could simply be the (retail) HDR transfer of Alien is darker than the original HD version (made more for highlighted light for the HDR version?).

EDIT: Range is set to Limited. Felt counterintuitive, but the original is set to Limited too.

Boulder

25th June 2021, 16:16

Could simply be the (retail) HDR transfer of Alien is darker than the original HD version (made more for highlighted light for the HDR version?).

Definitely. Reading the blu-ray.com review of the UHD version tells me that because of the very dark surroundings, many enhancements are in the details in the shadows, which probably were mostly flat in the SDR release. It must be remembered that a dim image is quite tough on the viewing conditions. Your room should be very dark or many of the finer details will easily be lost. But then again, a bright part of the image can make your eyes water at the same time :eek:

benwaggoner

25th June 2021, 18:45

I think the problem with --aq-mode 3 is that it is based on --aq-mode 2 which itself is not optimal at all with higher bitrates. --aq-mode 1 + bias for dark areas would be a much better combination. Once again, the adaptive quantization stuff is mostly just ported from x264 and the x265 devs have really not done anything special about them.
For sure! This stuff needs to be refactored

hevc-aq should be aq-mode 5, not a whole different parameter that overrides the existing one
Selecting the aq algorithm and luma bias should be different parameters, with --aq-mode 3 just aliasing to a default combination of the two.
Luma bias itself should have parameters to better tune it. It's hard to see how a single default value will work well across content and scenarios.

benwaggoner

25th June 2021, 18:49

This is bullshit. No matter how many times I tried aq-mode 3, it never got rid of banding in dark areas. In theory it may work but in practive it does not. psy-rd and psy-rdoq are FAR more effective in getting rid of banding and halo effects in darker scenes. Try it yourself if you don't believe it (I use psy-rd 4 and psy-rdoq 15)
I've certainly seen it be helpful with some 8-bit SDR content, although a hybrid approach also using other parameters is superior. Banding issues aren't always caused by a somewhat too high QP, and aq-mode 3 won't help the kinds of banding that occur at other luma levels. Reducing low luma basis pattern blocking is where it helps the most, and only to the degree that having lowered the overall CRF a few steps would have helped.

rwill

25th June 2021, 19:11

For sure! This stuff needs to be refactored

hevc-aq should be aq-mode 5, not a whole different parameter that overrides the existing one
Selecting the aq algorithm and luma bias should be different parameters, with --aq-mode 3 just aliasing to a default combination of the two.
Luma bias itself should have parameters to better tune it. It's hard to see how a single default value will work well across content and scenarios.

Different Modes and parameters to tune them are a bad idea. Unless you want to found an Encode Manufactory with hand crafted streams and that special "magic" the goal for an encoder should be to have as few parameters and settings as possible and it should Just Work for everything.

benwaggoner

26th June 2021, 00:07

Different Modes and parameters to tune them are a bad idea. Unless you want to found an Encode Manufactory with hand crafted streams and that special "magic" the goal for an encoder should be to have as few parameters and settings as possible and it should Just Work for everything.
That is the dream! And there are certainly lots of interesting projects going on using improved metrics and machine learning to better adapt parameters to different kinds of content.

But fundamentally, those are all layers above low-level encoder decisions. The parameters need to exist, at least at the API level, for more advanced tuning to take advantage.

The existence of different EOTFs like 709, PQ, and HLG also change the psychovisual impact and importance of code value ranges. Optimizing for 709 and PQ are quite different in ways that a traditional encoder that just inputs arrays of pixel values is blind to.

ghostshadow

28th June 2021, 11:09

I just built znver3 optimized binary using gcc10.3, you can get it here (https://github.com/DJATOM/x265-aMod/releases/download/3.5%2B20/x265-x64-v3.5+20-aMod-gcc10.3.0-opt-znver3.7z).

Hello DJATOM, I've been using your x265 builds for some time. I recently took a Ryzen 5900x instead of my 3900x. So I used the zen3 version of x265. On a same in code with the 5900x in x265 zen2 and zen3, the zen2 version of x265 is faster. Did you notice that too? Thank you, and bravo for your contributions

LeXXuz

28th June 2021, 13:31

Hello DJATOM, I've been using your x265 builds for some time. I recently took a Ryzen 5900x instead of my 3900x. So I used the zen3 version of x265. On a same in code with the 5900x in x265 zen2 and zen3, the zen2 version of x265 is faster. Did you notice that too? Thank you, and bravo for your contributions

Hm... you're quite right. Thanks for bringing that up.

https://abload.de/img/2021-06-28w0k8l.pnghttps://abload.de/img/2021-06-281pykx7.png

1) x265-x64-v3.5+20-aMod-gcc10.3.0-opt-znver3
2) x265-x64-v3.5+20-aMod-gcc10.2.1-opt-znver2

Both done on Ryzen 5950x. That seems to be more than just an error of measurement. Odd. Problem with gcc10.3 maybe? :confused:

quietvoid

28th June 2021, 14:16

I had done a quick test using GCC 11.1 which brought Zen3 optimizations and it was a little over 1% slower on my 5900X.
Not sure GCC 10.3 has anything for Zen3 actually.

charliebaby

28th June 2021, 17:43

Hm... you're quite right. Thanks for bringing that up.

https://abload.de/img/2021-06-28w0k8l.pnghttps://abload.de/img/2021-06-281pykx7.png

1) x265-x64-v3.5+20-aMod-gcc10.3.0-opt-znver3
2) x265-x64-v3.5+20-aMod-gcc10.2.1-opt-znver2

Both done on Ryzen 5950x. That seems to be more than just an error of measurement. Odd. Problem with gcc10.3 maybe? :confused:

hello you have a ryzen 9 5950x and your Fps and 5 aie it's not normal, I have the same cpu as you but my encoding is between 13-17Fps all the heart active in second pass in placebo

GCC 11.1 http://msystem.waw.pl/x265/x265-3.5+10-82786fc_gcc111-AVX2.7z

LeXXuz

28th June 2021, 19:34

hello you have a ryzen 9 5950x and your Fps and 5 aie it's not normal, I have the same cpu as you but my encoding is between 13-17Fps all the heart active in second pass in placebo

Speed depends on what and how you encode. ;)
Those numbers are correct. I use a very complex filtering script that uses up to 60% of the cpu's workload.

charliebaby

29th June 2021, 06:24

oh it's a shame to use only 60% of the processor :scared: , I put 100% at home

https://nsa40.casimages.com/img/2021/06/29/mini_210629073603709537.png

LigH

29th June 2021, 08:57

I believe LeXXuz does not mean "60% CPU utilization at all (and 40% idle)", but 60% of the 100% is done in AviSynth (and the 40% rest in the encoder).

LeXXuz

29th June 2021, 09:01

I believe LeXXuz does not mean "60% CPU utilization at all (and 40% idle)", but 60% of the 100% is done in AviSynth (and the 40% rest in the encoder).

Exactly. :)

DJATOM

30th June 2021, 23:40

Zen3 support might be not polished enough, I don't have zen3 cpu to benchmark with.
That build provided since It's easy to add new definition into compile script, but if -march=znver2 is faster than znver3, pick whatever is faster for you :)
There's no handcrafted stuff for zen3, so nothing to fix by my side.

LeXXuz

1st July 2021, 11:20

I'll stick with your Zen2 optimization for the time being. Thanks for the info. :)

LeXXuz

4th July 2021, 08:12

I'm thinking about a memory upgrade for my systems.

Does anyone know if bandwidth or latency has an impact on x265 performance, if any at all, on AMD machines?

Khun_Doug

4th July 2021, 17:03

In general, Ryzen seems to do really well with fast memory. On my 5950X system the XMP implementation is broke. By manually adjusting memory speeds, I was able to gain x265 encode performance with the same CPU clock. Best advice on this is to check the motherboard memory list and choose from that list. You may have trouble using the XMP feature. You should consider tools like Cinebench to gauge performance improvements between existing memory and the replacements.

DJATOM

4th July 2021, 17:25

I have 16GBx4 modules (https://www.amazon.com/Ballistix-Single-PC4-24000-288-Pin-Memory/dp/B07MGPGXPS) with overclock to 3533 MHz and timings 16-18-16-36 / 1.395 V. It can work with 3700 MHz on those voltage / timings, but I got bad memory controller with my CPU.

Boulder

5th July 2021, 09:06

I'm thinking about a memory upgrade for my systems.

Does anyone know if bandwidth or latency has an impact on x265 performance, if any at all, on AMD machines?
On x265, the difference is very small if it's even measurable. It's good to just have enough memory, I've seen 4K sources use up to 16GB for Avisynth+ and x265 depending on the script.

With Zen2 at least, you'll want to have the IF and memory speed match. In that sense, a 3600MHz kit should be the best bang for buck.

LeXXuz

5th July 2021, 10:55

Thanks for your input guys. :)

_kermit

6th July 2021, 00:03

benwaggoner

6th July 2021, 19:19

I've encoded a few UHD movies using wrong parameters.
Instead of providing the proper values for mastering display luminance, like:

Mastering display luminance : min: 0.0001 cd/m2, max: 1000 cd/m2
Maximum Content Light Level : 4000 cd/m2
Maximum Frame-Average Light Level : 1391 cd/m2

instead of:
Mastering display luminance : min: 0.0050 cd/m2, max: 4000 cd/m2
Maximum Content Light Level : 4000 cd/m2
Maximum Frame-Average Light Level : 1391 cd/m2

Are those values always used directly in the encoding process, hence I have to redo them or are those properties I could somehow set properly?

Those values are pure metadata, and don't impact how the video is encoded at all. The only impact would be if a display used the mastering_display metadata to change how it tone maps.

AFAIK, no displays use mastering_display anyway, so I doubt there would be any impact from your only somewhat incorrect values. MaxCLL is the only one that gets used much, and even then only on some TVs (LG in particular).

A tool could just "remux" the current encode and replace the metadata SEI messages with the correct ones without actual reencoding. Maybe ffmpeg? But you're safe to just ignore the issue entirely, as you got the important metadata right.

Asmodian

6th July 2021, 19:46

AFAIK, no displays use mastering_display anyway, so I doubt there would be any impact from your only somewhat incorrect values.

And now they will never be able to, since there are too many files in the wild with bad metadata. :scared: ;)

benwaggoner

6th July 2021, 20:09

And now they will never be able to, since there are too many files in the wild with bad metadata. :scared: ;)
Yeah, it was a mistake to make master_display mandatory. MaxCLL and MaxFALL are optional (setting to 0 is undefined). But everything has to have master_display, so if it isn't known, some random example gets pasted in or set to a default.

Of course in those early days of HDR, we didn't really know what metadata we needed or how it could be used. Turns out that the metadata we really wanted was dynamic metadata.

_kermit

7th July 2021, 16:30

Those values are pure metadata, and don't impact how the video is encoded at all. The only impact would be if a display used the mastering_display metadata to change how it tone maps.

AFAIK, no displays use mastering_display anyway, so I doubt there would be any impact from your only somewhat incorrect values. MaxCLL is the only one that gets used much, and even then only on some TVs (LG in particular).

A tool could just "remux" the current encode and replace the metadata SEI messages with the correct ones without actual reencoding. Maybe ffmpeg? But you're safe to just ignore the issue entirely, as you got the important metadata right.

I've confirmed that even my Video Processor only utilizes MaxCLL, so that means I don't have to re-encode or even update some metadata, I guess.
That's great.

thanks!

benwaggoner

7th July 2021, 17:34

I've confirmed that even my Video Processor only utilizes MaxCLL, so that means I don't have to re-encode or even update some metadata, I guess.
That's great.
Yep. AFAIK only MaxCLL is ever used for anything, and only on some devices.

_kermit

10th July 2021, 11:56

Yep. AFAIK only MaxCLL is ever used for anything, and only on some devices.

as it turns out, not that easy.
Quote from the processor vendor:

"I should have noted that the Radiance Pro also uses the Mastering Monitor Max to limit the MaxCLL to no more than the Mastering Monitor Maximum reported in the HDR10 Metadata.

One example of why is Mad Max Fury Road. It was mastered on a 4000 nit monitor, but has a specified MaxCLL around 9900 (if my memory serves on this). On a 4000 nit monitor anything above 4000 nits would be clamped to the maximum the monitor can reproduce. Nothing over 4000 nits would have been seen in Post Production. So there is no reason to set any scene maximum higher than the Mastering Monitor Maximum.

While we have seen a lot of content where MaxCLL is wrong, as I recall, when reported (so not 0 which would be ignored by the Pro), the Max Mon Metadata has been correct."

Hence this:
Mastering display luminance : min: 0.0050 cd/m2, max: 4000 cd/m2
Maximum Content Light Level : 1655 cd/m2
Maximum Frame-Average Light Level : 117 cd/m2

which I've encoded using 0.0001 and 1000 instead

would also result in 1000 max for CLL :(

This might be the solution though:
https://github.com/YouTubeHDR/hdr_metadata

a custom solution to update that, and not just the header at the beginning, it goes through the whole file, resulting in this, using just some random values.

mkvmerge.exe -o output.mkv --min-luminance 0:0.0001 --max-luminance 0:2000 input.mkv

took a few minutes on an SSD, which resulted in that:

Mastering display luminance: min: 0.0001 cd/m2, max: 2000 cd/m2
MasteringDisplay_Luminance_Original: min: 0.0050 cd/m2, max: 1000 cd/m2
Maximum Content Light Level: 1000 cd/m2
Maximum Frame-Average Light Level: 293 cd/m2

so, it updated "Mastering display luminance" and made a "backup" entry.

I'm testing this on the processor and if it reports the correct values, I think this should be fine.
Still assuming the values don't play a role during encoding, that they are just written into the stream.

EDIT

Is there a database with that HDR-metadata for movies?

----

It's not working.
While I get the proper values, the processor still reports the old ones.

I've tried the other tool which provides even better values (without "original")

Mastering display luminance : min: 0.0050 cd/m2, max: 4000 cd/m2
Maximum Content Light Level : 1000 cd/m2
Maximum Frame-Average Light Level : 162 cd/m2

Still, I get the old ones reported from the video processor

Can the old values still be embedded in the stream? They must be..
Does anyone have ideas?

SEI_rbsp_stop_one_bit : Missing

_kermit

11th July 2021, 15:09

benwaggoner

12th July 2021, 15:25

can x265 be brought to re-encode without actually re-encoding?
Basically, process the file, create a new one, but copy everything?

Meant as a solution for my problem with wrong metadata.
The only thing it needs to do is to not "encode/analyze" but update the metadata while doing so.
x265 vanilla can't do this, but could probably be hacked to. Tools like mp4box or ffmpeg are more commonly used for this kind of muxing-like stuff.

MrVideo

13th July 2021, 06:23

Before I go digging into x265, does it support AVISynth (AVISynth+) script file input like x264? I'm set up on my Win7 box to use AVIsynth, not the "+" version.

Also is there a good "how to" out there? I have the readdocs PDF, but that is a command reference, not a nice training tool.

LigH

13th July 2021, 07:32

Vanilla x265 without modifications does not support AviSynth scripts or encoded media files as inputs, only raw video files (with or without YUV4MPEG header).

There are modified builds (DJATOM or Yuuki-Asuna) which include AviSynth and libav input modules.

I don't know any detailed guide. But the best advice I can give is: Start with presets and tunings only; do not use extreme presets instead of convenient bitrate, there are no miracles.

It is not important at the moment to insist in the very latest builds, as there are little changes for Windows PC users during the last few months. A version around 3.3-3.4 is fine.

filler56789

13th July 2021, 09:17

Before I go digging into x265, does it support AVISynth (AVISynth+) script file input like x264? I'm set up on my Win7 box to use AVIsynth, not the "+" version.

The "normal" builds of x265.exe support Avisynth scripts only indirectly, through command-line programs like avs2yuv and avs2pipe(mod).

MrVideo

13th July 2021, 09:56

Thanks for the info guys.

_kermit

13th July 2021, 14:15

x265 vanilla can't do this, but could probably be hacked to. Tools like mp4box or ffmpeg are more commonly used for this kind of muxing-like stuff.

I've tested with ffmpeg, that seems to do the trick.

tonemapped

21st July 2021, 03:05

The biggest issue with x265 is is that the most capable people left a long time ago and even patch validation has had issues with features getting worse with development.

Is this the general consensus? If so, how incredibly disappointing.

Gravitator

25th July 2021, 21:18

Selur

26th July 2021, 04:57

How to encode to 12 bit?
Using ffmpeg -y -loglevel fatal -noautorotate -nostdin -threads 8 -ignore_editlist true -i "G:\TestClips&Co\files\ProRes\Test Patterns Resolve 4444 12-bit.mov" -map 0:0 -an -sn -vf zscale=rangein=tv:range=tv -pix_fmt yuv420p12le -strict -1 -vsync 0 -f yuv4mpegpipe - | x265 --input - --output-depth 12 --y4m --profile main444-12 --limit-modes --no-early-skip --no-open-gop --opt-ref-list-length-pps --crf 18.00 --opt-qp-pps --cbqpoffs -2 --crqpoffs -2 --limit-refs 0 --ssim-rd --psy-rd 2.50 --rdoq-level 2 --psy-rdoq 10.00 --aq-mode 0 --deblock=-1:-1 --limit-sao --no-repeat-headers --range limited --colormatrix bt709 --output "E:\Temp\2021-07-26@05_29_19_2410_01.265"
seems to feed 12bit to the encoder.
But looking at the output with MediaInfo it reports:
Video
ID : 1
Format : HEVC
Format/Info : High Efficiency Video Coding
Format profile : Main 10@L3@Main
Codec ID : hvc1
Codec ID/Info : High Efficiency Video Coding
Duration : 600 ms
Bit rate : 96.5 kb/s
Width : 720 pixels
Height : 576 pixels
Display aspect ratio : 4:3
Frame rate mode : Constant
Frame rate : 25.000 FPS
Standard : PAL
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 10 bits
Bits/(Pixel*Frame) : 0.009
Stream size : 7.07 KiB (85%)
Title : Core Media Video
Writing library : x265 3.5+10-82786fccc:[Windows][GCC 10.3.0][64 bit] 10bit
Encoding settings : cpuid=1111039 / frame-threads=5 / numa-pools=32 / wpp / no-pmode / no-pme / no-psnr / no-ssim / log-level=2 / input-csp=1 / input-res=720x576 / interlace=0 / total-frames=0 / level-idc=0 / high-tier=1 / uhd-bd=0 / ref=3 / no-allow-non-conformance / no-repeat-headers / annexb / no-aud / no-eob / no-eos / no-hrd / info / hash=0 / no-temporal-layers / no-open-gop / min-keyint=25 / keyint=250 / gop-lookahead=0 / bframes=4 / b-adapt=2 / b-pyramid / bframe-bias=0 / rc-lookahead=20 / lookahead-slices=0 / scenecut=40 / hist-scenecut=0 / radl=0 / no-splice / no-intra-refresh / ctu=64 / min-cu-size=8 / no-rect / no-amp / max-tu-size=32 / tu-inter-depth=1 / tu-intra-depth=1 / limit-tu=0 / rdoq-level=2 / dynamic-rd=0.00 / ssim-rd / signhide / no-tskip / nr-intra=0 / nr-inter=0 / no-constrained-intra / strong-intra-smoothing / max-merge=3 / limit-refs=0 / limit-modes / me=1 / subme=2 / merange=57 / temporal-mvp / no-frame-dup / no-hme / weightp / no-weightb / no-analyze-src-pics / deblock=-1:-1 / sao / no-sao-non-deblock / rd=3 / selective-sao=4 / no-early-skip / rskip / no-fast-intra / no-tskip-fast / no-cu-lossless / b-intra / no-splitrd-skip / rdpenalty=0 / psy-rd=2.50 / psy-rdoq=10.00 / no-rd-refine / no-lossless / cbqpoffs=-2 / crqpoffs=-2 / rc=crf / crf=18.0 / qcomp=0.60 / qpstep=4 / stats-write=0 / stats-read=0 / ipratio=1.40 / pbratio=1.30 / aq-mode=1 / aq-strength=0.00 / cutree / zone-count=0 / no-strict-cbr / qg-size=32 / no-rc-grain / qpmax=69 / qpmin=0 / no-const-vbv / sar=255 / sar-width / : / sar-height=59:54 / overscan=0 / videoformat=5 / range=0 / colorprim=2 / transfer=2 / colormatrix=1 / chromaloc=0 / display-window=0 / cll=0,0 / min-luma=0 / max-luma=1023 / log2-max-poc-lsb=8 / vui-timing-info / vui-hrd-info / slices=1 / opt-qp-pps / opt-ref-list-length-pps / no-multi-pass-opt-rps / scenecut-bias=0.05 / hist-threshold=0.03 / no-opt-cu-delta-qp / no-aq-motion / no-hdr10 / no-hdr10-opt / no-dhdr10-opt / no-idr-recovery-sei / analysis-reuse-level=0 / analysis-save-reuse-level=0 / analysis-load-reuse-level=0 / scale-factor=0 / refine-intra=0 / refine-inter=0 / refine-mv=1 / refine-ctu-distortion=0 / limit-sao / ctu-info=0 / no-lowpass-dct / refine-analysis-type=0 / copy-pic=1 / max-ausize-factor=1.0 / no-dynamic-refine / no-single-sei / no-hevc-aq / no-svt / no-field / qp-adaptation-range=1.00 / scenecut-aware-qp=0conformance-window-offsets / right=0 / bottom=0 / decoder-max-rate=0 / no-vbv-live-multi-pass
Encoded date : UTC 2021-07-26 03:52:24
Tagged date : UTC 2021-07-26 03:52:24
Color range : Limited
Matrix coefficients : BT.709
Codec configuration box : hvcC
Looking a 'x265 --help' it properly shows:
x265 [info]: HEVC encoder version 3.5+10-82786fccc
x265 [info]: build info [Windows][GCC 10.3.0][64 bit] 8bit+10bit+12bit

Is this a bug or am I missing something?

Cu Selur

filler56789

26th July 2021, 07:38

@Selur:

MAYBE you should a try a diferent version of MediaInfo? :confused:
(mine is 19.07)

OR/and......

try an old version of x265? :confused:
(mine is 3.4+7-g38774073d)

Selur

26th July 2021, 13:39

WTF,... I restarted my machine and when I run the same command now the output is reported as:
Format/Info : High Efficiency Video Coding
Format profile : Format Range@L3@Main
with:
Writing library : x265 3.5+10-82786fccc:[Windows][GCC 10.3.0][64 bit] 12bit
I don't get it how this could change,... but okay, since I can't reproduce the issue anymore it seems to have fixed itself. (cosmic rays?)

Thanks anyways.

Cu Selur

benwaggoner

26th July 2021, 19:56

I am looking for x265 maximum setting to get closer to the quality of AOM (above placebo).
Download the encoded sample & orig (size 6.4MB) > AOM (https://files.videohelp.com/u/227452/AOM.rar) (1Mbps/10bit was applied and the maximum preset was simplified by lowering the blocks from 128 to 64 - like in x265 ;)).
They are different codecs, and AV1 has some tools HEVC doesn't. There will be cases where x265 can't match AV1. Beamr's HEVC encoder seems to handle the scenarios where libaom > x265 well.

If you want to through MIPS at the wall, here's some "try everything" ultraplacebo values, to include in a 3-pass encode.

--profile main10 --preset placebo --no-wpp --pmode --subme 7 --bframes 16 --tskip --cu-lossless --rd-refine --multi-pass-opt-analysis --multi-pass-opt-distortion --opt-qp-pps --opt-ref-list-length-pps --multi-pass-opt-rps --opt-cu-delta-qp

It'd be glacially slow and probably <5% better than placebo for typical film/video content. Clean animation, CGI, and motion graphics could benefit a lot more. The less noise, the greater compression potential and the more extreme settings can benefit.

The source seems a good fit for x265 and HEVC in general. Can you share your current command line and output?

tonemapped

26th July 2021, 21:30

Beamr's HEVC encoder seems to handle the scenarios where libaom > x265 well.

Any idea on the cost for a single-user, non-commercial licence? I've looked through the company's website, but I'm guessing it's a case of 'if you have to ask, you can't afford it' 😋

benwaggoner

26th July 2021, 21:48

Any idea on the cost for a single-user, non-commercial licence? I've looked through the company's website, but I'm guessing it's a case of 'if you have to ask, you can't afford it' 😋
I believe they were working on ways to make it available for noncommercial use, but haven't checked in on that recently.

COVID-19's disruption of all the trade shows has left me less on top of the industry than I'm used to.

tonemapped

27th July 2021, 01:12

I believe they were working on ways to make it available for noncommercial use, but haven't checked in on that recently.

COVID-19's disruption of all the trade shows has left me less on top of the industry than I'm used to.

Thank you for replying. It looks like a fascinating plugin

benwaggoner

27th July 2021, 20:19

Say, what are folks' experiences with --lookahead-threads? I don't recall much or any discussion of this feature added back in 2.3

--lookahead-threads <integer>
Use multiple worker threads dedicated to doing only lookahead instead of sharing the worker threads with frame Encoders. A dedicated lookahead threadpool is created with the specified number of worker threads. This can range from 0 upto half the hardware threads available for encoding. Using too many threads for lookahead can starve resources for frame Encoder and can harm performance. Default is 0 - disabled, Lookahead shares worker threads with other FrameEncoders .
Values: 0 - disabled(default). Max - Half of available hardware threads.

For "ultraplacebo" encoding without frame threading it sounds like a potentially useful tool to take advantage of more cores. I've kicked off a test encode, and it does appear to improve --preset placebo speed some.

Anyone tried it or have any thoughts on it?

tonemapped

27th July 2021, 21:36

Say, what are folks' experiences with --lookahead-threads? I don't recall much or any discussion of this feature added back in 2.3

For "ultraplacebo" encoding without frame threading it sounds like a potentially useful tool to take advantage of more cores. I've kicked off a test encode, and it does appear to improve --preset placebo speed some.

Anyone tried it or have any thoughts on it?

I tried it on lower-end hardware (my 4C/4T Xeon) and more reasonable hardware (workstation with 8700K and 9700K). Setting it to 2, it seemed to slow down all encodes. I suppose that's to be expected - what's the C/T count of the CPU you use for testing?

benwaggoner

27th July 2021, 23:58

I tried it on lower-end hardware (my 4C/4T Xeon) and more reasonable hardware (workstation with 8700K and 9700K). Setting it to 2, it seemed to slow down all encodes. I suppose that's to be expected - what's the C/T count of the CPU you use for testing?
2x18/36, with most cores unused for this encode.

benwaggoner

28th July 2021, 01:52

guest

28th July 2021, 11:09

I have been trying / testing different x265 builds to see if the optimized AVX & AVX2 builds make any difference to encoding times.

Please check here, for my basic findings :-

https://forum.doom9.org/showthread.php?p=1948617#post1948617

Any comments would be more than welcome.

Cheers

Barough

31st July 2021, 07:25

x265 v3.5+11-c8905a7450 (https://www.mediafire.com/file/g0cqgbbsg4ywrkb/x265-3.5%252B11-c8905a745_Win_GCC103.7z/file) (32 & 64-bit 8/10/12bit Multilib Windows Binaries) (GCC 10.3.0)

https://bitbucket.org/multicoreware/x265_git/commits/branch/master

charliebaby

31st July 2021, 08:24

x265 v3.5+11-c8905a7450 (https://www.mediafire.com/file/dlimoxr9z9lc6n1/x265-3.5%252B9-bf91444e0_Win_GCC102.7z/file) (32 & 64-bit 8/10/12bit Multilib Windows Binaries) (GCC 10.3.0)

https://bitbucket.org/multicoreware/x265_git/commits/branch/master

this 3.5.9 not 3.5.11

Gravitator

31st July 2021, 10:17

Here's the best I could do throwing a bunch of MIPS at it: https://1drv.ms/v/s!AlvIQZWsyeO-k9scZ-4jLg_MNpsqTA?e=68IRmr
Your quality is terrible!:eek:
Here are my > results (https://files.videohelp.com/u/227452/x265%20%28rskip0_rskip2%29.rar)

Barough

31st July 2021, 11:07

this 3.5.9 not 3.5.11

Fixed now. Thnx for the headsup. :)

charliebaby

31st July 2021, 11:09

Fixed now. Thnx for the headsup. :)

tk you :-)

DJATOM

31st July 2021, 11:29

Your quality is terrible!:eek:
Here are my > results (https://files.videohelp.com/u/227452/x265%20%28rskip0_rskip2%29.rar)

I think yours not ideal too. I'm noticing slight blurring on reflector (or whatever it is), while original sample look transparent to me.

Gravitator

31st July 2021, 11:47

I think yours not ideal too. I'm noticing slight blurring on reflector (or whatever it is), while original sample look transparent to me.
Can you try to improve it? Your processor should quickly iterate through the settings. Slightly falls short of AOM quality. :rolleyes:

DJATOM

31st July 2021, 13:37

https://easyupload.io/djss6b | password: x265
I think going beyond crf 19 is bad and leads to blurred spots on reflector. I tried to optimize quantization by adjusting qp-adaptation-range and qcomp, It doesn't help much. Also I made few experiments with psy-rd, 1.7 looks sane for that sample.

Gravitator

31st July 2021, 14:29

https://easyupload.io/djss6b | password: x265
I think going beyond crf 19 is bad and leads to blurred spots on reflector.
The task was to keep within 1Mbps.

DJATOM

1st August 2021, 15:21

>The task was to keep within 1Mbps.
I somehow missed that. With such low bitrate the quality indeed suffers and washed details are inevitable.

benwaggoner

2nd August 2021, 19:44

Your quality is terrible!:eek:
Here are my > results (https://files.videohelp.com/u/227452/x265%20%28rskip0_rskip2%29.rar)
Yeah, I was experimenting with what "throwing MIPS at it" could do more than any clip-specific settings.

What are the settings you used for the rskip 0/2 clips?

benwaggoner

2nd August 2021, 19:54

this 3.5.9 not 3.5.11
I get
C:\Users\benwagg>C:\Users\benwagg\Desktop\x265\x265.exe --version
x265 [info]: HEVC encoder version 3.5+11-c8905a745
x265 [info]: build info [Windows][GCC 10.3.0][64 bit] 8bit+10bit+12bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast LZCNT SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
Biggest recent change in MASTER appears to be improvements in --hist-scenecut. And --hist-scenecut has been made the default? I'll have to test to confirm.

New text:
+ option:`--hist-scenecut` enables scenecut detection using the histograms.
+ It also uses the intra and inter cost info to arrive at a scenecut decision from the default scenecut method.

+ /* Flag to turn on/off traditional scenecut detection in histogram based scenecut detection.
+ * When false, only spatial properties are used for scenecut detection. Default true */
+ int bEnableTradScdInHscd;

Boulder

2nd August 2021, 20:24

Gravitator

3rd August 2021, 07:40

Yeah, I was experimenting with what "throwing MIPS at it" could do more than any clip-specific settings.

What are the settings you used for the rskip 0/2 clips?
R0
x265 --frame-threads 0 --wpp --no-pmode --no-pme --log-level 2 --input - --output-depth 10 --y4m --profile main10 --high-tier --min-cu-size 8 --ctu 64 --qg-size 64 --opt-cu-delta-qp --tu-intra-depth 4 --tu-inter-depth 4 --limit-tu 0 --max-tu-size 32 --hme --hme-search star,star,star --hme-range 16,32,48 --subme 4 --merange 57 --no-analyze-src-pics --no-limit-modes --rect --amp --max-merge 2 --no-early-skip --rskip 0 --temporal-mvp --rdpenalty 0 --tskip --no-tskip-fast --strong-intra-smoothing --no-constrained-intra --open-gop --gop-lookahead 0 --opt-ref-list-length-pps --keyint 250 --min-keyint 0 --max-ausize-factor 1 --radl 0 --bframes 6 --no-weightb --no-b-intra --bframe-bias 0 --b-adapt 2 --b-pyramid --ref 4 --weightp --rc-lookahead 150 --slices 1 --lookahead-threads 0 --qpstep 4 --qpmin 0 --qpmax 69 --qblur 0.50 --cplxblur 20.0 --bitrate 1100 --crf-min 0.00 --crf-max 0.00 --no-strict-cbr --no-opt-qp-pps --no-rc-grain --cbqpoffs 0 --crqpoffs 0 --ipratio 1.40 --pbratio 1.30 --nr-intra 0 --nr-inter 0 --qpfile GENERATED_QP_FILE --limit-refs 0 --rd 5 --no-fast-intra --no-ssim-rd --no-rd-refine --psy-rd 1.00 --rdoq-level 1 --psy-rdoq 10.00 --signhide --no-splitrd-skip --qcomp 0.60 --qp-adaptation-range 1 --no-aq-motion --aq-mode 2 --aq-strength 2 --cutree --no-cu-lossless --no-const-vbv --vbv-maxrate 0 --vbv-bufsize 0 --vbv-init 0.90 --vbv-end 0.000 --min-vbv-fullness 50.00 --max-vbv-fullness 80.00 --no-hrd --no-aud --info --no-idr-recovery-sei --deblock=0:0 --sao --selective-sao 4 --no-sao-non-deblock --no-limit-sao --repeat-headers --no-temporal-layers --log2-max-poc-lsb 8 --no-fades --psnr --no-ssim --no-interlace --range limited --colormatrix bt709 --no-single-sei --no-hdr10 --output OUTPUTFILE

R2
x265 --frame-threads 0 --wpp --no-pmode --no-pme --log-level 2 --input - --output-depth 10 --y4m --profile main10 --high-tier --min-cu-size 8 --ctu 32 --qg-size 32 --opt-cu-delta-qp --tu-intra-depth 4 --tu-inter-depth 4 --limit-tu 0 --max-tu-size 32 --hme --hme-search hex,umh,star --hme-range 16,32,48 --subme 4 --merange 57 --no-analyze-src-pics --no-limit-modes --rect --no-amp --max-merge 5 --no-early-skip --rskip 2 --rskip-edge-threshold 1 --temporal-mvp --rdpenalty 0 --tskip --no-tskip-fast --strong-intra-smoothing --no-constrained-intra --open-gop --gop-lookahead 0 --opt-ref-list-length-pps --keyint 250 --min-keyint 0 --max-ausize-factor 1 --radl 0 --bframes 6 --no-weightb --no-b-intra --bframe-bias 0 --b-adapt 2 --b-pyramid --ref 4 --weightp --rc-lookahead 150 --slices 1 --lookahead-threads 0 --lookahead-slices 0 --qpstep 4 --qpmin 0 --qpmax 69 --qblur 0.50 --cplxblur 20.0 --bitrate 1100 --crf-min 0.00 --crf-max 0.00 --no-strict-cbr --no-opt-qp-pps --no-rc-grain --cbqpoffs 0 --crqpoffs 0 --ipratio 1.40 --pbratio 1.30 --nr-intra 0 --nr-inter 0 --qpfile GENERATED_QP_FILE --limit-refs 0 --rd 5 --no-fast-intra --no-ssim-rd --no-rd-refine --psy-rd 1.00 --rdoq-level 1 --psy-rdoq 10.00 --signhide --no-splitrd-skip --qcomp 0.60 --qp-adaptation-range 1 --no-aq-motion --aq-mode 2 --aq-strength 2 --cutree --cu-lossless --no-const-vbv --vbv-maxrate 0 --vbv-bufsize 0 --vbv-init 0.90 --vbv-end 0.000 --min-vbv-fullness 50.00 --max-vbv-fullness 80.00 --no-hrd --no-aud --info --no-idr-recovery-sei --deblock=0:0 --sao --selective-sao 4 --no-sao-non-deblock --no-limit-sao --repeat-headers --no-temporal-layers --log2-max-poc-lsb 8 --no-fades --psnr --no-ssim --no-interlace --range limited --colormatrix bt709 --no-single-sei --no-hdr10 --output OUTPUTFILE

benwaggoner

3rd August 2021, 20:53

I hope they didn't screw up and make --hist-scenecut a default option. It's broken - the old method is not perfect and seems it's impossible to tune at all to catch scene changes better, but it's still the better option.
There have been other improvements in --hist-scenecut. It's odd the documentation doesn't specify which is default.

Boulder

4th August 2021, 05:22

There have been other improvements in --hist-scenecut. It's odd the documentation doesn't specify which is default.

Yes, but as mentioned earlier in this thread, the "improvements" broke the feature even more.

Gravitator

4th August 2021, 07:27

Yes, but as mentioned earlier in this thread, the "improvements" broke the feature even more.
It's like effective managers slow down old smartphones for the sake of new ones (stimulating the free market) :o

fadedmaple

8th August 2021, 14:54

Did some tests on ARM, x265 is not optimized for ARM or not enough.
Intel haswell 4 core without HT vs neoverse N1 4 core , compiled with GCC 9.3.0

x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
x265 [info]: using cpu capabilities: MMX2 SSE2Fast LZCNT SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2

x264 [info]: using cpu capabilities: ARMv8 NEON
x265 [info]: using cpu capabilities: NEON

ffmpeg is installed from Ubuntu(20.04) packages

ffmpeg -i input.mp4 -f yuv4mpegpipe - | x264 --demuxer y4m - -o output.264
ffmpeg -i input.mp4 -f yuv4mpegpipe - | x265 --y4m - -o output.265

x264 speed Intel is 1.47X of ARM
x265 speed Intel is 2.82X of ARM

quietvoid

8th August 2021, 14:59

vpupkind

9th August 2021, 22:06

Yes, but as mentioned earlier in this thread, the "improvements" broke the feature even more.

In my understanding, the latest patch makes some of the improvements conditioned on an extra command line option.
This means you can use either "traditional"-only (the original x264 scenecut, which just looked at bit cost), or histogram-only (more-or-less the initial version of it), or a combination of the two (scenecut invoked if histogram-based method is unsure whether something is a cut or not).

benwaggoner

9th August 2021, 22:12

If you're interested in ARM, there is ongoing work for an arm64 port.
You can find all the patches: https://mailman.videolan.org/pipermail/x265-devel/2021-August/013370.html

The full patch is also at the end of that message.

Not sure anyone has compiled them into a build yet..
My fellow Amazonian Sebastian Pop has been doing a whole lot to get ARM performance up! I'll ask him if he has any aggregate numbers for those improvements.

Barough

12th August 2021, 17:12

x265 v3.5+12-0983cffc5 (https://www.mediafire.com/file/8sjhyfwgalofcr5/x265-3.5%252B12-0983cffc5_Win_GCC103.7z/file) (32 & 64-bit 8/10/12bit Multilib Windows Binaries) (GCC 10.3.0)

https://bitbucket.org/multicoreware/x265_git/commits/branch/master

LeXXuz

14th August 2021, 09:41

With all those versions out there and all that talk about things being broken or gotten worse in never versions, I'm really struggling what version I should go for with archival purposes in mind. :rolleyes:

With DJ ATOM's 3.5+20 version (I still don't understand where the +20 comes from anyway) I've noticed very rare and random encoding errors on Ryzen 5950x. Those errors show as a single, (sometimes 2-3 consecutive) frame(s) being almost completely grey.

Worst part is those frames are not really broken and perfectly decodeable which makes them very hard to spot. Usually I let ffmpeg just run through an encoded video stream to spot any decoding errors.
Here I have to watch the entire video closely to check if the encode is flawless or not. Needless to say that's quite a tedious task. :(

DJATOM

14th August 2021, 11:27

That +20 is refers to my patches.

filler56789

14th August 2021, 12:43

With all those versions out there and all that talk about things being broken or gotten worse in never versions, I'm really struggling what version I should go for with archival purposes in mind. :rolleyes:

My suggestion: get version 3.3+10 from http://msystem.waw.pl/x265/ ,

OR perhaps versions 3.3+1 /+4 /+7 /+12 /+24 /+27 from the x265.exe thread at Videohelp:

https://forum.videohelp.com/threads/357754-[HEVC]-x265-EXE-mingw-builds

LeXXuz

15th August 2021, 11:27

That +20 is refers to my patches.

Ah. Thanks for the explanation. :)

My suggestion: get version 3.3+10 from http://msystem.waw.pl/x265/ ,

I'll try my luck with that one. :thanks:

benwaggoner

16th August 2021, 23:50

rwill

17th August 2021, 00:07

With DJ ATOM's 3.5+20 version (I still don't understand where the +20 comes from anyway) I've noticed very rare and random encoding errors on Ryzen 5950x. Those errors show as a single, (sometimes 2-3 consecutive) frame(s) being almost completely grey.

Worst part is those frames are not really broken and perfectly decodeable which makes them very hard to spot. Usually I let ffmpeg just run through an encoded video stream to spot any decoding errors.
Here I have to watch the entire video closely to check if the encode is flawless or not. Needless to say that's quite a tedious task. :(

Have you tried decoding the stream with a software decoder like the one in ffmpeg ( maybe ffplay ) or the reference software decoder ( ldecode, HM ) to check if its not some decoder side problem ?

DJATOM

17th August 2021, 00:41

On my observation rtx 2070's decoder has rare glitches with bframes > 6. Maybe it's also option specific issue...

benwaggoner

17th August 2021, 18:25

On my observation rtx 2070's decoder has rare glitches with bframes > 6. Maybe it's also option specific issue...
I've played a whole lot of --bframes 8 content on a Quadro 6000 (basically a 2080 with extra VRAM) without any issues.

LeXXuz

17th August 2021, 23:32

Is there something specific to your GPU about those errors?

Have you tried decoding the stream with a software decoder like the one in ffmpeg ( maybe ffplay ) or the reference software decoder ( ldecode, HM ) to check if its not some decoder side problem ?

Hm. Fair point. I forgot to check if this may be decoder related. I already deleted those faulty encodes and encoded those files again with a different version. They look okay now.

If I have time I will try to reproduce that error and check if it's decoder related.

rwill

18th August 2021, 09:13

Hm. Fair point. I forgot to check if this may be decoder related. I already deleted those faulty encodes and encoded those files again with a different version. They look okay now.

If I have time I will try to reproduce that error and check if it's decoder related.

It could be a muxer problem too, for example if keyframes are marked as inter and non-keyframes are marked as random access points...

Sometimes I get strange decode errors with VLC too but when I seek to a position prior to the garbage frames and try to reproduce the problems the frames come out ok, sometimes.

LigH

18th August 2021, 09:19

Modern decoders should know to seek to IDR frames only, one would assume...

rwill

18th August 2021, 16:01

Modern decoders should know to seek to IDR frames only, one would assume...

Well decoders can discard bitstream data until they encounter an IDR but this prevents the case where there are no IDR/CRA/BLA pictures and Intra Refresh is done with constrained Intra encoding like for live communication applications.

The player is relying, in most cases, on what is signaled in the container which contains the bitstream and starts feeding from a point which is marked as a keyframe there.

LigH

19th August 2021, 09:21

Thanks for this alternative point of view, @rwill ... video encoding can be more complex than expected when you find a use case where usually optimal techniques don't work perfectly.

charliebaby

25th August 2021, 15:24

x265 3.5.0.13 GCC 11.20

LINK : https://www.mediafire.com/file/bzeagt5sbzghwg6/x265-3.5%252B13-4c6880d_gcc112-AVX2.7z/file

tormento

2nd September 2021, 11:10

x265 3.5.0.13 GCC 11.20
Is AVX2 mandatory? A Sandy Bridge compile would be nice :)

Barough

16th September 2021, 16:58

x265 v3.5+15-4bf31dc15 (https://www.mediafire.com/file/xhsb6tmnvndu14g/x265-3.5+15-4bf31dc15_Win_GCC103.7z/file) (32 & 64-bit 8/10/12bit Multilib Windows Binaries) (GCC 10.3.0)

https://bitbucket.org/multicoreware/x265_git/commits/branch/master

LeXXuz

18th September 2021, 11:09

How much is fixed in 3.5+15 and how much is broken this time? :rolleyes:

DJATOM

18th September 2021, 19:43

Latest commits almost did nothing for x86 builds, so not much differs from 3.5 release...

FranceBB

1st October 2021, 09:03

Hi there,
I always forget who's maintaining those: http://msystem.waw.pl/x265/

but... can I ask for an AVX-512 Windows x64 build?

DJATOM

1st October 2021, 10:40

Afaik all builds supports AVX512, but you have to enable it explicitly (using --avx512 option). And it benefits only in 4k encodes, so it's not really useful for common user.

FranceBB

1st October 2021, 11:34

Afaik all builds supports AVX512, but you have to enable it explicitly (using --avx512 option). And it benefits only in 4k encodes, so it's not really useful for common user.

Ah so that's why it's not using them by default, I have to specify it from the command line, got it, now I remember, it's:

--asm=avx512

sometimes I learn something and that pushes out something else I already knew xD

FranceBB

1st October 2021, 17:39

New issue: why are 2 pools being created (56 + 56 = 112) but only 1 CPU (56) is used?

https://i.imgur.com/eWyQo1R.png

#Indexing the HDR10 PQ BT2100 4:2:2 10bit planar source
video=FFVideoSource("\\10.64.6.11\avisynth\Scambio\WARNER_MOVIES\BatmanBegins\HDR\BATMANBEGINS_INTL_UHD_HDR_PQ_REC2020_2398_240_PR4444_ITA51_ITA20_EN51_EN20-20210508155521.mov")
ch1=FFAudioSource("\\10.64.6.11\avisynth\Scambio\WARNER_MOVIES\BatmanBegins\HDR\BATMANBEGINS_INTL_UHD_HDR_PQ_REC2020_2398_240_PR4444_ITA51_ITA20_EN51_EN20-20210508155521.mov", track=1)
ch2=FFAudioSource("\\10.64.6.11\avisynth\Scambio\WARNER_MOVIES\BatmanBegins\HDR\BATMANBEGINS_INTL_UHD_HDR_PQ_REC2020_2398_240_PR4444_ITA51_ITA20_EN51_EN20-20210508155521.mov", track=2)
ch3=FFAudioSource("\\10.64.6.11\avisynth\Scambio\WARNER_MOVIES\BatmanBegins\HDR\BATMANBEGINS_INTL_UHD_HDR_PQ_REC2020_2398_240_PR4444_ITA51_ITA20_EN51_EN20-20210508155521.mov", track=3)
ch4=FFAudioSource("\\10.64.6.11\avisynth\Scambio\WARNER_MOVIES\BatmanBegins\HDR\BATMANBEGINS_INTL_UHD_HDR_PQ_REC2020_2398_240_PR4444_ITA51_ITA20_EN51_EN20-20210508155521.mov", track=4)
ch5=FFAudioSource("\\10.64.6.11\avisynth\Scambio\WARNER_MOVIES\BatmanBegins\HDR\BATMANBEGINS_INTL_UHD_HDR_PQ_REC2020_2398_240_PR4444_ITA51_ITA20_EN51_EN20-20210508155521.mov", track=5)
ch6=FFAudioSource("\\10.64.6.11\avisynth\Scambio\WARNER_MOVIES\BatmanBegins\HDR\BATMANBEGINS_INTL_UHD_HDR_PQ_REC2020_2398_240_PR4444_ITA51_ITA20_EN51_EN20-20210508155521.mov", track=6)
audio=MergeChannels(ch1, ch2, ch3, ch4, ch5, ch6)
AudioDub(video, audio)

#From 4:2:2 10bit to 4:2:2 16bit planar
ConvertBits(16)

#From 4:2:2 16bit planar Narrow Range to RGB Planar 16bit Narrow Range
z_ConvertFormat(pixel_type="RGBP16", colorspace_op="2020:2020:2020:limited=>rgb:2020:2020:limited", dither_type="none")

#From PQ to HLG with 16bit precision
Cube("I:\Scambio\WarnerBros\WarnerBros_PQToHLG_MaxCLL_1000.cube", fullrange=true)

#From RGB 16bit planar Narrow Range to YUV420 16bit planar Narrow Range
z_ConvertFormat(pixel_type="YUV420P16", colorspace_op="rgb:2020:2020:limited=>2020:2020:2020:limited", dither_type="none")

#Spatial denoise with 16bit planar precision
dfttest(sigma=64, tbsize=1, lsb_in=false, lsb=false, Y=true, U=true, V=true, opt=0, dither=0)

ffmpeg.exe -i "D:\Temp\AVS Script.avs" -vf scale=out_color_matrix=bt2020nc:out_h_chr_pos=0:out_v_chr_pos=0 -pix_fmt yuv420p16le -strict -1 -an -f yuv4mpegpipe - | x265.exe --y4m - --dither --preset medium --level 5.0 --tune fastdecode --no-high-tier --ref 4 --profile main10 --bitrate 25000 --deblock -1:-1 --min-luma 64 --max-luma 940 --chromaloc 2 --range limited --videoformat component --colorprim bt2020 --transfer arib-std-b67 --colormatrix bt2020nc --overscan show --no-open-gop --min-keyint 1 --keyint 24 --repeat-headers --rd 3 --vbv-maxrate 25000 --vbv-bufsize 25000 --wpp --pass 1 --asm=avx512 -o NUL

ffmpeg.exe -i "D:\Temp\AVS Script.avs" -vf scale=out_color_matrix=bt2020nc:out_h_chr_pos=0:out_v_chr_pos=0 -pix_fmt yuv420p16le -strict -1 -an -f yuv4mpegpipe - | x265.exe --y4m - --dither --preset medium --level 5.0 --tune fastdecode --no-high-tier --ref 4 --profile main10 --bitrate 25000 --deblock -1:-1 --min-luma 64 --max-luma 940 --chromaloc 2 --range limited --videoformat component --colorprim bt2020 --transfer bt2020-10 --colormatrix bt2020nc --overscan show --no-open-gop --min-keyint 1 --keyint 24 --repeat-headers --rd 3 --vbv-maxrate 25000 --vbv-bufsize 25000 --wpp --pass 2 --asm=avx512 -o "D:\Temp\raw_video.hevc"

C:\Program Files\MKVToolNix\mkvmerge.exe --output "D:\Temp\raw_video.mkv" --language 0:und ^"^(^" D:\Temp\raw_video.hevc ^"^)^"

ffmpeg.exe -i "D:\Temp\AVS Script.avs" -map 0:1 -af loudnorm=I=-24:LRA=14:tp=-2 -c:a ac3 -b:a 384k -ar 48000 "D:\Temp\audio.ac3"

ffmpeg.exe -i "D:\Temp\AVS Script.avs" -map 0:1 -af loudnorm=I=-24:LRA=14:tp=-2 -c:a libtwolame -b:a 192k -ar 48000 "D:\Temp\audio.mp2"

ffmpeg.exe -i "D:\Temp\raw_video.mkv" -i "D:\Temp\audio.ac3" -i "D:\Temp\audio.mp2" -map 0:0 -map 1:0 -map 2:0 -c:v copy -c:a copy -c:a copy -f mpegts -metadata provider_name="Sky" -metadata service_provider="Sky" -metadata service_name="Sky" -mpegts_original_network_id 0x1122 -mpegts_transport_stream_id 0x3344 -mpegts_service_id 0x5566 -mpegts_pmt_start_pid 0x1500 -mpegts_start_pid 0x150 -mpegts_service_type "digital_tv" -mpegts_flags "resend_headers" "D:\Temp\final.ts"

pause

LeXXuz

2nd October 2021, 19:52

Hi there,
I always forget who's maintaining those: http://msystem.waw.pl/x265/

Btw who IS maintaining those?

FranceBB

4th October 2021, 08:14

Why am I getting VBV violation? (the command line is the one above). x265 3.5+13-4c6880d

https://i.imgur.com/jSLRrzL.png

Boulder

4th October 2021, 12:27

Based on those QPs, your source is just too compressible and the encoder cannot keep up with the requested bitrate at those points?

FranceBB

4th October 2021, 15:26

Based on those QPs, your source is just too compressible and the encoder cannot keep up with the requested bitrate at those points?

Makes sense.
The movie is 1h and 30 and it looks like it's near the end, which is pure black with static credits, so it's very likely.
The CBR 1 Gbit/s ProRes is almost definitely zero filled towards that point.
Nobody asked for CBR, so VBR 25 Mbit/s with underflow should be ok. I'm gonna send those over anyway, wish me good luck :P

Mr.Rippley

4th October 2021, 17:05

Hi! I decided to learn how to efficiently encode video using HEVC. What is needed for this? Some special software or coding settings should be set "very slowly". Now I use StaxRip 2.8. I would like to achieve the quality and size of the video as on the PSA website, there are just magicians(IMHO). Thank you in advance for your help!

rwill

4th October 2021, 17:16

Makes sense.
The movie is 1h and 30 and it looks like it's near the end, which is pure black with static credits, so it's very likely.
The CBR 1 Gbit/s ProRes is almost definitely zero filled towards that point.
Nobody asked for CBR, so VBR 25 Mbit/s with underflow should be ok. I'm gonna send those over anyway, wish me good luck :P

Please check one of the following:
[ ] I know what a VBV underflow means.
[ ] I do not know what a VBV underflow means.

FranceBB

4th October 2021, 20:04

Please check one of the following:
[ ] I know what a VBV underflow means.
[X] I do not know what a VBV underflow means.

Checked, but I thought it was implied by the fact that I asked you, guys, about it xD
I get that that any vbv warning means that there's something off with the bitrate, however I generally used to get vbv errors when I was using like a very low CRF and there's a very complex scene like rain or snow etc and the bitrate wanted to skyrocket and fly high but I was limiting it with some constraints to stay much lower than it should, however I feel like this time it's a completely different situation, so if you could clarify what the encoder is actually telling me other than the fact that there's something wrong and it can't abide my constraints, that would be very helpful! :)

rwill

4th October 2021, 21:48

Video Buffer Verifier Model: Two buckets with one being above the other and having a hole in it. The buckets
hold bits. The encoder encodes pictures and puts them in the upper bucket. From there they flow through the
hole at a constant rate ( the vbv maxrate ) to the lower bucket. The lower bucket is the video buffer. When the
decoder decodes a picture it takes bits from the lower bucket ( video buffer ), most of the time this happens
at a constant frame rate.

A video buffer overflow happens when a decoder is not taking enough bits from the lower bucket and it flows over.
Now this is not really problematic unless its caused by encoder and decoder clocks drifting apart. If encoder
and decoder share a common clock the encoder should always put pictures at a framerate in its upper bucket
so that they can be removed at the same framerate from the lower bucket. Padding the flow from upper to lower
bucket when the upper bucket is empty may prevent that nothing flows but today thats not really needed anymore,
it was needed in the past though. When replacing the encoder and upper bucket by some instant access storage
( like a file ) one would just stop reading and so stop filling the lower bucket once it is full.

Now a problem arises where the encoder produces pictures that are so large that they cannot make it fast enough
through the hole to the lower bucket and once they made it there they are quicky removed by the decoder at the
fixed rate. There might arise a situation where the decoder tries to fetch a picture from the lower bucket but
the bucket does not contain a complete picture as it still needs some refill from the upper bucket. The past
pictures were to large and got removed previously -> the video buffer is empty, underrun, underflow -> "Buffering".

Video encoders try to prevent such a situation ( mostly at all costs because violating the VBV model will get your
stream get kicked from broadcast multiplex for example ). x265 tries to increase the QP within a picture when it
detects that encoding takes more bits than anticipated while encoding a picutre. I guess the pictures in question
( the ones with the underflows ) show lower quality in the lower pictures parts, maybe even complete breakage.
It appears to be not enough to prevent the underflow though. I think x265 just does not see it coming. Encoding
at such low QPs is quite delicate.

I wonder what the content looks like that x265 is able to maintain an average QP of ~3.2 at that rate. Are you
sure you got your bitdepth and clipping right ? I see in your script that you use
"-pix_fmt yuv420p16le -f yuv4mpegpipe". I never used y4m but wouldnt this flag the y4m as 16 bit depth ?
x265 gets no explicit input bit depth set too. And your .AVS script is doing 10 bit with limited range clipping.
I am confused.

Have you decoded the encoder output ? Does it look ok ?

FranceBB

4th October 2021, 22:15

First of all thank you so much for the detailed explanation, it makes a lot of sense now!
Still, I get that x265 just doesn't see it coming, but I kinda thought it would given that it's a two pass encode and this is the second pass.

I wonder what the content looks like that x265 is able to maintain an average QP of ~3.2 at that rate. Are you
sure you got your bitdepth and clipping right ?

Given that it's an old digital movie upscaled by the studio who sent it for the HDR UHD Remaster, I think there are no details whatsoever so it's just very easy to encode for x265.

I see in your script that you use
"-pix_fmt yuv420p16le -f yuv4mpegpipe". I never used y4m but wouldnt this flag the y4m as 16 bit depth ?
x265 gets no explicit input bit depth set too.

Yeah I forgot to post the final AVS, I have updated it now.
And yes, I tend to deliver 16bit planar to x265 and let it dither down to 10bit planar and use x265 itself to perform clipping at 64, 940 Limited TV range (not that HLG ever reaches the upper part of the available 940 anyway 'cause the movie ain't 1000 nits in the first place, but I just play safe). About the complexity, I don't like grain, like, at all, it doesn't look good on anything unless you have a very high bitrate (which I never have), therefore I always always always get rid of it, either spatially or temporally and sometimes both. This also makes things much easier for x265, hence the low qp.

Have you decoded the encoder output ? Does it look ok ?

It looks ok from the RDP given that all this is going on in a server, however I'll be able to tell once the encode is over and the file has been moved to a shared RAID6 instead of a puny local RAID1.

rwill

5th October 2021, 05:38

It looks ok from the RDP given that all this is going on in a server, however I'll be able to tell once the encode is over and the file has been moved to a shared RAID6 instead of a puny local RAID1.

I would look at least around timecode 00:34:21, 01:05:02, 01:21:16 and 01:29:24 and check
for visual problems.

Maybe lowering the average rate to 20Mbit or doing more passes will help. Or
increase minQP. x265 does not give many options in such a case, I think its
being too enthusiastic about reaching the average rate of 25Mbit and due to
local failure in frame size prediction it produces underflows.

Barough

7th October 2021, 00:15

x265 v3.5+16-7955046e7 (https://www.mediafire.com/file/b0e406dgzwdrcwi/x265-3.5+16-7955046e7_Win_GCC103.7z/file) (32 & 64-bit 8/10/12bit Multilib Windows Binaries) (GCC 10.3.0)

https://bitbucket.org/multicoreware/x265_git/commits/branch/master

Mr.Rippley

10th October 2021, 14:02

Thanks!

FranceBB

11th October 2021, 08:30

Please, help me!

We don't have enough info. Do you want device playback compatibility or not? 'cause if you don't and you just want to compress, then it's much easier. You can try with something like:

--preset placebo --ref 16 --me full --subme 7 --crf 22

to start and then you can work from there.

Mr.Rippley

11th October 2021, 18:04

We don't have enough info. Do you want device playback compatibility or not? 'cause if you don't and you just want to compress, then it's much easier. You can try with something like:

--preset placebo --ref 16 --me full --subme 7 --crf 22

to start and then you can work from there.

Thanks for the answer! I tried the placebo preset, coded it for several days) And what kind of encoder program do you use?Or do you consider it the best?

FranceBB

11th October 2021, 18:43

Thanks for the answer! I tried the placebo preset, coded it for several days) And what kind of encoder program do you use?Or do you consider it the best?

x265.exe via command line called through a BAT file.
If you're looking for builds, you'll find them here: http://msystem.waw.pl/x265/

LeXXuz

12th October 2021, 08:32

'Placebo' preset is a little bit over the top, isn't it?

If encoding time and energy consumption is of no concern, well go for it then.

But I think for private every day use the 'Slower' or 'Very Slow' presets are more than sufficient enough. I highly doubt most people would see any difference, especially on consumer displays.

RanmaCanada

12th October 2021, 14:34

Please, help me!

This particular topic is for discussing releases of x265. If you want actual help in learning how to encode with it you could either read the forum itself as there is a tonne of information in it, or create a topic in the forum requesting help while posting as much information as possible.

Barough

13th October 2021, 00:38

x265 v3.5+18-747a079f7 (https://www.mediafire.com/file/plqbcfef5j3uml3/x265-3.5+18-747a079f7_Win_GCC103.7z/file) (32 & 64-bit 8/10/12bit Multilib Windows Binaries) (GCC 10.3.0)

https://bitbucket.org/multicoreware/x265_git/commits/branch/master

tuanden0

17th October 2021, 02:13

Everyone can access to this site http://msystem.waw.pl/x265/?
I got error 404 :(

RanmaCanada

17th October 2021, 05:49

Everyone can access to this site http://msystem.waw.pl/x265/?
I got error 404 :(
works here.

charliebaby

17th October 2021, 08:17

x265-3.5+19-8003e43_gcc112-AVX2
https://www.mediafire.com/file/zms2i5loo85ca5u/x265-3.5+19-8003e43_gcc112-AVX2.7z/file

Dogway

2nd November 2021, 12:38

quietvoid

2nd November 2021, 12:51

What bitdepth and packing format is used for UHDs? I ask because I plan to do encodes in ICtCp or maybe its variant IPTPQc2. Are there any consumer media in these color models? Do they use 10 or 12-bit, I420, NV12 or something else?

UHDs? As in UHD Blu-ray discs?

The only media in IPTPQc2 is premium content on streaming services. It requires access to the Dolby Engine for encoding.
It is a 10 bit video file in the end along with Dolby Vision metadata, and requires playback on a Dolby Vision capable device and display.

There's no consumer media over 10 bit, and it's probably all YUV420.

Dogway

2nd November 2021, 13:01

benwaggoner

2nd November 2021, 22:58

I haven't looked into IPTPQc2 prerequisites for encoding, but I know x265 supports ICtCp. With these models 12-bit is encouraged but I'm not sure what kind of hindrances this carries later for compatibility (personal use). As I see all them use Planar 420 (fourcc: I420)
I don't know of any hardware that can decode 12-bit ICtCp. That's the nominal internal color volume used by DoVi in its intermediate stage, but I've never heard of it being directly encoded nor directly displayed. It's the input for DoVi's tone mapper to get to native panel space.

Dogway

3rd November 2021, 00:51

As per the papers it's a consumer oriented storage color model since it saves bits (and bandwidth) specially for low luma values and luma/chroma decorrelation is better than YCbCr or YcCbcCrc for HDR.
Yes I don't expect any hardware to be on the edge but I guess if premium stream services include it it must be the streaming apps that decode it to RGB. I personally use the LAVFilters+madVR combo so as far it can be decoded with that I'm fine except madVR's future isn't very bright (not FLOSS).

nevcairiel

3rd November 2021, 11:40

ICtCp is definitely used by streaming services for Dolby Vision. Its just metadata though, and you can decode it with any ordinary 10/12-bit capable HEVC decoder (depending on the stream), interpretation of the color data is then typically up to the implementation (ie. by using shaders), or using pass-through to the display.

Barough

17th November 2021, 06:35

x265 v3.5+21-7155a58f7 (https://www.mediafire.com/file/hvggc6r6pehjwpx/x265-3.5%252B21-7155a58f7_Win_GCC112.7z/file) (32 & 64-bit 8/10/12bit Multilib Windows Binaries) (GCC 10.3.0)

https://bitbucket.org/multicoreware/x265_git/commits/branch/master

charliebaby

19th November 2021, 11:02

LigH

21st November 2021, 20:29

tormento

7th December 2021, 12:39

Is there any option to optimize greyscale (i.e. BW movies) encoding. mostly about speed?

benwaggoner

7th December 2021, 20:03

Is there any option to optimize greyscale (i.e. BW movies) encoding. mostly about speed?
That was something that MCW had been looking at some years ago, but it doesn't look like it made it in.

The good news is that, with 4:2:0 chroma subsampling, only 1/3rd of the samples in a frame are color. And there is coarser quantization as well, so chroma compute is probably less than 33% on net. Setting --cbqpoffs and --crqpoffs to 12 would provide chroma with much coarser quantization, which speeds up some things (but would make actual chroma look pretty terrible).

You'd thus want to filter your source to luma-only before encoding to avoid that kind of problem. And IIRC there are some early exits that turn on when a chroma plane is packed with zeros. Give it a shot.

asarian

17th December 2021, 11:55

New upload: x265 3.5+20-17839cc0d (https://www.mediafire.com/file/cy3ma8e9nr4vm88/x265_3.5+20-17839cc0d.7z/file)

[Windows][GCC 11.2.0][32/32XP/64 bit] 8bit+10bit+12bit

Bugs not yet fixed:

NASM 2.15.05 multi-line macro warnings (reported August 2020)

FYI, I am currently using this version. Even though I have --frames 211420 on the command line, your version says 'unknown frame count'.

LigH

18th December 2021, 03:37

Don't blame me, I only run the compilation. The reason may be related to processing a raw video input stream via pipe instead of physical source file. Using x265 linked in a current ffmpeg may give you a better environment.

asarian

18th December 2021, 04:47

Boulder

18th December 2021, 09:02

Isn't the '--frames 211420' supposed to tell x265 how many frames there are?

The message you are seeing is from the input handling. I don't think I've ever seen a number of frames there when using piping.

The number of frames set in the parameters is only used for calculating the ETA.

asarian

18th December 2021, 09:34

The message you are seeing is from the input handling. I don't think I've ever seen a number of frames there when using piping.

The number of frames set in the parameters is only used for calculating the ETA.

Exactly. My entire command line goes like this:

VSPipe -c y4m "f:\jobs\test.vpy" - | x265 --y4m --input - --lossless --preset ultrafast --input-depth 10 --output-depth 10 --crf 10 --colorprim 9 --transfer 16 --colormatrix 9 --master-display "G(8500,39850)B(6550,2300)R(35400,14600)WP(15635,16450)L(10000000,1)" --max-cll "0,0" --frames 211420 --chromaloc 2 --aq-mode 3 --output "f:\video\test.hevc"

I've always done it like this. I do get the ETA at the end, though

x265 [info]: tools: strong-intra-smoothing lslices=8 deblock
[73.4%] 155199/211420 frames, 1.97 fps, 1008625.25 kb/s, eta 7:54:32

But the top line says

y4m [info]: 3840x2072 fps 24000/1001 i420p16 unknown frame count

Boulder

18th December 2021, 09:58

But the top line says

y4m [info]: 3840x2072 fps 24000/1001 i420p16 unknown frame count

This is because it cannot get the amount of frames from the piped stream.

Boulder

18th December 2021, 13:30

Without limit-tu, higher values of tu-intra and tu-inter only increase compression, not visual quality. limit-tu 1 + tu-inter 4 + tu-intra 4 should give better quality than using limit-tu 0.

I ran into this remark while once again searching for information regarding TU depth and related things.

Is it actually so that increasing the TU depth only affects compression and that in fact, at least in CRF mode, it can potentially lead to lower detail retention? I've always thought that it is one component which retains detail since it possibly splits into smaller pieces when needed. Or is CTU itself already the major thing in detail retention?

Asmodian

19th December 2021, 02:52

Is it actually so that increasing the TU depth only affects compression and that in fact, at least in CRF mode, it can potentially lead to lower detail retention?

It isn't that limit-tu 0 lowers detail retention, but that tu-inter 4 and tu-intra 4 increases detail retention while limit-tu 1 keeps the speed up.

limit-tu 0 with tu-inter 4 and tu-intra 4 would be great, but also slow.

Boulder

19th December 2021, 12:25

It isn't that limit-tu 0 lowers detail retention, but that tu-inter 4 and tu-intra 4 increases detail retention while limit-tu 1 keeps the speed up.

limit-tu 0 with tu-inter 4 and tu-intra 4 would be great, but also slow.
There the paradox lies. A higher depth is supposed to increase detail retention, but only if it is limited :confused:

Blade Runner 2049
--tu-inter-depth 4 --tu-intra-depth 4
CTU 64
limit-tu 4 - 4847,24 kbps - 19.57
limit-tu 3 - 4634,72 kbps - 19.54
limit-tu 2 - 4766,69 kbps - 19.61
limit-tu 1 - 4328,30 kbps - 19.82
limit-tu 0 - 3970,48 kbps - 20.15

CTU 32
limit-tu 4 - 5329,45 kbps - 19.29
limit-tu 3 - 5325,54 kbps - 19.28
limit-tu 2 - 5289,30 kbps - 19.31
limit-tu 1 - 5092,24 kbps - 19.40
limit-tu 0 - 5128,37 kbps - 19.47

And if I reduce the depths step by step, depth 1 will produce the highest filesize and also the lowest average QP. The difference is not much, but it's there. Depth 1 is also much faster, about 10% difference in my tests.

Based on a snippet of ~2000 frames, the average QP is much higher the less TU recursion is limited. I've also found out earlier that the combination of CTU 64, rskip 2 and limit-tu 0 is very much b0rked.

Asmodian

20th December 2021, 00:16

Shouldn't you compare at the same size? Lower CRF if a setting decreases the size, or better, compare settings with 2-pass so the bitrates are the same.

Are you assuming a setting retains more detail based on how it affects size at the same CRF? It is always possible to make a file bigger and retain more detail. :confused:

Boulder

20th December 2021, 06:16

I'd expect this setting to be one which doesn't require varying CRF since it's more like affecting the number of iterations. My question more or less rises from the fact that in x265 development, the focus in some parameters seems to be compressing more and more instead of detail retention. TU depth is one of those I cannot just understand well enough.

Asmodian

22nd December 2021, 23:27

I'd expect this setting to be one which doesn't require varying CRF since it's more like affecting the number of iterations.

This doesn't make sense to me. Why would affecting the number of iterations not require changing the CRF?

It changes the number of configurations tried, but only one is picked in the end. What "best" means when selecting the best option might not be the one with the highest detail retention, but it could also be that the other options simply don't compress as well.

Unless you compare visually at the same size, I don't see how you can come to any conclusion.

Boulder

27th December 2021, 14:18

benwaggoner

3rd January 2022, 21:43

This doesn't make sense to me. Why would affecting the number of iterations not require changing the CRF?
Why would it? higher --tu-i* depths allow for increasingly smaller TUs relative to --ctu. --tu-intra/inter-depth 4 with --ctu 64 and --tu-inter/intra-depth 3 with --ctu, both allowing a minimum TU size of 4x4.

It changes the number of configurations tried, but only one is picked in the end. What "best" means when selecting the best option might not be the one with the highest detail retention, but it could also be that the other options simply don't compress as well.
Yeah, intra-frame RC can get quite complex. And the smaller the TU, the higher QP can be at similar subjective quality, as there's fewer pixels for ringing or blocking to become visible.

Unless you compare visually at the same size, I don't see how you can come to any conclusion.
I fully agree that comparisons need to be done at the same ABR to really understand the pros/cons. Using CRF adds ABR as a confounding factor.

In my experience, the higher tu depths can be really helpful with low-noise content with sharp edges, like animation, graphics, and text.

They don't do much with grainy/noisy content because so much of the energy of potential TUs is in randomly distributed pixels. This is particularly of grain that has a lot of per-pixel variance. More natural grain that isn't per-pixel randomization is better.

benwaggoner

3rd January 2022, 21:48

Can anyone explain why --no-sao and --selective-sao 0 are two different things? What is the encoder doing when --selective-sao 0 is applied but SAO is still enabled? I was doing some testing with --selective-sao 1 and made a comparison encode with --selective-sao 0. I then noticed that it produced a larger file than when --no-sao is used..

--selective-sao 1 - 6635.40 kbps, avg QP 21.72
--selective-sao 0 - 6816.47 kbps, avg QP 21.62
--no-sao - 6641.23 kbps, avg QP 21.72

EDIT: hmm, apparently --selective-sao + --sao is the same as "old --sao".
--sao is also the "old sao" ;).

--selective-sao just turns off SAO for specified classes of frames. The only one that really matters is --selective-sao, which only applies SAO to I and P frames. SAO wasn't designed for bidirectional prediction and provides almost no benefit to B and b frames. So --selective-sao 2 is an essentially free minor speed boost. This is pretty orthogonal to the detail preservation/ringing suppression features of SAO, and has little or no visual impact on how SAO operates in x265.

I'd love to have an --sao-parameters that could actually tune the SAO parameters; x265 just uses fixed SAO parameters, so we really only can control in being on/off. An --adaptive-sao would be even better. I don't know of any actual research done in this area, though.

Boulder

4th January 2022, 06:34

--sao is also the "old sao" ;).

--selective-sao just turns off SAO for specified classes of frames.
In that case, --selective-sao 0 --sao should disable SAO since --selective-sao 0 is "Disable SAO for all slices" as per the documentation. Looks like a combination which really has not been tested while evaluating the patch..

I've now settled for using --selective-sao 1. SAO has a too strong effect on P-frames to my liking, it clearly removes detail. I-frames are affected only very slightly, which I believe is the real intent of the functionality. Even though they are not to be blindly trusted, distortion related metrics like MDSI and GMSD show improvement in the whole GOP with value 1 compared to other values, or --no-sao. So I think using it only in the first frame of the GOP is most beneficial.

benwaggoner

4th January 2022, 19:19

In that case, --selective-sao 0 --sao should disable SAO since --selective-sao 0 is "Disable SAO for all slices" as per the documentation. Looks like a combination which really has not been tested while evaluating the patch..
I concur. If x265 ever gets refactored, both parameters should be replaced with a single --sao-level,

I've now settled for using --selective-sao 1. SAO has a too strong effect on P-frames to my liking, it clearly removes detail. I-frames are affected only very slightly, which I believe is the real intent of the functionality. Even though they are not to be blindly trusted, distortion related metrics like MDSI and GMSD show improvement in the whole GOP with value 1 compared to other values, or --no-sao. So I think using it only in the first frame of the GOP is most beneficial.
That's an interesting result, and not one I've replicated. What sort of content, bitrates, and settings did you test with?

Boulder

4th January 2022, 20:07

The source is Fellowship of the Ring EE, frames 140330-141401 in this case. Some static scenes with closeups and panning motion.

https://drive.google.com/file/d/1SnSLipsVQRUNh25xFjRTjcsFjGDNcu45/view?usp=sharing --no-sao
https://drive.google.com/file/d/1pp2MlMauWswcqY6ttbrudSsHMjOGUWFv/view?usp=sharing --selective-sao 1
https://drive.google.com/file/d/13wDN1VVUbgcWiHrmFlGqodqCj3WTl4y5/view?usp=sharing --selective-sao 2

This is the MediaInfo output. --preset slower as the base, then some additional touches. CRF 18, very slight denoising in the script, I'm also doing the green tint fix with a GIMP curve.

Video
ID : 1
Format : HEVC
Format/Info : High Efficiency Video Coding
Format profile : Main 10@L4@Main
Codec ID : V_MPEGH/ISO/HEVC
Duration : 44 s 711 ms
Bit rate : 6 508 kb/s
Width : 1 920 pixels
Height : 800 pixels
Display aspect ratio : 2.40:1
Frame rate mode : Constant
Frame rate : 23.976 (24000/1001) FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 10 bits
Bits/(Pixel*Frame) : 0.177
Stream size : 34.7 MiB (98%)
Writing library : x265 3.5+21-30cbfda6a:[Windows][GCC 11.2.0][64 bit] 10bit
Encoding settings : cpuid=1111039 / frame-threads=4 / numa-pools=24 / wpp / no-pmode / no-pme /
no-psnr / no-ssim / log-level=2 / input-csp=1 / input-res=1920x800 / interlace=0 / total-frames=0 / level-idc=0 / high-tier=1 / uhd-bd=0 / ref=4 / no-allow-non-conformance /
no-repeat-headers / annexb / no-aud / no-eob / no-eos / no-hrd / info / hash=0 / no-temporal-layers / open-gop / min-keyint=5 / keyint=480 / gop-lookahead=0 / bframes=10 / b-adapt=2 /
b-pyramid / bframe-bias=0 / rc-lookahead=40 / lookahead-slices=0 / scenecut=40 / no-hist-scenecut / radl=0 / no-splice / no-intra-refresh / ctu=32 / min-cu-size=8 / rect / amp /
max-tu-size=32 / tu-inter-depth=4 / tu-intra-depth=4 / limit-tu=4 / rdoq-level=1 / dynamic-rd=0.00 / no-ssim-rd / signhide / no-tskip / nr-intra=0 / nr-inter=0 / no-constrained-intra /
strong-intra-smoothing / max-merge=2 / limit-refs=1 / limit-modes / me=3 / subme=4 / merange=26 / temporal-mvp / no-frame-dup / no-hme / weightp / weightb / no-analyze-src-pics /
deblock=-1:-1 / sao / no-sao-non-deblock / rd=6 / selective-sao=1 / no-early-skip / rskip / rskip-edge-threshold=0.020000 / no-fast-intra / no-tskip-fast / no-cu-lossless / b-intra /
splitrd-skip / rdpenalty=0 / psy-rd=1.80 / psy-rdoq=5.00 / no-rd-refine / no-lossless / cbqpoffs=-3 / crqpoffs=-3 / rc=crf / crf=18.0 / qcomp=0.60 / qpstep=4 / stats-write=0 / stats-read=0 /
ipratio=1.35 / pbratio=1.25 / aq-mode=1 / aq-strength=1.00 / cutree / zone-count=0 / no-strict-cbr / qg-size=32 / no-rc-grain / qpmax=69 / qpmin=0 / no-const-vbv / sar=1 / overscan=0 / videoformat=5 /
range=0 / colorprim=1 / transfer=1 / colormatrix=1 / chromaloc=0 / display-window=0 / cll=0,0 / min-luma=64 / max-luma=940 / log2-max-poc-lsb=8 / vui-timing-info / vui-hrd-info / slices=1 /
no-opt-qp-pps / no-opt-ref-list-length-pps / no-multi-pass-opt-rps / scenecut-bias=0.05 / hist-threshold=0.03 / no-opt-cu-delta-qp / no-aq-motion / no-hdr10 /
no-hdr10-opt / no-dhdr10-opt / no-idr-recovery-sei / analysis-reuse-level=0 / analysis-save-reuse-level=0 / analysis-load-reuse-level=0 / scale-factor=0 / refine-intra=0 / refine-inter=0 / refine-mv=1 /
refine-ctu-distortion=0 / no-limit-sao / ctu-info=0 / no-lowpass-dct / refine-analysis-type=0 / copy-pic=1 / max-ausize-factor=1.0 / no-dynamic-refine / no-single-sei / no-hevc-aq / no-svt / no-field /
qp-adaptation-range=1.00 / scenecut-aware-qp=0 / conformance-window-offsets / right=0 / bottom=0 / decoder-max-rate=0 / no-vbv-live-multi-pass
Default : Yes
Forced : No
Color range : Limited
Color primaries : BT.709
Transfer characteristics : BT.709
Matrix coefficients : BT.709

Kill3rWolf

5th January 2022, 02:21

Boulder

5th January 2022, 06:51

benwaggoner

5th January 2022, 22:09

The differences in motion are very subtle. However, in frame-by-frame comparison, to me the areas with edges look very slightly better with --selective-sao 1.

Here's --selective-sao 1 --limit-sao: https://drive.google.com/file/d/1zYPp4nZTuRtHbbZ_A-Q1l2NciOeILRDI/view?usp=sharing
Subtle changes only visible in frame-by-frame aren't really psychovisually relevant with moving image content. And x265 certainly has optimizations that presume full-speed playback.

benwaggoner

5th January 2022, 22:15

Also, I believe --limit-sao is supposed to be just a performance improvement via early exit, and shouldn't change the look of SAO. But stuff can have unanticipated impacts! Have you seen a material difference with it on versus off?

_kermit

8th January 2022, 12:16

what are the current recommended tuning options for 1080p and 4K (HDR) , assuming slow or slower is used?

considering that only motion and not still pictures are of interest?
what are you using?

tormento

8th January 2022, 18:00

Guys, as far as I can read on hevc papers and academic studies, sao is now completely safe to be used but in a very few cases.

rwill

8th January 2022, 18:20

Guys, as far as I can read on hevc papers and academic studies, sao is now completely safe to be used but in a very few cases.

SAO in HEVC was set in stone in 2013. What do you mean with 'now' ?

Boulder

8th January 2022, 19:37

I'd say the problem is in the implementation, not in the feature itself.

rwill

8th January 2022, 20:12

I'd say the problem is in the implementation, not in the feature itself.

True, my x265 SAO example (https://forum.doom9.org/showthread.php?p=1949376#post1949376) still stands it seems.

_kermit

9th January 2022, 13:57

microchip8

9th January 2022, 15:27

rwill

9th January 2022, 15:54

Massive tearing, at least I think that's what it's called: https://ibb.co/fSYXTQ0

it's a specific scene of Mission Impossible - Rogue Nation starting at 1.09.56 (original: https://ibb.co/9GhV3wx)

any ideas how to prevent that?
are my params the reason (hence the question above)?

Looks like encoder/decoder mismatch to me. Try to write out x265 recon to check if this is what x265 had in mind for these pictures or try another decoder ( maybe the reference software ) to see if the problem persists.

Boulder

9th January 2022, 15:58

IIRC, rskip 2 doesn't work (well) with a CTU of 64 which is the default if not set

At least if --limit-tu 0 is used.

I'm still trying to decide whether I should use CTU 64 with 1080p or 1440p encodes, but I'm a bit unsure. Based on some testing, CTU 32 does not seem to create as strong "floating noise in the flat background" effect as CTU 64 with standard quality sources, so I'm leaning towards CTU 32.

Kill3rWolf

9th January 2022, 17:42

but in a very few cases.

@tormento, Which few cases?

I've just encoded the same movie again using this (skipped a few params):
x265.exe - --y4m --output-depth 10 --preset slower --crf 22 --colorprim bt2020 --transfer smpte2084 --colormatrix bt2020nc --vbv-bufsize 160000 --vbv-maxrate 160000 --rdoq-level 2 --cu-lossless --max-merge 3 --rc-lookahead 25 --lookahead-slices 4 --ref 4 --range limited --hdr --hdr-opt --repeat-headers --aud --deblock -1:-1 --no-strong-intra-smoothing --aq-mode 1 --rskip 2 --rskip-edge-threshold 3 --psy-rd 4 --psy-rdoq 15

What (rskip-edge-threshold 3) actaully do? When should increase psy-rd 4 & psy-rdoq? Reason for deblock -1:-1?

Explaining here &/or providing links will be beneficial for me.

microchip8

9th January 2022, 18:09

@tormento, Which few cases?

What (rskip-edge-threshold 3) actaully do? When should increase psy-rd 4 & psy-rdoq? Reason for deblock -1:-1?

Explaining here &/or providing links will be beneficial for me.

rskip threshold sets the threshold for rskip 2. The lower, the better/more precise rskip 2 will work.

High values of psy-rd and psy-rdoq help eliminate banding issues far better than using aq-mode 3

deblock -1,-1 lowers the strength of the in-loop deblocker and thus provides a sharper image.

Kill3rWolf

9th January 2022, 23:59

High values of psy-rd and psy-rdoq help eliminate banding issues far better than using aq-mode 3.

Currently, I'm using only aq-mode 3 without touching the psy-rd and psy-rdoq (default or auto-detect). Is there any chance higher psy-rd and psy-rdoq values will provide bad quality or any kind of accident?

deblock -1,-1 lowers the strength of the in-loop deblocker and thus provides a sharper image.

What about deblock=1:-3:-3? I don't know why some people (encoders) biased/tendency to use deblock=1:-3:-3. Which one is the safest option to use almost all contents? Less or almost zero Artifacts with sharper image & well detail?

apophis906

10th January 2022, 02:01

I've just encoded the same movie again using this (skipped a few params):
x265.exe - --y4m --output-depth 10 --preset slower --crf 22 --colorprim bt2020 --transfer smpte2084 --colormatrix bt2020nc --vbv-bufsize 160000 --vbv-maxrate 160000 --rdoq-level 2 --cu-lossless --max-merge 3 --rc-lookahead 25 --lookahead-slices 4 --ref 4 --range limited --hdr --hdr-opt --repeat-headers --aud --deblock -1:-1 --no-strong-intra-smoothing --aq-mode 1 --rskip 2 --rskip-edge-threshold 3 --psy-rd 4 --psy-rdoq 15

with the same result.
Massive tearing, at least I think that's what it's called: https://ibb.co/fSYXTQ0

it's a specific scene of Mission Impossible - Rogue Nation starting at 1.09.56 (original: https://ibb.co/9GhV3wx)

any ideas how to prevent that?
are my params the reason (hence the question above)?

As far as I can tell that is a decoder issue. I have had that happen on a few myself. With both Mortal Kombat animated UHD and then Halloween 2018 UHD. It would mess up like that on my htpc but not the pc I encoded it with, or when I used software decoding. My htpc has an EVGA GTX 1060 SC 6GB, and I have two htpc's for different rooms and they both react the same way.
After the third one I tested and changed from slow to slower due to it not happening with that change. Even with that it still happened with Shang Chi. So then I started to test a bunch of changes and decided to try changing the IP ratio to 1.3 and PB ratio to 1.2. That got all of those to do fine, but then caused Black Widow to mess up when it had not before. In the end I put the IP and PB back to stock and turned on selective SAO 1 and that got all of them to not do it. So far I have not have any UHD show that problem, but only been using that setting a few weeks so far.
I did have an HD show do it last night though. It was the remastered Gundam Seed episode 47. It cuts to a computer screen and you see a radar image, then a solid yellow frame for one frame and then a weapons chart for 7 frames. Then you get another solid yellow frame for one frame and a selection menu. On playback with my settings it was all yellow with just the letters all messed up trying to show. My solution was to add the two frames right after the yellow single frame to the QP file and set them to an I frame. That cleared it up and had it play fine.
My advice is you just have to test settings to see what works for you and your playback device. If this is the only movie that has given you any trouble then the simplest option is to find the frame that is at the start of the error and add that to a QP file and set it to an I frame. That should clear it up for you. Just know if you use Staxrip like I do and you chunk encode, that any QP setting after the first chunk doesn't go through and will only take effect for the first chunk. Now if this happens to a few movie then you will need to play around with settings and see what avoids this with your playback device.

microchip8

10th January 2022, 08:11

Currently, I'm using only aq-mode 3 without touching the psy-rd and psy-rdoq (default or auto-detect). Is there any chance higher psy-rd and psy-rdoq values will provide bad quality or any kind of accident?

it's safe to use high values of psy-rd/psy-rdoq. I use them all the time and have yet to see something wrong. Make sure you use aq-mode 1 when using high values.

What about deblock=1:-3:-3? I don't know why some people (encoders) biased/tendency to use deblock=1:-3:-3. Which one is the safest option to use almost all contents? Less or almost zero Artifacts with sharper image & well detail?

-3:-3 is a good balance between deblocking and preserving detail/sharpness. The default in x265 softens the image too much. I also use -3:-3 for all my encodes

_kermit

10th January 2022, 13:36

IIRC, rskip 2 doesn't work (well) with a CTU of 64 which is the default if not set

I removed it, seems to work, thanks!
But it also cut FPS in half.
How can I improve that again?

Boulder

10th January 2022, 13:48

Use --ctu 32. Rskip 2 is useful for retaining details, CTU 64 much less (if at all) so.

_kermit

10th January 2022, 13:50

it's safe to use high values of psy-rd/psy-rdoq. I use them all the time and have yet to see something wrong. Make sure you use aq-mode 1 when using high values.

-3:-3 is a good balance between deblocking and preserving detail/sharpness. The default in x265 softens the image too much. I also use -3:-3 for all my encodes

what are high values for psy-..?

microchip8

10th January 2022, 15:01

what are high values for psy-..?

3 and above for psy-rd, 5 and above for psy-rdoq. I personally use 4 for psy-rd and 15 for psy-rdoq. This due to encountering a lot of banding while encoding Blade Runner 2049 blu-ray. aq-mode 3 couldn't cut it so I went over to psy-* which worked perfectly. Ever since then I've been encoding different content with these psy values and never had banding again (unless it's present in the source itself)

microchip8

10th January 2022, 15:04

I removed it, seems to work, thanks!
But it also cut FPS in half.
How can I improve that again?

use rskip 1 and ctu 32

_kermit

10th January 2022, 17:28

use rskip 1 and ctu 32

thanks for both tips!

Kill3rWolf

10th January 2022, 23:20

3 and above for psy-rd, 5 and above for psy-rdoq. I personally use 4 for psy-rd and 15 for psy-rdoq. This due to encountering a lot of banding while encoding Blade Runner 2049 blu-ray. aq-mode 3 couldn't cut it so I went over to psy-* which worked perfectly. Ever since then I've been encoding different content with these psy values and never had banding again (unless it's present in the source itself)

Do these tricks apply to both CRF & Target Bitrate encoding?

microchip8

11th January 2022, 00:13

Do these tricks apply to both CRF & Target Bitrate encoding?

Yes they do

Kill3rWolf

11th January 2022, 00:17

Use --ctu 32. Rskip 2 is useful for retaining details, CTU 64 much less (if at all) so.

use rskip 1 and ctu 32

Got confusion. (Boulder) Rskip 2 & (microchip8) rskip 1. Same topic, but with a different value.

Kill3rWolf

11th January 2022, 00:56

https://i.ibb.co/s5KJWHf/HEVC-Tier.png (https://ibb.co/B4yKnNp)
Could you tell me perfect vbv-maxrate= & vbv-bufsize= value for Level 4.1 Main tier (Blu-ray encoding)?
Should I set 20,000 for both?
Can I do vbv-maxrate=12000 & vbv-bufsize=20000 without any error?

I saw that an encoder did vbv-maxrate=18000 & vbv-bufsize=24000 for Main 10@L5.1@Main.

Boulder

11th January 2022, 06:03

Got confusion. (Boulder) Rskip 2 & (microchip8) rskip 1. Same topic, but with a different value.

--rskip 2 is better for detail retention. --rskip 1 is the same as the old --rskip. I recommend using --ctu 32 regardless of the rskip setting.

_kermit

11th January 2022, 08:55

--rskip 2 is better for detail retention. --rskip 1 is the same as the old --rskip. I recommend using --ctu 32 regardless of the rskip setting.

with --rskip 2 I had tearing, not using that anymore.

rwill

11th January 2022, 09:24

with --rskip 2 I had tearing, not using that anymore.

Tearing ? You mean what you reported here ?:
https://forum.doom9.org/showthread.php?p=1960919#post1960919

You had quite a lot of parameters. By that logic you don't set any of these now ?

Kill3rWolf

11th January 2022, 10:41

Guys, need help regarding HEVC Level, Tier vbv-maxrate= & vbv-bufsize= value.

Gravitator

11th January 2022, 11:56

excellentswordfight

11th January 2022, 21:24

[url=https://ibb.co/B4yKnNp]
Could you tell me perfect vbv-maxrate= & vbv-bufsize= value for Level 4.1 Main tier (Blu-ray encoding)?

The "perfect" values depends on your target device & playback variables.

Should I set 20,000 for both?

You could, that is also what the encoder sets if you specify that level.

Can I do vbv-maxrate=12000 & vbv-bufsize=20000 without any error?

As long as its not above the max value of the level you want to use, yes.

I saw that an encoder did vbv-maxrate=18000 & vbv-bufsize=24000 for Main 10@L5.1@Main.

Ok?

_kermit

11th January 2022, 22:36

Tearing ? You mean what you reported here ?:
https://forum.doom9.org/showthread.php?p=1960919#post1960919

You had quite a lot of parameters. By that logic you don't set any of these now ?

yes, that screen (btw how is that called properly?)

and I kept everything just skipped --rskip 2 (which I'm still wondering if that is a good thing, but those I collected over time. I'd prefer to keep it more simple or better ones if possible)

that above went away, I try now with --rskip 1 to get more FPS.

_kermit

11th January 2022, 22:37

Can you provide a sample (30 sec)?

the first screenshot shows it (if still online). That specific scene is all like that, while there is like "doublevision" of her (screen two) in the original.

rwill

11th January 2022, 23:15

yes, that screen (btw how is that called properly?)

and I kept everything just skipped --rskip 2 (which I'm still wondering if that is a good thing, but those I collected over time. I'd prefer to keep it more simple or better ones if possible)

that above went away, I try now with --rskip 1 to get more FPS.

If the screen is covered with tiny blocks or garbage that do not fit the bitrate like in this case I would call it encoding or decoding error or encoder/decoder mismatch. Its an error that is caused by the encoder trying to do one thing and writing it out but the decoder is doing another thing upon reading it in, leading to complete disaster. Once broken by such an error the encoding/decoding process most likely does not recover until the next random access point. In simpler terms it is a miscommunication of picture contents between encoder and decoder.

This can be caused by some bug or out of spec behavior in the encoder or decoder where the standard ( here h.265 ) is not followed correctly. If you used hardware decoding to produce that output the most possible cause lies in the encoder because hardware decoders are tested very thoroughly before being cast in silicon.

These type of problems can be debugged by cutting the source down to some small sample which triggers the effect every time it is decoded with one specific or all decoders. So to fix it one needs the small input sample, the encoder in a specific version in source form, the encoder configuration and the decoder in source form in a specific version. Oh and the H.265 standard. Then you can see at which point the encoder and decoder start to differ in the encoding/decoding process. Large samples where things start to happen one hour in are useless because their are very cumbersome to debug.

Could be that the rskip 2 codepath triggers some problem but it might also be that due to all the dependencies a bitstream and encoder / decoder state has a very rare condition somewhere else in the encoder or decoder is triggered by pure chance.

Then again maybe just some frame is too large or the decoder application is feeding DXVA2 wrong at your 160MBit rate.

benwaggoner

13th January 2022, 03:32

with --rskip 2 I had tearing, not using that anymore.
Turning down the --rskip-edge-threshold some should fix the tearing while still preserving some of the speed improvements. I find 2-3 works pretty well with the content I've tested with it.

rwill

13th January 2022, 09:11

Turning down the --rskip-edge-threshold some should fix the tearing while still preserving some of the speed improvements. I find 2-3 works pretty well with the content I've tested with it.

Have you seen the screenshot he provided ?

Kill3rWolf

16th January 2022, 23:04

I have a question, Which one is better, max-luma=255 or max-luma=1023? or depend on the source?

Which one provides bad quality, I mean incorrect colors & too much unacceptable brightness.

GEfS

17th January 2022, 10:36

I have a question, Which one is better, max-luma=255 or max-luma=1023? or depend on the source?

Which one provides bad quality, I mean incorrect colors & too much unacceptable brightness.

the value already tells the reason, 255 (0-255, 256 values, 2^8) is 8 bit, 1023 (0-1023, 1024 value, 2^10) is 10bit, it depends on your in and output

Kill3rWolf

17th January 2022, 21:02

FranceBB

18th January 2022, 09:40

I have a question, Which one is better, max-luma=255 or max-luma=1023? or depend on the source?

Which one provides bad quality, I mean incorrect colors & too much unacceptable brightness.

That's full range, though.
for 10bit Limited TV Range I use:

--min-luma 64 --max-luma 940

Kill3rWolf

18th January 2022, 10:25

That's full range, though.
for 10bit Limited TV Range I use:

--min-luma 64 --max-luma 940

What's the actual benefit of doing it? I mean behind the reasons.

excellentswordfight

18th January 2022, 11:56

What's the actual benefit of doing it? I mean behind the reasons.
It sets what values the encoder excepts for the incomming signal, values above/below are clipped. I.e. if you set this incorrectly you will mess upp the luma values of your encode.

It's very unlikely that you need to specify this manually unless you have good reason for it.

Kill3rWolf

18th January 2022, 13:01

It's very unlikely that you need to specify this manually unless you have good reason for it.

Yeah, that's why ain't gonna touch it.

higher

19th January 2022, 19:09

microchip8

19th January 2022, 19:32

Boulder

19th January 2022, 20:30

Or post a MediaInfo output of both cases so it's to see what the basic difference is.

higher

19th January 2022, 21:07

Thanks to both of you for the help. I was using the slow preset and some extra command line options. Upon comparing the encoding settings of a new and an older file via MediaInfo I saw no difference. I reset hybrid and now the encode is H/W accelerated in mpc-hc. Don't know what went wrong. :)

Kill3rWolf

20th January 2022, 18:36

MeteorRain

23rd January 2022, 17:10

This is because it cannot get the amount of frames from the piped stream.

It is possible but requires a patch to the y4m input handler code.

y4m [info]: 2560x1064 fps 24000/1001 i420p10 frames 0 - 70330 of 70331

benwaggoner

24th January 2022, 03:05

https://ibb.co/2NRL0P9

@Ben Waggoner, Should I use aq-mode=4 only for 4k HDR content? What about 1080p SDR content? aq-mode=4, Is it still unstable? Can be used for almost all content?

https://iopscience.iop.org/article/10.1088/1742-6596/2078/1/012029/meta
Yeah, --aq-mode 4 can be used with all color spaces. The only one that is really color space specific is --aq-mode 3, which is --aq-mode 2 with a bias for lower QPs near black, to address SDR's insufficient code values in low luma.

The right aq mode can depend on content and other parameters, but I generally find --aq-mode 4 to be the best default these days. In particular, it generally handles high grain/noise better than the default of --aq-mode 2.

benwaggoner

24th January 2022, 03:10

It sets what values the encoder excepts for the incomming signal, values above/below are clipped. I.e. if you set this incorrectly you will mess upp the luma values of your encode.

It's very unlikely that you need to specify this manually unless you have good reason for it.
And there is a downside to clamping with content that has excursions below nominal black, like analog sources.

Even if black averages out at Y'=64 or =16, noise can result in some pixels having values somewhat lower and others somewhat higher. Clamping to nominal black will leave around half the pixels at nominal black and the others somewhat higher. That results in black areas actually averaging brighter than nominal black, and can leave a "boiling oil" appearance. And the resulting sharp edges are harder to encode, increasing artifacts and/or wasting bits.

I don't know that I've ever set those values in a real encode. Setting a maximum value would be less fraught because SDR has slight code differences be much less visible near white.

If I had to set a minimum value for some reason, I'd want to set it at maybe 75% of nominal black to allow for some random noise without clamping. So maybe 48 for 10-bit and 12 for 8-bit? It would be both context and content dependent.

benwaggoner

24th January 2022, 03:12

Hello Gents!

I discovered that my x265 encodes (1080p, 2160p) lately are not gpu H/W accelerated in mpc-hc and are decoded on the cpu. My older encodes are playing with gpu H/W acceleration. Is there a flag that I have messed with or what could be the culprit? I'm using a gui called hybrid and mkvmerge.
Perhaps you're using 10-bit with 8-bit only capable hardware, or exceeding the hardware decoders' maximum Profile @ Level support?

First step is to compare before/after in MediaInfo. I'd guess Profile or Level are higher in the newer encodes.

benwaggoner

24th January 2022, 03:15

Does gamma increase at a certain point bother you while watching a movie/tv-series? Can you identify with your open eyes without any tools while watching? Do you every time check you have achieved correct gamma after encoding a video with any tools/soft?

I'm talking'bout gamma increase due to encoding error, wrong parameters/values. How can I prevent this problem? I mean a specific settings/value/command line.
http://xahlee.info/img/what_is_gamma_correction.html
Accidentally changing gamma in encoding isn't that common. A lot more common source of different apparent brightness is a full/limited range mismatch. For example, encoding limited range 16-235 as full range 0-255. That'll yield a file where the darkest pixel is 16 points above actual black, and the brightest 20 below actual white. It also reduces saturation.

Lots of variants of that problem happen all too often, like when a device decodes 16-235 and assumes it is 0-255 RGB. Or when reencoding presuming the wrong source color space.

higher

30th January 2022, 22:01

I have been trying to encode the Taxi Driver UHD Blu-Ray. I have some issues and I think it has to do with the heavy grain. There are cyan patches during the intro sequence in the smoke and also throughout the movie. Please see the linked screenshot which is tonemapped to SDR by madVR. The original doesn't have it and frankly I have seen it before with some encodes. This particular encode is around 40Mbit/s and I used the slow preset. I was trying different aq-modes and strength, tune grain but all have this issue. Is there anything else I could experiment with?

Taxi Driver (https://ibb.co/v15SzkL)

quietvoid

30th January 2022, 22:21

You could try lowering chroma QP offsets (--cbqpoffs, --crqpoffs. try around -2 to -4), but this is really a x265 weakness with heavy grain.
It has no problem retaining the luma noise but chroma noise is just a mess.

higher

31st January 2022, 00:09

Thank you for the suggestion. Unfortunately it made no difference. It's what it is then? :)

RanmaCanada

31st January 2022, 05:59

Thank you for the suggestion. Unfortunately it made no difference. It's what it is then? :)

Sadly yes. I've stated it before and I'll state it again, x265 just can't handle grain as well as x264 can. It has gotten better, but it still can't handle it properly. Maybe AV1 or VVC will be able to match x264 in regards to grain handling once they become mature.

Boulder

31st January 2022, 06:24

Sadly yes. I've stated it before and I'll state it again, x265 just can't handle grain as well as x264 can. It has gotten better, but it still can't handle it properly. Maybe AV1 or VVC will be able to match x264 in regards to grain handling once they become mature.

As far as I've understood, they will happily suck all the details as well.

To the OP: did you use --hdr10-opt? It will do some internal changes in encoding.

higher

31st January 2022, 21:30

Of course I did. Upon further inspection it looks slightly better with --tune-grain but not so much. :)
What are they using for the disc encodes that outperforms x265 in these situations?

microchip8

31st January 2022, 21:57

What are they using for the disc encodes that outperforms x265 in these situations?

Ateme or MainConcept or Beamr

rwill

31st January 2022, 22:37

Sadly yes. I've stated it before and I'll state it again, x265 just can't handle grain as well as x264 can. It has gotten better, but it still can't handle it properly. Maybe AV1 or VVC will be able to match x264 in regards to grain handling once they become mature.

You are mixing standards with standard implementations.

rwill

31st January 2022, 22:38

Ateme or MainConcept or Beamr

Or In-House encoders which are actually tuned for such high rates.

markiemarcus

1st February 2022, 13:29

Thank you for the suggestion. Unfortunately it made no difference. It's what it is then? :)

I've also seen this before. In my experience some AQ modes exacerbate the problem and are more prone to it than others. This actually came up on an animated source in a shot not dissimilar to this.

Edit: It's been a while, but IIRC AQ mode 4 worked well on flat gradients with lots of grain. Have you tried this mode or just 1-3?

RanmaCanada

1st February 2022, 18:48

You are mixing standards with standard implementations.

The only reason I am is because even their current implementations of grain retention, suck hard. Until the individual encoders properly address it, I think it's fair to lump them together.

benwaggoner

1st February 2022, 23:52

excellentswordfight

2nd February 2022, 13:26

Of course I did. Upon further inspection it looks slightly better with --tune-grain but not so much. :)
What are they using for the disc encodes that outperforms x265 in these situations?
Could you upload a 1min sample from the source file and your re-encode?

jauh

3rd February 2022, 07:05

--preset grain needs a complete refactor. I recently had some quality issues with grainy HDR stuff that got a big improvement by removing --tune grain. Some things that work better:
using --rd 4 instead of --rd 6
lowering --ipratio and --pbratio some
Increasing --psy and --psyrdoq some
Using --rskip 2 instead of --rskip 1, and with a lower threshold than the default
A lot of --nr-inter with --nr-intra maybe 25% of that

After a fair bit of experimenting, I've got this to replicate most of the film grain (where the grain is akin to salt-and-pepper noise) on the default preset (so no psyrdoq) for 1080p with reasonable bitrates (6~10Mbps):-

--crf 20 \
--pmode --pme \
--ref 6 --bframes 16 --weightb --deblock -4:-1 --lookahead-slices 1 \
--min-keyint 24 --keyint 240 --b-intra \
--analyze-src-pics --me star --no-early-skip --rskip 0 --fades \
--no-limit-modes --rd 6 --subme 5 --rc-lookahead 240 \
--merange 58 --max-merge 5 --qpstep 1 \
--rd-refine \
--tu-intra-depth 4 --tu-inter-depth 4 \
--psy-rd 4 \
--aq-mode 2 \
--no-cutree \
--ipratio 1.1 \
--no-sao \
--limit-refs 0 \
--qg-size 64

but even with that I'm struggling with "giant grain" like the pilot of TBBT where the rates shoot up past 20Mbps.

I found that lowering --pbratio was just wasting too much bandwidth. The main things that "turned the tide" were --no-sao and --no-cutree (cutree created artefacts where non-patterned objects would have a "noise halo" when they moved against a patterned object (like a door with woodgrain) when film grain was present on the patterned object). I also initially had --psy-rd at 4.5 but dropped it down to 4 as the extra .5 didn't seem to make that much difference.

benwaggoner

4th February 2022, 19:56

@jauh
Did you try my suggestions? If so, with what results?

Also?
What preset are you using?
Are you actually getting a speed increase with --pme? If so, what resolution and how many cores?
How did you come up with --deblock -4:-1?
--refs 6 and especially --bframes 16 seem unlikely to help with grain much. Did you compare with lower values?
Why --analyze-src-pics? What impact did it give? That's normally a performance optimization, not a quality one.
Why --me-range 58 instead of 57?
--qg size 64 is interesting, but makes some sense as it would reduce fine QP variation.
Did you see any benefit from --fades?

Also:
--rd-refine shouldn't do anything in a 1-pass encode
Was even --pbratio 1.3 counterproductive?

jauh

5th February 2022, 04:49

@benwaggoner:-

I made modifications to my usual config off your suggestions using the "scientific poke" method, but not tried any one specific in isolation. I'm running on the default preset, i. e. no explicit --preset option.

It's 1080p material, and on a 16C/32T CPU pme makes a huge difference in churning frames (sub-1fps to over 2fps).

Deblock was a trial and error based on a scene where over-blurring/blocking was evident, so I just fiddled till the balance was right.

--ref 6 was a hang-over from non-grain/noise material, and you're right, if the material is sufficiently noisy you're not going to get many back refs, but in that case it should make no difference what the number is anyway

--bframes 16 is purely an economics decision based on the cost of signalling of B frames (for most part, I get 3--5 consecutive B frames, but even with the fractional percentage of double-digit consecutive B frames you end up saving some space)

I'm all up for squeezing even fractional performance, so --analyse-src-pics is in ;)

--me-range is a 58 because the -1 is a penalty for hex, since I'm using star there's no point penalising self (although I'm experimenting with hme, see later on).

--rd-refine is doing something even in 1-pass, not certain whether what it's doing is necessarily good or bad though:-

with --rd-refine

x265 [info]: frame I: 8, Avg QP:18.69 kb/s: 35639.33
x265 [info]: frame P: 88, Avg QP:20.08 kb/s: 19036.22
x265 [info]: frame B: 404, Avg QP:21.80 kb/s: 5604.48
x265 [info]: Weighted P-Frames: Y:1.1% UV:1.1%
x265 [info]: Weighted B-Frames: Y:3.0% UV:1.0%
x265 [info]: consecutive B-frames: 8.3% 7.3% 2.1% 26.0% 15.6% 16.7% 8.3% 11.5% 1.0% 0.0% 0.0% 0.0% 0.0% 1.0% 0.0% 0.0% 2.1%

encoded 500 frames in 202.93s (2.46 fps), 8449.02 kb/s, Avg QP:21.45

without --rd-refine

x265 [info]: frame I: 8, Avg QP:18.67 kb/s: 35633.43
x265 [info]: frame P: 88, Avg QP:20.38 kb/s: 19123.22
x265 [info]: frame B: 404, Avg QP:21.88 kb/s: 5997.30
x265 [info]: Weighted P-Frames: Y:1.1% UV:1.1%
x265 [info]: Weighted B-Frames: Y:3.0% UV:1.0%
x265 [info]: consecutive B-frames: 8.3% 7.3% 2.1% 26.0% 15.6% 16.7% 8.3% 11.5% 1.0% 0.0% 0.0% 0.0% 0.0% 1.0% 0.0% 0.0% 2.1%

encoded 500 frames in 183.40s (2.73 fps), 8781.64 kb/s, Avg QP:21.57

I had a scene where an opening door cast a progressing shadow on a wall, tried increasing crf till x265 produced blocking artefacts, that happened at crf 22, so I dropped pbratio to even 1.0 and that made no difference at crf 22, at crf 21, pbratio of 1.4 was adequate, so I didn't really bother fiddling with that. x265 has a weird rate control mechanism:- I noticed that when I dropped ipratio, the average bitrate of the completed encode didn't change much (<<10%), the thing that did change was the average cost of each type of frame, go figure!

The "see later on" part:-
I've noticed on one of the encodes that grain/noise over a patterned object (again, think door with woodgrain) resulted in a paintbrush smear effect when the camera strictly tilted up during a scene, hme seems to have mitigated that somewhat, but I'm still working on eliminating that completely (not present in the original encode) without throwing a ton of bandwidth at it.

edit:- the culprit of the paintbrush smear effect seem to have been --limit-tu 0, and the effect is gone even at --limit-tu 1, the flip side of increasing --limit-tu is a bump in the average signalling cost

limit-tu 0, paintbrush smear:-

x265 [info]: frame I: 7, Avg QP:18.43 kb/s: 34261.16
x265 [info]: frame P: 95, Avg QP:19.89 kb/s: 21423.34
x265 [info]: frame B: 398, Avg QP:21.42 kb/s: 6820.33
x265 [info]: Weighted P-Frames: Y:2.1% UV:2.1%
x265 [info]: Weighted B-Frames: Y:3.0% UV:1.3%
x265 [info]: consecutive B-frames: 7.8% 4.9% 2.9% 37.3% 12.7% 18.6% 9.8% 2.9% 0.0% 0.0% 0.0% 0.0% 0.0% 1.0% 0.0% 0.0% 2.0%

encoded 500 frames in 319.25s (1.57 fps), 9979.07 kb/s, Avg QP:21.09

limit-tu 1, no paintbrush smear:-

x265 [info]: frame I: 7, Avg QP:18.43 kb/s: 34261.16
x265 [info]: frame P: 95, Avg QP:19.84 kb/s: 22190.47
x265 [info]: frame B: 398, Avg QP:21.27 kb/s: 7802.09
x265 [info]: Weighted P-Frames: Y:2.1% UV:2.1%
x265 [info]: Weighted B-Frames: Y:3.0% UV:1.3%
x265 [info]: consecutive B-frames: 7.8% 4.9% 2.9% 37.3% 12.7% 18.6% 9.8% 2.9% 0.0% 0.0% 0.0% 0.0% 0.0% 1.0% 0.0% 0.0% 2.0%

encoded 500 frames in 317.05s (1.58 fps), 10906.31 kb/s, Avg QP:20.96

higher

6th February 2022, 14:45

--preset grain needs a complete refactor. I recently had some quality issues with grainy HDR stuff that got a big improvement by removing --tune grain. Some things that work better:
using --rd 4 instead of --rd 6
lowering --ipratio and --pbratio some
Increasing --psy and --psyrdoq some
Using --rskip 2 instead of --rskip 1, and with a lower threshold than the default
A lot of --nr-inter with --nr-intra maybe 25% of that

A lof of suggestions to try. Need to decrypt the disc again to try these as I already deleted the original.

By the way some of the Amazon Video 4K HDR encodes also have these cyan patches but even on light grain. Recently I have noticed it on Reacher but I remember the 4K HDR version of Wrath of Man on Amazon also have these discolorations unlike the UHD disc version.
Are you using x265 for 4K HDR on Amazon Video, Ben?

Reacher (https://ibb.co/pjy68Lc) - screenshot tonemapped to SDR by MadVR; I have a Prime sub but I had to download it from elsewhere to be able to make screens. :)

Check the wall under the picture and the sheets on the table. Looks quite ugly.

Prime Video's 4K bitrates of 14-15 Mbit/sec should also be higher. It doesn't really cut it with grainy, high motion materials.

Look at this mess. (https://ibb.co/MRNnK8s) There's hardly any motion here yet it looks terrible.

AppleTV+ is around 23-24 Mbit/sec and it definitely shows.

Boulder

6th February 2022, 15:29

The "see later on" part:-
I've noticed on one of the encodes that grain/noise over a patterned object (again, think door with woodgrain) resulted in a paintbrush smear effect when the camera strictly tilted up during a scene, hme seems to have mitigated that somewhat, but I'm still working on eliminating that completely (not present in the original encode) without throwing a ton of bandwidth at it.

edit:- the culprit of the paintbrush smear effect seem to have been --limit-tu 0, and the effect is gone even at --limit-tu 1, the flip side of increasing --limit-tu is a bump in the average signalling cost

The combination of rskip mode 2, CTU 64 and --limit-tu 0 is broken, and I'd expect rskip mode 0 to be broken as well since it means just disabling all recursion skipping instead of limiting it (mode 2).

https://forum.doom9.org/showthread.php?p=1919347#post1919347

Kuler087

6th February 2022, 15:51

Reacher (https://ibb.co/pjy68Lc) - screenshot tonemapped to SDR by MadVR :)

Check the wall under the picture and the sheets on the table. Looks quite ugly.

yep, amazon HEVC encoding is horrible. Much worse than Netflix and Disney+ while using roughly the same bitrate

jauh

6th February 2022, 20:04

The combination of rskip mode 2, CTU 64 and --limit-tu 0 is broken, and I'd expect rskip mode 0 to be broken as well since it means just disabling all recursion skipping instead of limiting it (mode 2).

https://forum.doom9.org/showthread.php?p=1919347#post1919347

I've tested your hypothesis with --rskip 1 and --limit-tu 0, and while the smear effect is considerably milder (if I hadn't known to look for it, chances are, I'd not have spotted it) it's still there, so it's likely that there's more than just that combination of switches.

Boulder

6th February 2022, 20:26

I've tested your hypothesis with --rskip 1 and --limit-tu 0, and while the smear effect is considerably milder (if I hadn't known to look for it, chances are, I'd not have spotted it) it's still there, so it's likely that there's more than just that combination of switches.

Try switching to CTU 32 and I'm sure it will look better.

jauh

6th February 2022, 23:24

Try switching to CTU 32 and I'm sure it will look better.

I'll give that a try, but I think it's chasing the wrong goal: rskip >0 gives an encoding speed improvement at the cost of coding efficiency, and lowering CTU to 32 lowers the latter further, whereas limit-tu 1 stops recursion based on the relative cost of a TU split. So, so far as I can tell you get better, albeit slower, coding efficiency at CTU=64 with rskip=0 and limit-tu=1 cf. CTU=32, rskip=1, and limit-tu=0.

I could be wrong, so I'll try it, just need to wait for the current encode to finish and it's chugging along at 1.43fps...

jauh

7th February 2022, 02:12

Try switching to CTU 32 and I'm sure it will look better.

So, I gave CTU=32 a go, and as you hypothesised, no paintbrush smear, but:-

CTU=64,QG=64,rskip=0,limit-tu=1:

x265 [info]: frame I: 7, Avg QP:18.46 kb/s: 33742.92
x265 [info]: frame P: 95, Avg QP:19.68 kb/s: 21594.99
x265 [info]: frame B: 398, Avg QP:21.23 kb/s: 7305.39
x265 [info]: Weighted P-Frames: Y:2.1% UV:2.1%
x265 [info]: Weighted B-Frames: Y:3.0% UV:1.3%
x265 [info]: consecutive B-frames: 7.8% 4.9% 2.9% 37.3% 12.7% 18.6% 9.8% 2.9% 0.0% 0.0% 0.0% 0.0% 0.0% 1.0% 0.0% 0.0% 2.0%
encoded 500 frames in 366.59s (1.36 fps), 10390.54 kb/s, Avg QP:20.89

CTU=32,QG=32,rskip=1,limit-tu=0:

x265 [info]: frame I: 7, Avg QP:18.46 kb/s: 33622.09
x265 [info]: frame P: 95, Avg QP:19.60 kb/s: 22259.21
x265 [info]: frame B: 398, Avg QP:21.10 kb/s: 8580.94
x265 [info]: Weighted P-Frames: Y:2.1% UV:2.1%
x265 [info]: Weighted B-Frames: Y:3.0% UV:1.3%
x265 [info]: consecutive B-frames: 7.8% 4.9% 2.9% 37.3% 12.7% 18.6% 9.8% 2.9% 0.0% 0.0% 0.0% 0.0% 0.0% 1.0% 0.0% 0.0% 2.0%
encoded 500 frames in 374.67s (1.33 fps), 11530.39 kb/s, Avg QP:20.78

CTU=32,QG=32,rskip=0,limit-tu=0:

x265 [info]: frame I: 7, Avg QP:18.46 kb/s: 33622.09
x265 [info]: frame P: 95, Avg QP:19.61 kb/s: 22226.53
x265 [info]: frame B: 398, Avg QP:21.10 kb/s: 8577.44
x265 [info]: Weighted P-Frames: Y:2.1% UV:2.1%
x265 [info]: Weighted B-Frames: Y:3.0% UV:1.3%
x265 [info]: consecutive B-frames: 7.8% 4.9% 2.9% 37.3% 12.7% 18.6% 9.8% 2.9% 0.0% 0.0% 0.0% 0.0% 0.0% 1.0% 0.0% 0.0% 2.0%
encoded 500 frames in 372.18s (1.34 fps), 11521.40 kb/s, Avg QP:20.78

So it does help, but at a cost... In the circumstances, I think CTU=QG=64,rskip=0,limit-tu=1 is the optimal variant...

Boulder

7th February 2022, 06:07

asarian

7th February 2022, 09:03

jauh

7th February 2022, 10:48

Looks like all recent x265 binaries are broken. Whichever one I use, keeps saying

y4m [info]: 3840x2160 fps 24000/1001 i420p10 unknown frame count
raw [info]: output file: q:\video\arti.hevc
x265 [info]: HEVC encoder version 3.5+20-17839cc0d
x265 [info]: build info [Windows][GCC 11.2.0][64 bit] 10bit

(With --frames on the command line, of course).

Only the yuuki one I still have, from April last year, still shows the frames. Did something change?

It's not a bug, it's literally telling you that the pipe input doesn't have a clue about how many frames it has (how would a pipe know how many frames exist?) The progress bar will display [n/specified frames] because that's how many frames x265 is expecting. If you specify a yuv/y4m file (not a pipe) x265 will work out how many frames the file has based on the file size. At least that's my experience.

jauh

7th February 2022, 10:52

If the filesize bothers you, you can always raise CRF to compensate. The fact just is that CTU 64 has these strange side effects in x265. Personally I go for quality first while the x265 development has gone size first :p

I think I was being too laconic which led to a possible confusion: CTU=QG=32 eliminates the paintbursh smear with limit-tu=0 and rskip!=1, but visually, the result is indistinguishable from CTU=QG=64 with limit-tu=1 and rskip=0. The size of 32 only increases the signalling cost. ;)

asarian

7th February 2022, 10:54

It's not a bug, it's literally telling you that the pipe input doesn't have a clue about how many frames it has (how would a pipe know how many frames exist?) The progress bar will display [n/specified frames] because that's how many frames x265 is expecting. If you specify a yuv/y4m file (not a pipe) x265 will work out how many frames the file has based on the file size. At least that's my experience.

Oh, it's a bug alright. :)

The pipe knows, the way it always does: because I tell it how many frames there are (like I said); like:

VSPipe -c y4m "f:\jobs\contact.vpy" - | x265 --y4m --input - --preset medium --input-depth 8 --output-depth 10 --crf 10 --frames 215323 --output "g:\video\contact.hevc"

It's worked like that for years.

jauh

7th February 2022, 11:00

Oh, it's a bug alright. :)

The pipe knows, the way it always does: because I tell it how many frames there are (like I said); like:

VSPipe -c y4m "f:\jobs\contact.vpy" - | x265 --y4m --input - --preset medium --input-depth 8 --output-depth 10 --crf 10 --frames 215323 --output "g:\video\contact.hevc"

It's worked like that for years.

--frames tells x265 how many frames to expect to encode (so you get a % progress report), a pipe can never know how many things will be fed to it in the future, it only knows what it's been fed so far.

asarian

7th February 2022, 11:14

--frames tells x265 how many frames to expect to encode (so you get a % progress report), a pipe can never know how many things will be fed to it in the future, it only knows what it's been fed so far.

Exactly. --frames is parsed to x265. Look how yuuki build does it:

y4m [info]: 3840x2160 fps 25/1 i420p10 frames 0 - 10194 of 10195
x265 [info]: Using preset medium & tune none
raw [info]: output file: q:\video\opera.hevc
x265 [info]: HEVC encoder version 3.5+2-g2b25c9ba0+45
x265 [info]: build info [Windows][GCC 10.2.0][64 bit] Yuuki 10bit

And as long as memory serves, x265 has always shown the frames like that, whatever build (until recently, that is).

Boulder

7th February 2022, 11:43

I think I was being too laconic which led to a possible confusion: CTU=QG=32 eliminates the paintbursh smear with limit-tu=0 and rskip!=1, but visually, the result is indistinguishable from CTU=QG=64 with limit-tu=1 and rskip=0. The size of 32 only increases the signalling cost. ;)

CTU 64 will make a mess out of noisy flat backgrounds compared to CTU 32 (qg-size 32 in both cases since x265 doesn't use 64 by default for CTU 64).

Boulder

7th February 2022, 11:44

Exactly. --frames is parsed to x265. Look how yuuki build does it:

y4m [info]: 3840x2160 fps 25/1 i420p10 frames 0 - 10194 of 10195
x265 [info]: Using preset medium & tune none
raw [info]: output file: q:\video\opera.hevc
x265 [info]: HEVC encoder version 3.5+2-g2b25c9ba0+45
x265 [info]: build info [Windows][GCC 10.2.0][64 bit] Yuuki 10bit

And as long as memory serves, x265 has always shown the frames like that, whatever build (until recently, that is).

That build probably has a patch applied, in vanilla x265 it's always been like it is now as far as I can remember.

I don't know why it is so important to you since the only place where it really matters is the ETA calculation which works regardless of the piping info string.

jauh

7th February 2022, 11:49

CTU 64 will make a mess out of noisy flat backgrounds compared to CTU 32 (qg-size 32 in both cases since x265 doesn't use 64 by default for CTU 64).

It doesn't, that's what I mean by visually indistinguishable.

Boulder

7th February 2022, 12:08

It doesn't, that's what I mean by visually indistinguishable.

Do you have any samples? It would be interesting to see them as my experiences are the exact opposite, also based on recent tests.

jauh

7th February 2022, 12:38

Do you have any samples? It would be interesting to see them as my experiences are the exact opposite, also based on recent tests.

TBBT season 1 episode 2 is great with (the first 500 frames are more than enough to see that flat noisy areas are just fine):-

--crf 20 \
--ref 6 --limit-refs 0 \
--bframes 16 --weightb --b-intra --b-adapt 2 \
--lookahead-slices 1 --rc-lookahead 240 \
--min-keyint 24 --keyint 240 \
--pmode --pme \
--rect --amp --no-limit-modes \
--me star --merange 58 --subme 5 --max-merge 5 \
--analyze-src-pics --no-early-skip --rskip 0 --fades \
--tu-intra-depth 4 --tu-inter-depth 4 --limit-tu 1 \
--rdoq-level 0 --psy-rd 4 --rd-refine \
--deblock -2:-1 \
--no-cutree \
--aq-mode 2 --aq-strength 1.0 \
--no-sao \
--ctu 64 --qg-size 64 --qpstep 8 \
--opt-qp-pps --opt-ref-list-length-pps ...

Boulder

7th February 2022, 13:16

TBBT season 1 episode 2 is great with (the first 500 frames are more than enough to see that flat noisy areas are just fine):-

--crf 20 \
--ref 6 --limit-refs 0 \
--bframes 16 --weightb --b-intra --b-adapt 2 \
--lookahead-slices 1 --rc-lookahead 240 \
--min-keyint 24 --keyint 240 \
--pmode --pme \
--rect --amp --no-limit-modes \
--me star --merange 58 --subme 5 --max-merge 5 \
--analyze-src-pics --no-early-skip --rskip 0 --fades \
--tu-intra-depth 4 --tu-inter-depth 4 --limit-tu 1 \
--rdoq-level 0 --psy-rd 4 --rd-refine \
--deblock -2:-1 \
--no-cutree \
--aq-mode 2 --aq-strength 1.0 \
--no-sao \
--ctu 64 --qg-size 64 --qpstep 8 \
--opt-qp-pps --opt-ref-list-length-pps ...

I'd rather you post some samples so the rest of us don't need to dig the source from anywhere. I can myself compare on the Chernobyl sample I posted in that linked answer earlier and which I have already available.

AQ-mode 2 is already something I avoid along with a large max-merge. I tested disabling cu-tree just a while ago and it caused quite a lot of lost details in places. Of course, these are all subjective things.

jauh

7th February 2022, 13:20

I'd rather you post some samples so the rest of us don't need to dig the source from anywhere. I can myself compare on the Chernobyl sample I posted in that linked answer earlier and which I have already available.

AQ-mode 2 is already something I avoid along with a large max-merge. I tested disabling cu-tree just a while ago and it caused quite a lot of lost details in places. Of course, these are all subjective things.

What can take a 43MB APNG file? You've disabled things that I've expressly enabled on purpose and vice versa, of course your results are going to be wildly different to mine...

edit: here you go:-
edit 2: uploaded a longer clip (5s) with a scene change so that ffmpeg has to render all frames and not stop abruptly without properly resolving all references

https://ufile.io/pusv5xt0

Boulder

7th February 2022, 14:36

What can take a 43MB APNG file? You've disabled things that I've expressly enabled on purpose and vice versa, of course your results are going to be wildly different to mine...

edit: here you go:-
edit 2: uploaded a longer clip (5s) with a scene change so that ffmpeg has to render all frames and not stop abruptly without properly resolving all references

https://ufile.io/pusv5xt0

Why not just regular source and result samples?

jauh

7th February 2022, 14:44

Why not just regular source and result samples?

What do you expect to see??? It's far simpler for you to encode your problematic scene with the options that I specified and see if it makes a difference, is it not?

Boulder

7th February 2022, 15:06

Then, please provide an unprocessed source as is.

jauh

7th February 2022, 15:16

Then, please provide an unprocessed source as is.

Why? I'm not making an archive copy of the BD (I have the BD for that!), is there anything in the clip that worsens your psychovisual experience or is it that it's not got the artefacts that you expected and you now want to knit pick that tiniest portions of grain are out of phase? But here, I've no dog in the fight:-

https://ufile.io/01rl8ik1

I'll even tell you the clip zone for the episode:- TBBT S1 E2 @

-ss 00:00:08 -t 5 -vf crop=600:200:1200:0

Boulder

7th February 2022, 15:37

Why? I'm not making an archive copy of the BD (I have the BD for that!), is there anything in the clip that worsens your psychovisual experience or is it that it's not got the artefacts that you expected and you now want to knit pick that tiniest portions of grain are out of phase? But here, I've no dog in the fight:-

https://ufile.io/01rl8ik1

I'll even tell you the clip zone for the episode:- TBBT S1 E2 @

-ss 00:00:08 -t 5 -vf crop=600:200:1200:0

I'm not going to investigate ways of opening APNG files in Avisynth, simple as that. As I've said, I have my preferences and you probably have different ones. I am just interested in seeing what kind of results your settings produce on the source compared to mine - maybe there is something to look deeper in.

jauh

7th February 2022, 15:48

I'm not going to investigate ways of opening APNG files in Avisynth, simple as that. As I've said, I have my preferences and you probably have different ones. I am just interested in seeing what kind of results your settings produce on the source compared to mine - maybe there is something to look deeper in.

You don't need to use Avisynth for APNG, MS Edge plays them fine (drag and drop), and so would any Chromium clone, by extension, I suspect... And if you really wanted to see the difference between the two settings, there's literally nothing stopping you from encoding a scene of your choosing from your own sources and seeing the difference...

Boulder

7th February 2022, 15:55

You don't need to use Avisynth for APNG, MS Edge plays them fine (drag and drop), and so would any Chromium clone, by extension, I suspect... And if you really wanted to see the difference between the two settings, there's literally nothing stopping you from encoding a scene of your choosing from your own sources and seeing the difference...

Well, the point was to take a look at your results as they often depend on the source. And as I mentioned, I can compare things on the couple of sample clips I have ready.

Boulder

7th February 2022, 16:23

TBBT season 1 episode 2 is great with (the first 500 frames are more than enough to see that flat noisy areas are just fine):-

--crf 20 \
--ref 6 --limit-refs 0 \
--bframes 16 --weightb --b-intra --b-adapt 2 \
--lookahead-slices 1 --rc-lookahead 240 \
--min-keyint 24 --keyint 240 \
--pmode --pme \
--rect --amp --no-limit-modes \
--me star --merange 58 --subme 5 --max-merge 5 \
--analyze-src-pics --no-early-skip --rskip 0 --fades \
--tu-intra-depth 4 --tu-inter-depth 4 --limit-tu 1 \
--rdoq-level 0 --psy-rd 4 --rd-refine \
--deblock -2:-1 \
--no-cutree \
--aq-mode 2 --aq-strength 1.0 \
--no-sao \
--ctu 64 --qg-size 64 --qpstep 8 \
--opt-qp-pps --opt-ref-list-length-pps ...

Are you sure you didn't mess something up in those settings? I did a test encode using them and using my baseline, and your encode is a lot smaller and looks just really ugly in motion. I did compare the MediaInfo output to make sure I didn't misplace something, but that was not the case.

Mine: https://drive.google.com/file/d/1JvmaK9YKUGqcDdd_J2fTvGb8z63yCJox/view?usp=sharing
Yours: https://drive.google.com/file/d/1qP0-McP8aJzC6M_URd2kf_BvHVQpBwzJ/view?usp=sharing

My settings from MediaInfo: https://pastebin.com/bsqX1S4c

jauh

7th February 2022, 18:12

Are you sure you didn't mess something up in those settings? I did a test encode using them and using my baseline, and your encode is a lot smaller and looks just really ugly in motion. I did compare the MediaInfo output to make sure I didn't misplace something, but that was not the case.

Mine: https://drive.google.com/file/d/1JvmaK9YKUGqcDdd_J2fTvGb8z63yCJox/view?usp=sharing
Yours: https://drive.google.com/file/d/1qP0-McP8aJzC6M_URd2kf_BvHVQpBwzJ/view?usp=sharing

My settings from MediaInfo: https://pastebin.com/bsqX1S4c

Well, for starters you're massively(!) overcompensating on chroma QP offsets (I have no idea what effect that has on CRF decisions) whereas I'm not, you also set frame-threads=4, I do not, and claim they are my settings.

Conversely, your "default" is at CRF=18, mine's at 20, c'mon, that's not even like for like, my settings explode to similar bitrates at CRF 18, so if you're going to compare and whine at least try to compare like for like!

edit: as it happens, I have Chernobyl, so just tell me what episode and what frame range that clip is, and I'll do it on my end...

Boulder

7th February 2022, 18:50

Well, for starters you're massively(!) overcompensating on chroma QP offsets (I have no idea what effect that has on CRF decisions) whereas I'm not, you also set frame-threads=4, I do not, and claim they are my settings.

Conversely, your "default" is at CRF=18, mine's at 20, c'mon, that's not even like for like, my settings explode to similar bitrates at CRF 18, so if you're going to compare and whine at least try to compare like for like!

Chill out, even if this is the internet. The point has been comparing the results, which I did and I was only asking if the settings are correct.

Besides, you said that those setting produce a satisfying output, so naturally I try them first. They work much better with my Hot Fuzz (crisp film) and The Hobbit (all CGI) samples but there's a lot of motion there or very little noise or actual detail so it is much harder to find out the problems. With x265, the hardest parts are flat, darker coloured surfaces since the floating noise patterns start appearing quite easily, especially with a lower quality source where some might already be present.

Your settings are so different that the same CRF does not apply. In x264, there is a clear correlation; basically more effective options just reduce the filesize and keep a similar quality level, but it's not there in x265. Just to please you, I did try CRF 18 and it produced a much bigger file than CRF 20, but the defects are still there if less obvious. The difference between the base encodes probably comes mostly from the AQ mode. I did test aq-mode 2 with my settings and it fails to produce a satisfying result and the filesize is a lot smaller than with mode 1.

The chroma QP offsets do very little to the average bitrate. The difference when removing the offset is -2,3% in this case.

Frame threads from 4 to 1 affect the filesize by a whopping 0,01% in this case so it's meaningless and makes the encoding only slower.

One final problem with your settings is that they are mostly so placebo that the encoding gets dead slow, 2.45 fps vs 7.09 fps. Add any filtering to the formula and it's definitely not for everyday use.

Here are the original clips I've got:
Hot Fuzz sample: https://drive.google.com/file/d/19cIzFGAocSHYlBNJdZJ08OaQbfBIHUjm/view?usp=sharing
Chernobyl sample: https://drive.google.com/file/d/1nEa9H70d7Ekeue2omnwLDToVK7LaxZ_t/view?usp=sharing
The Hobbit sample: https://drive.google.com/file/d/1LHYBAHsKLAeHuSYu79r_uoRTM4iyjC6G/view?usp=sharing

jauh

7th February 2022, 18:59

Chill out, even if this is the internet. The point has been comparing the results, which I did and I was only asking if the settings are correct.

Besides, you said that those setting produce a satisfying output, so naturally I try them first. They work much better with my Hot Fuzz (crisp film) and The Hobbit (all CGI) samples but there's a lot of motion there or very little noise or actual detail so it is much harder to find out the problems. With x265, the hardest parts are flat, darker coloured surfaces since the floating noise patterns start appearing quite easily, especially with a lower quality source where some might already be present.

Your settings are so different that the same CRF does not apply. In x264, there is a clear correlation; basically more effective options just reduce the filesize and keep a similar quality level, but it's not there in x265. Just to please you, I did try CRF 18 and it produced a much bigger file than CRF 20, but the defects are still there if less obvious. The difference between the base encodes probably comes mostly from the AQ mode. I did test aq-mode 2 with my settings and it fails to produce a satisfying result and the filesize is a lot smaller than with mode 1.

The chroma QP offsets do very little to the average bitrate. The difference when removing the offset is -2,3% in this case.

Frame threads from 4 to 1 affect the filesize by a whopping 0,01% in this case so it's meaningless and makes the encoding only slower.

One final problem with your settings is that they are mostly so placebo that the encoding gets dead slow, 2.45 fps vs 7.09 fps. Add any filtering to the formula and it's definitely not for everyday use.

Here are the original clips I've got:
Hot Fuzz sample: https://drive.google.com/file/d/19cIzFGAocSHYlBNJdZJ08OaQbfBIHUjm/view?usp=sharing
Chernobyl sample: https://drive.google.com/file/d/1nEa9H70d7Ekeue2omnwLDToVK7LaxZ_t/view?usp=sharing
The Hobbit sample: https://drive.google.com/file/d/1LHYBAHsKLAeHuSYu79r_uoRTM4iyjC6G/view?usp=sharing

Those settings are completely inappropriate for crips/cgi films, I'd definitely not use them for that!

But given that the episodes of TBBT on my settings are churning out at about 1MBps (8000kbps) and grain/noise is well preserved, I don't know what you're doing on your end but the SEI info in cherno_jauh.hevc is that it's not what I'm doing, which is why I said give me the episode number and frame range and I'll encode it on my end... <-- never mind, found it, I'll encode the scene (together with scene cuts) next.

Edit: are you also saying that you're filtering prior to encoding???

Boulder

7th February 2022, 19:58

Edit: are you also saying that you're filtering prior to encoding???

Not in these tests, but normally yes. Ultra light denoising and usually downscaling to 720p (HD) or 1080p/1440p (UHD).

benwaggoner

8th February 2022, 00:31

So, I gave CTU=32 a go, and as you hypothesised, no paintbrush smear, but:-

So it does help, but at a cost... In the circumstances, I think CTU=QG=64,rskip=0,limit-tu=1 is the optimal variant...
You'll want to increase --tu-intra-depth and --tu-inter-depth by 1 for --ctu 64 versus --ctu 32, so you can have the same minimum TU size.

--qg-size 64 seems too big to be actually optimal, unless you're going for very little adaptive quant. I've never seen 64/64 be better than 32/32 , and 32/32 can certainly be better than 64/64.

If you're doing --csv-log-level 2, you'll get per frame breakdowns of block types. Comparing different --limit-tu and --tu-*-depth modes can yield some interesting results in that data, which correlate with visible results in even more interesting ways.

jauh

8th February 2022, 04:03

Chill out, even if this is the internet. The point has been comparing the results, which I did and I was only asking if the settings are correct.

<snip>

So I literally have no idea how you used my settings and got a clip at around 2.7MB where as when I encoded the same clip (well, you didn't give me the start frame and duration so I had to guess, I also noticed your encode is at 25fps where as my BD source is 24000/1001 (so actually fewer frames than you would've needed), my encode starts from frame 52600 and lasts 600 frames) from BD source and got a much bigger (~13.5MB) file...

https://ufile.io/sjwmnuqu

The slight wobble of the two lines of the panel on the door (top-centre frame, from 10s till 17s) looks like a bug in x265 that my settings hit and yours don't, because I can't think of a reason why things would move in a frame during coding when they actually haven't moved at all in the master...

As for grinding to a halt, you can remove --pmode --pme --rect and --amp if you don't like the speed, I don't mind them as otherwise my cores sit there twiddling their... bits...

@benwaggoner
just for completeness I ran an encode with --qg-size 32 as well, that resulted in a slightly lower signalling cost, but still with the wobble:

https://ufile.io/zmdq52d7

Boulder

8th February 2022, 06:40

So I literally have no idea how you used my settings and got a clip at around 2.7MB where as when I encoded the same clip (well, you didn't give me the start frame and duration so I had to guess, I also noticed your encode is at 25fps where as my BD source is 24000/1001 (so actually fewer frames than you would've needed), my encode starts from frame 52600 and lasts 600 frames) from BD source and got a much bigger (~13.5MB) file...
Different releases so there might be a big difference in bitrate. Naturally this can affect the source quality quite a lot as well and present problems to the encoder.

The slight wobble of the two lines of the panel on the door (top-centre frame, from 10s till 17s) looks like a bug in x265 that my settings hit and yours don't, because I can't think of a reason why things would move in a frame during coding when they actually haven't moved at all in the master...

That's a side effect of the "floating noise" problem. If you look at my original sample clip, the lines already flicker slightly in it and encoding just amplifies it a lot.

Aq-mode > 1 is very prone to make it happen. I think the low-frequency noise is a hard case for those auto variance methods and the encoder just quantizes those CTUs to death. If I use my settings but change to aq-mode 2, the bitrate drops by ~30% at the same CRF so they are not very comparable. Then, in the Hobbit and Hot Fuzz clips, it overshoots the bitrate (+29% and +24%) without bringing anything extra to the overall quality.

I still don't understand why that mode was made default at the last big changes they made. They didn't even do anything to the actual method, just changed the default value.

jauh

8th February 2022, 13:03

<snip>

That's a side effect of the "floating noise" problem. If you look at my original sample clip, the lines already flicker slightly in it and encoding just amplifies it a lot.

<snip>

Try this one, see what you think:

https://ufile.io/l5o1hv52

Boulder

8th February 2022, 17:52

benwaggoner

9th February 2022, 02:11

Increasing --psy and --psy-rdoq can help with grain, as they preserve the energy and the general texture. That can help with visible banding, grain swirling, etcetera.

I suggest always testing with 2-pass VBR with grain tuning, as lots of the knobs to tweak can change both file size and CRF requirements significantly.

jauh

9th February 2022, 13:44

Definitely looks better. Your source probably has higher frequency grain, at least it looks like that so it won't start floating nearly as bad as the low freq type. Still the aq-mode 2 induced static areas are there, but they are not as apparent. I don't know how much the psychovisual options could help with that though, I've never really tested switching psy-rdoq off like you have and using only psy-rd. I borrowed my psy settings from littlepox's "tune film" thread.

It's the same source as produced noticeable wobble with other settings. The switch that made the difference was --aq-motion.

Increasing --psy and --psy-rdoq can help with grain, as they preserve the energy and the general texture. That can help with visible banding, grain swirling, etcetera.

I suggest always testing with 2-pass VBR with grain tuning, as lots of the knobs to tweak can change both file size and CRF requirements significantly.

In the x264 days, I used to do a 1st pass CRF then a second pass using the ABR from the CRF pass, I tried the same with x265's 3.5+20 version, but every single encode I tried, the video starts to frame skip at non-deterministic timecodes... Is this a known issue?

Boulder

9th February 2022, 16:42

In the x264 days, I used to do a 1st pass CRF then a second pass using the ABR from the CRF pass, I tried the same with x265's 3.5+20 version, but every single encode I tried, the video starts to frame skip at non-deterministic timecodes... Is this a known issue?

I've done this and it did work, albeit not with a recent build. It definitely should work since the stats file is the only thing used by the following passes and the rest comes from your source anyway.

benwaggoner

9th February 2022, 23:23

It's the same source as produced noticeable wobble with other settings. The switch that made the difference was --aq-motion.
Yeah, --aq-motion was an experiment abandoned years ago. It was introduced back five years ago in x265 2.3, and I don't know that it ever got any more engineering work. I'd anticipate some weird and suboptimal behaviors with changes and new parameters introduced since. And it started weird and suboptimal.

--aq-motion has the Experimental Feature warning for a reason! It never got close to something that could be used as a default setting. --aq-mode 4 gives better and much more reliable improvements.

benwaggoner

9th February 2022, 23:24

In the x264 days, I used to do a 1st pass CRF then a second pass using the ABR from the CRF pass, I tried the same with x265's 3.5+20 version, but every single encode I tried, the video starts to frame skip at non-deterministic timecodes... Is this a known issue?
I don't know what the issue is, but I'd recommend against it. Reusing first pass data is okay if only bitrate is changing, but things can diverge when other parameters are being played with.

jauh

10th February 2022, 19:09

Yeah, --aq-motion was an experiment abandoned years ago. It was introduced back five years ago in x265 2.3, and I don't know that it ever got any more engineering work. I'd anticipate some weird and suboptimal behaviors with changes and new parameters introduced since. And it started weird and suboptimal.

--aq-motion has the Experimental Feature warning for a reason! It never got close to something that could be used as a default setting. --aq-mode 4 gives better and much more reliable improvements.

That's the thing: --aq-motion is the thing that stopped the wobble!

I don't know what the issue is, but I'd recommend against it. Reusing first pass data is okay if only bitrate is changing, but things can diverge when other parameters are being played with.

I feared someone might say something along those lines.

I'm trialling a new strategy: avoiding psy altogether and going for plain CQP.

jauh

13th February 2022, 13:41

Does x265 have a calculation error in PSNR? For example in the Y-plane, encoder.cpp has (just grepping through source; similar for Cr and Cb planes):

int maxvalY = 255 << (X265_DEPTH - 8);
...
double refValueY = (double)maxvalY * maxvalY * size;
...
psnrY = (ssdY ? 10.0 * log10(refValueY / (double)ssdY) : 99.99);

except that in material with Bt.709 limited range, none of the planes can ever swing the full range of byte values, so does MAX_I not end up being off by quite a bit, esp. in the Y plane?

rwill

13th February 2022, 15:50

Does x265 have a calculation error in PSNR?

No. It does not.

jauh

13th February 2022, 16:14

No. It does not.

How so, when max effective value (nominal peak - black) for the Y plane, for example, is 219 for 8bit (cf. 255) and 876 for 10bit (cf. 1024)?

rwill

13th February 2022, 21:56

How so, when max effective value (nominal peak - black) for the Y plane, for example, is 219 for 8bit (cf. 255) and 876 for 10bit (cf. 1024)?

Thats not how it works you know...

The 255 << (BITDEPTH-8) comes from the reference software.
One could argue that it should be (1<<BITDEPTH)-1 but what you are proposing is .. I don't know what to write.. baseless ?

jauh

14th February 2022, 03:08

Thats not how it works you know...

The 255 << (BITDEPTH-8) comes from the reference software.
One could argue that it should be (1<<BITDEPTH)-1 but what you are proposing is .. I don't know what to write.. baseless ?

Why is it baseless, you don't have (1<<BITDEPTH)-1 possible values in the limited range, you have far fewer, so the magnitude of error relative to the total range is greater, i. e. 20log10(255)-10log10(5) != 20log10(219)-10log10(5)?

nevcairiel

14th February 2022, 08:48

jauh

14th February 2022, 14:32

Pixel values can still use the full range in an image, only limited by the actual physical bitdepth of 8 or 10-bit. If you signal the image to be "limited range", then these values are called BTB (Blacker than Black) or WTW (Whiter than White)
Thats why you always use the full 8-bit range for PSNR calculations, in every reference you can find for PSNR.

Even if you ignore that, the difference between a 255 range and a 219 range is barely even a third of a bit.

That explanation makes no sense: if you signal outside the permitted range the receiver clips the signal and neither BTB nor WTW convey anything meaningful (think speakers clip sound when DAC generates a wave outside speakers' range) so why would meaningless data contribute to SNR computation, let alone to the bettering of the metric, when it's actually axiomatically noise. Think about it: if you have a perfect sine wave between ±1 and then add noise to the wave, the peaks will overshoot the ±1 peaks of the sine wave, arguing that the overshooting conveys meaning is curious.

The argument that 255 vs 219 is barely even a third of a bit is neither here nor there: the peak of the signal affects PSNR. Let's take the peak at 255 and MSE at 5, the PSNR is 20log10(255)-10log10(5), or 41.1411 dB, whereas if you take peak at 219, the PSNR is 20log10(219)-10log10(5), or 39.8192 dB.

Just because some code does it, even if majority does it, doesn't mean it's the right way to do (the bandwagon fallacy); it's certainly the right way to computer PSNR for the full swing of 8 bits, there's nothing to say the code wasn't simply lifted from an image compression comparison code without considering the peaks of the signal that is being measured. Case in point: ffmpeg still creates mp4 files with 'isom' as major_brand, for example, when 14496-12 has for a long time (at least six years) said "[Annex E] brands should not be used as the major brand..."

charliebaby

17th February 2022, 07:38

New Version 3.5+29-3415705 :)

GCC 11.20
https://www.mediafire.com/file/nxmmgimcxxl5dgd/x265-3.5+29-3415705_gcc112-AVX2.7z/file

VS 2022
https://www.mediafire.com/file/j5blcthttfcnnlq/x265-3.5+29-3415705_vs2022-AVX2.7z/file

Barough

23rd February 2022, 07:28

benwaggoner

24th February 2022, 03:23

x265 v3.5+35-287466181
https://www.mediafire.com/file/257hl0ulvf479qo/x265-3.5%252B35-287466181_Win_GCC112.7z/file

x265 Note : The commit value is wrong due to something upstream at MulticoreWare. I recall that something like this have happen earlier also.
Are there any actual changes other than commit value in this build? It appears all 2022 commits (https://bitbucket.org/multicoreware/x265_git/commits/) have been for ARM (courtesy of AWS).

benwaggoner

24th February 2022, 19:42

Where are people at with using --fades in x265 3.5? Anyone see it make a difference, positive or negative?

From the readthedocs documenation:
--fades, --no-fades
Detect and handle fade-in regions. Default disabled.

From the 3.1 release notes:
--fades can detect and handle fade-in regions. This option will force I-slice and initialize RC history for the brightest frame after fade-in.

charliebaby

26th February 2022, 22:29

I did a test with the --fades but I don't see any difference

FranceBB

27th February 2022, 22:12

I did a test with the --fades but I don't see any difference

Check the GOP Structure with:

ffprobe.exe -show_frames -i "Test.ts" | findstr pict_type

pause

It will show you each frame and which kind of frame it is, so basically I, P or B.
Technically each scene change should be an I.
Encode the same content twice, same parameters, one with --fades the other without and compare the GOP and see if it changes. If it does, then it's working.

Blue_MiSfit

1st March 2022, 00:17

Are there any actual changes other than commit value in this build? It appears all 2022 commits (https://bitbucket.org/multicoreware/x265_git/commits/) have been for ARM (courtesy of AWS).

Very cool, thanks Amazon :)

I've been meaning to build an ARM ffmpeg docker image and give the latest Graviton instances a whirl!

quietvoid

4th March 2022, 19:07

H.274 film grain characteristics SEI support in x265: https://mailman.videolan.org/pipermail/x265-devel/2022-March/013459.html
Also made available, a tool to generate the metadata from YUV: https://bitbucket.org/multicoreware/libfgm

Have fun!

Boulder

4th March 2022, 21:09

Very interesting! Does it require specific support from the decoder or is it a standard requirement in the specification?

quietvoid

4th March 2022, 21:57

I'm pretty sure it does require support at the decoder/renderer. I don't know whether current hardware supports it.
FFmpeg can decode and apply one of the types at 8 bits only as far as I can tell.

nevcairiel

4th March 2022, 22:11

Very interesting! Does it require specific support from the decoder or is it a standard requirement in the specification?

It is not a standard requirement for HEVC, it is optional SEI data.
AV1 has made it a baseline function, for what its worth, including support in HW decoders.

charliebaby

21st March 2022, 19:36

New Version x265-3.5.0.36-3415763_gcc112-AVX2

GCC 11.20

https://www.mediafire.com/file/ssl3me9ghx6s4x7/x265-3.5%252B36-3415763_gcc112-AVX2.exe/file

Barough

21st March 2022, 21:41

x265 v3.5+37-9af013bb3
Built on March 21, 2022, GCC 11.2.0

https://www.mediafire.com/file/hokjj9zy2afu0p0/x265-3.5%252B37-9af013bb3_Win_GCC112.7z/file

Note :

The commit value is wrong due to something upstream at MulticoreWare.

Correct commit number for this release is

9b59d45

benwaggoner

23rd March 2022, 03:44

It is not a standard requirement for HEVC, it is optional SEI data.
AV1 has made it a baseline function, for what its worth, including support in HW decoders.
...although one of the most popular first generation AV1-capable SoCs had a bug with FGS which makes it problematic to safely use. I think it can be fixed with a driver update, but lots of devices don't regularly update those sorts of components.

LazyNcoder

2nd April 2022, 19:02

rwill

3rd April 2022, 00:11

benwaggoner

7th April 2022, 22:29

Guys, using aq-mode 4 with CRF, I'm getting much smaller file. Is this OK?
Obviously we're losing some details here. Should I choose a lower number for CRF to keep the same quality? Is it enough? maybe some more aq-strength? little bite psy-rd and psy-rdoq?
It's okay if it's giving you better bang for the bit, which is hard to figure out when both bitrate and detail drops. The only real way to test is to compare two different 2-pass VBR encodes using the same average bitrate but different AQ modes.

DKILLER1

24th May 2022, 04:47

My anime encoding settings is (720p)

--ctu 32 --max-tu-size 16 --tu-inter-depth 3 --splitrd-skip --tu-intra-depth 3 --b-intra --ssim-rd --weightb --cbqpoffs -3 --crqpoffs -3 --no-open-gop --max-merge 4 --rc-lookahead 90 --lookahead-slices 0 --limit-refs 1 --limit-modes --rdpenalty 1 --ref 5 --me 3 --psy-rd 2.40 --subme 4 --bframes 6 --merange 50 --psy-rdoq 3 --rdoq-level 2 --deblock -3 --aq-mode 2 --aq-strength 1.30 --rd 4 --no-strong-intra-smoothing --no-sao --limit-tu 2 --qg-size 16 --qcomp 0.70 --fades

Is there anything I should change to further improve the quality of the anime?
Please anyone help here

benwaggoner

24th May 2022, 22:17

My anime encoding settings is (720p)
Is there anything I should change to further improve the quality of the anime?
Please anyone help here
How did you wind up with these settings? A lot seems pretty random. For example, --rdpenalty only applies to 64x64 CUs, but you're using --ctu 32. And I've never seen --max-tu-size 16 used for anything. Did you actually see improvements by setting that?
Or --ssim-rd, which didn't work well for general content, and I would think would be less applicable to anime.

You're not specifying --profile or --level-idc, which can be important for compatibility.

I'd start with --preset slower --tune animation and iterate from there. It's rare that you'd need or should set so many parameters manually.

LigH

28th May 2022, 22:27

New upload: x265 3.5+37-0d865ebe2 (https://www.mediafire.com/file/znuc3jsmtsnx7jc/x265_3.5+37-0d865ebe2.7z/file)

[Windows][GCC 12.1.0][32/32XP/64 bit] 8bit+10bit+12bit

Bugs not yet fixed:

NASM 2.15.05 multi-line macro warnings (reported August 2020)

LoRd_MuldeR

28th May 2022, 23:30

Jamaika

29th May 2022, 11:05

https://github.com/netwide-assembler/nasm
How can I compile or download nasm 2.16?
There are supposedly newer nasm replacements. Where and from where can I download?

LigH

29th May 2022, 12:34

Looks like the right place. Pull it with git, adding .git to the URL. Or download an archive from the green button [ Code ] - "Download ZIP". Then ./configure, make ... as usual. I guess.

I should possibly suggest to the MSYS2 team that nasm 2.16 is available.

Or not. This tag is not "released" yet. The "latest" snapshot is 2.16rc0-20201104 ... hmm, it aged well.

LoRd_MuldeR

29th May 2022, 14:20

https://github.com/netwide-assembler/nasm
How can I compile or download nasm 2.16?
There are supposedly newer nasm replacements. Where and from where can I download?

What about?
https://www.nasm.us/pub/nasm/snapshots/latest/win64/

Jamaika

29th May 2022, 15:27

Sorry snapshots has bugs and it isn't 2022.

tuanden0

8th June 2022, 14:38

Khun_Doug

8th June 2022, 15:23

My build was using a 3950X but I upgraded to a 5950X, 16 cores / 32 threads. The prices have dropped as availability increased. I encode with grain and slow selected, usually at CRF 20. That 5950X comes to life and moves along quite well. One factor that effects speed is the number of filters you have applied. I almost always encode with AVS, and set the AVS threads to 14. My experience was that anything higher than 14 on HD source didn't yield much benefit. On SD source I use a thread count of 6.

rwill

8th June 2022, 15:40

RanmaCanada

9th June 2022, 06:51

I am building my new PC for x265 video encoding because my PC is broken by thunderstruck accident.

Could everyone give me some advice?

My old CPU is R7 5800x with encode --preset veryslow about 6.2 fps (x265 version 3.5), I want to increase the encode speed at least 14 fps

On very slow nothing at a "reasonable" price will give you that. I would honestly say for now to get an inexpensive unit (sub $200 USD refurb) to get you buy until Zen 4 is released, as Raptor Lake will also be released "soon". It's possible there may be some insane increases in encode speeds (15% IPC at least rumoured). Even if there is not, Zen3 prices should drop some more and Intel 12th gen should as well as they have learned they can no longer demand stupid prices with AMD beating them.

tuanden0

9th June 2022, 12:01

Thank @Khun_Doug, @rwill, @RanmaCanada

I will follow RanmaCanada's idea to wait for AMD or Intel release new gen CPU to check perf/price to buy or waiting old gen price down price

Jamaika

3rd July 2022, 09:47

I am going to ask this brazen question. Is this project finished?

rwill

3rd July 2022, 18:05

I am going to ask this brazen question. Is this project finished?

It might be done for.

RanmaCanada

4th July 2022, 02:02

It might be done for.

Well hasn't it been in maintenance mode for quite some time now with no major updates? it's possible we might need to move to VVC or AV1 and completely relearn how to encode haha.

filler56789

4th July 2022, 03:58

I am going to ask this brazen question. Is this project finished?

We may say it is effectively-dead. :-|

* 2013
† 2022

Time flies indeed.

Blue_MiSfit

4th July 2022, 07:53

My understanding is that lots of talent left MulticoreWare, so the pace of development has slowed quite a lot.

FranceBB

4th July 2022, 09:17

excellentswordfight

4th July 2022, 20:39

Well I mean x265 is very mature now, so what I'm concerned about is actually x266.
It's been a while now since the official specs have been finalized and there are encoders like VVEnc from Fraunhofer which are somewhat usable compared to the original reference encoder which was meant to be used for tests only.
Decoders are also slowly but surely popping up and there are plugins that can be compiled and integrated in VLC etc.

And yet... no public x266 repository.
Given that there hasn't been any progress on x265 for a while, I assume Multicoreware is going hands down on x266 and focusing on it, but at this point I'm not sure.

H.266 VVC specs were finalized on the 6 July 2020, so in 2 days it's gonna be two years of no progress with x265 and still no x266. :(
I just had a look at MulticoreWares website and it seems a lot more focused on AI/ML than video encoding, same goes for their job openings.

But looks like they had a x266 demo at IBC this year though.

We may say it is effectively-dead. :-|

* 2013
† 2022

Time flies indeed.
Tbh it has pretty much been dead since 3.0 in 2019.

FranceBB

4th July 2022, 21:07

But looks like they had a x266 demo at IBC this year though.

IBC? I think you meant NAB. It can't be IBC 'cause it's in September and I'm gonna be there in person this year, hence my post: https://forum.doom9.org/showthread.php?t=184144

(which by the way is still valid as I'm gonna meet with steipal, momocampo and emcodem, but I'd love to met with more people from Doom9).

But anyway I'd love to see x266 on display there by the Multicoreware guys 'cause I'm gonna ask them a few questions and report everything here.
One of them is also gonna be whether they're gonna create an account called x266_Project like they did for the x265_Project from which they used to post everything regarding the development etc.

Barough

4th July 2022, 22:21

Double post

Barough

4th July 2022, 22:25

My understanding is that lots of talent left MulticoreWare, so the pace of development has slowed quite a lot.

After Tom left MulticoreWare so did it start to go downhill with the development. He was the Head of the x265 project and the Head of MulticoreWare's video business. He left back in the beginning of 2018.

excellentswordfight

5th July 2022, 12:10

IBC? I think you meant NAB. It can't be IBC 'cause it's in September and I'm gonna be there in person this year, hence my post: https://forum.doom9.org/showthread.php?t=184144

(which by the way is still valid as I'm gonna meet with steipal, momocampo and emcodem, but I'd love to met with more people from Doom9).

Yes, typo, i ofc meant NAB.

Are they confirmed for this years IBC as well? Not sure if I'm going this year, but if I do I guess I will have look as well.

ksec

5th July 2022, 15:32

After Tom left MulticoreWare so did it start to go downhill with the development. He was the Head of the x265 project and the Head of MulticoreWare's video business. He left back in the beginning of 2018.

He went to Beamr, which is arguably the best HEVC encoder. And now he is at Kaleidescape.

He doesn't tweet about Video Codec anymore. I have a feeling that he simply got fed up with the whole VVC licensing situation and simple left Video Codec field.

rwill

5th July 2022, 20:05

Well Video Compression is not a good business.

No one wants to pay more than a couple bucks for some encoder license, I mean there are ones free available.
If the free encoders suck people will complain but no one wants to sponsor development.
No one wants to pay for decoders, they already paid for the content. Some dont even pay for the content.
No one wants to buy extra hardware for encoding, why is a quad core CPU from 2012 not sufficient again ?
Some People with no clue mistake video compression for modern magic and then think they have three wishes free.
People then have elevated expectations about video quality, cant be hard to do better right ?
People don't want to spent time to learn about quality tunes, rather they want something like WinZip.

and I could continue this list ...

I mean I had the case where someone complained about quality and then someone asked him if he could try a higher bitrate and this person then was really thankful because a higher bitrate apparently improved quality a lot in his case. Quite the trick once you know it....

And the people you have to work with...

There are a lot of Pretenders that just have mastered the art of Bullshit Bingo.
The most capable people tend to be legally insane.

Does not mean that other areas of entrepreneurship do not have similar issues, but because software is immaterial and a video encoder or decoder is a non interactive black-box their value is hard to grasp for the average populance.

benwaggoner

5th July 2022, 21:04

I just had a look at MulticoreWares website and it seems a lot more focused on AI/ML than video encoding, same goes for their job openings.
That's always been a big focus of MCW, along with performance engineering in general. Codecs were never more than a fraction of their total business.

Jamaika

5th July 2022, 21:04

I mean I had the case where someone complained about quality and then someone asked him if he could try a higher bitrate and this person then was really thankful because a higher bitrate apparently improved quality a lot in his case. Quite the trick once you know it....
As they say in third world countries. Who uses low bitrate? Software pirate that used xvid interlaced DVD.
He doesn't know much about filming, framing, editing but he tries to sell something under the table.
There is still a 4 EURO DVD, not necessarily with voiceover. Then came x264 and x265. For years it wasn't as successful as xvid. Market collusion?
Initially x265 was better for low bitrates than Mainconcept HEVC in low-cost editors. Everyone disliked the fact that the copy of the original isn't webcam quality.
Youtube is now censored. Fired journalists pretend to contribute something in 4K. It no longer needs xvid, x264, x265.
Who needs Bluray for 16EURO today in Europe. There is no video DVD rental. We have pay TV for any high bitrate programs, websites, metadata as ECO

benwaggoner

5th July 2022, 21:04

benwaggoner

5th July 2022, 21:11

Well Video Compression is not a good business.

No one wants to pay more than a couple bucks for some encoder license, I mean there are ones free available.
If the free encoders suck people will complain but no one wants to sponsor development.
No one wants to pay for decoders, they already paid for the content. Some dont even pay for the content.
No one wants to buy extra hardware for encoding, why is a quad core CPU from 2012 not sufficient again ?
Some People with no clue mistake video compression for modern magic and then think they have three wishes free.
People then have elevated expectations about video quality, cant be hard to do better right ?
People don't want to spent time to learn about quality tunes, rather they want something like WinZip.
If you are talking about a consumer market, yes, there's not a lot of money to be made in compression. The general class of workstation encoding tools largely vanished around a decade ago. To my regret; there are all sorts of high-touch operations that were easier to do back then which are much more challenging to do now. I barely did any command-line encoding the first half of my career, and now I barely do anything but.

However, enterprise and broadcast encoding is still a huge markets. At scale, bandwidth becomes quite expensive. And for RF-limited transmission like cable/sat/OTA, better compression=more channels=more revenue. A big cable company can spend $100M upgrading encoders with a healthy ROI.

Consumers and hobbyists largely have to make due to with plugging in scraps of open-source code funded by bigger corps these days. Advanced preprocessing is much more an enthusiast domain, but codecs themselves have gotten too complex for small ad-hoc teams to do the kinds of intense innovation like x265 had in its early days.

FranceBB

5th July 2022, 23:18

Who uses low bitrate? Software pirate

Not really, given how crowded satellites like HotBird are and that many many many TVs all over the globe rely on them to broadcast their contents, it makes sense to keep the development of new codecs ongoing.
And remember, it's those TVs that in the end are gonna pay the big bucks to get their fair share of bitrate and with ever growing resolutions, bit depth and frame-rates, encoders just need to get better.

Then you have a totally separate market which is VOD with streaming services which are also trying to get the best possible compression at the lowest possible bitrate 'cause CNDs ain't cheap either. I often say that satellite bitrate is hugely expensive, but think about companies like Amazon, Netflix etc and how many users they have and how much bandwidth they "spend" with their CDNs to serve all the customers.

I'm planning on going myself! Let's have a meetup.

Totally! :D
I'll PM my number just in case and I'll come up with more detailed info as the date approaches. :)

enterprise and broadcast encoding is still a huge markets. At scale, bandwidth becomes quite expensive. And for RF-limited transmission like cable/sat/OTA, better compression=more channels=more revenue. A big cable company can spend $100M upgrading encoders with a healthy ROI.

Absolutely.
I work one step ahead of the final distribution guys, so the files I encode are all high bitrate mezzanine files that are gonna be re-encoded for broadcasting by live encoders most of the time, but I know that the final distribution guys are always trying to get the best possible result 'cause unlike people in the team I work with, they're gonna have to deliver to the actual consumers, so they can't have the "luxury" to be like: "Oh, UHD, H.264, 500 Mbit/s 10bit, that will do" like the files I create but they're gonna have to find the best possible way to squeeze all that in a 25 Mbit/s H.265 stream encoded live.

Anyway, I'm digressing, the thing is that you're right, there's still a big market besides the consumer-niche.

Jamaika

6th July 2022, 05:28

Not really, given how crowded satellites like HotBird are and that many many many TVs all over the globe rely on them to broadcast their contents, it makes sense to keep the development of new codecs ongoing.
Compare the markets from 20 years ago. How many satellites have come and changed?
I've been to orange recently. You must have TV decoder, this will lower your bills. How does a TV set-top box work? Yes, the decoder is h264 but if you don't have optical fiber for high bitrates, you won't watch TV. What low data rate and why did you buy 16K TV? Is this data downsizing on satellites?
Last, how does this relate to the x265 codec. It's a codec used in simple converters.

FranceBB

6th July 2022, 13:35

Compare the markets from 20 years ago. How many satellites have come and changed?

Not many have been launched cause it's crazily expensive to do that.

I've been to orange recently. You must have TV decoder, this will lower your bills. How does a TV set-top box work? Yes, the decoder is h264 but if you don't have optical fiber for high bitrates, you won't watch TV.

I'm not familiar with orange and their decoders, but most set top box work with satellite dishes and rely on internet only for on demand stuff. In other words, any time you wanna watch a linear channel, you're gonna use the satellite dish. Now, when it comes to transmissions, almost every TV has MPEG-2 for SD, H.264 for FULL HD and H.265 for UHD. In our case it's around 4 Mbit/s for SD, 12 Mbit/s for FULL HD and 25 Mbit/s for UHD but as you probably are aware of, given that no new satellites have been launched and that literally the overwhelming majority of EU TVs are all on Hotbird 13B/13C/13E at 13.0°E whose bandwidth is the same as when the satellites have been originally launched, you can see why getting the lowest possible bitrate is important.
Besides, to those who say that streaming is the future, sure, granted, but not everyone has reliable broadband everywhere, especially those living in rural areas and that's where satellite comes in handy, but even then the biggest limiting factor is the cost: everyone wants UHD channels and there's just not enough bitrate in the satellites to grant it to everyone.

What low data rate and why did you buy 16K TV? Is this data downsizing on satellites?

Consumers will always want the latest and greatest technologies. If you stick your head in the sand and play dumb, you're never gonna gain new customers or indeed keep the ones that you already have.
Think about it: if no broadcaster updated its infrastructure, we would have been stuck with SD which was no longer competitive. People were buying HD TVs and wanted to watch HD contents. Sure, the bitrate was a problem then like it is now, but broadcasters had to cope with it and begin airing in HD. Then it came FULL HD, adding more struggle to the infrastructure. Right now, it's all about UHD, HDR 50p H.265 which is the standard. You know, imagine a world in which people bought UHD TVs and broadcasters kept airing in FULL HD BT709 SDR 25i or 30i H.264. People would just stop watching TV altogether or indeed complain about it and about the fact that they bought a new shiny TV and that their favourite sport is in FULL HD. And the circle goes on and on and on.

Last, how does this relate to the x265 codec. It's a codec used in simple converters.

I think you're by far underestimating how good x26x have become. This is not 2003 when there were dvix/xvid used by private consumers to re-encode their legally purchased MPEG-2 DVDs, this is 2022 where x264 and x265 are at the heart of contents encoded by companies all over the world, especially VOD.

Barough

6th July 2022, 16:14

x265 v3.5+39-58f46fa4a
Built on July 06, 2022, GCC 11.2.0

https://bitbucket.org/multicoreware/x265_git/commits/branch/master

DL:
https://www.mediafire.com/file/wc1t5q2ckc6390j/

Note :

The commit value is wrong due to something upstream at MulticoreWare.

Correct commit number for this release is
20255e6

benwaggoner

7th July 2022, 05:07

Last, how does this relate to the x265 codec. It's a codec used in simple converters.
x265 absolutely gets used in enterprise products and for enterprise-scale content. It's not as dominant as x264 is for H.264 encoding (where pretty much everyone but Beamr uses it under the hood), but still a very big deal.

Jamaika

9th July 2022, 07:56

Testing of new paid SAT functions LCEVC-X265. We don't have to go to the sat fair.

ffmpeg -y -i "Football.mp4" -g 60 -c:v lcevc_hevc -base_encoder x265 -r
29.97 -s 1920x1080 -b:v 0k -eil_params
"preset=veryslow;rc_pcrf=33;scenecut=0;min-keyint=60;frame-threads=4;residual_mode_priority_enabled=0;temporal_use_priority_map=0"
football_pCRF33_LCEVC_x265.mp4

benwaggoner

10th July 2022, 03:18

I've been avoiding doing actual compiling of code for 25 years, but now I've got a modified source version of x265 I need to compile for Windows 64-bit. I did the stuff in the ReadMe (yasm, etc).

I'm trying to run \x265_v3_5_mod\build\vc15-x86_64\make-solutions.bat, but I keep getting a "no visual studio 15" error, and I'm unsure on what Visual Studio I need to have installed. VS 2007 15.9 didn't work. Do I use "Build Tools for Visual Studio 2017 (version 15.0)" or something else/additionally?

As a former Microsoft employee, I wish to decry how tightly coupled Visual Studio is to Windows. I shouldn't have to reboot this many times ;)!

DJATOM

10th July 2022, 10:46

AFAIK you need a link with special altered environment (look for that in MSVC Programs group, I'm on Linux from work PC and don't want to boot my ryzen PC with win10 just to check exact link name), from that console you will generate solution files. Also after some modification you could use even MSVC 2019 - https://github.com/DJATOM/x265-aMod/tree/aMod-3.5/build/vc16-x86_64

benwaggoner

11th July 2022, 05:10

AFAIK you need a link with special altered environment (look for that in MSVC Programs group
That sounds promising. Alas, I do not know what you mean by MSVC Programs group or what to look for there.

Does anyone have the link for that?

Sorry for such basic questions. The last time I really compiled something like this outside of some autobuild environment was before Visual Studio 97 was even released...

qyot27

11th July 2022, 18:13

Trying to remember off the top of my head because I'm booted into Ubuntu at the moment:

Start Menu->Programs->Visual Studio->Tools->Visual Studio Command Line Tools [x86|x64|arm64]

Basically, it's just the Start Menu shortcut to the relevant vcvarsall(%arch%).bat that loads up the compiler environment in cmd.exe. I *think* those are supposed to be automatically installed no matter what configuration options you've chosen for VS, but otherwise you may have to re-run the VS updater and select them.

In general, I wouldn't bother with the build scripts provided by x265, because those are by nature too highly coupled to a specific build environment configuration. If you know how to use CMake directly (and have notes on exactly which x265[mod]-specific configuration options you want), it's much more straightforward, since you can select exactly what you know you're working with (and you don't have to load up a GUI).

Boulder

12th July 2022, 12:45

Wasn't it so that you have to edit the bat files manually to support a specific VS version. I have Visual Studio 2019 Community installed on my PC, and it's apparently "Visual Studio 16 2019" for CMake.

benwaggoner

12th July 2022, 18:51

If anyone can give me an overview of the best way to go from a source directory to a Windows 64-bit binary, I'll owe you one! I don't mind going the GCC route if that's easier.

Boulder

12th July 2022, 20:38

I don't know if it helps, but I just cloned the repo and edited the make-solutions.bat in the build\vc15-x86_64 folder to this:

cmake -G "Visual Studio 16 2019" ..\..\source && cmake-gui ..\..\source

Then ran make-solutions in the console and it will proceed like it is instructed in the wiki. You might need to point to the path where NASM is and enable assembly in the configure part (assembly is disabled if CMake cannot find NASM). The resulting .sln file can be opened in Visual Studio 2019.

LigH

22nd July 2022, 12:06

@benwaggoner:

I enjoy using the media-autobuild suite in most cases. You can configure it down to building only what you need. Doesn't have to be a whole ffmpeg, just a separate x265.exe is fine too.

Blue_MiSfit

23rd July 2022, 19:21

Another vote for MABS. It's incredibly useful (when it's not broken, which is often lol)

Barough

23rd July 2022, 23:27

@benwaggoner

MABS gets my vote also

Selur

24th July 2022, 16:13

Yup, using MABS too when building Windows tools for Hybrid. :)

BuccoBruce

27th July 2022, 16:22

If anyone can give me an overview of the best way to go from a source directory to a Windows 64-bit binary, I'll owe you one! I don't mind going the GCC route if that's easier.

I don't know if it helps, but I just cloned the repo and edited the make-solutions.bat in the build\vc15-x86_64 folder to this:

cmake -G "Visual Studio 16 2019" ..\..\source && cmake-gui ..\..\source

Then ran make-solutions in the console and it will proceed like it is instructed in the wiki. You might need to point to the path where NASM is and enable assembly in the configure part (assembly is disabled if CMake cannot find NASM). The resulting .sln file can be opened in Visual Studio 2019.

Just a note about MABS, it will build with GCC and purely with GCC, unless you tell it to build with clang. Not an issue for most, and probably preferred if you're going to be using the libraries to link against anything else built with GCC. I still prefer it for building non-free ffmpeg (with ffmpeg's AAC, MP2, MP3, opus, and vorbis implementations disabled outright) to make an MPV build that can handle USAC and that de/encodes opus using libopus. Trying to build all those requirements separately with MSVC myself would take a few years off my life, and take forever.

For what it's worth, I've found on some machines that VS builds of x265.exe (and x264, aom-enc, SVT-AV1, to name a few) perform a bit better in some cases, but only negligibly. It does seemingly add up on Intel CPUs without AVX2 though, and it also allows you to use VS profiler guided optimizations (PGO) if you choose to, and in my case, disable things like Spectre slowdow...I mean mitigations, but only because I don't know how to pass that through to GCC.

I can confirm Boulder's edit worked for compiling under VS2019. I add -A x64 out of habit, so cmake -G "Visual Studio 16 2019" -A x64 ..\..\source && cmake-gui ..\..\source. You would presumably edit it to read "Visual Studio 17 2022" if you're using 2022.

Make sure you start a "x64 Native Tools Command Prompt for VS 2019" to run things from, and make sure NASM is in your PATH or you'll end up with no optimized assembly code.

BuccoBruce

27th July 2022, 16:49

TL;DR Is there some magic bullet for muxing an HEVC elementary stream with Open GOP using mp4box and getting Media Foundation to decode it properly?

I'm running into some weird issues with Open GOP HEVC + Media Foundation decoding in an MP4 container. Muxing with ffmpeg -movflags faststart+negative_cts_offsets seems to work fine most of the time. It complains about a lack of timestamps in the raw .hevc stream and outputs VFR, so -i has to be preceded with e.g. -r 60000/1001 to get around that, and I have to -loglevel error -stats or it will just quickly fill the console, ad infinitum, with:
[mp4 @ 000001da260000c0] Timestamps are unset in a packet for stream 0.
This is deprecated and will stop working in the future.
Fix your code to set the timestamps properly
[mp4 @ 000001da260000c0] pts has no valueB time=00:00:00.00 bitrate=N/A speed= 0x
Last message repeated 103 times

5760x2880, 5408x2704, 4800x2400, 4096x2048, 3840x1920, 3000x1500
Level 6 Main at the max, lower for smaller resolutions
Issue persists even with L5/Main 3000x1500 50 fps video, or 2160x2160 30 fps
8/10 bit doesn't matter
GOP length doesn't seem to matter, tried 60/600 (10 second rule), 30/300 (half), 25/250 (default)
Ref/b-frame count doesn't seem to matter, all within the limits of Level 6 or well below anyways
Thought it might be VBV limited CRF acting up and overflowing the DPB using the default Level 6 VBV, so I tried lowering the VBV, and lower bitrate ABR with a much longer RC Lookahead, issue persisted
CRF encodes ended up being fine anyways, since the issue seems to be limited to MP4Box+Media Foundation...
Disabling Open GOP magically fixes it most of the time.

Is it some kind of IDR signaling issue? Disabling Open GOP alone seemingly resolves all of the issues, but I would like to use Open GOP since these are static camera shots. Is it something really dumb like -inter 500 being too small? I guess that would make me really dumb. It's starting to seem more like a GPAC/mp4box issue, or more likely super-duper Dunning-Kreuger PEBKAC, but I don't know enough about HEVC bitstream output and signaling (PPS/SPS/VUI) to know any better so I wasted my time messing with encoder parameters.

MP4Box output is mostly unplayable, it doesn't seek, and it plays choppy, almost like what you'd expect to see when the decoder drops a temporal enhancement layer and plays back at half FPS. Using --forcesync with mp4box didn't help either. I tried MP4Box with an added --negctts but that just outputs "Arg negctts set but not used" in the console - whereas using negative_cts_offsets in ffmpeg seems to fix things?! Either way, the issue only seems to be with Media Foundation playback. Using an MKV and/or decoding with LAV works fine, as does decoding in software or using MPV or even ffplay.

---

What about any of the x265 bitstream options, could they help? Based on the documentation, --repeat-headers seems like it's only useful for trying to seek within the elementary stream output before muxing it. --aud? --eos? --hrd? I thought --idr-recovery-sei might help, but enabling it along with --repeat-headers seemingly made things worse. mp4box's output when trying to mux a stream made with those two options makes avidemux crash immediately, and makes ffmpeg (MPV) have serious issues playing the file too. This seems to be regardless of the parameters I tried with mp4box: -inter 0 to force a flat mp4, letting it do the default -inter 500, and trying with and without --forcesync for both options. I am pretty sure I tried --nosei, but that'd just be throwing away the extra stuff I asked x265 to write, and then I'd have to waste my time remembering how to re-signal bt709/limited. Trying to remux any of that mp4box output using ffmpeg results in a file that is entirely unplayable in anything, it skips back and forth randomly, you get intermittently decoded blocks, etc. It's also the only result that could technically allow posting a screenshot, since it's NSFW video...it's VR pr0n alright? I can mux directly from the raw .hevc stream if I use those two x265 options with ffmpeg but only with no other parameters, just forcing the FPS with -r 60000/1001 to prevent erroneous VFR output, and -c copy. I haven't tried -movflags faststart, and using negative_cts_offsets seemingly breaks these files too. I also have yet to try putting ffmpeg's output back through mp4box. I am streaming these from a NAS and would prefer to have the MOOV atom at the beginning of the file, so a working flat mp4 is only "half fixed".

What's even weirder is I can take a working mp4 from elsewhere at the same resolution, frame rate, and bitrate, and with seemingly identical x265 settings visible in the SEI, and remux it all I want with mp4box. It doesn't break playback under Media Foundation. The only difference is the version tag for x265 reading 0.0 - these working files were also seemingly muxed with ffmpeg (Lavf58.12.100) or even encoded directly with it using -c:v libx265. Looking at that file, it looks like the only options they passed to x265 were --bitrate 30000 --output-depth 10 --colormatrix=2 --colorprim=2 --transfer=2 --videoformat=5. Everything else is --preset medium defaults.

I've just been using --preset medium with some slower options selectively enabled, and some that I thought would help lower bitrate when I thought that was the issue.

--bitrate 20000 --output-depth 10 --level-idc 6 --no-high-tier
--rect --amp --tskip --tskip-fast --b-intra --limit-modes
--vbv-bufsize 30000 --vbv-maxrate 40000
--analyze-src-pics --rc-lookahead 120 --min-keyint 60 --keyint 600
--fades --video-signal-type-preset BT709_YCC
--opt-qp-pps --opt-ref-list-length-pps --opt-cu-delta-qp
--limit-sao --selective-sao 1 --sao-non-deblock

Plus either +,- or -,+ for pools on a dual socket system, and I've obviously tried with/without --repeat-headers --idr-recovery-sei . Adding/removing any of --b-intra --fades --analyze-src-pics --opt-qp-pps --opt-ref-list-length-pps --opt-cu-delta-qp didn't make a difference either - I'm just including the command I tried with the most options for completeness.

Boulder

29th July 2022, 12:23

A note for MABS users who build for Zen2/3: add -march=znver2 or -march=znver3 in custom_profile in the local64\etc directory. It gives a slightly better performance for those chips, I think I found it 3-4% better when I tested it on my 3900X.

LigH

30th July 2022, 20:06

New upload: x265 3.5+39-a599806d3 (https://www.mediafire.com/file/s6srzkosonu5yai/x265_3.5+39-a599806d3.7z/file)

[Windows][GCC 12.1.0][32/32XP/64 bit] 8bit+10bit+12bit

LeXXuz

30th July 2022, 20:42

I was wondering if someone could build me a Zen3 optimized and Zen2 optimized Windows version for my 5950x and 3950x CPUs. That would be much appreciated. :o :thanks:

RanmaCanada

1st August 2022, 15:34

I was wondering if someone could build me a Zen3 optimized and Zen2 optimized Windows version for my 5950x and 3950x CPUs. That would be much appreciated. :o :thanks:

Pretty sure DJATOM (https://github.com/DJATOM/x265-aMod) has the best. Yes it's an older build, but x265 has been in maintenance mode for well over a year now.

LeXXuz

1st August 2022, 19:20

Pretty sure DJATOM (https://github.com/DJATOM/x265-aMod) has the best. Yes it's an older build, but x265 has been in maintenance mode for well over a year now.

I know that's why I'd like an actual build for comparison. :)

benwaggoner

1st August 2022, 19:23

TL;DR Is there some magic bullet for muxing an HEVC elementary stream with Open GOP using mp4box and getting Media Foundation to decode it properly?
I've had some .hevc files that don't play properly when muxed in mp4box, but do when muxed in ffmpeg. ffmpeg complains enormously about missing PTS data, but seems to fix it fine.

They were all Closed GOP, though, so potentially unrelated to your issue.

BuccoBruce

1st August 2022, 22:19

I've had some .hevc files that don't play properly when muxed in mp4box, but do when muxed in ffmpeg. ffmpeg complains enormously about missing PTS data, but seems to fix it fine.

They were all Closed GOP, though, so potentially unrelated to your issue.

Might still be related, I re-encoded so many files I might have forgotten if something other than Open GOP was also causing it. Guess I'll stick to ffmpeg for HEVC and just use mp4box for AVC+HLS.

LeXXuz

2nd August 2022, 06:13

Is SAO still an issue for high quality encodes with actual builds or can this safely be activated now?

microchip8

2nd August 2022, 07:23

Is SAO still an issue for high quality encodes with actual builds or can this safely be activated now?

it's still an issue

LeXXuz

24th August 2022, 11:58

I'm tinkering around with my profiles to gain more speed out of my encodes. The significant rise in electricity cost here in Germany made that decision necessary. :(

I have a question regarding the "--limit refs" parameter. As there is a huge speed difference between mode 1 and 3 and I was told to better use mode 1 for better quality, I now also tested mode 2 which none of the presets seem to use by default.

I got a decent performance increase with mode 2 over mode 1 and tested this with quite a few examples. Can't say I've seen any notable differences in quality so far.

I read the docs about the differenct modes, but in all honesty I don't really understand what's written there and how that may affect quality.

I always do high bitrate encodes with the "slower" preset as a base and CRF values of 18 or even below. Is there any good reason NOT to use mode 2 over 1 for better performance? :o

benwaggoner

24th August 2022, 18:42

benwaggoner

24th August 2022, 19:03

I'm tinkering around with my profiles to gain more speed out of my encodes. The significant rise in electricity cost here in Germany made that decision necessary. :(
If you're looking for ways to reduce joules/pixel, --frame-threads 1 can really help. The overhead of frame threading can really reduce power efficiency, and doesn't always have that big of a speed boost depending on how many cores you have and the resolution you're encoding at.

If you use SAO, --selective-sao 2 saves a bit without material quality impact.

If you can share your current command line, we might have other suggestions.

In general, the --preset options are pretty well tuned for a typical range of content and scenarios as of x265 3.0. They don't include any features added in 3.1 or later, which is why no --selective-sao, --rskip 2, etcetera, even though those really should be the defaults.

LeXXuz

24th August 2022, 23:07

benwaggoner

25th August 2022, 01:49

Thank you for those suggestions. :thanks:

Right now I recode 1080p content

I use these settings:

--preset slower --crf 17.00 --qpfile "E:\WORK\chp.qpf"
--repeat-headers --input-depth 16 --output-depth 10 --dither
--ctu 32 --limit-refs 2 --psy-rdoq 5 --selective-sao 0 --no-sao
--colorprim bt709 --transfer bt709 --colormatrix bt709

CPUs used are Ryzen 5950x and 3950x.
--slower is already one of the better-balanced presets. Changing parameters from slower to ones from slow will speed things up, but all of them have quality impacts too.

There's no point to using --selective-sao if you're already using --no-sao.

I always like to set --profile and --level-idc so I'll get warnings if I violate the requirements. In your case that looks like --profile main10 --level-idc 4.0 or 4.1.

Using --psy-rdoq 5 without raising --psy as well is an uncommon configuration, but should work.

I'd use --rskip 2 to replace the default --rskip 1 because it's a better quality mode. I've not directly compared the speed. Higher --rskip-edge-threshold values are faster, but can reduce quality. I tend to use 2-3 in my stuff, but I'm more biased towards quality/efficiency than your use case.

What CPU are you running on?

The biggest thing to improve pixels/joule without any quality loss would be --frame-threads 1. Lower values can actually improve quality.

You can learn a lot from doing a --csv-log-level 2 and looking at the frame level data. For example, if there aren't a lot of TUs smaller than 8x8 you could reduce --tu-intra-depth and --tu-inter-depth by 1. Recursing all the way down is mostly helpful with content that has sharp details, like text and cel animation.

If you have a lot of RAM, increasing --rc-lookahead can improve quality when VBV-limited quite a lot without much negative speed impact.

LeXXuz

25th August 2022, 09:38

There's no point to using --selective-sao if you're already using --no-sao.
I was uncertain if I have to set it to 0 as well when I don't want to have SAO at all. I will remove that parameter.

I always like to set --profile and --level-idc so I'll get warnings if I violate the requirements. In your case that looks like --profile main10 --level-idc 4.0 or 4.1.
Again, I was unsure if I should let x265 decide on its own or put these in manually. Never thought about the violation warnings though which is a very good point. Will add these again.

Using --psy-rdoq 5 without raising --psy as well is an uncommon configuration, but should work.
Well, that is a longer story and the most subtle approach at the moment to fight banding with the quite clean source material I have. The --slower preset already uses --psy-rd 2. Raising that any higher added too much static noise into flat areas to my taste.
It's barely visible on 4k, but visible on 1080p and almost terrible on SD.
Without raising at least --psy-rdoq a little, x265 tends to produce banding in certain flat areas. And sadly my living room TV is very susceptible to that and tends to intensify even the slightest banding compared to my other TVs. So this is somewhat a personal compromise.

I'd use --rskip 2 to replace the default --rskip 1 because it's a better quality mode. I've not directly compared the speed. Higher --rskip-edge-threshold values are faster, but can reduce quality. I tend to use 2-3 in my stuff, but I'm more biased towards quality/efficiency than your use case.
I'll add --rskip 2 to my script.

What CPU are you running on?
AMD Ryzen 5950x and 3950x. Both with 16 cores/32 threads

The biggest thing to improve pixels/joule without any quality loss would be --frame-threads 1. Lower values can actually improve quality.
Doesn't that decrease speed a lot as it reduces parallel processing? Or am I mistaken here?

If you have a lot of RAM, increasing --rc-lookahead can improve quality when VBV-limited quite a lot without much negative speed impact.
The machines have 64GB. I think the default is 40 for the --slower preset? How much should I raise that?

Thanks again for your valued input Ben. :)

vpupkind

25th August 2022, 20:25

rc_lookahead -- at least 1s worth of frames

Immaculate

25th August 2022, 23:13

--slower is already one of the better-balanced presets. Changing parameters from slower to ones from slow will speed things up, but all of them have quality impacts too.

There's no point to using --selective-sao if you're already using --no-sao.

I always like to set --profile and --level-idc so I'll get warnings if I violate the requirements. In your case that looks like --profile main10 --level-idc 4.0 or 4.1.

Using --psy-rdoq 5 without raising --psy as well is an uncommon configuration, but should work.

I'd use --rskip 2 to replace the default --rskip 1 because it's a better quality mode. I've not directly compared the speed. Higher --rskip-edge-threshold values are faster, but can reduce quality. I tend to use 2-3 in my stuff, but I'm more biased towards quality/efficiency than your use case.

What CPU are you running on?

The biggest thing to improve pixels/joule without any quality loss would be --frame-threads 1. Lower values can actually improve quality.

You can learn a lot from doing a --csv-log-level 2 and looking at the frame level data. For example, if there aren't a lot of TUs smaller than 8x8 you could reduce --tu-intra-depth and --tu-inter-depth by 1. Recursing all the way down is mostly helpful with content that has sharp details, like text and cel animation.

If you have a lot of RAM, increasing --rc-lookahead can improve quality when VBV-limited quite a lot without much negative speed impact.

Thanks for the tips. --rskip 2 seems to improve grain "motion" quite a bit in some cases.

It's a shame that you have to fiddle with x265 to get an acceptable quality, when a simple --tune film --preset veryslow produces good results with x264. Of course, clean material isn't an issue, it's just that x264 looks better with noise/grain - out of the box.

FranceBB

28th August 2022, 20:24

A friend of mine bought a brand new Apple MacBook Pro that he wanted to use, so after mocking him a bit, I said: "you know what, I could run some benchmarks to see how ARM chips are gonna do in encoding" and I expected poor results, but... it looks like ARM chips are somehow closing the gap on x86 and I still can't figure out how nor why.

Anyway, here's the benchmark, enjoy.

Test Platform 1:

Apple MacBook Pro 2021
CPU: Apple M1 Max 10c/10th (8 performance + 2 energy) ARM
GPU: Apple M1 Max 32-core GPU
RAM: 64 GB DDR4
OS: Mac OS Monterey 12.5.1
Year: 2021

Test Platform 2:
Asus N552VX
CPU: Intel i7 6700HQ 4c/8th x86_64
GPU: NVIDIA GTX 950M 4GB GDDR5 (640CUDA Cores)
RAM: 64 GB DDR4
OS: Fedora Linux 36
Year: 2016

Test Platform 3:
CPU: Intel i7 10750H 6c/12th x86_64
GPU: NVIDIA RTX 2060 6GB GDDR6 (1920 CUDA Cores)
RAM: 16 GB DDR4
OS: Windows 10 Enterprise x64
Year: 2020

Test number 1: Video Decoding
Software used MPV

Source file: DCP (Digital Cinema Package)
Codec: MJPEG 2000
Profile: D-Cinema 4K (All Intra)
Bitrate: 124 Mbit/s
Resolution: 4096x1716 (2.40 LB)
Framerate: 24p
Colorspace: XYZ
Sampling: 4:4:4
Range: Full PC Range
Bit Depth: 12bit

Apple M1 Max 10c/10th ARM
CPU Usage: 100%
Total Frames of the clip: 480 frames
Number of dropped frames: 141

Intel i7 6700HQ 4c/8th 3.5GHz
CPU Usage: 100%
Total Frames of the clip: 480 frames
Number of dropped frames: 138

Test number 2: Video Encoding
Software used: x265 (with FFMpeg decoding)

Source file: Studio Masterfile
Codec: MJPEG 2000
Profile: D-Cinema 4K (All Intra)
Bitrate: 124 Mbit/s
Resolution: 4096x1716 (2.40 LB)
Framerate: 24p
Colorspace: RGB
Sampling: 4:4:4
Range: Full PC Range
Bit Depth: 12bit

Encoding target: H.265 HEVC 4:4:4 12bit YUV
Preset used: --placebo --crf 28

Apple M1 Max 10c/10th ARM
CPU Usage: 100%
Threadpool created using 10 threads
Using cpu capabilities: NEON
Speed: 0.6fps

https://i.imgur.com/H2ZuYyV.png

Intel i7 6700HQ 4c/8th 3.5GHz
CPU Usage: 100%
Threadpool created using 8 threads
Using cpu capabilities: MMX2 SSE2Fast LZCNT SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
Speed: 0.2fps

https://i.imgur.com/UKJ1tfv.png

Intel i7 10750H 6c/12th 2.6GHz
CPU Usage: 100%
Threadpool created using 12 threads
Using cpu capabilities: MMX2 SSE2Fast LZCNT SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
Speed: 0.4fps

https://i.imgur.com/bvpXqtK.png

To my very own surprise, even though the decoding benchmark was poor on both end, when it came to encoding, the Apple M1 10c/10th outperformed the old i7 4c/8th and the old i7 6c/12th by keeping a steady 0.6fps, while the i7 eventually reached 97°C, started throttling from the 3.5GHz frequency down to 2.5GHz, lowering the speed to 0.2fps and the other i7 reached around the same temperature going down from a turbo boost of 4.8GHz to 2.60GHz too, hence reaching 0.4fps...

So... looks like modern mobile ARM are indeed just slightly faster than 2016 mobile x86 CPUs, who would have thought that...
I've always been someone who thought x86 was gonna be faster, no matter what, due to the intrinsics / instructions set, but looks like I was wrong.

Still, given that ARM are actually meant to perform computationally low tasks like browsing the web, reading emails, sending messages in a chat etc, I don't think they'll ever make their way into a desktop, nor should they be used for encoding (and neither should laptop in general).
Still, for a laptop, this is quite remarkable, given that their own architecture is supposed to be against them... O_O

rwill

28th August 2022, 20:40

FranceBB

28th August 2022, 20:42

Ah, the daily Alternative Facts.

It should say "NEON" for cpu capabilities when running on ARM. You did it wrong.

Damn. Let me repeat the test and update it...

EDIT: Updated the other post with the new benchmark using brew install homebrew-ffmpeg --with-neon (source here (https://www.osxexperts.net/FFmpeg51ARM.zip)), the M1 won. O_O Why? How?! Daaaaaaaaaaaamn

rwill

28th August 2022, 20:51

If you look here:

https://bitbucket.org/multicoreware/x265_git/src/master/source/common/cpu.cpp

You somehow have to get it to compile with X265_ARCH_ARM64.

The other way is to offer an AMD64 binary to OSX and let it recompile the x86/SSE/AVX machine code to ARM/NEON at runtime. This is sub-optimal but still faster than x265 reporting cpu capabilities as "none"

I am actually carefully expecting ~0.7 to ~0.9 fps on an M1 then if you got ~0.2 before.

FranceBB

28th August 2022, 21:10

rwill

29th August 2022, 04:24

Yep, I've got it right now.
Thanks, I had no idea that I had to download the NEON build.
I've got 0.6fps steady now, with the fan going like crazy, while my poor i7 succumbed to 0.2fps and my slightly newer i7 went to 0.4fps... :|

Well I think your 'newer' Intel Core i7-10750H is still on a 14nm process node. So my guess is that it is hitting thermal or power limits. You could use HWINFO64 or similar monitoring software to find out. Intel kinda used the trick to run excessive power consumption before throttling very hard to win in benchmarks.

The M1 is a good chip, and Apple designing their stuff around it helps to deploy it optimally. In the mobile space anyway.

Amazon spent some time/money on optimizing x265 for ARM/NEON, I guess they run it on their Graviton AWS instances. So x265 was already optimized for a somewhat similar ARM platform.

Ritsuka

29th August 2022, 06:19

I think homebrew is missing the latest NEON optimizations available in the x265 master branch (or maybe not? it's not clear to me if it's using the 3.5 tarball or the git master branch). There is an additional patch to tune the thread pools for the M1 on https://github.com/HandBrake/HandBrake/blob/master/contrib/x265/A03-threads-pool-adjustments.patch

FranceBB

29th August 2022, 07:03

#if MACOS && X265_ARCH_ARM64
p->frameNumThreads = 8;

So if I got it right, it's gonna create a threadpool using the 8 real cores and not throw the 2 energy saving cores in, probably 'cause otherwise calcualtions would be divided equally and therefore the powerful cores would have to wait for the energy saving ones to finish and sync the results of the split multithreaded operations.

I think homebrew is missing the latest NEON optimizations available in the x265 master branch

Yeah if I understood the piece of code right, it definitely is 'cause it created a threadpool with 10 threads, thus using both the 8 power cores and the 2 energy saving cores.

nevcairiel

29th August 2022, 11:03

Both your x86 machines are old and mid-range at best, the MacBook is using a brand new 10 core high-end CPU. Its no surprise it wins. Apple has also been contributing some ARM improvements to a variety of multimedia projects to help accelerate it there, as historically most development was focused on x86 only.

excellentswordfight

29th August 2022, 11:14

Blue_MiSfit

29th August 2022, 19:57

Apple's hardware is remarkable, especially when you look at the performance per watt.

I'm also astonished at how well it performs even on non-native code via Rosetta 2 emulation. On the workstation I'm a Windows guy until the bitter end, but wow this is impressive.

The current generation of ARM servers is also quite good, but from what I've seen they're not on the same level as Apple for things like x265. More NEON optimizations in this and other similar apps will be a boon to media workloads.

TL;DR -- ARM is an absolute monster when the software is sufficiently optimized.

rwill

30th August 2022, 04:48

TL;DR -- ARM is an absolute monster when the software is sufficiently optimized.

Well ARM is just the instruction set here, most IP cores ARM offers remain low power.

There is nothing really stopping AMD or Intel to bolt some ARM or Risc-V instruction decoder in front of their CPU designs to get high performance CPUs without the x86 instruction set. Ok maybe they need to do some touch-ups here and there but still..

Both companies have experience with designing CPUs that perform rather well for a given power envelope. They have most CPU building blocks ready. AMD even had some ARM Opteron project a couple of years ago but that was canceled I think.

Have you tried running x265 on some Raspberry Pi 4 yet ?

benwaggoner

30th August 2022, 18:22

Thanks for the tips. --rskip 2 seems to improve grain "motion" quite a bit in some cases.
That's what it does, lowering --rskip-edge-threshold to 2-3 can further improve grain quality, although at a cost of some speed.

It's a shame that you have to fiddle with x265 to get an acceptable quality, when a simple --tune film --preset veryslow produces good results with x264. Of course, clean material isn't an issue, it's just that x264 looks better with noise/grain - out of the box.[/QUOTE]
Yeah, x265 hasn't had its presets refactored since 3.0, despite some new featured added since that should be defaults.

As a community, we could probably define some revised presets, ala "slower except --selective-sao 2 and --rskip 2 --rskip-threshold 3"

LigH

10th September 2022, 11:29

New upload: x265 3.5+39-3ca6a8197 (https://www.mediafire.com/file/794hz6035k00xdf/x265_3.5+39-3ca6a8197.7z/file)

[Windows][GCC 12.2.0][32/32XP/64 bit] 8bit+10bit+12bit

The git revision hash may be the only obvious change since my last upload; and the GCC version.

tormento

14th September 2022, 11:12

x265 3.5+39-3ca6a8197
Did you try to encode something with it?

I have tried x64 with StaxRip and it simply doesn't work, not throwing any error.

tormento

14th September 2022, 20:20

New upload
With

D:\Eseguibili\Media\StaxRip\Apps\FrameServer\VapourSynth\vspipe.exe "F:\In\Attacco dei giganti S2\01_temp\01.vpy" - --y4m | D:\Eseguibili\Media\StaxRip\Apps\Encoders\x265\x265.exe --frames 34816 --crf 22 --output-depth 10 --aq-mode 5 --fades --pools 10 --colorprim bt709 --colormatrix bt709 --transfer bt709 --range limited --qpfile "F:\In\Attacco dei giganti S2\01.qp" --y4m --output "F:\In\Attacco dei giganti S2\01_temp\01_out.hevc" -

I get

Error: fwrite() call failed when writing frame: 5, plane: 0, errno: 32

Barough

14th September 2022, 22:34

x265 v3.5+40-1ea20502b
Built on September 13, 2022, GCC 12.2.0

https://www.mediafire.com/file/kfmz9m3q4r4fc6l/

x265 Note :

The commit value on the binary is wrong due to something upstream at MulticoreWare.

Correct commit number for this release is
9311783

LigH

15th September 2022, 00:21

Did you try to encode something with it?

I just tested encoding a few physical Y4M files, that worked.

rwill

15th September 2022, 02:05

With

D:\Eseguibili\Media\StaxRip\Apps\FrameServer\VapourSynth\vspipe.exe "F:\In\Attacco dei giganti S2\01_temp\01.vpy" - --y4m | D:\Eseguibili\Media\StaxRip\Apps\Encoders\x265\x265.exe --frames 34816 --crf 22 --output-depth 10 --aq-mode 5 --fades --pools 10 --colorprim bt709 --colormatrix bt709 --transfer bt709 --range limited --qpfile "F:\In\Attacco dei giganti S2\01.qp" --y4m --output "F:\In\Attacco dei giganti S2\01_temp\01_out.hevc" -

I get

Error: fwrite() call failed when writing frame: 5, plane: 0, errno: 32

Check again, you should get something like

x265 [error]: Aq-Mode is out of range
x265 [error]: x265_encoder_open() failed for Enc,
x265 [error]: Failure generating stream headers in x265
Segmentation fault

Errno 32 is "Broken Pipe", x265 dies and vspipe.exe can no longer push data downstream.

tormento

15th September 2022, 09:36

Aq-Mode is out of range
Is there an updated x265 with aq-mode 5, beside the Patman ones?

Barough

15th September 2022, 12:42

Is there an updated x265 with aq-mode 5, beside the Patman ones?

Not by MulticoreWare. Only available in DJATOM's or Patman's compiles to my knowledge.

tormento

15th September 2022, 20:28

Only available in DJATOM's or Patman's compiles to my knowledge.
Unfortunately both have little or no time at the moment.

PoeBear

19th September 2022, 22:16

What are the best settings to tweak for better edge detection between elements? Eg. I've got a 1080p mixed AR source (16:9 and 4:3) and anytime there's a colorful part of the image that hits the black border, there's a 1px column that gets sapped of color. It also happens to a lesser effect for colorful objects against darker backgrounds

Here is a cropped example of the specific areas featuring both:
https://thumbs4.imagebam.com/02/ea/1e/MEDM6PN_t.PNG (https://www.imagebam.com/view/MEDM6PN) https://thumbs4.imagebam.com/4f/02/37/MEDM6PM_t.PNG (https://www.imagebam.com/view/MEDM6PM)
You'll notice the obvious colorless 1px line across the side, and on the curtain's edge you can see the color start to fade where it converges with the dimly lit room it's in:

I've compared various psy-rd/psy-rdoq levels, never higher than psy-rdoq 10, against the veryslow preset with a static bitrate of 30,000 for testing. None of those adjustments seem to make a dent in getting the color back, so I thought there might be a less obvious setting I should be tweaking

LigH

20th September 2022, 11:57

The reason is probably the YUV 4:2:0 chroma subsampling and the placement of the border at an odd numbered pixel column.

benwaggoner

20th September 2022, 16:20

The reason is probably the YUV 4:2:0 chroma subsampling and the placement of the border at an odd numbered pixel column.
That's a very sound theory. And if so, not much can be done about it.

That said, this sort of artifact should be extremely hard to detect visually, especially while playing and at a full-screen visible viewing distance.

PoeBear

20th September 2022, 16:30

The reason is probably the YUV 4:2:0 chroma subsampling and the placement of the border at an odd numbered pixel column.

Should that not show up on the source side as well then? It's a 10-bit 4:2:0 image out of the frameserver (which is what my screenshot is from). The non-border part of the image is mod8, as well as the border part

And you can also see it's affecting the cloth's edge, to a lesser degree, that's still within the main part of the image

That said, this sort of artifact should be extremely hard to detect visually, especially while playing and at a full-screen visible viewing distance.

I don't disagree, but some of my projects require static transparency (generally for future encoding use), plus solutions are always good to have in the pocket, you never know where else you can use them

benwaggoner

20th September 2022, 18:07

Should that not show up on the source side as well then? It's a 10-bit 4:2:0 image out of the frameserver (which is what my screenshot is from). The non-border part of the image is mod8, as well as the border part

And you can also see it's affecting the cloth's edge, to a lesser degree, that's still within the main part of the image

I don't disagree, but some of my projects require static transparency (generally for future encoding use), plus solutions are always good to have in the pocket, you never know where else you can use them
Ah, if it is from a 4:2:0 source, there are things that might be done.

If it is an issue from chroma compression, try reducing chroma QP.
--cbqpoffs -3 --crqpoffs -3

Those values are generally too strong for most real-world use, and may well make the video look worse overall as it takes bits out of luma. But if they fix your chroma issue, that suggests it is an encoding issue. If nothing else, find the closest-to-zero values for the above that fixes your issue.

Can you share your command line? I'd want to use nothing faster than --preset slow as that is the fastest that includes --rd 3+ and--subme 3+, which are the lowest levels where chroma is included.

Also, if you specify a --chromaloc that doesn't match the source, that could potentially make horizontal chroma a little off.

Boulder

20th September 2022, 19:26

Ah, if it is from a 4:2:0 source, there are things that might be done.

If it is an issue from chroma compression, try reducing chroma QP.
--cbqpoffs -3 --crqpoffs -3

Those values are generally too strong for most real-world use, and may well make the video look worse overall as it takes bits out of luma. But if they fix your chroma issue, that suggests it is an encoding issue. If nothing else, find the closest-to-zero values for the above that fixes your issue.

It's worth mentioning that the bitrate issue is not valid for CRF mode, it just compresses a bit less. I myself use -3 and -6 respectively (with non-HDR sources) but the reds are destroyed by the encoder no matter what offsets are used. Maybe this artifact is related to that.

I think x264 uses -3 and -3 internally.

PoeBear

21st September 2022, 05:19

Also, if you specify a --chromaloc that doesn't match the source, that could potentially make horizontal chroma a little off.

I think this might be it. Although in theory my chromaloc 2 matches my source, according to its MediaInfo: 4:2:0 (Type 2)

But if I throw a propSet("_ChromaLocation",2) at the very end of my script, the preview looks an awful lot like my encode output, with the colorless line now appearing. I'm guessing somehow after all of my plugins and scaling, the frame properties are being stripped/changed. When I set x265 to chromaloc 0 the line is back to near full color

After a bit of searching mid-post, it seems the culprit is somehow... the AddBorders (https://forum.doom9.org/showthread.php?t=176867) I was using to pad/match the 16:9 footage!? Who would have thought? But if I put AddBorders at the very end of my preview script, after it tonemaps with Cube, it no longer produces the faded line. Somehow AddBorders is changing the chromaloc, or something

Now to figure out how to change it back, or find an alternative border solution, so I can have my output/encode chromaloc match my input/source. If I use ConvertToYUV444 before AddBorders, and then ConvertToYUV420 right after, it lessens the color sapping by a significant amount, and looks about how the x265/chromaloc 0 setting looks, but still not "perfect". A z_ConvertFormat(chromaloc_op="bottom=>top_left") before the AddBorders also looks relatively similar to 444, but does shift the chroma in the rest of the image as well...

But I'm also thinking maybe the non-16:9 footage wasn't handled correctly in the source to begin with, because the ConverFormat call seems to look more natural, ie the color goes to the edges on the curtains on my previous source screen, and it doesn't seem to make anything else worse skipping through

I've got a lot to unpack/test :( Suggestions are welcome, but now it's drifting outside of this thread's focus

benwaggoner

22nd September 2022, 00:58

I think this might be it. Although in theory my chromaloc 2 matches my source, according to its MediaInfo: 4:2:0 (Type 2)

But if I throw a propSet("_ChromaLocation",2) at the very end of my script, the preview looks an awful lot like my encode output, with the colorless line now appearing. I'm guessing somehow after all of my plugins and scaling, the frame properties are being stripped/changed. When I set x265 to chromaloc 0 the line is back to near full color
Ha! I thought it was likely that. I think any theoretical value in having different chromaloc is probably swamped by the difficulty in getting end-to-end chromaloc-accurate processing.

Now to figure out how to change it back, or find an alternative border solution, so I can have my output/encode chromaloc match my input/source. If I use ConvertToYUV444 before AddBorders, and then ConvertToYUV420 right after, it lessens the color sapping by a significant amount, and looks about how the x265/chromaloc 0 setting looks, but still not "perfect". A z_ConvertFormat(chromaloc_op="bottom=>top_left") before the AddBorders also looks relatively similar to 444, but does shift the chroma in the rest of the image as well...
Yeah, your the best bet is to probably to do everything in RGB or Y'CbCr 4:4:4, and then only convert to the final 4:2:0 as the very last step before encoding with a known-good conversion tool.

Also, some decoders/rendering pipelines used to ignore chromaloc, and do everything as if it was chromaloc 0. More than once I've realized something worked only because source processing and decoding both ignored chromaloc, and so reversed the other. I've not tested that recently.

Launching HDR in 2015 was...interesting!

22nd September 2022, 14:09

When looking at the discussion on the forum, I often see these two parameters. I have read its documentation many times, but I can't understand the causal relationship between its skipping mechanism and the improvement of image quality?

The moderators say that --tskip and --tskip-fast can improve the quality of animations. Besides animations, what types of images will the effects of --tskip affect?
Are there any inappropriate types?

It is known that the file size is smaller when only --tskip is used, and the encoding speed is faster when both --tskip and --tskip-fast are used.

So what is the difference in the effect of these two combinations on improving the picture quality?

Regardless of the compression ratio, which option will improve the picture quality better?

rwill

22nd September 2022, 19:20

Also, if you specify a --chromaloc that doesn't match the source, that could potentially make horizontal chroma a little off.

This is somewhat unlikely. Chroma Loc 0,2,4 are left and Chroma Loc 1,3,5 are center on the horizontal axis.

The only Choma Locations that make sense in the current environment are 0 and 2, both left aligned. To find Chroma Loc 1,3,4 and 5 in the wild is no accident but almost borders on sabotage.

rwill

22nd September 2022, 19:31

The moderators say that --tskip and --tskip-fast can improve the quality of animations. Besides animations, what types of images will the effects of --tskip affect?
Are there any inappropriate types?

Porn

It is known that the file size is smaller when only --tskip is used, and the encoding speed is faster when both --tskip and --tskip-fast are used.

So what is the difference in the effect of these two combinations on improving the picture quality?

Regardless of the compression ratio, which option will improve the picture quality better?

The option with the highest compression ratio will improve picture quality the most, in theory anyway.

23rd September 2022, 15:11

The same type will have different characteristics, and different types will also have the same characteristics. So what I want to ask is the specific shape characteristics, rather than the noun used in commodity classification.

The option with the highest compression ratio will improve picture quality the most, in theory anyway.

Your theory is more applicable to X264, but X265 has a lot more subjective compression options than X264, and these options can't be summarized in a simple set of theories. Psycho -Visual Options is one example, and --tskip and --tskip-fast are another similar example.

benwaggoner

23rd September 2022, 17:02

This is somewhat unlikely. Chroma Loc 0,2,4 are left and Chroma Loc 1,3,5 are center on the horizontal axis.

The only Choma Locations that make sense in the current environment are 0 and 2, both left aligned. To find Chroma Loc 1,3,4 and 5 in the wild is no accident but almost borders on sabotage.
Sorry, yes. I meant vertical.

benwaggoner

23rd September 2022, 17:03

Your theory is more applicable to X264, but X265 has a lot more subjective compression options than X264, and these options can't be summarized in a simple set of theories. Psycho -Visual Options is one example, and --tskip and --tskip-fast are another similar example.
That said, I'm not aware of any cases where using --tskip causes quality regressions. Have you seen some?

microchip8

23rd September 2022, 17:15

That said, I'm not aware of any cases where using --tskip causes quality regressions. Have you seen some?

I've done almost a 100 encodes with --tskip enabled (--tskip 1) and honestly I can't see a difference nor are there any issues in all of the encodes.

I do have --tskip-fast disabled (--tskip-fast 0) so can't talk about that

kedautinh12

24th September 2022, 01:52

Source was 4K, downsampled to 1080p. These are the basic parameters that I've set:
--input-depth 16
--dither
--profile main10
--min-keyint 5
--keyint 480
--merange 44
--splitrd-skip
--preset veryslow
--rc-lookahead 60
--deblock -2:-2
--no-strong-intra-smoothing
--no-sao
--qcomp 0.8
--aq-mode 3
--aq-strength 0.8
--ctu 64
--max-tu-size 32
--rdpenalty 1
--qg-size 16
--tu-inter-depth 4
--tu-intra-depth 4
--limit-tu 4
--limit-refs 3
--max-merge 2
--rd-refine
--ref 6
--bframes 10
--crf 19
I've then also tested ctu 32 with max-tu-size 32 or 16.

In the 1080p encode, the bitrate is as follows:

CTU 64 / TU 32 - 15078 kbps
CTU 32 / TU 32 - 14608 kbps
CTU 32 / TU 16 - 14472 kbps

Of these, CTU 32 / TU 32 resembles the original the most. It's interesting that setting TU 16 also causes distortion in the same areas as CTU 64 / TU 32. I checked areas like eyes, hair etc. which have easily some sort of distortion because there are many fine lines and things that can be compared quite easily.

I've just started testing what CRF value is visually enough for 1080p so the final bitrate will probably be lower than what I got from my tests. I'd estimate CRF 20-21 would be the final value.

You still use those settings now??

kedautinh12

25th September 2022, 07:16

When i used settings about. The filesize of output file same as x264 or maybe larger than x264

benwaggoner

26th September 2022, 20:50

I've done almost a 100 encodes with --tskip enabled (--tskip 1) and honestly I can't see a difference nor are there any issues in all of the encodes.

I do have --tskip-fast disabled (--tskip-fast 0) so can't talk about that
For most content, tskip doesn't do much at all. It's only with really sharp, typically synthetic content where we see improvements. In a typical movie, I would only expect tskip to kick in for the opening and end credits, which are computer-rendered without any noise.

tskip might also help with burned-in subtitles, but I've not tested that deeply.

Stereodude

29th September 2022, 15:59

Do the latest builds that I see posted in the thread support opening a .avs directly or do they require piping?

Edit: I see they require piping. Where can I get a recent build that doesn't require piping? I think Wolfberry was posting them.

jpsdr

29th September 2022, 18:05

I've seen some new version posted here, but nothing related commited on the git : https://bitbucket.org/multicoreware/x265_git.git
Is it the proper git i'm using ?

quietvoid

29th September 2022, 18:19

I've seen some new version posted here, but nothing related commited on the git : https://bitbucket.org/multicoreware/x265_git.git
Is it the proper git i'm using ?

That's the correct official repo.
There are other forks with native frameserver support (AVS and VS).

benwaggoner

29th September 2022, 23:20

When i used settings about. The filesize of output file same as x264 or maybe larger than x264
Does the quality compare?

If you really want to understand quality @ bitrate, you'll need to compare 2-pass ABR encodes. There's certainly a high enough bitrate than x264 and x265 will converge on transparent quality.

Your settings have a lot of very specific stuff in them, and may not be an optimal combination of features. Starting with a preset and iterating from there is a simpler way to start where parameters have been extensively tested in that combination.

I've not seen anyone use --rdpenalty in at least five years. And --no-strong-intra-smoothing hasn't been demonstrated to actually improve anything, and can reduce detail if the reduced inefficiency pushed up QPs.

4th October 2022, 06:31

That said, I'm not aware of any cases where using --tskip causes quality regressions. Have you seen some?

I read the documentation and learned that --tskip bypass DCT processing, but I am not an expert in related fields, so I want to ask the approximate principle between bypass DCT and improving image quality, otherwise --tskip is too mysterious.

I understand --tskip and psycho visual options as the same category, the reason is not --tskip causes quality regressions,
The reason is that they all give up a part of the mathematically optimal compression, both of which will increase kbs, and I guess DCT is part of the mathematically optimal compression.

So there are two choices here, a lower crf with --no-tskip, or a higher crf with --tskip, assuming they are the same bitrate.

Not sure about the principle of -tskip, it's really hard to choose.

rwill

4th October 2022, 07:17

I understand --tskip and psycho visual options as the same category, the reason is not --tskip causes quality regressions,
The reason is that they all give up a part of the mathematically optimal compression, both of which will increase kbs, and I guess DCT is part of the mathematically optimal compression.

The Transform Skip Tool is more or less for compression efficiency enhancement. It does not relate to psycho visual options. If enabled the ability to skip the transform or not for a transform unit will become available under certain conditions and an encoder might check if it is beneficial for compression. If a transform unit is then better coded with transform skipped than with the frequency transform the encoder will/should code the transform unit as such. There is a slight overhead associated with signalling transform skip but for most use cases the option for an encoder to skip the transform rarely becomes available anyway, making overhead a non issue.

4th October 2022, 09:01

The Transform Skip Tool is more or less for compression efficiency enhancement.

Even though the original intention of designing the Transform Skip Tool is to reduce the time required for compression, the actual discussion and use are related to improving the subjective visual quality. DCT may have some adverse effects on subjective visual quality.
What exactly is the impact, it's not clear?

If a transform unit is then better coded with transform skipped than with the frequency transform the encoder will/should code the transform unit as such.

What is a better coding transform unit, and what criteria are used to judge it?

rwill

4th October 2022, 14:02

Even though the original intention of designing the Transform Skip Tool is to reduce the time required for compression
Actually it increases the time required for compression as more options for encoding become available by adding an encoding tool.

the actual discussion and use are related to improving the subjective visual quality.
I personally don't understand why though.

DCT may have some adverse effects on subjective visual quality.
What exactly is the impact, it's not clear?

Well, I suggest to stop worrying and just set a --preset, a --tune and --no-sao.

What is a better coding transform unit, and what criteria are used to judge it?
Rate/Distortion.

benwaggoner

4th October 2022, 16:58

Transform skip is described briefly in this post from the invaluable thread to understand how HEVC works under the hood: https://forum.doom9.org/showthread.php?p=1614299&highlight=transform+skip#post1614299

Yes, DCT is broadly optimal for natural image content. It converts from the pixel values to frequency values, which are much better tuned to the human visual system, which is very focused on seeing edges.

However, it is not optimal for coding sharp differences between adjacent pixels, which essentially have a infinite frequency. Hence transform skip, which can more efficiently code things like aliasing artifacts, alternating black/white checkerboard pixels, sharp edges of text, etcetera.

x265 will only pick tskip when it is more efficient than a DCT option. I'm not aware of it ever having any visual quality regression, but for live-action content, it rarely offers any benefit either. But digitally rendered scrolling end credits might save significant bits.

Barough

5th October 2022, 00:17

x265 v3.5+41-0b75c44
Built on October 05, 2022, GCC 12.2.0

https://bitbucket.org/multicoreware/x265_git/commits/branch/master

32-bit
x265 [info]: HEVC encoder version 3.5+41-c02eb2842
x265 [info]: build info [Windows][GCC 12.2.0][32 bit] 8bit+10bit+12bit

64-bit
x265 [info]: HEVC encoder version 3.5+41-4be2fd080
x265 [info]: build info [Windows][GCC 12.2.0][64 bit] 8bit+10bit+12bit

NOTE:

The commit value is wrong due to something upstream at MulticoreWare.

Correct commit number for this release is
0b75c44

DL :
https://www.mediafire.com/file/tye1v6btgm4j9s1/

ShortKatz

15th October 2022, 23:44

Here is HEVC encoder version 3.5+40-0b75c44c1 precompiled for macOS. It is a fat binary that includes architectures x86_64; arm64; arm64e.
Build on October 16, 2022.
x265 [info]: HEVC encoder version 3.5+40-0b75c44c1
x265 [info]: build info [macOS][clang 14.0.0][64 bit][noasm] 8bit+10bit+12bit

https://polysom.verilite.de/tmp/x265_macOS.tar.gz
SHA256: 68498485a84089ce376ff6362123e3381e32ce1c5262092b66699872b9c3c6f6

Ritsuka

16th October 2022, 07:39

Why is compiled without asm? That's going to be slow.

rwill

16th October 2022, 08:20

Why is compiled without asm? That's going to be slow.

I was told by Apple fans that applications running on Apple devices are always fast enough regardless of their optimizations. So they are never slow.

ShortKatz

16th October 2022, 09:53

Why is compiled without asm? That's going to be slow.

I don't know why it does say noasm. It is build with ENABLE_ASSEMBLY=1. But as rwill said, it will be fast enough.

Jamaika

16th October 2022, 10:38

LigH

16th October 2022, 18:16

Boy, these are warnings; it still works correctly. Just don't read the console while compiling. The only issue is that their output slows down the compilation process. But better that than a slow video encoding.
_

Oh, sorry, Jamaika; I assumed that was a reply to ShortKatz, but I doubt now.

Jamaika

16th October 2022, 19:53

rwill

16th October 2022, 21:19

No need to apologize. Another version of nasm.
What processors does x265 support for the AVX512?

Hey I can answer that one.

x265 checks cpuid eax=7, ecx=0 results for ebx=0xD0030000

see here:
https://bitbucket.org/multicoreware/x265_git/src/0b75c44c10e605fe9e9ebed58f04a46271131827/source/common/cpu.cpp#lines-204

now 0xd0030000 is 11010000000000110000000000000000b which you can check here:
https://en.wikipedia.org/wiki/CPUID#EAX=7,_ECX=0:_Extended_Features

so x265 requires: AVX-512 Foundation, AVX-512 Doubleword and Quadword Instructions, AVX-512 Conflict Detection Instructions, AVX-512 Byte and Word Instructions and AVX-512 Vector Length Extensions. Thats AVX512-*: F, DQ, CD, BW and VL in short.

This can be looked up here:
https://en.wikipedia.org/wiki/AVX-512#CPUs_with_AVX-512

So, Intel apparently has quite a few unless they disable it for.. "reasons" and AMD seems to support it with Zen4.

* Note: Wikipedia might not be a reliable information source.

*edit*
Intel CPU capabilities are very confusing as they enable and disable features with each generations quite randomly, especially on Desktop.
I think its best to check the Intel Ark website to check if a CPU supports AVX512 or not.
Here:
https://ark.intel.com/

FranceBB

17th October 2022, 09:32

What processors does x265 support for the AVX512?

For what it's worth, I can only test with my very own CPU and all I can say is that it works with my Intel Xeon Gold 6238R, however you have to enable it as it's disabled by default. I probably have a screenshot somewhere.

EDIT: Found it, I'm gonna quote myself from last year:

x265 3.5+13-4c6880d

https://i.imgur.com/jSLRrzL.png

Emulgator

17th October 2022, 12:35

i9-11900K here: AVX512 yes, and one got to enable it in x265 CL, as FranceBB mentioned.

LeXXuz

17th October 2022, 13:55

Is AVX512 really that much of a holy grail some people do claim - regarding performance?

LigH

17th October 2022, 14:05

No. That's why its support is not auto-detected by default. It is a heat risk with only a small performance gain over AVX2.

Jamaika

18th October 2022, 06:34

For what it's worth, I can only test with my very own CPU and all I can say is that it works with my Intel Xeon Gold 6238R, however you have to enable it as it's disabled by default. I probably have a screenshot somewhere.
How can AVX2 enable?
I turn on basic SIMD X265.
#define ARCH_X86_64
#define HAVE_SSE3
#define HAVE_SSSE3
#define HAVE_SSE4
The rest of the AVX2 and AVX512 are probably default.
ARCH_X86_64 && cpuflag(avx512)

benwaggoner

18th October 2022, 19:14

How can AVX2 enable?

AVX2 on down are automatically enabled if available. AVX512 is the special case, as it is more likely to reduce performance than increase it.

19th October 2022, 10:14

Will the current x265-3.5 encoder use ctu 64 parameters still cause 64x64 block textures to be weakened?

vpupkind

19th October 2022, 17:01

AVX2 on down are automatically enabled if available. AVX512 is the special case, as it is more likely to reduce performance than increase it.
I've seen <=5% improvement in motion estimation (umh) in Cascade Lake times, haven't tested on IceLake though. The problem used to be thermals -- a 900MHz frequency penalty due to use of AVX-512.

FranceBB

19th October 2022, 22:08

How?
ARCH_X86_64 && cpuflag(avx512)

You can enable any asm with the --asm command, like:

--asm avx512

the same goes if you want to enable only a specific set, like maybe you wanna try the speed differences between AVX512 and AVX so you can use

--asm avx

and so on and of course there's

--no-asm

for plain C.

AVX512 is the special case, as it is more likely to reduce performance than increase it.

...on consumer hardware.
Things are of course different in perfectly cooled down server rooms.

The problem used to be thermals -- a 900MHz frequency penalty due to use of AVX-512.

Correct. Temperatures skyrocket high and make the CPU throttle thus going down up to a point that it's no longer feasible and the speed gain is nullified.
This of course doesn't apply to server rooms where temperature and humidity is perfectly controlled and CPUs can keep a high enough clock under pressure like it happens with my encodes.

This is the situation while encoding a MJPEG2000 4:4:4 12bit HDR PQ IMF with x265 to create a consumer H.265 file (with AVX-512 enabled):

https://i.imgur.com/U6sWEAN.jpghttps://i.imgur.com/MpyQmOh.jpghttps://i.imgur.com/Sy45bRz.jpg

as you can see, clock fluctuates a bit, but given that the overall temperature of the server room is really low, the CPU is able to keep working at regime under sustained pressure and in this case AVX-512 really make sense, which is why they're enabled in all my workflows.

If it wasn't for the wide user-driven community and the open source nature of the project, I would argue AVX512 should be enabled by default just like other intrinsics, 'cause for companies it makes a whole lot of sense...

excellentswordfight

19th October 2022, 22:58

...on consumer hardware.
Things are of course different in perfectly cooled down server rooms.

? Not sure what you are on about, for example all Skylake-SP Xeons i've run avx512 code on downclocked ALOT more than my "consumer" Tiger-Lake laptop (and yeah the servers has ofc been in a proper server environment). This topic is a lot more complex then if you run the system in a cool room or not, and I'm pretty sure that most of the downclock on Xeons I've seen has been of powerlimit, not tempeture. AVX-512 behavior has been tweaked alot between models/platforms.

2,6Ghz looks rather good for 28C xeon under avx512 load. Mind sharing the frequency when you run x265 without it and the performance Numbers? Cause i also have some systems with Cascade Lake Refresh Xeons and i lost 5-10% when using avx512.

Edit. You pipe avs to x265 right? How CPU intesive is that script? How much of the load does the x265 process account for?

Emulgator

20th October 2022, 00:40

Is AVX512 really that much of a holy grail some people do claim - regarding performance?
Not that one should hold his breath or should spend more on such CPU, as LigH says.
Here on a i9-11900K notebook ("14nm", 70nm gate pitch, 125W, 5,3GHz single core turbo, 8 core steady 4,5GHz)
I remember to have seen like +10 to +20% x265 fps gain invoking x265 -avx512.
Sustained AVX512 CPU clock was above 4GHz, downclocking was avoided with 2 fans running full 4800rpm.
Nice to have, and I will stay there, but not worth a bigger fuss.
Will have to repeat a test encode and note down my comparison.

Now in 2022 a "5nm" (in reality 51nm gate pitch) CPU (AMD) will be the better investment, I guess.

benwaggoner

20th October 2022, 07:29

So, I finally picked up a 16" Apple MacBook Pro with a M1 Pro processor today.

What's the best way to get a well-optimized x265 binary to run on this hardware?

vpupkind

20th October 2022, 15:49

...on consumer hardware.
Things are of course different in perfectly cooled down server rooms.

Correct. Temperatures skyrocket high and make the CPU throttle thus going down up to a point that it's no longer feasible and the speed gain is nullified.
This of course doesn't apply to server rooms where temperature and humidity is perfectly controlled and CPUs can keep a high enough clock under pressure like it happens with my encodes.

I actually ran this on a bunch of 18-core Sky Lake and Cascade Lake Gold server CPUs. The problem is the internal CPU throttling. We locked it in the P0 state (no throttling overall), but use of AVX-512 still automagically reduced the whole CPU's frequency. Cascade Lake reduced the impact to a subset of cores physically close to the one using AVX-512.

Ritsuka

20th October 2022, 16:56

So, I finally picked up a 16" Apple MacBook Pro with a M1 Pro processor today.

What's the best way to get a well-optimized x265 binary to run on this hardware?

Unfortunately I think you will have to compile it yourself, and maybe apply a couple of patches from https://github.com/HandBrake/HandBrake/tree/master/contrib/x265

qyot27

20th October 2022, 17:32

So, I finally picked up a 16" Apple MacBook Pro with a M1 Pro processor today.

What's the best way to get a well-optimized x265 binary to run on this hardware?
I would have said 'MacPorts or Homebrew', except that it seems neither of them apply the NEON acceleration patch for x265 that Apple submitted to Handbrake. And I don't know if the contents of said patch would otherwise be superseded by whatever NEON stuff has been committed upsteam (if any has). I've not tried the patch on my M1 Mac Mini, so I can't say what the difference in performance is (I also don't remember if I bothered building x265 there, either).

There is this build script that handles several things - including the Apple patch - for building FFmpeg: https://github.com/Vargol/ffmpeg-apple-arm64-build. As an aside, the 'avisynth' branch on that repo confuses me, because it's up-to-date with the master branch and has no additional changes, and the master branch doesn't have it enabled, even though that's fully possible.

I actually ran this on a bunch of 18-core Sky Lake and Cascade Lake Gold server CPUs. The problem is the internal CPU throttling. We locked it in the P0 state (no throttling overall), but use of AVX-512 still automagically reduced the whole CPU's frequency. Cascade Lake reduced the impact to a subset of cores physically close to the one using AVX-512.
To expand on this,
https://en.wikipedia.org/wiki/AVX-512#Performance
https://en.wikipedia.org/wiki/Advanced_Vector_Extensions#Downclocking

AVX downclocking was present as actual modes in several generations, based on the width of the executed instructions. To wit, GCC and Clang prefer a vector width of 256 when using AVX-512, which would largely sidestep the issue. From the snippets I've read on the topic, this also seems to be the way Zen4 implements AVX-512 in hardware.

Skylake had three levels, Ice Lake had only two. But as of Rocket Lake, those explicit downclocking modes are gone. AVX-512 will not downclock on modern generations just because 512-wide vectors get used, but only because doing so may or may not hit standard thermal or power limits, same as any other intensive process.

benwaggoner

20th October 2022, 19:43

I would have said 'MacPorts or Homebrew', except that it seems neither of them apply the NEON acceleration patch for x265 that Apple submitted to Handbrake. And I don't know if the contents of said patch would otherwise be superseded by whatever NEON stuff has been committed upsteam (if any has). I've not tried the patch on my M1 Mac Mini, so I can't say what the difference in performance is (I also don't remember if I bothered building x265 there, either).

There is this build script that handles several things - including the Apple patch - for building FFmpeg: https://github.com/Vargol/ffmpeg-apple-arm64-build. As an aside, the 'avisynth' branch on that repo confuses me, because it's up-to-date with the master branch and has no additional changes, and the master branch doesn't have it enabled, even though that's fully possible.
And that worked just great on the first try! A lot easier than using autobuildsuite on Windows 10.

I'd still like to get a separate x265 binary not in ffmpeg so I can use identical syntax across platforms, but this is certainly enough for perf testing.

Ritsuka

20th October 2022, 20:12

x265 master branch already contains the Apple intrinsics patches, plus a lot of additional Neon optimizations provided by Amazon. No need to look for weird forks or branches. But there are still a couple of patches in the HandBrake repository that will make it run better.

LeXXuz

21st October 2022, 09:25

Pardon my ignorance, but why does AVX512 produce so much more heat than AVX2 mode in x265? :confused:

benwaggoner

22nd October 2022, 00:17

Wow, x265 got more commits nine hours ago than it got the rest of 2022!

https://bitbucket.org/multicoreware/x265_git/commits/

benwaggoner

22nd October 2022, 00:28

This is an interesting new command line added!

--[no-]mctf Enable GOP based temporal filter.

LigH

22nd October 2022, 15:53

Wow, x265 got more commits nine hours ago than it got the rest of 2022!

https://bitbucket.org/multicoreware/x265_git/commits/

I hope that fixed some of the issues I had to complain about. The last set of commits before that severely destroyed compilation or multilib linking in MSYS2 with GCC 12.2.
_

No, there is no fix yet; MABS does not build x265 anymore.

quietvoid

22nd October 2022, 16:54

The new SBRC patch forgot to free memory used from edge detection buffers, so it probably leaks when using --sbrc without AQ mode 4.
It's essentially the same as a 2 year old patch for auto-AQ varying by frame average brightness and edge density.

jpsdr

23rd October 2022, 19:09

@quietvoid
Can you point where in what file ?

I've created an x265 repo in my github with a branch with the avs support patchs.
No build yet, but will soon. A few month ago i've success building with VS, so, if nothing bad happened, still should be able to... :D

quietvoid

23rd October 2022, 19:51

@quietvoid
Can you point where in what file ?

The allocations here https://github.com/jpsdr/x265/blob/x265_mod/source/common/frame.cpp#L134
Have a different condition compared to the freeing here https://github.com/jpsdr/x265/blob/x265_mod/source/common/frame.cpp#L345

It's missing param->rc.frameSegment to free.

Also, you changed the upstream behaviour for sbrc without renaming the edge AQ fallback mode: https://github.com/jpsdr/x265/blob/x265_mod/source/encoder/slicetype.cpp#L1469
It's supposed to be AQ 1, not 5.

jpsdr

23rd October 2022, 20:30

Ok, one of the patches added a constant with the same name, i must admit that i was confused when trying to solve the conflict when i "fetched and rebased".
Push commit on my branch. VS build failed... :(
Have to test more...

ShortKatz

24th October 2022, 06:58

This is an interesting new command line added!

--[no-]mctf Enable GOP based temporal filter.

I've also have seen this. Could somebody point out to me, for which use cases this might be interesting? When to use this?

ShortKatz

24th October 2022, 07:00

It's essentially the same as a 2 year old patch for auto-AQ varying by frame average brightness and edge density.

I use this auto-AQ patches still with all of my own builds of HandBrake. What do you mean about "varying by frame average brightness and edge density"? Are those patches incomplete then?

LigH

24th October 2022, 10:06

The auto-AQ feature has different modes, and the patch is for one specific mode.

quietvoid

24th October 2022, 14:20

I use this auto-AQ patches still with all of my own builds of HandBrake. What do you mean about "varying by frame average brightness and edge density"? Are those patches incomplete then?

They just renamed the option to `--sbrc`, as far as I can tell the behaviour should be the same in both old and new patches.
So nothing is really incomplete. Instead of specifying both --aq-mode 5 and --auto-aq, you just do --sbrc now.

Now it analyses the brightness and edge density for every frame, and switches the AQ according to thresholds (bright frame = mode 2, otherwise 3. high edge density = mode 4, otherwise mode 1).

Boulder

24th October 2022, 14:51

ShortKatz

24th October 2022, 17:49

They just renamed the option to `--sbrc`, as far as I can tell the behaviour should be the same in both old and new patches.
So nothing is really incomplete. Instead of specifying both --aq-mode 5 and --auto-aq, you just do --sbrc now.

Now it analyses the brightness and edge density for every frame, and switches the AQ according to thresholds (bright frame = mode 2, otherwise 3. high edge density = mode 4, otherwise mode 1).

OK. Now I understand. They implemented a variation of the auto-aq patch now as --sbrc. Thats fine, then I don't need to apply the auto-aq patches anymore.

But I don't understand why this is "1" shouldn't this be "5"? This was "5" in the old auto-aq patch.
https://bitbucket.org/multicoreware/x265_git/src/8584bc7bd99262b8bd926476c866840fe0d9428a/source/x265.h#lines-584

LeXXuz

24th October 2022, 18:28

quietvoid

24th October 2022, 18:30

OK. Now I understand. They implemented a variation of the auto-aq patch now as --sbrc. Thats fine, then I don't need to apply the auto-aq patches anymore.

But I don't understand why this is "1" shouldn't this be "5"? This was "5" in the old auto-aq patch.
https://bitbucket.org/multicoreware/x265_git/src/8584bc7bd99262b8bd926476c866840fe0d9428a/source/x265.h#lines-584

It was a separate mode in the old patch, now it falls back to AQ mode 1 when the edge density is too low.
There's no more mode 4 + dark bias, as mode 5 was. At last in the upstream code, other forks still have mode 5.

quietvoid

24th October 2022, 18:37

jpsdr

24th October 2022, 20:15

Ah.... I've been too fast... ;)
Changed, thanks.

:thanks:

benwaggoner

24th October 2022, 21:07

It was a separate mode in the old patch, now it falls back to AQ mode 1 when the edge density is too low.
There's no more mode 4 + dark bias, as mode 5 was. At last in the upstream code, other forks still have mode 5.
Dark bias should really be its own independent parameter. Ideally a strength parameter, not just on/off.

Probably different for SDR and HDR content as well. In HDR it should tweak --hdr10-opt tendency to remove bits from dark regions.

benwaggoner

24th October 2022, 21:13

Are there any proper tests with the old patch? For any higher bitrate encode, I've always found mode 1 best but an auto switching mode could be ideal. Though HDR encodes are quite dark compared to SDR, which will make things harder.
"Dark" in the sense that the median code value is a lot further from median nits? HDR obviously can get way brighter than SDR (PQ can go up to 10K nits, versus a nominal 100 nits of SDR).

Psychovisual optimizations based around luminance aren't very fungible between SDR and HDR for that reason. --aq-mode 4 wastes bits like crazy with HDR, for example, as visible steps are much, much closer together near dark than with SDR.

quietvoid

24th October 2022, 21:35

Dark bias should really be its own independent parameter. Ideally a strength parameter, not just on/off.Well good thing adjustable bias strength is already a thing in the x265 forks.
It was added along with aq-mode 5, and it's just a multiplier on top of the AQ strength.

ShortKatz

24th October 2022, 22:04

It was a separate mode in the old patch, now it falls back to AQ mode 1 when the edge density is too low.
There's no more mode 4 + dark bias, as mode 5 was. At last in the upstream code, other forks still have mode 5.
Thanks for pointing this out. I found this which adds back mode 5.
https://gist.github.com/noizuy/fbd25590a4cd2ca3a1c635d3a01f59cf
I will give this a try.

LigH

24th October 2022, 22:55

Has anyone an actual compiled build with that patch for me to try? I'd like to test that feature on some problematic files.

I can't build multilib executables anymore since the last 14 patches on Friday, Oct. 21 :rolleyes:

Boulder

25th October 2022, 06:31

"Dark" in the sense that the median code value is a lot further from median nits?

Based on what I expect the encoder to "see" when it analyzes the stream. Somehow I believe that it uses the raw flat image you see if you open the script in VDub2 without any grading. So the material appears darker and more flat than it actually is upon playback on an HDR capable display.

benwaggoner

25th October 2022, 16:27

Based on what I expect the encoder to "see" when it analyzes the stream. Somehow I believe that it uses the raw flat image you see if you open the script in VDub2 without any grading. So the material appears darker and more flat than it actually is upon playback on an HDR capable display.
Gotcha.

It's not like SDR is more "real" pixels than HDR, it's just been the primary EOTF (Electro-optical transfer function) for the first decades of digital video. But gamma is a psychovisual optimization every bit as much as the PQ curve, just with simpler math that isn't optimal for a broader dynamic range.

It's really hard to work with HDR without a 10-bit HDR display and HDR-enabled tools, because what you see is not what you get. Encoders that are tuned for SDR aren't optimal for HDR, and vise versa. Optimal SDR needs lower QPs near black as code value steps are much more visible at the low end than the high end in SDR. HDR is much more perceptually uniform, and benefits from different optimizations like --hdr10-opt which would make near-dark look really bad if applied to SDR.

Boulder

25th October 2022, 18:03

Gotcha.

It's not like SDR is more "real" pixels than HDR, it's just been the primary EOTF (Electro-optical transfer function) for the first decades of digital video. But gamma is a psychovisual optimization every bit as much as the PQ curve, just with simpler math that isn't optimal for a broader dynamic range.

It's really hard to work with HDR without a 10-bit HDR display and HDR-enabled tools, because what you see is not what you get. Encoders that are tuned for SDR aren't optimal for HDR, and vise versa. Optimal SDR needs lower QPs near black as code value steps are much more visible at the low end than the high end in SDR. HDR is much more perceptually uniform, and benefits from different optimizations like --hdr10-opt which would make near-dark look really bad if applied to SDR.

Yeah, it's sometimes difficult to grasp the idea that the PC monitor can fool you big time with HDR encodes..

My thinking was more or less around the fact that for example aq-mode 3 has not been recommended for HDR encodes because it will start allocating bits into all the wrong places. Now if this new automatic method uses all modes as it seems necessary, things could go quite wrong. A movie like Blade Runner 2049 might be a good test since it has both ends of the spectrum.

benwaggoner

25th October 2022, 21:45

My thinking was more or less around the fact that for example aq-mode 3 has not been recommended for HDR encodes because it will start allocating bits into all the wrong places. Now if this new automatic method uses all modes as it seems necessary, things could go quite wrong. A movie like Blade Runner 2049 might be a good test since it has both ends of the spectrum.
Gotcha.

Lots of algorithms assume perceptual uniformity across code values, so the difference between 16 and 17 is as important as between 234 and 235. And while that's not true for SDR, it is true for PQ, so I've seen non-SDR optimized algorithms that work better with PQ than SDR. However, if an algorithm has a low-luma bias for SDR, it will work quite a bit worse with HDR.

Hopefully the algorithm either ignores dark bias, or is adaptive based on the color space being encoded.

In the early years of HDR encoding, we fundamentally encoded everything as if it were SDR, avoiding any SDR-optimized tuning, and it worked okay. Later we got stuff like --hdr10-opt which specifically optimizes for HDR.

Barough

26th October 2022, 08:13

x265 v3.5+57
Built on October 26, 2022, GCC 12.2.0

https://bitbucket.org/multicoreware/x265_git/commits/branch/master

DL :
https://www.mediafire.com/file/9ohiddzw2yftl7w

LigH

26th October 2022, 10:47

MABS included a patch which should be commited soon in the x265 repo.

Boulder

26th October 2022, 12:52

Later we got stuff like --hdr10-opt which specifically optimizes for HDR.

I'm afraid this optimization is just adjusting QP offsets, for example this small snippet of a code. From what I gather, any luma level related aq-mode would be doing similar things on top or before that, which cannot be good practice.

if (param->bHDR10Opt)
{
uint32_t sum = lumaSumCu(curFrame, blockX, blockY, param->rc.qgSize);
uint32_t lumaAvg = sum / (loopIncr * loopIncr);
if (lumaAvg < 301)
qp_adj += 3;
else if (lumaAvg >= 301 && lumaAvg < 367)
qp_adj += 2;
else if (lumaAvg >= 367 && lumaAvg < 434)
qp_adj += 1;
else if (lumaAvg >= 501 && lumaAvg < 567)
qp_adj -= 1;
else if (lumaAvg >= 567 && lumaAvg < 634)
qp_adj -= 2;
else if (lumaAvg >= 634 && lumaAvg < 701)
qp_adj -= 3;
else if (lumaAvg >= 701 && lumaAvg < 767)
qp_adj -= 4;
else if (lumaAvg >= 767 && lumaAvg < 834)
qp_adj -= 5;
else if (lumaAvg >= 834)
qp_adj -= 6;
}

jpsdr

26th October 2022, 13:42

Build a release of current version with Patman mods.

https://github.com/jpsdr/x265/releases

Stereodude

26th October 2022, 16:28

Build a release of current version with Patman mods.

https://github.com/jpsdr/x265/releases
Thanks for the builds. Sorry if this is a dumb question, but what's the difference between the standard build and the AVX2 builds? The standard one shows it has AVX2 enabled.

jpsdr

26th October 2022, 17:02

Standard is build with no specific arch instruction (for the C code part), so it will run on "any" CPU. The AVX2 build is build specifying AVX2 arch CPU instruction (for the C code part), so, it can't run on CPU without at least AVX2.

LeXXuz

26th October 2022, 19:22

Build a release of current version with Patman mods.

https://github.com/jpsdr/x265/releases

Thanks a lot for this. :thanks:

benwaggoner

26th October 2022, 20:00

I'm afraid this optimization is just adjusting QP offsets, for example this small snippet of a code. From what I gather, any luma level related aq-mode would be doing similar things on top or before that, which cannot be good practice.
Yes, it is exactly doing QP offsets for psychovisual improvements. IIRC, it's based on an IEEE paper from the mid 2010s. It's a pretty basic implementation with room for further enhancements, certainly. It can wind up raising QPs in the darks a little too much.

jpsdr

27th October 2022, 16:56

As i've said, i have some concern about possible flickering issue with sbrc. One possible "fix" would be some kind of "fuzzy" logic, but it's not always possible, another, easier to put is to have an hysteresis.

In my mod branch, i've made a patch to allow the possibility of an hysteresis in sbrc decision. Limit/range of hysteresis is for now totaly "out of the blue", no idea what to put (relative like i've done, static...).
No build made yet, your thoughts about it : https://github.com/jpsdr/x265/commit/14ec6023f9e564f516687c1b0220b89f8e78f255

Also, your thoughts about another possible sbrc tweak : add (another...) switch to allow X265_AQ_EDGE : X265_AQ_EDGE_BIASED instead of X265_AQ_EDGE : X265_AQ_EDGE_BIASED_SBRC.

Waiting your feedback... :D

jpsdr

27th October 2022, 17:04

Just thought !!!
Can sbrc be ported back to x264 ??? I don't think this feature exist.
If i'm so over it, it's because i found the idea (with some minor adjustments :D) realy interesting !

ShortKatz

27th October 2022, 17:05

Is there something special with --mcstf ? I'm getting a crash if I try to use it with my custom build HandBrake.

Edit: OK, all-clear. That is just a bug in the current code.
This patch does fix the crash: https://mailman.videolan.org/pipermail/x265-devel/2022-October/013521.html

Edit2: I think I will never use --mcstf for whole movies. This option does increase the encoding time for me 7 times.

benwaggoner

27th October 2022, 22:57

Just thought !!!
Can sbrc be ported back to x264 ??? I don't think this feature exist.
If i'm so over it, it's because i found the idea (with some minor adjustments :D) realy interesting !
There is a TON of great x265 features that should be back ported to x264. I know MCW contributed a lot of patches from x265 back, but not a lot made it into main. I would so love to have CSV logging, for example. And I know that code was contributed to x265 at least seven years ago.

LigH

27th October 2022, 23:18

New upload: x265 3.5+57-4d2890fe4 (https://www.mediafire.com/file/0wegkfl8rhshflu/x265_3.5+57-4d2890fe4.7z/file)

[Windows][GCC 12.2.0][32/32XP/64 bit] 8bit+10bit+12bit

News since v3.5+39:
--[no-]sbrc Enables the segment based rate control, using its scene statistics. Default disabled

--[no-]mcstf Enable GOP based temporal filter. Default 0

LeXXuz

28th October 2022, 08:38

jpsdr

28th October 2022, 15:44

Possible situation...
If the calculated value is at the threshold value, check here (https://forum.doom9.org/showthread.php?p=1977277#post1977277).

jpsdr

28th October 2022, 18:11

I've also added the option to use AQ-MODE 5 instead of AQ-MODE 1 in sbrc, and... i've made a build, it's on my github.
So you can play with both experimental options i've added... ;)

LeXXuz

29th October 2022, 09:20

I've also added the option to use AQ-MODE 5 instead of AQ-MODE 1 in sbrc, and... i've made a build, it's on my github.
So you can play with both experimental options i've added... ;)

M'kay, sbrc was worth a shot. I've seen that flickering now. Unbearable.
Not sure if aq-mode 5 + sbrc would change anything on that.

I guess I will test a little more with just aq-mode 4 and 5 now. aq-mode 3 does improve faint details in very dark scenes slightly where aq-mode 2 fails, even with generous CRF values of 18 and below. The opening scene of Thor (2011) is one of these rare cases where x265 doesn't spend enough bits with aq-mode 1/2 for that scene and the result looks awful. Even CRF 16 with the Very Low preset +rskip and -sao didn't change anything on that.

But I think aq-mode 3 just wastes too much bits in general compared to the other aq-modes. Even clips with no dark scenes at all get much bigger than with mode 2, which I don't quite understand to be honest.

jpsdr

29th October 2022, 15:19

LeXXuz

29th October 2022, 15:27

Sure flickering is because of sbrc ? I was just talking of a rare theoretical possible situation sbrc may produce...
But if it's because of that, the hysteresis option may help, it's for that case i've made it.

Well I've encoded that particular scene countless times before with all the kinds of settings. This was the first time with sbrc enabled. So I presumed it was the cause. I can give it another go, of course. How do I activate the hysteresis option?

EDIT: Never mind. Found it.

jpsdr

29th October 2022, 17:07

You activate hysteresis with : --sbrc-hyst
You activate replace AQ-MODE 1 with 5 with : --sbrc-aq5
Of course, you have to get my build on my github.

Edit : didn't see the edit...

I'm curious about the result.
If it doesn't help, i think i'll forget sbrc. (On the other hand, i've made the hysteresis range totaly "at random", maybe this part needs tunning).
Sometimes things which seem a good idea "on paper" are unfortunately not so good when in practical use...

LeXXuz

30th October 2022, 13:39

Looking good so far. Flickering gone. Encoded several films with auto-hyst-aq5 mode of 3.5+57. Will carefully watch them over next days and report back.

Will also do another encode of that one film with 3.5+56 again just to make sure that flickering I saw really was related to the mod and not something else.

jpsdr

30th October 2022, 17:10

@LeXXuz
Thanks for the feedback. Waiting your compare with 3.5+56.

Tuning hysteresis is not realy easy. Too big and you'll prevent switch that should have happened, too small and it's useless...
I think (even if version used by LeXXuz may has solved his issue) that the hysteresis range was a little small, so i've made it just a little bigger.
There is a new build avaible.

Boulder

30th October 2022, 18:59

M'kay, sbrc was worth a shot. I've seen that flickering now. Unbearable.
Not sure if aq-mode 5 + sbrc would change anything on that.

I guess I will test a little more with just aq-mode 4 and 5 now. aq-mode 3 does improve faint details in very dark scenes slightly where aq-mode 2 fails, even with generous CRF values of 18 and below. The opening scene of Thor (2011) is one of these rare cases where x265 doesn't spend enough bits with aq-mode 1/2 for that scene and the result looks awful. Even CRF 16 with the Very Low preset +rskip and -sao didn't change anything on that.

But I think aq-mode 3 just wastes too much bits in general compared to the other aq-modes. Even clips with no dark scenes at all get much bigger than with mode 2, which I don't quite understand to be honest.

Are your tests done with HDR or SDR sources? I'd be interested in hearing how it works out since mode 1 has been the best general choice for both around CRF 16 (HDR) - 18 (SDR).

LeXXuz

30th October 2022, 19:40

Are your tests done with HDR or SDR sources? I'd be interested in hearing how it works out since mode 1 has been the best general choice for both around CRF 16 (HDR) - 18 (SDR).

SDR Blu-ray as source. Mode 1 never convinced me in dark scenes.

Stereodude

30th October 2022, 20:28

Is there a general write-up or guidelines somewhere of the pro/cons / what each one excels at for the AQ modes in x265?

Boulder

31st October 2022, 07:18

@jpsdr: Would it be beneficial to disable using the dark bias aq-modes for HDR? They might be counter-productive AFAIK.

http://forum.doom9.net/showthread.php?p=1857767#post1857767

So for HDR, the auto switching modes would be 1, 2 and 4 and all modes would be available for SDR.

EDIT: Thinking some more, it might be useful if the value used for the frame could be exported in the CSV log as well. Otherwise a pain to debug :)

benwaggoner

31st October 2022, 18:06

@jpsdr: Would it be beneficial to disable using the dark bias aq-modes for HDR? They might be counter-productive AFAIK.

http://forum.doom9.net/showthread.php?p=1857767#post1857767
Yeah, --aq-mode is definitely counterproductive with PQ, which has vastly more code values near black. It'd either raise ABR a bunch with --crf, or reduce quality in midtones and brighter with --bitrate.

So for HDR, the auto switching modes would be 1, 2 and 4 and all modes would be available for SDR.
Makes good sense to me.

EDIT: Thinking some more, it might be useful if the value used for the frame could be exported in the CSV log as well. Otherwise a pain to debug :)
Good idea! ++ on that feature request.

jpsdr

31st October 2022, 18:30

@jpsdr: Would it be beneficial to disable using the dark bias aq-modes for HDR? They might be counter-productive AFAIK.

Sorry, i don't know, i'm very new to x265 encodes, i may just have made around 10 encodes. When it takes 10 to 12 days to encode 1h30, it's verryyyyy slow to build experience... ;)
benwaggoner is a lot more suited than me to answer.

I can see how to add on my mod branch version these features request. I must say that i said to myself at almost the begining that having a log of the the edgeIntensity and brightnessIntensity could be very interesting. And if there is the hysteresis, the current choice also. But had no idea how to add this.
No idea yet how to detect HDR/SDR for choosing switching mode, it's probably on some kind of frame property. Adding --sbrc- any idea ? switch.
Will see what if i can do something later, for now, other things to do...

Boulder

31st October 2022, 19:57

Sorry, i don't know, i'm very new to x265 encodes, i may just have made around 10 encodes. When it takes 10 to 12 days to encode 1h30, it's verryyyyy slow to build experience... ;)
[b]benwaggoner[\b] is a lot more suited than me to answer.

I can see how to add on my mod branch version these features request. I must say that i said to myself at almost the begining that having a log of the the edgeIntensity and brightnessIntensity could be very interesting. And if there is the hysteresis, the current choice also. But had no idea how to add this.
No idea yet how to detect HDR/SDR for choosing switching mode, it's probably on some kind of frame property. Adding --sbrc- any idea ? switch.
Will see what if i can do something later, for now, other things to do...
The encoder does some detection probably based on the set color primaries, I think it will warn if you don't have --hdr set but set the appropriate values for matrix, transfer and primaries.

Maybe --sbrc-hdr and --sbrc-sdr so anyone can choose to use all modes regardless. Then you wouldn't need to even detect the source characteristics.

And yes, the more you can add to the log, the better :D For example, it's strange that you don't know how many refs are actually used while the information is available in x264. Or CTU/TU sizes etc. which would be useful in tweaking the encoder based on content.

jpsdr

31st October 2022, 21:00

Maybe --sbrc-hdr and --sbrc-sdr so anyone can choose to use all modes regardless. Then you wouldn't need to even detect the source characteristics.

Ooohhh. Good one !
:thanks:

ShortKatz

1st November 2022, 01:19

Where do I see the flickering you mentioned with --sbrc? I've encoded some 4k HDR movies and SDR movies with the official x265, but I do not see any flickering compared to the source material. I've also encoded lots and lots of movies before with the old --auto-aq patch and did not see any flickering.

benwaggoner

1st November 2022, 05:55

The encoder does some detection probably based on the set color primaries, I think it will warn if you don't have --hdr set but set the appropriate values for matrix, transfer and primaries.

Maybe --sbrc-hdr and --sbrc-sdr so anyone can choose to use all modes regardless. Then you wouldn't need to even detect the source characteristics.
We already have --hdr10 and --dolby-vision-profile. Maybe we should have --sdr and --hlg parameters too, so as to unambiguously specify what colorimetry is intended. That could let lots of other filters know how to apply color volume specific optimizations.

And yes, the more you can add to the log, the better :D For example, it's strange that you don't know how many refs are actually used while the information is available in x264. Or CTU/TU sizes etc. which would be useful in tweaking the encoder based on content.
There's a lot of CU block type data if you use --csv-log-level 2. Wow, 94 columns of data. When testing with really low bitrates, I've wound up with a .csv larger than its .hevc

Boulder

1st November 2022, 06:04

We already have --hdr10 and --dolby-vision-profile. Maybe we should have --sdr and --hlg parameters too, so as to unambiguously specify what colorimetry is intended. That could let lots of other filters know how to apply color volume specific optimizations.
True, that would be a good shortcut :)

There's a lot of CU block type data if you use --csv-log-level 2. Wow, 94 columns of data. When testing with really low bitrates, I've wound up with a .csv larger than its .hevc
I would like to see basic CTU/TU data in the summary after the encode, it could be blurted out like similar things in x264. Csv is good but always requires some manual work for calculating the average per type etc.

jpsdr

1st November 2022, 17:09

So for HDR, the auto switching modes would be 1, 2 and 4 and all modes would be available for SDR.

My mod patched code is :

if (preFrame->m_frameSegment_thrs_edge == SBRC_THRS_LOW)
preFrame->m_frameSegment = (preFrame->m_frameSegment_thrs_bright == SBRC_THRS_HIGH) ? X265_AQ_AUTO_VARIANCE : X265_AQ_AUTO_VARIANCE_BIASED;
else
{
if (preFrame->m_param->rc.frameSegment_aq5)
preFrame->m_frameSegment = (preFrame->m_frameSegment_thrs_bright == SBRC_THRS_HIGH) ? X265_AQ_EDGE : X265_AQ_EDGE_BIASED;
else
preFrame->m_frameSegment = (preFrame->m_frameSegment_thrs_bright == SBRC_THRS_HIGH) ? X265_AQ_EDGE : X265_AQ_EDGE_BIASED_SBRC;
}

How do you see it for HDR ?
Like this ?

if (preFrame->m_frameSegment_thrs_edge == SBRC_THRS_LOW)
preFrame->m_frameSegment = (preFrame->m_frameSegment_thrs_bright == SBRC_THRS_HIGH) ? X265_AQ_AUTO_VARIANCE : X265_AQ_AUTO_VARIANCE;
else
{
if (preFrame->m_param->rc.frameSegment_aq5)
preFrame->m_frameSegment = (preFrame->m_frameSegment_thrs_bright == SBRC_THRS_HIGH) ? X265_AQ_EDGE : X265_AQ_VARIANCE;
else
preFrame->m_frameSegment = (preFrame->m_frameSegment_thrs_bright == SBRC_THRS_HIGH) ? X265_AQ_EDGE : X265_AQ_VARIANCE;
}

Boulder

1st November 2022, 18:54

My mod patched code is :

if (preFrame->m_frameSegment_thrs_edge == SBRC_THRS_LOW)
preFrame->m_frameSegment = (preFrame->m_frameSegment_thrs_bright == SBRC_THRS_HIGH) ? X265_AQ_AUTO_VARIANCE : X265_AQ_AUTO_VARIANCE_BIASED;
else
{
if (preFrame->m_param->rc.frameSegment_aq5)
preFrame->m_frameSegment = (preFrame->m_frameSegment_thrs_bright == SBRC_THRS_HIGH) ? X265_AQ_EDGE : X265_AQ_EDGE_BIASED;
else
preFrame->m_frameSegment = (preFrame->m_frameSegment_thrs_bright == SBRC_THRS_HIGH) ? X265_AQ_EDGE : X265_AQ_EDGE_BIASED_SBRC;
}

How do you see it for HDR ?
Like this ?

if (preFrame->m_frameSegment_thrs_edge == SBRC_THRS_LOW)
preFrame->m_frameSegment = (preFrame->m_frameSegment_thrs_bright == SBRC_THRS_HIGH) ? X265_AQ_AUTO_VARIANCE : X265_AQ_AUTO_VARIANCE;
else
{
if (preFrame->m_param->rc.frameSegment_aq5)
preFrame->m_frameSegment = (preFrame->m_frameSegment_thrs_bright == SBRC_THRS_HIGH) ? X265_AQ_EDGE : X265_AQ_VARIANCE;
else
preFrame->m_frameSegment = (preFrame->m_frameSegment_thrs_bright == SBRC_THRS_HIGH) ? X265_AQ_EDGE : X265_AQ_VARIANCE;
}

Hmm, I believe it's correct if we consult quietvoid's answer here: https://forum.doom9.org/showthread.php?p=1977129#post1977129

Not related to your patch/mod, but how does the encoder decide to start checking for edges? Is there a range of medium brightness which will then trigger that check?

benwaggoner

1st November 2022, 18:54

I would like to see basic CTU/TU data in the summary after the encode, it could be blurted out like similar things in x264. Csv is good but always requires some manual work for calculating the average per type etc.
Yeah, I do spend quite a lot of time in Excel analyzing and graphing data from .csv logs.

A --csv-log-level 2 for an hour long episode is 60-70 MB. Saving as .xlsb (Binary workbook) is the smallest Excel file format, and is still 50 MB. Even with 128 GB of RAM and 2x18/36 Xeon CPUs, it still takes seconds to open the document. And it can take 10+ sec to change just one element of a chart.

As I am a multi-spectrum compression nerd, I see that 7z LZMA2 Ultra is able to get it down to 20 MB, only 40% the size as Excel's default Deflate compression. Of course, it's a lot slower to write.

Boulder

1st November 2022, 19:20

Can anyone of the resident experts say what is the use case for --rd-refine?

jpsdr

2nd November 2022, 18:30

Not related to your patch/mod, but how does the encoder decide to start checking for edges? Is there a range of medium brightness which will then trigger that check?

Is this answering your question ?
/* SBRC */
if (preFrame->m_param->rc.frameSegment)
{
int heightL = preFrame->m_lowres.lines;
int widthL = preFrame->m_lowres.width;
pixel *lumaPlane = preFrame->m_lowres.fpelPlane[0];
intptr_t stride = preFrame->m_lowres.lumaStride;
double brightnessIntensity = 0, edgeIntensity = 0;

/* Edge plane computation */
memset(preFrame->m_lowres.lowresEdgePlane, 0, stride * (heightL + (preFrame->m_fencPic->m_lumaMarginY * 2)) * sizeof(pixel));
pixel* lowresEdgePic = preFrame->m_lowres.lowresEdgePlane + preFrame->m_fencPic->m_lumaMarginY * stride + preFrame->m_fencPic->m_lumaMarginX;
computeEdge(lowresEdgePic, lumaPlane, NULL, stride, heightL, widthL, false);

/*Frame edge percentage computation */
edgeIntensity = computeEdgeIntensity(lowresEdgePic, widthL, heightL, stride);

/* Frame Brightness percentage computation */
brightnessIntensity = computeBrightnessIntensity(lumaPlane, widthL, heightL, stride);

If sbrc is enabled, edge computation is always done.

Boulder

2nd November 2022, 18:40

Is this answering your question ?
/* SBRC */
if (preFrame->m_param->rc.frameSegment)
{
int heightL = preFrame->m_lowres.lines;
int widthL = preFrame->m_lowres.width;
pixel *lumaPlane = preFrame->m_lowres.fpelPlane[0];
intptr_t stride = preFrame->m_lowres.lumaStride;
double brightnessIntensity = 0, edgeIntensity = 0;

/* Edge plane computation */
memset(preFrame->m_lowres.lowresEdgePlane, 0, stride * (heightL + (preFrame->m_fencPic->m_lumaMarginY * 2)) * sizeof(pixel));
pixel* lowresEdgePic = preFrame->m_lowres.lowresEdgePlane + preFrame->m_fencPic->m_lumaMarginY * stride + preFrame->m_fencPic->m_lumaMarginX;
computeEdge(lowresEdgePic, lumaPlane, NULL, stride, heightL, widthL, false);

/*Frame edge percentage computation */
edgeIntensity = computeEdgeIntensity(lowresEdgePic, widthL, heightL, stride);

/* Frame Brightness percentage computation */
brightnessIntensity = computeBrightnessIntensity(lumaPlane, widthL, heightL, stride);

If sbrc is enabled, edge computation is always done.

So there is then later some check to see if the frame should go to the "bright vs. dark" or "edges vs. no edges" path and the proper aq-mode can be selected?

switches the AQ according to thresholds (bright frame = mode 2, otherwise 3. high edge density = mode 4, otherwise mode 1

quietvoid

3rd November 2022, 02:26

Either way it appears MCW is partially reverting some of the SBRC changes to "improve" it.
Didn't look too deep into it, will update when it's pushed.

Also we're getting an updated histogram scene cut algorithm, it seems.

Boulder

3rd November 2022, 05:49

Also we're getting an updated histogram scene cut algorithm, it seems.

This could be a good change. I wonder why it was left as it was; totally unfinished and not working at all.

LeXXuz

3rd November 2022, 09:37

@LeXXuz
Thanks for the feedback. Waiting your compare with 3.5+56.

The second encode was free of flickering as well. Could've been a problem with one of the CUDA prefilters I use. They are picky sometimes. Good thing you were sceptical sbrc being the cause. :o

jpsdr

3rd November 2022, 17:26

So there is then later some check to see if the frame should go to the "bright vs. dark" or "edges vs. no edges" path and the proper aq-mode can be selected?
Yes, later there is first "edges vs. no edges", and for each of these cases there is "bright vs. dark".
It seems there's been new commits, didn't check them yet.

Edit: commits are only for aarch64, so no need for new build. I thought it was what quietvoid was talking about, but not. Will wait for the MCW sbrc improve, before making any change on it (i was about to see to add --sbrc-hdr, but will wait to see what will change).

LeXXuz

3rd November 2022, 20:51

Would --frame-threads >1 have any negative impact with --sbrc on quality in CRF mode? I need at least 2 frame-threads to saturate my CPUs with the prefiltering I do. Can't run two encodes at once as it would eat up too much memory.

LigH

4th November 2022, 13:17

New upload: x265 3.5+68-536f6a796 (https://www.mediafire.com/file/4ba6yvp79vklgtf/x265_3.5+68-536f6a796.7z/file)

[Windows][GCC 12.2.0][32/32XP/64 bit] 8bit+10bit+12bit

News since v3.5+57:
Improved histogram based scene change detection (--hist-threshold removed); fixed memory leak in SBRC

ShortKatz

4th November 2022, 18:05

Boulder

4th November 2022, 18:22

The new scene cut algorithm seems to be broken as well. I tested it briefly on my Hot Fuzz test clip with 15 scene changes, the "classic" mode detected 10 and the new --hist-scenecut just 6 of them. The old histcut method found 8 scene cuts, part of those different from the new version. The scene cuts are very obvious, clear changes in broad daylight throughout those frames.

vpupkind

6th November 2022, 15:31

Boulder

6th November 2022, 15:41

Can you please share the sequence?

Sure, here it is: https://drive.google.com/file/d/19cIzFGAocSHYlBNJdZJ08OaQbfBIHUjm/view?usp=share_link

These entries I get for the QP file using StainlessS's tool for detecting scene cuts, I think they are all good and none are missing.

0 I
55 I
87 I
114 I
140 I
193 I
248 I
289 I
345 I
380 I
430 I
462 I
528 I
557 I
582 I
620 I

LeXXuz

6th November 2022, 16:45

Ashok Kumar Mishra

7th November 2022, 10:15

Ashok Kumar Mishra

7th November 2022, 10:29

Sure, here it is: https://drive.google.com/file/d/19cIzFGAocSHYlBNJdZJ08OaQbfBIHUjm/view?usp=share_link

These entries I get for the QP file using StainlessS's tool for detecting scene cuts, I think they are all good and none are missing.

0 I
55 I
87 I
114 I
140 I
193 I
248 I
289 I
345 I
380 I
430 I
462 I
528 I
557 I
582 I
620 I

Able to decode only 63 frames using ffmpeg. Can you please share the yuv file or the command line used to decode the complete hotfuzz.vc1.
It will be helpful for us to see the performance of histogram-based scene change feature.

Boulder

7th November 2022, 11:22

Able to decode only 63 frames using ffmpeg. Can you please share the yuv file or the command line used to decode the complete hotfuzz.vc1.
It will be helpful for us to see the performance of histogram-based scene change feature.

I'm using DGSource with Avisynth to decode the stream, see https://www.rationalqm.us/dgdecnv/dgdecnv.html. I can try other methods when I get back home.

Boulder

7th November 2022, 16:31

Able to decode only 63 frames using ffmpeg. Can you please share the yuv file or the command line used to decode the complete hotfuzz.vc1.
It will be helpful for us to see the performance of histogram-based scene change feature.
Please try with this one, looks like ffmpeg doesn't like the VC1 stream cut by DGIndex while demuxing.

https://drive.google.com/file/d/1zNUnmukafWs6JWflxlnbjT2FEeC0K2VS/view?usp=share_link

jpsdr

7th November 2022, 16:33

In one of the last commit, they replaced :

x265_log(param, X265_LOG_INFO, "Keyframe min / max / scenecut / edge threshold : %d / %d / %d / %.2lf\n",
param->keyframeMin, param->keyframeMax, param->bHistBasedSceneCut, param->edgeTransitionThreshold);

with

x265_log(param, X265_LOG_INFO, "Keyframe min / max / scenecut : %d / %d / %d / %.2lf\n",
param->keyframeMin, param->keyframeMax, param->bHistBasedSceneCut);

Should not it be this ?

x265_log(param, X265_LOG_INFO, "Keyframe min / max / scenecut : %d / %d / %d\n",
param->keyframeMin, param->keyframeMax, param->bHistBasedSceneCut);

LigH

7th November 2022, 22:32

Well spotted, I guess.

There are also cases of copy/paste mistakes printing a default of 0 for a boolean switch. Quality assurance is not their greatest talent.

Ashok Kumar Mishra

8th November 2022, 08:39

Please try with this one, looks like ffmpeg doesn't like the VC1 stream cut by DGIndex while demuxing.

https://drive.google.com/file/d/1zNUnmukafWs6JWflxlnbjT2FEeC0K2VS/view?usp=share_link

We decoded the VC1 stream and there are 1192 frames. There is total 13 scenecuts in the video clip.

11 scenecuts in "classic" scenecut method:
5, 55, 112, 279, 333 ,365, 414, 444, 508, 536, 598

13 scenecuts in histogram based scenecut method:
55, 87, 112, 138, 242, 279, 333, 365, 414, 444, 508, 536, 598

rwill

8th November 2022, 11:37

We decoded the VC1 stream and there are 1192 frames. There is total 13 scenecuts in the video clip.

11 scenecuts in "classic" scenecut method:
5, 55, 112, 279, 333 ,365, 414, 444, 508, 536, 598

13 scenecuts in histogram based scenecut method:
55, 87, 112, 138, 242, 279, 333, 365, 414, 444, 508, 536, 598

I actually watched the video and counted 16 scenecuts.

jpsdr

8th November 2022, 12:00

Well spotted, I guess.

No, just big pure luck. If i saw it it's because this line was in conflict with one of the mod patch of my x256_mod version, and so when with turtoise git i handle the conflict, i saw it, otherwise, i would never realized it....

---------------------

For scencut, it seems that according the video, sometimes new histogram is better, sometimes classic is better.

Ashok Kumar Mishra

8th November 2022, 12:24

I actually watched the video and counted 16 scenecuts.

New scenecut method is better compared to classic method in this case.

Ashok Kumar Mishra

8th November 2022, 12:25

Boulder

8th November 2022, 14:03

New scenecut method is better compared to classic method in this case.

Please post your command line so I can try to doublecheck. I used a fresh git build (using Media Autobuild Suite) and my normal command line with --hist-scenecut added and got only those 6 scene cuts. I did downscale to 1280x536 though.

There should be 1220 frames.

Ashok Kumar Mishra

8th November 2022, 14:52

Please post your command line so I can try to doublecheck. I used a fresh git build (using Media Autobuild Suite) and my normal command line with --hist-scenecut added and got only those 6 scene cuts. I did downscale to 1280x536 though.

There should be 1220 frames.

Below is the command line we used for new scene cut algorithm.

ffmpeg -y -v -10 -i "hotfuzz.mkv" -an -sn -threads 8 -vsync 0 -r 50000/1000 -pix_fmt yuv420p -f rawvideo - | x265 --input - --input-res 1920x1080 --fps 50 --input-depth 8 --input-csp i420 --psnr --ssim --crf 26 --csv HotFuzz_1920x1080_8bit_26_hist-latest.csv --csv-log-level 2 --output HotFuzz_1920x1080_8bit_26_hist-latest.hevc --hist-scenecut

and compared with the default scene cut method (--scenecut 40)

Boulder

8th November 2022, 15:41

Below is the command line we used for new scene cut algorithm.

ffmpeg -y -v -10 -i "hotfuzz.mkv" -an -sn -threads 8 -vsync 0 -r 50000/1000 -pix_fmt yuv420p -f rawvideo - | x265 --input - --input-res 1920x1080 --fps 50 --input-depth 8 --input-csp i420 --psnr --ssim --crf 26 --csv HotFuzz_1920x1080_8bit_26_hist-latest.csv --csv-log-level 2 --output HotFuzz_1920x1080_8bit_26_hist-latest.hevc --hist-scenecut

and compared with the default scene cut method (--scenecut 40)

I'm unable to replicate the behaviour. I'm getting the same amount of detected SCs for my and your command line, though I did not set any framerate etc. since it's all autodetected. Besides, you have it wrong there, this is a regular 23.976 fps film source.

Classic detects 114, 193, 289, 345, 380, 430, 528, 557 and 620. Hist-scenecut detects 55, 87, 140, 248, 289. If I crop the borders and downscale to 1280x536, the classic mode misses 5 scenechanges (87, 140, 380, 462 and 582).

The csv log files are here:
https://drive.google.com/file/d/1Gymy3rIJq5GFLULsiumraKp0_E6ml4hA/view?usp=share_link (Classic)
https://drive.google.com/file/d/1umTC4Q1t9MiT0PxDcJOMjxizjVFa0cJ7/view?usp=share_link (same CL as classic but with --hist-scenecut)
https://drive.google.com/file/d/1MHiPNUynUiHrg2jnNDrtqgq2AieG3vCe/view?usp=share_link (the doublecheck)

I'm only loading the source without any cropping etc. in the avs script.

Ashok Kumar Mishra

8th November 2022, 16:23

I'm unable to replicate the behaviour. I'm getting the same amount of detected SCs for my and your command line, though I did not set any framerate etc. since it's all autodetected. Besides, you have it wrong there, this is a regular 23.976 fps film source.

Classic detects 114, 193, 289, 345, 380, 430, 528, 557 and 620. Hist-scenecut detects 55, 87, 140, 248, 289. If I crop the borders and downscale to 1280x536, the classic mode misses 5 scenechanges (87, 140, 380, 462 and 582).

The csv log files are here:
https://drive.google.com/file/d/1Gymy3rIJq5GFLULsiumraKp0_E6ml4hA/view?usp=share_link (Classic)
https://drive.google.com/file/d/1umTC4Q1t9MiT0PxDcJOMjxizjVFa0cJ7/view?usp=share_link (same CL as classic but with --hist-scenecut)
https://drive.google.com/file/d/1MHiPNUynUiHrg2jnNDrtqgq2AieG3vCe/view?usp=share_link (the doublecheck)

I'm only loading the source without any cropping etc. in the avs script.

We converted .mkv which you shared into .yuv and there are 1220 frames.

Classic: 10 scenecuts(54, 113, 192, 288, 344, 379, 429, 527, 556, 619)

Histogram: 15 scenecuts(54, 86, 113, 139, 192, 247, 288, 344, 379, 429, 461, 527, 556, 581, 619)

Command line used:

Classic:
x265 --input hotfuzzn.yuv --input-res 1920x1080 --fps 50 --input-depth 8 --input-csp i420 --psnr --ssim --crf 26 --csv HotFuzz_1920x1080_8bit_26_hist.csv --csv-log-level 2 --output HotFuzz_1920x1080_8bit_26_hist.hevc --scenecut 40

Histogram:
x265 --input hotfuzzn.yuv --input-res 1920x1080 --fps 50 --input-depth 8 --input-csp i420 --psnr --ssim --crf 26 --csv HotFuzz_1920x1080_8bit_26_hist.csv --csv-log-level 2 --output HotFuzz_1920x1080_8bit_26_hist.hevc --hist-scenecut

Boulder

8th November 2022, 16:45

I think I got it -- please try encoding using the Main10 profile. It will give you very different results compared to Main.

Ashok Kumar Mishra

8th November 2022, 18:06

I think I got it -- please try encoding using the Main10 profile. It will give you very different results compared to Main.

It's 8-bit video, so using Main. Is it practical using main 10?

Boulder

8th November 2022, 18:27

It's 8-bit video, so using Main. Is it practical using main 10?

Yes. I do all my processing (denoising, resizing, debanding) in 16 bits, then feed that to the encoder and encode with the Main10 profile. No use encoding in 8 bits since pretty much all the current HW decoders support Main10 as well.

jpsdr

8th November 2022, 19:28

tormento

8th November 2022, 19:40

It's 8-bit video, so using Main. Is it practical using main 10?
HW decoders rarely can process 8 bit HEVC.

Moreover, 10 bit encoding has (since the infamous Hi10P days) proven to increase compression with additional benefits regarding banding and no disadvantages (at least HEVC files).

nevcairiel

8th November 2022, 19:58

HW decoders rarely can process 8 bit HEVC.

Wut?
Any HW decoder that can support HEVC at all will support 8-bit, and most of them also 10-bit.

benwaggoner

8th November 2022, 21:42

HW decoders rarely can process 8 bit HEVC.
You mean can rarely process 8-bit HEVC without also supporting 10-bit HEVC, right?

Moreover, 10 bit encoding has (since the infamous Hi10P days) proven to increase compression with additional benefits regarding banding and no disadvantages (at least HEVC files).
The benefit of 10-bit encoding for 8-bit sources is less with HEVC than with H.264, as 8-bit internal precision was improved. But it can still give a few percentage improvement and can reduce banding artifacts.

tormento

9th November 2022, 09:52

You mean can rarely process 8-bit HEVC without also supporting 10-bit HEVC, right?.
So picky you are. :p

mister_no

9th November 2022, 11:37

For those interested, made a new build of my x265_mod version.
-------------------------------------------------------------------------
Will wait the results of "classic" vs "histogram" to see if i change my encoding scripts, as i'm only doing 10 bits video / main10 encodes.

Can you please make a build without the extanded progress bar? My GUI doesn't show progress with the extanded progress bar.

jpsdr

9th November 2022, 14:20

Euh... Sorry but no, don't have time to try (don't even know if i would be able to) to build specific for X, Y, Z, etc...

jpsdr

9th November 2022, 19:28

Boulder

10th November 2022, 06:27

About "histogram" vs "classic", what about hrd-pq ? If Boulder's video is 8 bit, i assume it sdr, so the "issue" is maybe main10 / 10 bits sdr video...
Has someone made tests to see if "histogram" behaves better than "classic" with main10 / 10 bits video in hdr-pq ?

It looks like the histogram mode works quite well with PQ sources. I tested a 3000-frame part from Harry Potter and the Philosopher's Stone, and classic produced 10 scene changes while the histogram mode found 24, which should be very close to the real value. There are some difficult parts with slo-mo in the clip which confuses the tool I use for creating a QP file.

So the issue is probably seen with SDR sources encoded with the Main10 profile.

jpsdr

10th November 2022, 19:16

Ok, thanks for the feedback. In that case, i'll change my encode scripts to use histogram mode.

benwaggoner

10th November 2022, 22:03

Wow, that's big news if the new histogram mode is good enough to be the default.

Anyone test --mctf or the new segment based rate control?

vpupkind

11th November 2022, 16:57

Wow, that's big news if the new histogram mode is good enough to be the default.

Anyone test --mctf or the new segment based rate control?

SBRC is a bit of a misnomer right now. It does some aq-related stuff, hopefully some actual rate control stuff will follow.

ShortKatz

12th November 2022, 22:07

Wow, that's big news if the new histogram mode is good enough to be the default.

Anyone test --mctf or the new segment based rate control?

Tested --mctf just once. It increased my encoding time 7-fold. So I did not do any further tests.

LeXXuz

13th November 2022, 10:17

Ok, thanks for the feedback. In that case, i'll change my encode scripts to use histogram mode.

Could you possibly add 8-bit support to your version? I'd like to do some comparisons with aq5-mode between 8 and 10bit output.

jpsdr

13th November 2022, 13:19

.... There is not...? I ran cmake to create a VS projet and checked all possible options, i thought all the supports were enabled.
Setting --profile main, -D 8 (and all others related stuff saying that video is 8 bits) on the command line is not working ?

If not, for now, unfortunately i don't know what to do for... :(

quietvoid

13th November 2022, 15:36

If you enabled HIGH_BIT_DEPTH when configuring the build, it only builds 10 bit.
12 bit if you do MAIN12 too.

There's a multilib script to get a statically linked executable with all bit depths.
I can't really help for Windows/VS though.

LeXXuz

13th November 2022, 15:56

.... There is not...? I ran cmake to create a VS projet and checked all possible options, i thought all the supports were enabled.
Setting --profile main, -D 8 (and all others related stuff saying that video is 8 bits) on the command line is not working ?

No there isn't. x265 exits with this error if forced to work in 8-bit:
x265 [FLAW]: main profile not supported, internal bit depth 10.

jpsdr

13th November 2022, 16:29

Ah.... Don't know how to do it for now, don't realy have time for now, sorry. Try to see if i can figure out something when there will be the need for a new build.
Otherwise, anyone can also make a build from my repo.

quietvoid

13th November 2022, 16:52

I just noticed that the new histogram scene cut commit removed the logging of consecutive B frames %, seemingly for no reason.
If anyone had a use for it, here's a diff: https://github.com/quietvoid/x265/commit/43e460330af001365c8508d430a35be4377e390e

Should probably be renamed from m_histogram now but I just copied the old code back.

Boulder

13th November 2022, 16:53

Here's a very crude tool written in Python to get those average figures out of the csv log file written with csv-log-level 2 set. It expects that SSIM or PSNR is not enabled as those change the column structure. Only tested with two csv files I had on my HDD ;)

Usage: "python logtool.py filename.csv". It will then write a .log file to the path where the csv file is and output the same information in the command prompt window.

import pandas as pd
import os
import sys

filename = os.path.abspath(sys.argv[1])

log_content = pd.read_csv(filename, sep=",", decimal=".", header=0)
log_content = log_content.iloc[:-4,11:-9]
log_content.columns = log_content.columns.str.replace('.1', '', regex=True)
log_content.columns = log_content.columns.str.replace('66', ' 16x16', regex=True)
log_content.columns = log_content.columns.str.replace('326', ' 32x16', regex=True)
log_content.columns = log_content.columns.str.replace('6x32', ' 16x32', regex=True)
log_content.columns = log_content.columns.str.replace('6x8', ' 16x8', regex=True)
log_content.columns = log_content.columns.str.replace('86', ' 8x16', regex=True)
log_content.columns = log_content.columns.str.replace('AMP6', 'AMP 16', regex=True)
log_content = log_content.replace('%', '', regex=True).astype(float)
average = log_content.describe().loc['mean'].to_string()

print (average)
with open(filename+str('.log'), mode='w') as outputfile:
outputfile.write(str(average))
outputfile.close()

_kermit

14th November 2022, 15:02

excellentswordfight

14th November 2022, 15:28

piping to x265 with powershell

sorry if that is too off-topic...

this works just fine in a command prompt:

ffmpeg.exe -i FILE -pix_fmt yuv420p10le -strict -1 -f yuv4mpegpipe - | x265.exe - --y4m .....etc.

Now I wanted to use Powershell for that and the piping isn't working, the x265 process doesn't even start, while ffpmpeg is running:

It seems that using the pipe character isn't working here.
Does anyone have an idea?
There is nothing "wrong" with the pipe command, but piping does not work in the same way in powershell as in cmd so you cannot pipe raw data in the same way.

There are some workaround and more info here https://stackoverflow.com/questions/50456424/using-ffmpeg-and-ffplay-piped-together-in-powershell

_kermit

14th November 2022, 15:45

There is nothing "wrong" with the pipe command, but piping does not work in the same way in powershell as in cmd so you cannot pipe raw data in the same way.

There are some workaround and more info here https://stackoverflow.com/questions/50456424/using-ffmpeg-and-ffplay-piped-together-in-powershell

thanks for confirming, thought this might be the issue.
Looks rather messy doing it that way and the only reason I wanted to use PS was to retrieve values from the MKV and use them in the script.
Probably too much of a hassle now.

In case anyone is interested (pretty basic, with debug code...).
params: file, CRF, presetname

$file = $args[0]
$crf = $args[1]
$preset = $args[2]

$temp = $(&D:\temp\Mediainfo\MediaInfo.exe "--Inform=Video;%MaxCLL%" $file).tostring().split(" ")
$MaxCLL= $temp[0].trim()

$temp = $(&D:\temp\Mediainfo\MediaInfo.exe "--Inform=Video;%MaxFALL%" $file).tostring().split(" ")
$MaxFALL = $temp[0].trim()

$temp = $(&D:\temp\Mediainfo\MediaInfo.exe "--Inform=Video;%MasteringDisplay_Luminance%" $file).tostring().split(" ")
if ($temp) {
$LumMin = [int]([decimal]$temp[1].trim() * 10000)
$LumMax = [int]$temp[4].trim() * 10000}
else
{$LumMin = 0 ; $LumMax = 0}

$x265Params = "- --y4m --output-depth 10 --preset $preset --crf $crf --colorprim bt2020 --transfer smpte2084 --colormatrix bt2020nc --master-display ""G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L($LumMax,$LumMin)"" --vbv-bufsize 160000 --vbv-maxrate 160000 -o ""$file." + "$crf" + "_" + $preset + """ --rdoq-level 2 --cu-lossless --max-merge 3 --rc-lookahead 25 --lookahead-slices 4 --ref 4 --range limited --max-cll ""$maxCLL,$maxFALL"" --hdr --hdr-opt --repeat-headers --aud --deblock -3:-3 --no-strong-intra-smoothing --aq-mode 1 --rskip-edge-threshold 3 --psy-rd 4 --psy-rdoq 15 --ctu 32 --rskip 2 --deblock -3:-3"

"crf: $crf"
"preset: $preset"
"maxcll: $maxcll"
"maxfall: $maxfall"
"low: $LumMin"
"high: $LumMax"
"x265Params: $x265Params"

&D:\temp\ffmpeg-4.4-full_build\bin\ffmpeg.exe -i ""$file"" -pix_fmt yuv420p10le -strict -1 -f yuv4mpegpipe - | d:\temp\x265\x265.exe $x265Paramsffmpeg.exe -i %1 -pix_fmt yuv420p10le -strict -1 -f yuv4mpegpipe - | d:\temp\x265\x265.exe $x265Params

quietvoid

15th November 2022, 16:02

From a quick comparison of a full length encode in PQ/HDR, --hist-scenecut seems pretty good now.

Out of 628 expected shots..
Default: 180 false positives, 214 missed (or not exact frame), 414 frame perfect. 66% correct.
--hist-scenecut: 12 false positives, 121 missed (or not exact frame), 507 frame perfect. 81% correct.

YMMV.

stax76

15th November 2022, 17:51

@_kermit

This powershell code works:

$env:ffmpeg = 'D:\ffmpeg.exe'
$env:x265 = 'D:\x265.exe'
$env:avs = 'D:\test.avs'
$env:output = 'D:\test.hevc'

$env:ffmpeg_args = '-f yuv4mpegpipe -strict -1 -loglevel fatal -hide_banner'

$env:x265_args = '--crf 18 --output-depth 10 --y4m'

cmd /s /c --% ""%ffmpeg%" -i "%avs%" %ffmpeg_args% - | "%x265%" %x265_args% --output "%output%" -"

In short:

cmd /s /c --% "<cmd style command>"

benwaggoner

15th November 2022, 22:18

quietvoid

16th November 2022, 01:38

It seems there's a chance the current upstream x265 segfaults with --hist-scenecut, at the end of an encode.

#0 0x0000555555641be9 in x265::Lookahead::detectHistBasedSceneChange (this=this@entry=0x555555be6a70,
frames=frames@entry=0x7fff8521b870, p0=p0@entry=7, p1=p1@entry=8, p2=p2@entry=9)
at x265/source/encoder/slicetype.cpp:2848
2848 absIntDiffFuturePast = (uint8_t)X265_ABS((int16_t)futureFrame->averageIntensityPerSegment[segmentInFrameWidthIndex][segmentInFrameHeightIndex][0] - (int16_t)previousFrame->averageIntensityPerSegment[segmentInFrameWidthIndex][segmentInFrameHeightIndex][0]);

Seems futureFrame is a null pointer.

(gdb) p futureFrame
$1 = (x265::Lowres *) 0x0
(gdb) p previousFrame
$2 = (x265::Lowres *) 0x7ffef110f6c8

Should probably be reported.

_kermit

16th November 2022, 01:45

Boulder

16th November 2022, 06:02

It seems there's a chance the current upstream x265 segfaults with --hist-scenecut, at the end of an encode.

Seems futureFrame is a null pointer.

Should probably be reported.

I've now had one full movie encode crash twice and judging by the ETA time left in the console window, it could be the exact same frame so maybe it's this bug causing it.

Ashok Kumar Mishra

17th November 2022, 13:23

I've now had one full movie encode crash twice and judging by the ETA time left in the console window, it could be the exact same frame so maybe it's this bug causing it.

I observed the issue and sent the commit to be pushed.

jpsdr

17th November 2022, 21:13

Odd... (and lucky...?) I'm using also --hist-scenecut but didn't observe the issue. Made some very quick test with very small file and now a multi-pass encode is running since a few days, the 1rst pass went fine, second is running for now. I just hope i'll not have the issue just at the end of the encode after several days... :(
In the meantime, I hope the commit will be avaible in the official release.

=================================================

@LeXXuz
Don't know if you'll be able to use it, but i realized that the x86 version in my current releases is a 8 bits version. When you choose Win32 target, the HIGH_BIT_DEPTH option is not avaible in the Cmake config options... Don't know why, but with CMake/VisualStudio, if you build a x86 version of x265, you can do only 8bits. I'll try to build a 8bits x64 version when the commit will be made public.

benwaggoner

17th November 2022, 21:59

@LeXXuz
Don't know if you'll be able to use it, but i realized that the x86 version in my current releases is a 8 bits version. When you choose Win32 target, the HIGH_BIT_DEPTH option is not avaible in the Cmake config options... Don't know why, but with CMake/VisualStudio, if you build a x86 version of x265, you can do only 8bits. I'll try to build a 8bits x64 version when the commit will be made public.
I have some dim recollection of 10-bit not being implemented for 10-bit as the extra memory requirements being even less feasible within the 4 GB process limit.

jpsdr

17th November 2022, 22:30

Ok. I had a suspicion of something like that.

vpupkind

18th November 2022, 01:03

quietvoid

18th November 2022, 01:43

How did you determine the 628 expected shots?

Extracted from Dolby Vision RPU, since the shot starts are flagged with "scene_refresh_flag".

qyot27

18th November 2022, 02:37

I have some dim recollection of 10-bit not being implemented for 10-bit as the extra memory requirements being even less feasible within the 4 GB process limit.
More or less that reasoning from what I recall, but it is possible to build 10-bit and 12-bit x265 (and of course, 8-10-12 multilib) for IA32 targets. You just have to disable ASM when building for 10 or 12 bit (thus, not really worth it to do it, but it is still possible).

benwaggoner

18th November 2022, 03:11

More or less that reasoning from what I recall, but it is possible to build 10-bit and 12-bit x265 (and of course, 8-10-12 multilib) for IA32 targets. You just have to disable ASM when building for 10 or 12 bit (thus, not really worth it to do it, but it is still possible).
Yeah. Running a 32-bit Windows is so limiting these days I'm always startled to realize anyone is still doing it. So many apps aren't available at all, and the ones that do exist are almost always slower than their 64-bit equivalent. And running 32-bit on 32-bit limits apps to only 2 GB, 3 GB if Large Address Aware (LAA). A 64-bit OS allows a 32-bit app to have 4 GB in LAA, and so is still preferable.

I've not had any 32-bit Windows system in my household for over a decade now. And that's been a couple of dozen by this point, what with all the kids I have.

Boulder

18th November 2022, 06:19

I've tried using --hist-scenecut on a PQ B/W movie (Mr. Smith Goes to Washington from 1939) and in this case it fails badly. Lots of missed scene changes even if they are rather sharp ones. I didn't compare against the classic mode yet.

LeXXuz

18th November 2022, 12:08

@LeXXuz
Don't know if you'll be able to use it, but i realized that the x86 version in my current releases is a 8 bits version.

Thank you but my entire workflow is in 64bit.

vpupkind

18th November 2022, 19:08

Extracted from Dolby Vision RPU, since the shot starts are flagged with "scene_refresh_flag".
Can you recommend any tool for this? Also, how do you know this is the ground truth? Is it coming from an EDL, or a DolbyVision tool?

quietvoid

18th November 2022, 19:28

Can you recommend any tool for this? Also, how do you know this is the ground truth? Is it coming from an EDL, or a DolbyVision tool?

You can just use dovi_tool if you have a binary RPU.
The flag is pretty reliable in the RPU, as long as the RPU source is authored properly.
So the shots are probably coming from the XML metadata.

This can be easily verified by looking at the video at the shot boundaries indicated.
The only exception is for user generated content and "long play mode" metadata, which doesn't have shots.

For dovi_tool, there's this simple workflow to obtain the shot frame numbers: https://forum.makemkv.com/forum/viewtopic.php?p=123221&sid=039dcffa96a306f14f96cd47121a1f60#p123221

First, convert the RPU to json:

dovi_tool export -i RPU.bin -o RPU.json

Second, use jq (https://stedolan.github.io/jq/) to get a list of frames that have the scene_refresh_flag set:

jq "to_entries | .[] | select(.value.vdr_dm_data.scene_refresh_flag == 1) | .key" RPU.json > RPU-scene-frames.txt

The result with be a file with one (zero-based) frame number per line. Then I'll pick a few throughout the file, look at those frames in the video, and verify that each one is indeed the start of a new shot.

Boulder

19th November 2022, 10:20

jpsdr

19th November 2022, 12:22

I observed the issue and sent the commit to be pushed.

Where can i find it ?
I searched on https://bitbucket.org/multicoreware/x265_git/src/master/ but didn't find anything.

rwill

19th November 2022, 16:22

Here's a test clip for hist-scenecut with B/W PQ sources, where it fails miserably.
https://drive.google.com/file/d/19FyX-yJGPqultNHbfaFDQpR-xL5Bjilu/view?usp=share_link

Hist-scenecut was able to find only one scene change, the classic mode got 9 and not all in the correct place. The correct amount is 17 if I didn't miss any.

I decoded the MKV stream and there are 1441 frames. There is total 0 scenecuts in the video clip.

This can easily be confirmed visually because there is almost no change in color throughout the stream.

Boulder

19th November 2022, 16:28

Using DGSource to decode and loading in VDub, these should be the scene cut frames.

45
263
375
405
492
647
668
728
772
866
898
1111
1202
1253
1285
1324
1403

jpsdr

20th November 2022, 09:55

How do you make a pull request for x265 ? Or can someone make the following PR fix :
In encoder.cpp line 4288, apply the video preset BEFORE testing bHDR10Opt just in line 4274 !!!!!!!!

ShortKatz

20th November 2022, 13:16

A pull request for x265 is quite a bit tricky and in my experiences is also mostly ignored. But you need to send it to the mailing list x265-devel@videolan.org with a specific formatting. For details see
https://bitbucket.org/multicoreware/x265_git/wiki/Contribute

jpsdr

20th November 2022, 15:22

Tricky...
I think i'll for now just do it on my personnal mod branch... If any current dev read this, he can do the PR.

jpsdr

20th November 2022, 20:44

FranceBB

20th November 2022, 22:54

Hi Jean Philippe, I'll be back at work after Thanksgiving.
I already have access to your FTP.
We'll prepare a formal email together when I'm back (Sky is a member of the multicoreware partner program) ;)

jpsdr

21st November 2022, 01:46

I've allready put everything on it.

vpupkind

21st November 2022, 02:31

--multi-pass-opt-rps broken !!!
I have a small clip which can reproduce the issue, this option produces a broken, unplayable h265 stream.
I can give :
- The clip
- The encode script
- The broken encode result
I can put all these data on my FTP server, so any dev can PM me to have the server information to retrieve all of these.

I encountered this issue a few years ago. This option results in different PPS's for different GOPs, which is perfectly legal. If a decoder assumes that PPS does not change, it will end up incorrectly decoding at some point. What did you use to package and decode?

In any case, just try using `hev1` (and not `hvc1`, where just one copy of PPS is stored) packaging in ISO-BMFF, MP4Box is your friend here.
If this doesn't help, just turn it off. It is a very elegant optimization, but not that significant in the grand scheme of things.

jpsdr

21st November 2022, 13:13

I'm opening the raw stream with DGIndexNV.

vpupkind

21st November 2022, 17:15

I'm opening the raw stream with DGIndexNV.
Can you please share the stream? I can look at it in an analyzer.

jpsdr

21st November 2022, 19:49

Thank for that. I've sent you information by PM.
Interested of course by feedback.

Majorlag

22nd November 2022, 18:34

--multi-pass-opt-rps broken !!!
I have a small clip which can reproduce the issue, this option produces a broken, unplayable h265 stream.
I can give :
- The clip
- The encode script
- The broken encode result
I can put all these data on my FTP server, so any dev can PM me to have the server information to retrieve all of these.

I use --multi-pass-opt-rps with all my encodes. I use it with --multi-pass-opt-analysis --multi-pass-opt-distortion in my command lines, in 2 pass encodes. Don't think those options work correctly in 1 pass encodes since they are specifically for multi pass. I also use mkv containers if that helps.

I have never run into issues since they where introduced awhile back using LigH encoder

jpsdr

22nd November 2022, 19:38

Didn't have --multi-pass-opt-analysis in my script, tested, didn't change the issue, result stream unplayable.

Majorlag

22nd November 2022, 20:15

the only other time I have had any issues with playback was my end resolution height was not devisable by minimum of 2, had the wrong crop set which ended in a 1. Have you tried FFVideoSource instead of DGIndexNV to test cpu only decode instead of hardware acceleration decode?

jpsdr

22nd November 2022, 21:18

If i mux to mkv and play with Mediaplayer Classic Home-Cinema or VLC it still output garbage, if i remove the option everything is fine.
Also, i didn't trig the issue at the begining, i first made tests on a small clip to figure out the options. It's when i encoded the whole movie that the bug trigged. The first 80000 frames where good, and then after a specific scene everything was garbage.
Lucky, the bug is trigged just by extracting a small clip with the scene, don't need the whole movie !
At this point, and with all these elements, for me --multi-pass-opt-rps is broken, period !

mister_no

23rd November 2022, 12:17

@jpsdr
Do I have to set all 3 options (--sbrc --sbrc-aq5 and --sbrc-hdr) or only --sbrc and --sbrc-hdr with Source HDR?

vpupkind

23rd November 2022, 17:16

jpsdr

23rd November 2022, 19:09

Not before a week, i've started an encode on my PC Video and i have to wait for it to finish.
What have you seen on your side with my file ?

ShortKatz

23rd November 2022, 22:24

I'm wondering why no one fixes the broken Apple build on M1/M2 architecture. Its broken since 2022-11-02.
https://mailman.videolan.org/pipermail/x265-devel/2022-November/013536.html
https://bitbucket.org/multicoreware/x265_git/issues/623/missing-underscore-for-asm-function-needed
HandBrake has a patch for this issue: https://github.com/HandBrake/HandBrake/blob/master/contrib/x265/P00-darwin-build-fix.patch

jpsdr

24th November 2022, 09:32

Finaly i've been able to made some others tests play remuxing, i haven't the bad output with DGIndexNV, but i still have some unexpected "hicup" during the play.

At this point, it realy needs someone who have the proper tools to fully/totaly analyse the result x265 stream to check proper compliancy.

ShortKatz

26th November 2022, 08:42

Where can i find it ?
I searched on https://bitbucket.org/multicoreware/x265_git/src/master/ but didn't find anything.

I assume its this one: https://mailman.videolan.org/pipermail/x265-devel/2022-November/013544.html
Does this fix the segfault you are seeing?

jpsdr

26th November 2022, 09:47

I personnaly don't have the crash issue, but thanks for the links.
It's just one line, i'll add it for now in my "mod" version and make a new build with it.

Boulder

26th November 2022, 10:04

I assume its this one: https://mailman.videolan.org/pipermail/x265-devel/2022-November/013544.html
Does this fix the segfault you are seeing?

I've tested that change and it didn't fix the crash I have. I guess I need to try to find out the exact spot where it happens to get a small sample for debugging.

jpsdr

26th November 2022, 12:28

Will wait in that case before new builds, as they took a little time to make...

Boulder

26th November 2022, 15:12

I've now got the sample clip, the encoder crashes silently around frame 450 or so if hist-scenecut is enabled.

https://drive.google.com/file/d/1u8d7ClC793AWMuu0iqRpmDIQCEchfmW-/view?usp=share_link

quietvoid

26th November 2022, 16:02

I've now got the sample clip, the encoder crashes silently around frame 450 or so if hist-scenecut is enabled.

https://drive.google.com/file/d/1u8d7ClC793AWMuu0iqRpmDIQCEchfmW-/view?usp=share_link

I can't reproduce the crash with the patch.
It crashes with master only.

I'd make sure you patched x265 properly.

Boulder

26th November 2022, 18:37

I can't reproduce the crash with the patch.
It crashes with master only.

I'd make sure you patched x265 properly.

I got it now.. I had mistakenly put the custom patch part in the wrong phase of MABS compilation. MABS doesn't seem to give a really verbose log of what happens during compilation so I had no clue :(

And yes, the patched version works on that sample so it looks good for production use :)

jpsdr

27th November 2022, 11:08

Ah... Good news, so i'll try today to make builds of my "mod" version.

jpsdr

27th November 2022, 13:23

Still about the possibility of --multi-pass-opt-rps broken.
A question from someone investigating on the stream, is the fact of having different PPSs with the same PPS id H265 compliant ?

rwill

27th November 2022, 13:43

Still about the possibility of --multi-pass-opt-rps broken.
A question from someone investigating on the stream, is the fact of having different PPSs with the same PPS id H265 compliant ?

Why don't you look this up yourself?

Took me just seconds to find this in 7.4.2.4.2 of the HEVC standard ("Order of VPS, SPS and PPS RBSPs and their activation"):

Any PPS NAL unit containing the value of pps_pic_parameter_set_id for the active PPS RBSP for a coded picture (and consequently
for the layer containing the coded picture) shall have the same content as that of the active PPS RBSP for the coded picture,
unless it follows the last VCL NAL unit of the coded picture and precedes the first VCL NAL unit of another coded picture.

jpsdr

27th November 2022, 14:13

Took me just seconds to find this
Because it seems that you, contrary of me, you know what this question was talking about, and where and what to search...

Thanks for the answer.

Edit:
The fact is... My H265 knowlegde is just not enough to understand the answer if it's compliant or not, and so if the stream x265 produced is H265 compliant or not. :(
I would said it's NOT compliant (so in that case a bug in x265), unless...
But i don't know if my stream is in the unless... case.

jpsdr

27th November 2022, 16:23

Released new builds of my "mod" version (so, including the histogram cut-scene fix) on my github.

iAvoe

27th November 2022, 21:49

Still about the possibility of --multi-pass-opt-rps broken.
A question from someone investigating on the stream, is the fact of having different PPSs with the same PPS id H265 compliant ?

With --opt-qp-pps, --opt-ref-list-length-pps, I have encountered heavy artifact that teared the scene apart as the motion progresses (block and motion residual based), only the beginning still frames were all good. However I'm not very certain if PPSs caused this, as mainwhile I've tried changing from --me star to --me umh and disabled --asm. I don't encode often so this could be just me.

LigH

28th November 2022, 17:55

New upload: x265 3.5+70-3177961fb (https://www.mediafire.com/file/3a687fzj9qf06m9/x265_3.5+70-3177961fb.7z/file)

[Windows][GCC 12.2.0][32/32XP/64 bit] 8bit+10bit+12bit

News since v3.5+68:
Fix encoder crash with histogram scenecut

LeXXuz

29th November 2022, 14:33

I'm kinda lost with this hist-scenecut issue.

Is this relevant for 1080p SDR encodings or only HDR? Is it enabled by default with 3.5+68 and up or not?

Boulder

29th November 2022, 17:44

Released new builds of my "mod" version (so, including the histogram cut-scene fix) on my github.

There is the aq-fast-edge patch (https://gist.github.com/noizuy/909ceff9230a24e54073837aaa09e9f7), would it make any sense to incorporate it to SBRC? I suppose the edge detection will always be done in that case so I don't know if the patch fits it straight away.

Boulder

29th November 2022, 17:45

I'm kinda lost with this hist-scenecut issue.

Is this relevant for 1080p SDR encodings or only HDR? Is it enabled by default with 3.5+68 and up or not?

--hist-scenecut works for regular HDR (except for my B/W testclip) and for 8-bit SDR encodes.

jpsdr

29th November 2022, 18:39

There is the aq-fast-edge patch
I'll see if i can add this to my mod version.

The only warning i would think of it's without blur, the edge detection result/value will change, and so probably change the resulted computed AQ value.
Edges will have more effects, if the actual values of AQ modes are tuned for using the blur edge version results, this may have a negative effect.

benwaggoner

29th November 2022, 19:38

I'm kinda lost with this hist-scenecut issue.

Is this relevant for 1080p SDR encodings or only HDR? Is it enabled by default with 3.5+68 and up or not?

Huh, yeah. --hist-scenecut doesn't list its default value, unlike the other parameters. A documentation miss:
--hist-scenecut, --no-hist-scenecut
Indicates that scenecuts need to be detected using luma edge and chroma histograms. --hist-scenecut enables scenecut detection using the histograms and disables the default scene cut algorithm. --no-hist-scenecut disables histogram based scenecut algorithm.

If it is on by default, it would only be in the current Master branch. It is off by default in the current 3.5 release and before.

quietvoid

29th November 2022, 19:41

The new --hist-scenecut doesn't have a tunable parameter anymore. There are around 6 hardcoded thresholds for it.
It's also disabled by default.

iAvoe

30th November 2022, 14:46

This is an interesting new command line added!

--[no-]mctf Enable GOP based temporal filter.

It was recently changed to --[no-]mcstf.

In my research, mctf builds a lift-scheme network among every GOP, and then creates predict (estimated) & update (residuals) function among the GOP boundary.
The network depth is depending on GOP size, and a decent network must be established on accurate scenecut.
Some of my tests shows performance slowdowns as the memory (16GB) was full (could that be mctf actually stores the whole GOP into RAM?), and film sourced encoding speed from 2.8fps dropped to 0.6fps on a 10700K.

The testing were done with these options:

--mcstf --tu-intra-depth 4 --tu-inter-depth 4 --limit-tu 1 --me star --subme 3 --merange 48 --analyze-src-pics --weightb --ref 3 --max-merge 4 --no-open-gop --min-keyint 3 --keyint 312 --fades --bframes 14 --b-adapt 2 --radl 3 --hist-scenecut --constrained-intra --b-intra --crf 21.8 --qpmin 8 --crqpoffs -3 --ipratio 1.2 --pbratio 1.5 --rdoq-level 2 --aq-mode 4 --aq-strength 1 --qg-size 8 --rd 5 --limit-refs 0 --rskip 0 --rect --amp --psy-rd 1.6 --rd-refine --qp-adaptation-range 3 --deblock 0:0 --limit-sao --sao-non-deblock --selective-sao 3 --hash 2 --allow-non-conformance

benwaggoner

1st December 2022, 00:52

In my research, mctf builds a lift-scheme network among every GOP, and then creates predict (estimated) & update (residuals) function among the GOP boundary.
The network depth is depending on GOP size, and a decent network must be established on accurate scenecut.
Great description!

Some of my tests shows performance slowdowns as the memory (16GB) was full (could that be mctf actually stores the whole GOP into RAM?), and film sourced encoding speed from 2.8fps dropped to 0.6fps on a 10700K.
Do you mean your machine only has 16 GB of physical RAM? Were you able to determine if the slowdown was due to virtual memory kicking in?

The testing were done with these options:
--mcstf --tu-intra-depth 4 --tu-inter-depth 4 --limit-tu 1 --me star --subme 3 --merange 48 --analyze-src-pics --weightb --ref 3 --max-merge 4 --no-open-gop --min-keyint 3 --keyint 312 --fades --bframes 14 --b-adapt 2 --radl 3 --hist-scenecut --constrained-intra --b-intra --crf 21.8 --qpmin 8 --crqpoffs -3 --ipratio 1.2 --pbratio 1.5 --rdoq-level 2 --aq-mode 4 --aq-strength 1 --qg-size 8 --rd 5 --limit-refs 0 --rskip 0 --rect --amp --psy-rd 1.6 --rd-refine --qp-adaptation-range 3 --deblock 0:0 --limit-sao --sao-non-deblock --selective-sao 3 --hash 2 --allow-non-conformance
That's an interesting mix of some unusual parameters that might be triggering some non-standard paths and states. Anime optimized? I'm curious how you came to those settings, and what you were optimizing for.

I can imagine any of bframes >8, radl, and keyint >250 could all trigger some odd combinations that could use extra memory or bypass some optimizations.

A quick check using just --preset slow or slower (your preset is sort of blend of the two, fundamentally) with --mcstf on and off would be less likely to trigger anything odd, and would give a clearer sense of what the typical impact of --mcstf would be.

jpsdr

1st December 2022, 11:50

Some of my tests shows performance slowdowns as the memory (16GB) was full (could that be mctf actually stores the whole GOP into RAM?)
mctf disable multi-threading, this is probably the slowdown.

iAvoe

1st December 2022, 16:35

Great description!

Do you mean your machine only has 16 GB of physical RAM? Were you able to determine if the slowdown was due to virtual memory kicking in?

That's an interesting mix of some unusual parameters that might be triggering some non-standard paths and states. Anime optimized? I'm curious how you came to those settings, and what you were optimizing for.

I can imagine any of bframes >8, radl, and keyint >250 could all trigger some odd combinations that could use extra memory or bypass some optimizations.

A quick check using just --preset slow or slower (your preset is sort of blend of the two, fundamentally) with --mcstf on / off would be less likely to trigger anything odd, and would give a clearer sense of what the typical impact of --mcstf would be.

I swapped memory sticks (2x16 to 2x8, but both have the same speed & latency) on my PC, the slowdown still kicks in, but 2x8GB kits get to experience more/longer usage spikes.

My theory is that even with film source, you still could get scenes that matches better with anime presets, so the setting still could get extra compression there.

I borrowed x264 settings for --bframes & --keyint, since maintaining longer GOP could result to better compression, but I could be wrong. I wasn't getting into memory usages much, could you elaborate?

I was also unbale to find how much is too much, and I see people testing high values without getting artifacts, so kind of passively accepted them :D

Boulder

1st December 2022, 18:39

With grainy content, x265 tends to use plenty of B-frames compared to clean ones. I've always found this a bit odd, and I don't know the reason for the behaviour. Just encode any old crispy B/W movie and it's double digits for the 10 consecutive B-frames stat.

Boulder

2nd December 2022, 05:57

Boulder

2nd December 2022, 12:54

I can see how to add on my mod branch version these features request. I must say that i said to myself at almost the begining that having a log of the the edgeIntensity and brightnessIntensity could be very interesting. And if there is the hysteresis, the current choice also. But had no idea how to add this.

Have you had the chance to look at this (adding AQ-mode and that other SBRC related data to the csv log)? I was thinking of maybe doing some peeking on the code this weekend but as I'm not a coder, it takes time :D

jpsdr

2nd December 2022, 13:59

Boulder

2nd December 2022, 16:23

Not realy, and for the very short glimpse i have made, it doesn't seem easy (or my glimpse was bad). And, unfortunately for you, as it's not a feature i'm interesting on, i'm not realy interesting in spending a lot of time for this...

Hmm.. I can affect what to write in the csv file, but I cannot figure out how to get the chosen variable value output there :mad:

benwaggoner

7th December 2022, 19:51

Something I've noticed while testing --limit-refs and --limit-modes a bit: --no-limit-modes increases the effect of --amp quite a lot, based on what --csv-log-level 2 shows. I'd even say that without --no-limit-modes, it really makes no sense enabling --amp. I don't know if --limit-refs should affect the partition decisions at all, but --limit-refs 1 produced noticably bigger amounts of amp partitions compared to 3 and 0 in this one test of mine.
The behavior certainly makes sense. --limit-modes (and --limit-refs) are early-exit optimizations that less aggressively check --amp and --rect opportunities in order really reduce the speed impact of those modes. Those limits mean that amp and rect will mostly get used where they provide a more obvious benefit. In theory (and apparently practice), while you get fewer of those CU types, the ones you'll get are the ones with the biggest benefit, so the speed/quality tradeoff is better.

When trying to juggle a bunch of settings for optimal quality@perf, it's really a combination of things. I'd generally expect an --amp --rect --limit-modes --limit-refs to yield better net results in less time that just using --amp OR --rect with --no-limit-*. Similarly --tskip without --tskip-fast or --tu-inter- and -intra-depth 4 without --limit-tu 4 can slow things down to much. Better to get 75% of the value of the features in 25% of the time than none of the benefits at all.

These all tend to be pretty content specific (high detail, low noise, sharp edges), so leaving them on by default without --limit* options can waste a whole lot of power & time without material benefit for the majority of GOPs.

LigH

11th December 2022, 17:38

Attention:

Patches for a "BBAQ" feature with configuration files are being proposed; prepare for their commitment soon™...

Boulder

11th December 2022, 18:41

Attention:

Patches for a "BBAQ" feature with configuration files are being proposed; prepare for their commitment soon™...

I have to ask - what is BBAQ?

LigH

11th December 2022, 19:06

It is related to "Scene Cut Aware QP", could not yet spot an explanation for the acronym though...

File containing scenecut-aware-qp mode, window duration and offsets settings required for the masking. Works only with --pass 2

vpupkind

12th December 2022, 22:50

It is related to "Scene Cut Aware QP", could not yet spot an explanation for the acronym though...
Bidirectional Boundary Aware Quantization, if I'm not mistaken

jpsdr

13th December 2022, 01:18

For those interested, i've made a new build of my mod version.

LeXXuz

13th December 2022, 07:42

For those interested, i've made a new build of my mod version.

Always. And thx for the 8bit version btw. :)

James_b

14th December 2022, 14:40

xxxxx

14th December 2022, 16:24

Is there a program I can see the size of the frames in a video?

excellentswordfight

14th December 2022, 16:48

Hi Guys!
Does anyone know if i get better picture quality whith HEVC than HuffYUV? I think about from a video editor, both set at 100% (lowest compression). This is for max out before Youtube conversion sets in.

Some say VP9 gives better dynamic and colors than AVC but is there something like this in above?

Thank's for any advice!
HuffYUV is a lossless codec, so nothing will get you better picture quality. HEVC in lossless mode can although give you the same at a lower filesize but will be alot slower, but I doubt any NLE have a lossless mode for HEVC implemented.

jpsdr

14th December 2022, 19:08

Does anyone know if i get better picture quality whith HEVC than HuffYUV?
If you're looking for a lossless codec, HuffYUV is a little old and not always supported, and slow.
Choose UTVideo or MagicYUV, they're the best lossless codec i know, fast and efficient (doesn't mean it doesn't exist others lossless codec even better).

LigH

14th December 2022, 21:16

Is there a program I can see the size of the frames in a video?

Such tools, in general, are called "Bitrate Viewer". Not all of them support every source format, though. And please better learn what a GOP is and why the size of I, P, and B frames differs much more than the quality.

LigH

15th December 2022, 08:17

Just committed:

Pushed BBAQ 3 patches to master branch

LigH

15th December 2022, 08:33

I need advice ... I have a shell script which is supposed to update my local copy of x265 when an update is available. This used to work flawlessly before. But now a vi opens, trying to create a text file explaining the reason for a merge. So I guess there is a flaw in my git command, and I do not understand the concept of git good enough to tell apart the meaning of many similar terms in the heap of options... How do I avoid asking back? I do not even want to commit anything on the git server, only update my local copy.

#!/bin/sh
git -C /build/x265_git-git fetch -v --progress origin
git -C /build/x265_git-git pull -v --no-rebase --progress origin "${1:-master}"

The desired result is: If I submit a commit hash as parameter to this script, it shall be synced with that specific commit, else it shall be updated to the latest commit of the master branch.
_

PS: Maybe I should try to source /build/media-suite_helper.sh and then try to use do_vcs "https://bitbucket.org/multicoreware/x265_git.git"; but there is also a patch related to cmake library paths with L vs. l, maybe that one interfers.

LigH

15th December 2022, 14:13

Boulder

15th December 2022, 14:24

New upload: x265 3.5+74-1ec26b9f6 (https://www.mediafire.com/file/kh7loyrqrgv3zx7/x265_3.5+74-1ec26b9f6.7z/file)

[Windows][GCC 12.2.0][32/32XP/64 bit] 8bit+10bit+12bit

News since v3.5+70:
--scenecut-qp-config <file> File containing scenecut-aware-qp mode, window duration and offsets settings required for the masking. Works only with --pass 2

I guess we need a sample file to know how it works...

Yes, the functionality looks a bit confusing to me. Hopefully someone could provide a simple use case for this.

LigH

15th December 2022, 14:38

Unfortunately, readthedocs (https://x265.readthedocs.io/en/master/cli.html) is not yet up to date.

jpsdr

15th December 2022, 22:24

I was using --scenecut-aware-qp 3 in my scripts, and now with the new version i have a FLAW error, "invalid argument", but according the --fullhelp it's stil a valid argument...

Edit :
Has anyone else tested to check if --scenecut-aware-qp 3 is still working after these commits ?
For me this release is broken somewhere, all my encoding scripts are not running anymore.

benwaggoner

16th December 2022, 17:54

Unfortunately, readthedocs (https://x265.readthedocs.io/en/master/cli.html) is not yet up to date.
FWIW, there is an "Edit on Bitbucket" button at the top of all those pages. We can submit edits and additions ourselves for things that MCW hasn't documented yet or needs tweaking. I just submitted a commit to edit the weird discontinuity in the options list for --force-flush as a test.

ShortKatz

16th December 2022, 18:04

New upload: x265 3.5+74-1ec26b9f6 (https://www.mediafire.com/file/kh7loyrqrgv3zx7/x265_3.5+74-1ec26b9f6.7z/file)

[Windows][GCC 12.2.0][32/32XP/64 bit] 8bit+10bit+12bit

News since v3.5+70:
--scenecut-qp-config <file> File containing scenecut-aware-qp mode, window duration and offsets settings required for the masking. Works only with --pass 2

I guess we need a sample file to know how it works...

There is now a sample file here: https://bitbucket.org/multicoreware/x265_git/downloads/scenecut_qp_config.txt

Its from https://mailman.videolan.org/pipermail/x265-devel/2022-December/013560.html

benwaggoner

16th December 2022, 19:20

There is now a sample file here: https://bitbucket.org/multicoreware/x265_git/downloads/scenecut_qp_config.txt

Its from https://mailman.videolan.org/pipermail/x265-devel/2022-December/013560.html
Huh. It seems odd to use an external file for just a few static parameters. Normally x265 uses external files (multi-pass-opt-analysis, qpfile) for data that can vary per-frame.

Perhaps there will be a way to specify different parameters for different frame ranges?

Greenhorn

16th December 2022, 21:25

Looking at the docs for the new file on BitBucket, it can get a lot more complicated that sample, so I can see why they used a file for it even if it's still static values.

(docs/rEST/cli.rst, and you have to click "Open Raw" in the little menu next to the "Edit" button since the fancy markdown version you see by default is ancient. Search for masking-strength.)

quietvoid

16th December 2022, 21:54

You can specify the different QP adjustments per window, and there are 6 windows now.
So the file is useful for that, but hardly IMO.

My CLI for example is already 1000 characters, so it wouldn't bother me.

LigH

16th December 2022, 22:14

We can submit edits ...

... but better do so only when we actually have a clue. I do not, in this case.

But I noticed a mailing list entry so it will happen.

benwaggoner

16th December 2022, 23:31

... but better do so only when we actually have a clue. I do not, in this case.

But I noticed a mailing list entry so it will happen.
It still goes through the normal commit process, so MCW has a chance to review any edits. I focused on some formatting deficiencies that I'd noticed had been a common source of confusion.

ghostshadow

17th December 2022, 12:17

jpsdr

17th December 2022, 13:33

I have, just for testing, replaced :
OPT("scenecut-aware-qp") p->bEnableSceneCutAwareQp = x265_atoi(value, bError);
with
OPT("scenecut-aware-qp") p->bEnableSceneCutAwareQp = atoi(value);
No change, my log file has only one line :
x265 [FLAW]: invalid argument: scenecut-aware-qp = 3

Boulder

17th December 2022, 15:41

Do we still need quietvoid's patch (probably a new version of it) to make the functionality work in CRF mode?

benwaggoner

19th December 2022, 02:12

I thought there was an option to activate an automatic filling of this file with different options for each scene but the need to put them by hand from what I understand
ex : --scenecut-aware-qp 3 --masking-strength 500,2,0,200,-1,-1 in the file scenecut_qp_config.txt.
so we have to manually enter all the options per scene?
While it can be hand-edited, which is very useful for testing and early days evaluations like we are today. Production implementations would automatically generate the file based on preanalysis, like how qpfiles are most commonly used. Perhaps a future x265 will generate the file, which could be analyzed and edited as needed.

jpsdr

20th December 2022, 09:59

Ok, i think i understand my mistake.
I thought the new patch offered the possibility of having an external file to configure the scenecut-qp stuff.
But not, it's a necessity !
For just using --scenecut-aware-qp 3 i have to create an external file, acces it with --scenecut-qp-config and put --scenecut-aware-qp 3 inside it... :eek:
I will in my mod version restore the possibility of being able to just put --scenecut-aware-qp directly in command inside a script without the necessity to have an external file just for it (and make all my encoding scripts working again...:D)

LigH

20th December 2022, 16:23

:confused: That's all? It sounded like there is a zone control.

jpsdr

20th December 2022, 19:29

What i ment, it's that now you can't anymore just put --scenecut-aware-qp 3 directly in the command line, you have to put it in an external file. There is probably a lot of others possibilities with this new patch (i didn't dig to see what it's doing because i'm not using these stuff), and having the possibility of using an external file for a lot of zone description is a good thing of course, but for someone like me who was just using --scenecut-aware-qp 3, it breaks all my encode script, and to have to add an extra file just to put one command inside... :(

ShortKatz

20th December 2022, 23:15

jpsdr

22nd December 2022, 20:33

I've made a new build on my mod version (check my github).
As i have en encode running, i couldn't test it.

madey83

23rd December 2022, 07:51

LigH

23rd December 2022, 13:02

The documentation says:

Works only with --pass 2

So according to that one: No. It will most probably require a statistics file gathered in a first pass, to tune scenecuts in a second/last pass of two/multiple passes.

Boulder

23rd December 2022, 14:54

If you use a config file with only --scenecut-aware-qp 3 in it, it changes the output in CRF mode even though there's a warning that it only works with --pass 2 :scared:

ShortKatz

26th December 2022, 15:44

A bunch of new patches have been uploaded to the mailing list. I quite don't understand what this patches are doing, its something about temporal layers, miniGOP and NAL type.

LigH

26th December 2022, 16:24

Temporal sublayer for unreferenced B frames: Decoders may skip them if decoding time is tight, without introducing decoding artifacts.

A "miniGOP" is only the P(B...) portion of a GOP. I frames are essential as reference to many following P frames (and maybe some B frames too). P frames are often referenced by following P and intermediate B frames, so they are semi-essential too, missing one of these in the decoding sequence usually means skipping to the next IDR frame to recover the video stream decoding.

ShortKatz

26th December 2022, 19:15

Ah, thanks for your explanation. :-)

ksec

27th December 2022, 08:37

https://rigaya.github.io/vq_results/

Some H.264 / H.265 and AV1 encoding results.

Fishman0919

28th December 2022, 17:15

Commit: Niranjan Bala

Niranjan Bala committed 65eaa5a 2022-12-15

View source

Add BBAQ to CRF and fix qgSize 0 issue.

Works with CRF now

Boulder

28th December 2022, 19:37

Still the same error with a new build :D

if (param->bEnableSceneCutAwareQp)
{
if (!param->rc.bStatRead)
{
param->bEnableSceneCutAwareQp = 0;
x265_log(param, X265_LOG_WARNING, "Disabling Scenecut Aware Frame Quantizer Selection since it works only in pass 2\n");
}

Boulder

29th December 2022, 07:46

I retested the feature by bypassing that check. With default settings (i.e. setting only --scenecut-aware-qp 1/2/3 in the config file), the filesize difference for a 3000-frame sample was very small, much less than 1%. Does anyone else have any results? I didn't check the csv log yet for changes in QPs.

LigH

29th December 2022, 17:17

Some clean-ups have been committed.

jpsdr

29th December 2022, 18:51

Euh... Does it mean SBRC is removed ? It didn't live for long...:confused:

quietvoid

29th December 2022, 19:30

No, it's reimplemented in another patch.

ShortKatz

30th December 2022, 00:11

I also don't get it, why did they remove the current SBRC implementation and then try to reimplement it with another patch? What different will this make? What was wrong with the first SBRC implementation?

quietvoid

30th December 2022, 00:14

The new SBRC patch is totally different.
The first one didn't really make sense.

Boulder

30th December 2022, 12:07

The new SBRC patch is totally different.
The first one didn't really make sense.

I suppose the new patch is still pending a push to the master branch. Looking at the proposed patch, I find it a bit odd that a closed GOP is now required and that there seems to be a fixed GOP length :confused:

jpsdr

30th December 2022, 18:09

Ok, will wait for this new SBRC patch...

ShortKatz

31st December 2022, 17:47

I was wondering what the HISTOGRAM_BINS variable is used for, which was introduced in the latest SBRC removal patch. It does not occur in the new SBRC patch. Maybe there is more to come?

LeXXuz

2nd January 2023, 14:56

I have a question regarding ctu sizes in CRF mode.

I did some testing with SD and HD content to compare differences between encodes with --ctu 64 and --ctu 32 setting in crf mode.
I read in the docs and on Wikipedia that larger coding tree units usually increase encoding efficiency, therfore should decrease filesize in crf-mode, right?

But I noticed quite the contrary where ctu=32 results in noticeabley smaller file sizes. See screenshot:
https://abload.de/thumb/vergleich64-1vs64-2vsxcemz.png (https://abload.de/image.php?img=vergleich64-1vs64-2vsxcemz.png)
top-left: ctu=64, framethreads=1
top-right: ctu=64, framethreads=2
bottom-left: ctu=32, framethreads=1
bottom-right: ctu=32, framethreads=2
(This is an example with SD video. The behaviour is similar with 1080p content. I also repeated this with different content with less or more complexity to rule out this was just coincidence.)

Okay, as mentioned in the docs, additional frame-threads reduce efficiency and slightly increase filesize. Regarding the noticeable speed increase I think the additional thread is well spend.
And I understand that ctu=32 gives a big speed increase on lower resolutions because there is more parallelism.

But why does the filesize decrease that much which clearly should have an impact on visual quality, right?

The only reason I can think of is that crf values scale differently with different ctu sizes. If so, could I just decrease crf values to get similar filesizes and, more importantly, similar visual results?

Or will ctu=64 still give better visual quality and should always be prefered?

Boulder

2nd January 2023, 20:38

benwaggoner

3rd January 2023, 19:41

For those comparing --ctu 32 versus 64, have you tried using --ctu 64 with --rdpenalty 1? That raises the cost of a 32x32 TU by 4x, theoretically meaning they'll be selected a lot less often, and only when benefits are unambiguous.

However, it's a really old parameter from x265 circa 2014, and I don't know that anyone has tested with it in ages.

--rdpenalty <0..2>
When set to 1, transform units of size 32x32 are given a 4x bit cost penalty compared to smaller transform units, in intra coded CUs in P or B slices.

When set to 2, transform units of size 32x32 are not even attempted, unless otherwise required by the maximum recursion depth. For this option to be effective with 32x32 intra CUs, --tu-intra-depth must be at least 2. For it to be effective with 64x64 intra CUs, --tu-intra-depth must be at least 3.

Note that in HEVC an intra transform unit (a block of the residual quad-tree) is also a prediction unit, meaning that the intra prediction signal is generated for each TU block, the residual subtracted and then coded. The coding unit simply provides the prediction modes that will be used when predicting all of the transform units within the CU. This means that when you prevent 32x32 intra transform units, you are preventing 32x32 intra predictions.

Default 0, disabled.

Values: 0:disabled 1:4x cost penalty 2:force splits

LeXXuz

3rd January 2023, 21:44

For those comparing --ctu 32 versus 64, have you tried using --ctu 64 with --rdpenalty 1?

Thanks Ben. I never heard or noticed that parameter before. So no, I haven't. :o
I'll give it a try and see how this may affect speed and file size.

Still, it would be great to find an explanation to those differences in size between 32 and 64. Like Boulder said, there must be more to it.

vpupkind

3rd January 2023, 22:34

Thanks Ben. I never heard or noticed that parameter before. So no, I haven't. :o
I'll give it a try and see how this may affect speed and file size.

Still, it would be great to find an explanation to those differences in size between 32 and 64. Like Boulder said, there must be more to it.
The lower the resolution the less probability of 64x64 being selected. It's pointless at HD resolutions. Even 32x32 is selected in a tiny % of cases in 1080p and nearly never with lower resolutions.

LeXXuz

3rd January 2023, 23:35

The lower the resolution the less probability of 64x64 being selected. It's pointless at HD resolutions. Even 32x32 is selected in a tiny % of cases in 1080p and nearly never with lower resolutions.

That makes sense. But why those differences then? Shouldn't those results between --ctu 32 and --ctu 64 setting be much closer if x265 wouldn't use 64x64 at those resolutions anyway? Or does the --ctu 64 setting actually force x265 to use 64x64 where it wouldn't normally do? I thought this is more like a 'maximum allowed size' setting, or am I wrong?

vpupkind

3rd January 2023, 23:58

That makes sense. But why those differences then? Shouldn't those results between --ctu 32 and --ctu 64 setting be much closer if x265 wouldn't use 64x64 at those resolutions anyway? Or does the --ctu 64 setting actually force x265 to use 64x64 where it wouldn't normally do? I thought this is more like a 'maximum allowed size' setting, or am I wrong?
It will always go for CTU 64x64, and then evaluate a 32x32 split.

Maxiuca

8th January 2023, 05:56

I compiled the x265 binary yesterday and noticed today that it gave me a strange error ("unknown option -- no-temporal-layers") when I was trying to rerender something, so I had a look at the latest commits and it looks like this option has been retired and the way the temporal layers are configured also changed.

So I just wanted to give everyone a heads-up and hopefully save some time.

Here's the new syntax:

option: --temporal-layers <integer>

Enable specified number of temporal sub layers. For any frame in layer N,
all referenced frames are in the layer N or N-1.A decoder may choose to drop the enhancement layer
and only decode and display the base layer slices.Allowed number of temporal sub-layers
are 2 to 5.(2 and 5 inclusive)

When enabled,temporal layers 3 through 5 configures a fixed miniGOP with the number of bframes as shown below
unless miniGOP size is modified due to lookahead decisions.Temporal layer 2 is a special case that has
all reference frames in base layer and non-reference frames in enhancement layer without any constraint on the
number of bframes.Default disabled.

+----------------+--------+
| temporal layer | bframes|
+================+========+
| 3 | 3 |
+----------------+--------+
| 4 | 7 |
+----------------+--------+
| 5 | 15 |
+----------------+--------+

ShortKatz

8th January 2023, 16:18

I've tried the most recent x265 with the new sbrc patch applied. I got
x265 [warning]: Segment based RateControl requires closed gop structure. Enabling closed GOP.
x265 [warning]: Segment based RateControl requires fixed gop length. Force set min-keyint equal to keyint.

I was wondering what the reason for the closed GOP and min-keyint = keyint might be? The old patch did not set this.

For my test it did set it to
x265 [info]: Keyframe min / max / scenecut / bias : 240 / 240 / 40 / 5.00

Boulder

9th January 2023, 07:14

I was wondering what the reason for the closed GOP and min-keyint = keyint might be? The old patch did not set this.

To me it looks like they are looking at it in a very different way this time. The functionality name itself was a bit strange considering that it was actually an auto AQ function. Now it really looks like what it says -- and I don't think it's a good idea to make things fixed like this. It would be much better to allow the old functionality as well and set those fixed values only in case of VBV being enabled.

benwaggoner

9th January 2023, 18:17

To me it looks like they are looking at it in a very different way this time. The functionality name itself was a bit strange considering that it was actually an auto AQ function. Now it really looks like what it says -- and I don't think it's a good idea to make things fixed like this. It would be much better to allow the old functionality as well and set those fixed values only in case of VBV being enabled.
This SBRC implementation may be for use cases where per fragment max size is the fundamental limitation, not VBV. I've seen that for low latency live solutions, where VBV isn't a great tool for controlling worst-case latency. It would also open up GOP level parallelism since the prior GOP's VBV state isn't relevant.

That's certainly the behavior I'd imagine for a "Segment Based Rate Control" - it's based on each segment/fragment, not on VBV.

vpupkind

9th January 2023, 18:40

This SBRC implementation may be for use cases where per fragment max size is the fundamental limitation, not VBV. I've seen that for low latency live solutions, where VBV isn't a great tool for controlling worst-case latency. It would also open up GOP level parallelism since the prior GOP's VBV state isn't relevant.

That's certainly the behavior I'd imagine for a "Segment Based Rate Control" - it's based on each segment/fragment, not on VBV.

The idea is that HLS and DASH measure bitrate in units of segments, rather than sliding window. You have to obey the VBV in any case, but can redistribute bits more freely this way.

vpupkind

9th January 2023, 18:41

Boulder

9th January 2023, 18:52

It is not an auto AQ function. They submitted a wrong parch initially.

So is the very old auto AQ patch, which has been submitted but never pushed to the master, the one to use for that purpose? I've not seen anything strange in my tests with the incorrect patch.
(https://mailman.videolan.org/pipermail/x265-devel/2020-February/012882.html)

Boulder

9th January 2023, 18:54

This SBRC implementation may be for use cases where per fragment max size is the fundamental limitation, not VBV. I've seen that for low latency live solutions, where VBV isn't a great tool for controlling worst-case latency. It would also open up GOP level parallelism since the prior GOP's VBV state isn't relevant.

That's certainly the behavior I'd imagine for a "Segment Based Rate Control" - it's based on each segment/fragment, not on VBV.

I based my understanding on this comment in the patch which is not yet in master: 3. Reset RateControl (CRF/ABR) at the segment beginning

benwaggoner

9th January 2023, 22:16

I based my understanding on this comment in the patch which is not yet in master: 3. Reset RateControl (CRF/ABR) at the segment beginning
Yep, that nails it.

VBV-based encoding makes sense for Video on Demand, as bits saved in one fragment mean the next fragment can be downloaded sooner to increase client-side fragment buffer duration.

But with adaptive streaming where encoder latency~segment duration, there's not any way to "save" bits for the encoder to use on a fragment that hasn't happened yet. Rebuffer chance is proportional to the maximum fragment size of the lowest bitrate rendition. That's hard and limiting to control through VBV itself, and SBC does a better job of maximizing quality for a given max fragment size.

ShortKatz

9th January 2023, 23:09

It is not an auto AQ function. They submitted a wrong parch initially.

Thats a pity, because HandBrake 1.6.0 does now contain the wrong sbrc patch. If someone now makes a preset with the sbrc option, than this patch will give different results if HandBrake 1.7.0 will be released some day, because the new patch reuses the same option name than the old patch.

So is the very old auto AQ patch, which has been submitted but never pushed to the master, the one to use for that purpose? I've not seen anything strange in my tests with the incorrect patch.
(https://mailman.videolan.org/pipermail/x265-devel/2020-February/012882.html)

Yes, I very much like this auto-aq feature. Was wondering why this patch never made it into master. I still use it with my own builds.

Boulder

12th January 2023, 09:06

Barough

12th January 2023, 10:17

x265 v3.5+85
Built on January 12, 2023, GCC 12.2.0

https://bitbucket.org/multicoreware/x265_git/commits/branch/master

DL :
https://www.mediafire.com/file/9qs4smpj0m6tjzy/

benwaggoner

13th January 2023, 19:04

Looks like the final version was now pushed to master.

Here's hoping that jpsdr can include the old auto-aq feature (+ the hysteresis part) in his mod since SBRC is now something very different from what it was..
I propose --aq-auto or --aq-mode auto.

Yeah, Auto AQ and SBRC are orthogonal and additive features.

jpsdr

13th January 2023, 22:06

I'll try to put it back and replace the name of all the commands :
--srbc => --aq-auto
--srbc-hyst => --aq-auto-hyst
--srbc-aq5 => --aq-auto-aq5
--srbc-hdr => --aq-auto-hdr

benwaggoner

14th January 2023, 07:31

I'll try to put it back and replace the name of all the commands :
--srbc => --aq-auto
--srbc-hyst => --aq-auto-hyst
--srbc-aq5 => --aq-auto-aq5
--srbc-hdr => --aq-auto-hdr
So, --aq-auto itself would be Rec. 709 tuned?

Is there not a way to have hdr + hyst?

ShortKatz

14th January 2023, 11:19

So is the very old auto AQ patch, which has been submitted but never pushed to the master, the one to use for that purpose? I've not seen anything strange in my tests with the incorrect patch.
(https://mailman.videolan.org/pipermail/x265-devel/2020-February/012882.html)

I have updated the original autoAQ patch to the current x265 master. I also included the memory leak fix from the first sbrc approach, AQ mode 5 and the --aq-bias-strength option from a GitHub patch from noizuy.
Everyone feel free to give it a try: https://gist.github.com/Nomis101/9953a2feb69e9f4b96a73fabca2d5ff8

jpsdr

14th January 2023, 12:15

So, --aq-auto itself would be Rec. 709 tuned?
I don't know, i just put back as it was in the first place.

Is there not a way to have hdr + hyst?
Yes, put the 2 commands options.

Should explain more.
First, you have to enable it with --aq-auto, and then, you tune with --aq-auto-xxx. Using for example --aq-auto-hdr without --aq-auto has no effect, as the feature is just not enabled.

I have pushed things on my github, didn't have time yet to build and test.

Boulder

14th January 2023, 14:05

I have updated the original autoAQ patch to the current x265 master. I also included the memory leak fix from the first sbrc approach, AQ mode 5 and the --aq-bias-strength option from a GitHub patch from noizuy.
Everyone feel free to give it a try: https://gist.github.com/Nomis101/995...a73fabca2d5ff8

I don't know, i just put back as it was in the first place.

Yes, put the 2 commands options.

Should explain more.
First, you have to enable it with --aq-auto, and then, you tune with --aq-auto-xxx. Using for example --aq-auto-hdr without --aq-auto has no effect, as the feature is just not enabled.

I have pushed things on my github, didn't have time yet to build and test.

Thank you to both :)

Are the "false SBRC patch" and "old auto-aq" the same or are there some fundamental differences there?

jpsdr

14th January 2023, 17:17

From what i've seen (quickly), my custom release includes all the patchs of ShortKatz. I don't know what was the "old auto-aq".

Boulder

15th January 2023, 08:20

From what i've seen (quickly), my custom release includes all the patchs of ShortKatz. I don't know what was the "old auto-aq".

It was this one: https://mailman.videolan.org/pipermail/x265-devel/2020-February/012882.html

jpsdr

15th January 2023, 09:36

ShortKatz

15th January 2023, 18:58

Is it possible to include support for DOVI profile 7 support to x265? I think I've seen a patch for this somewhere, but I cannot find it, so I may be wrong.

I have pushed things on my github, didn't have time yet to build and test.

Thanks for your effort. I think it is quite a pity that we need to make mod versions to bring x265 to the feature level it actually should be.

vpupkind

15th January 2023, 19:13

@Boulder
At very quick first glance, it seems similar, so there is a good chance it was the same.

QUESTION
With the new SBRC there is :
x265 [warning]: Segment based RateControl requires fixed gop length. Force set min-keyint equal to keyint.
Does it mean that there is not anymore I-Frames inserted on scene change detection and so GOP have always the same length ?
In that case, is it realy an interesting thing ?

It is still interesting for live, as in streaming production you typically have fixed segment duration. I bet this can be extended to variable length segments, especially for 2-pass encoding.

excellentswordfight

16th January 2023, 11:31

@Boulder
Does it mean that there is not anymore I-Frames inserted on scene change detection and so GOP have always the same length ?
In that case, is it realy an interesting thing ?
You can still have i-frames inserted on scene change with fixed GOP size, as you can have non-IDR I-frames.

Fixed GOP sizes are still very relevant for broadcast/streaming/VOD.

FranceBB

16th January 2023, 19:54

I think it is quite a pity that we need to make mod versions to bring x265 to the feature level it actually should be.

You don't, please commit back and the Multicoreware guys will look at everything.
If no one opens pull requests, then the changes will never be merged...

Ritsuka

16th January 2023, 20:08

I guess all the the unapplied patches on the mailing-list and on bitbucket are just in my mind ;P

jpsdr

16th January 2023, 21:54

I've made a new build of my mod version (check my github). Quick test seems fine.

ShortKatz

16th January 2023, 23:18

You don't, please commit back and the Multicoreware guys will look at everything.
If no one opens pull requests, then the changes will never be merged...

I've opened several pull requests on this mailing list. Just one got applied, half a year later. And there are so many more patches on this list that got never merged, not even reviewd.
Here are 3 examples of interesting patches that never got applied:
https://mailman.videolan.org/pipermail/x265-devel/2020-February/012907.html
https://mailman.videolan.org/pipermail/x265-devel/2020-February/012908.html
https://mailman.videolan.org/pipermail/x265-devel/2020-October/013158.html

And then there are sadly some bug fixes that were opened at the wrong place:
https://github.com/videolan/x265/pulls

(And I don't even talk about all those open bugs in the bug tracker that never got fixed)

vpupkind

17th January 2023, 06:09

I've opened several pull requests on this mailing list. Just one got applied, half a year later. And there are so many more patches on this list that got never merged, not even reviewd.
Here are 3 examples of interesting patches that never got applied:
https://mailman.videolan.org/pipermail/x265-devel/2020-February/012907.html
https://mailman.videolan.org/pipermail/x265-devel/2020-February/012908.html
https://mailman.videolan.org/pipermail/x265-devel/2020-October/013158.html

And then there are sadly some bug fixes that were opened at the wrong place:
https://github.com/videolan/x265/pulls

(And I don't even talk about all those open bugs in the bug tracker that never got fixed)

Two of them seem to be of limited non-academic use.

Yomiko

17th January 2023, 11:24

What are the params that are allowed to change for encoding two slices of video and later losslessly concatenating them? Are they merely the ones specified in --zones?

benwaggoner

17th January 2023, 18:24

Should explain more.
First, you have to enable it with --aq-auto, and then, you tune with --aq-auto-xxx. Using for example --aq-auto-hdr without --aq-auto has no effect, as the feature is just not enabled.
Did you consider having the different modes as parameters to --aq-auto? Ala:

--aq-auto none (aka --no-auto-aq)
--aq-auto hyst
--aq-auto "hdr,hyst"

Or

--auto-aq-mode for the parameters?

Those would seem to match x265 style a little better. I'm not sure what the limitations are in command line structure.

jpsdr

17th January 2023, 18:47

Not realy, i made it "quick"... Don't realy interested in spending time to figure out how to analyse command line with several parameters. Could see others command line to figure out how it's done.
This configuration is working and is fine for me. Yes, it could be better and more x265 friendly, but not in the mood...
It's in the first place an own custom build, and i share it in case people could be interested with.
After, if someone is motivated enough to make a pull request, no problemo... ^_^

jpsdr

17th January 2023, 19:18

Are these patches interesting (and correct, not adding issues) to commit in my custombuild ?
I checked my code, they are not implemented, so the bugs (if realy) are still here. As i'm using tskip, if the 1rst is to be commited, i'm very interested by a validation.

https://github.com/videolan/x265/pull/6/commits/51da5f6781e052521cf42ab14bdbc25d50cec5d8 (very interested if this one is to be applyed)
https://github.com/videolan/x265/pull/5/commits/cf043d2a47ac9a2cfa29f518550c8ffe57bc08b6
https://github.com/videolan/x265/pull/4/commits/3b99bbe6306eb3dd3eed677a840cc988a0928a11 (this one is big... Maybe not so easy to check the effects ?)
https://github.com/videolan/x265/pull/4/commits/3d29c2497b3763c797cd97ba4d953a189859ff8b

vpupkind

17th January 2023, 19:53

Did you consider having the different modes as parameters to --aq-auto? Ala:

--aq-auto none (aka --no-auto-aq)
--aq-auto hyst
--aq-auto "hdr,hyst"

Or

--auto-aq-mode for the parameters?

Those would seem to match x265 style a little better. I'm not sure what the limitations are in command line structure.

Talk to MulticoreWare? I would expect them to post it again, with a different parameter name.

Boulder

17th January 2023, 19:55

vpupkind

17th January 2023, 22:04

The third one could well be interesting to many. If you could do the first pass in like half the resolution of the actual encode in the second pass, it might speed things up considerably. Especially with 4K sources, that is.

I think (looking at GitHub) that the reason for non-inclusion is CLA (derived from the x265 CLA)

jpsdr

18th January 2023, 09:55

I think for now i'll not try to include the big one, but i need to be sure that the 3 others are validated to be bug fixes.

For aq-auto, what i'll see if easy to do is the following :
--aq-auto <integer>
0 : Default, disable. <>0 enabled.
Bit 1 : If set to 1 enable hysteresis.
Bit 2 : If set to 1 enable HDR mode.
Bit 3 : If set to 1 enable AQ5.
The others bits (0, 4 to 7) if set to 1 will just enable the aq-auto mode.
=>
--aq-auto 1 : Just enable the aq-auto mode.
--aq-auto 2 is exactly the same than --aq-auto 3.

LeXXuz

18th January 2023, 10:02

Anyone else having issues with x265 halting about 4 seconds before the actual source file ends? It doesn't happen every time but quite a few times now on different systems.
Currently using jp's version: 3.5+75+0-f3c013983 [Mod by Patman]

benwaggoner

18th January 2023, 19:58

jpsdr

18th January 2023, 20:21

I've put the 2 first patches. And no, they were not fixed.
Also put the change on --aq-auto command. Now, i have to build and make my usual small quick encode test.

madey83

18th January 2023, 21:27

I've put the 2 first patches. And no, they were not fixed.
Also put the change on --aq-auto command. Now, i have to build and make my usual small quick encode test.

Hi,

is this "--aq-auto-hdr" can be used for dolby vision encode use case?

jpsdr

18th January 2023, 23:23

@madey83
Yes.

New build with the modified --aq-auto command described in post #8949, and also include the two old bug fix patches. Check my Github.
Quick small encode test, seems fine.

Boulder

19th January 2023, 06:47

It's a bit confusing :D Are we talking about an 8-bit integer here? I was just quickly testing it, and expected --aq-auto 3 to enable hysteresis and HDR mode (calculated 1+2) but it only enabled hysteresis. --aq-auto 6 brought up the desired combination while I assumed it would enable all of the three.

jpsdr

19th January 2023, 09:42

With 3, you set Bits 0 & 1 at 1.
With 6, you set Bits 1 & 2 at 1.

Zebulon84

19th January 2023, 09:43

obsolete

jpsdr

19th January 2023, 19:04

What is obsolete ???

Boulder

19th January 2023, 20:06

With 3, you set Bits 0 & 1 at 1.
With 6, you set Bits 1 & 2 at 1.

Oh, I thought it was to calculate "0+1+2+4" for the first four bits. It's been 25 years since I practiced any of this stuff :D

ShortKatz

19th January 2023, 21:07

That said, it's from 2018. I wouldn't be surprised if it had already been fixed.

Nope, the patch still does apply. So, I was wondering, if this bug is so big and there since (minimum) 2018, why did no one else notice?

Barough

20th January 2023, 11:09

x265 v3.5+87
Built on January 20, 2023, GCC 12.2.0

https://www.mediafire.com/file/iz8fqi7rqpi55sm

benwaggoner

20th January 2023, 19:12

Nope, the patch still does apply. So, I was wondering, if this bug is so big and there since (minimum) 2018, why did no one else notice?
Have been a before/after analysis to show the benefits of the fix?

jpsdr

20th January 2023, 21:52

I've made a new build of my custom version, small quick encode test seems fine.
Check my Github.

guest

21st January 2023, 05:25

I've made a new build of my custom version, small quick encode test seems fine.
Check my Github.

I have done a lot of testing, you maybe interested in the results:-

https://forum.doom9.org/showthread.php?p=1981517#post1981517
&
https://forum.doom9.org/showthread.php?p=1981553#post1981553

But only up to build 85.

LeXXuz

21st January 2023, 23:45

I've made a new build of my custom version, small quick encode test seems fine.
Check my Github.

jpsdr, I have been using v+75 with extra switches: "--aq-mode 5 --sbrc --sbrc-aq5 --sbrc-hyst" for 1080p SDR content.

Is this still valid with v+87? A lot of talk about new switch names here which confuses me a little.

jpsdr

22nd January 2023, 10:40

LeXXuz

22nd January 2023, 12:37

@LeXXuz
For the same command line effect, you have to replace --sbrc --sbrc-aq5 --sbrc-hyst with --aq-auto 10 assuming you don't want the new sbrc (wich has totaly nothing to do with the first patch) and just keep the auto AQ feature.
(And still check with the log file ;))

Thank you jpsdr. Actually I'm not sure if I want or need the new sbrc as I don't fully understand what it does and if it is beneficial for my usage. :o

So with the latest version I just add "--aq-mode 5 --aq-auto 10", is that right? :)

EDIT: Btw the clang version shows "x265 [INFO]: build info [Windows][MSVC 1928][64 bit] 10bit", is that correct :confused:

Boulder

23rd January 2023, 06:17

So with the latest version I just add "--aq-mode 5 --aq-auto 10", is that right? :)

If you want to use the automatic AQ mode with AQ5 + hysteresis enabled, just use --aq-auto 10. --aq-auto 6 for HDR (enables auto mode + hysteresis, no AQ3 or AQ5 used since it's counterproductive).

jpsdr

23rd January 2023, 18:34

EDIT: Btw the clang version shows "x265 [INFO]: build info [Windows][MSVC 1928][64 bit] 10bit", is that correct :confused:
Yes, because it's build with llvm but still with Visual Studio (so using clang compiler instead).

Boulder

30th January 2023, 05:59

@jpsdr: I've seen mentioned in some places that AQ mode 4 needs a bit lower aq-strength than other modes. Would it be possible to have a separate aq-strength parameter for it in case aq-auto is enabled?

jpsdr

30th January 2023, 21:22

@Boulder
Euh... I don't know. I'll see but absolutely not promises... Don't have time for now.

Boulder

31st January 2023, 05:57

@Boulder
Euh... I don't know. I'll see but absolutely not promises... Don't have time for now.

Thank you :)

I was thinking that it might be as simple as adding the parameter and then conditionally using it as the base strength when the mode for the frame is either X265_AQ_EDGE or X265_AQ_EDGE_BIAS.

jpsdr

31st January 2023, 18:29

@Boulder
For X265_AQ_EDGE the calcul is:

inclinedEdge = curFrame->m_lowres.edgeInclined[blockXY];
qp_adj = curFrame->m_lowres.qpCuTreeOffset[blockXY];
if(inclinedEdge && (qp_adj - avg_adj > 0))
qp_adj = ((strength + AQ_EDGE_BIAS) * (qp_adj - avg_adj));
else
qp_adj = strength * (qp_adj - avg_adj);

So, you want something that can change the value of strength when aq-auto is enabled ?
If it's that, 2 options. Somethings like strength2 = strength*coeff or strength2 = strength + bias with coeff=1 or bias=0 when aq-auto is disabled and use strength2 instead of strength in calcul.
And if i add some kind of AQ4 specific parameter, it can be applied to AQ 4 mode, with or without aq-auto enable, if default value keeps original behavior.

Boulder

31st January 2023, 19:20

What I was thinking was basically that - having AQ 4 its own strength parameter. Maybe making it equal to aq-strength in case aq-auto is not enabled so any command line would work normally.

I was wondering if it would make sense to just change the calculation of 'strength' here, I haven't had the time to get the whole idea to work because the compiled exe won't accept my new parameters for some reason. What I did was just carbon copy the existing aq-strength and aq-bias-strength parameters in various files and adjust the variable names accordingly, but probably I've missed something. In this example there's also a different value for AQ 5 bias strength.

if (param->rc.aqMode == X265_AQ_EDGE || param->rc.aqMode == X265_AQ_EDGE_BIASED)
strength = param->rc.aqStrengthEdge * avg_adj;
else
strength = param->rc.aqStrength * avg_adj;
avg_adj = avg_adj - 0.5f * (avg_adj_pow2 - modeTwoConst) / avg_adj;
if (param->rc.aqMode == X265_AQ_EDGE_BIASED)
bias_strength = param->rc.aqBiasStrengthEdge * param->rc.aqStrengthEdge;
else
bias_strength = param->rc.aqBiasStrength * param->rc.aqStrength;

jpsdr

1st February 2023, 20:35

Is it what you want ?
https://github.com/jpsdr/x265/commit/3a56c08d4e451f8837ef67e9ca6dc36cef1d1880

Boulder

2nd February 2023, 05:51

Is it what you want ?
https://github.com/jpsdr/x265/commit/3a56c08d4e451f8837ef67e9ca6dc36cef1d1880

Yes, that looks exactly what I thought of doing :) Thank you so much, I really appreciate it :thanks:

jpsdr

2nd February 2023, 14:21

I don't know when i'll be able to make a build (as it takes time), maybe this evening, maybe in few days, but you still can make your own build frommy github.

Boulder

2nd February 2023, 19:20

I don't know when i'll be able to make a build (as it takes time), maybe this evening, maybe in few days, but you still can make your own build frommy github.

No worries, I created a diff file to use for patching in Media Autobuild Suite so I got a compiled exe to test. First impression looks good, so I believe it's working. I didn't do any frame by frame compares but just watched an encoded clip to see if anything's weird.

jpsdr

5th February 2023, 11:43

If you have your own build, in that case, i'll realy take my time to make a new build... ;)

Boulder

5th February 2023, 11:55

If you have your own build, in that case, i'll realy take my time to make a new build... ;)

Yeah, no problem with that :)

If anyone wants to test the modifications, here's the build I made. 10-bit only, and optimized for Zen 3. I can also provide the patch file I used if needed.

https://drive.google.com/file/d/1WGiv0vzrpqrPI3_atczSWGuQqZJUdo3N/view?usp=share_link

vadlerg

5th February 2023, 17:30

Thank You for sharing the build.

--aq-auto 10 works. It is about 3% faster than --aq-mode 4 and may preserve a bit more detail.

When using --rd 4 --dynamic-rd 3 options I've got the warning
x265 [WARN]: Dynamic-rd disabled, requires RD <= 4, VBV and aq-mode enabled

That seems to be not rational. There must be some typo in the source, the build itself is OK.

Boulder

5th February 2023, 19:40

Thank You for sharing the build.

--aq-auto 10 works. It is about 3% faster than --aq-mode 4 and may preserve a bit more detail.

When using --rd 4 --dynamic-rd 3 options I've got the warning

That seems to be not rational. There must be some typo in the source, the build itself is OK.

The error message is a bit bad English to me. Do you have VBV enabled as well? I believe the AQ-mode checks ok if AQ auto is enabled.

if (m_param->dynamicRd && (!bIsVbv || !p->rc.aqMode || p->rdLevel > 4))
{
p->dynamicRd = 0;
x265_log(p, X265_LOG_WARNING, "Dynamic-rd disabled, requires RD <= 4, VBV and aq-mode enabled\n");
}

vadlerg

6th February 2023, 08:07

VBV is set automatically because of the parameters --crf 18 --preset slower --profile main10 --level-idc 52 .
aqMode should be 2 by default, but --aq-auto 10 sets it to AQ: mode / str(edg) / qg-size / cu-tree : auto-hyst-aq5 / 1.0(1.0) / 32 / 1
rdLevel is set to 4 and encoder still disabling dynamic-rd, even if I set --rd to 3.
In the INFO section I see
x265 [INFO]: tools: rect amp limit-modes rd=4 psy-rd=2.00 rdoq=2 psy-rdoq=1.00
x265 [INFO]: tools: rskip mode=2 rskip-edge-threshold=0.03 limit-tu=4 signhide
x265 [INFO]: tools: tmvp b-intra strong-intra-smoothing deblock sao
x265 [INFO]: tools: selective-sao

Boulder

6th February 2023, 09:06

aqMode should be 2 by default

Then it must be because of --aq-auto. I'll take a look at it when I'm back home from work and adjust that check in the code accordingly to make it work again.

LeXXuz

6th February 2023, 09:35

aqMode should be 2 by default, but --aq-auto 10 sets it to

Isn't that what it is supposed to do?
https://forum.doom9.org/showthread.php?p=1981600#post1981600

vadlerg

6th February 2023, 09:41

Of course it is what it supposed to be but this should not disable dynamicRd.

Boulder

6th February 2023, 13:18

Of course it is what it supposed to be but this should not disable dynamicRd.

Yes, it's probably just that validation which has not been updated accordingly. I don't remember I've ever seen the dynamic RD option used by anyone so it's definitely understandable.

Boulder

6th February 2023, 17:47

I found the cause of the issue, it's not auto-aq related. The validation is almost ok, it just doesn't understand that --level-idc enables VBV. If you add the VBV parameters to the command line, it will work.

EDIT: Here's a build where the validation is fixed (if VBV is not set and --level-idc is 0, it will disable dynamic-rd): https://drive.google.com/file/d/1U9cz3xsdp06ji79ZHyGA3oyuz29tYLul/view?usp=share_link

benwaggoner

7th February 2023, 03:58

Yes, it's probably just that validation which has not been updated accordingly. I don't remember I've ever seen the dynamic RD option used by anyone so it's definitely understandable.
I've used it before, with high resolution CRF with VBV encodes. It lets the encoder run faster when it can achieve CRF, and then slows down to improve quality when VBV limitations keep CRF from being hit.

It can improve throughput across many clips, but perf improvements for any given encode are unpredictable, varying with how often the content would hit the VBV.

Of course, this will increase ABR versus just using full complexity the whole time, so it's quite scenario specific. There is also the risk of the different modes causing some "stylistic" discontinuities.

vadlerg

7th February 2023, 18:36

Thank You. That's it.

jpsdr

12th February 2023, 10:10

I'm stuck with an issue i've no idea for now how to solve, and i may think that's there is just no solution at all (unless a loooooot of rework of the code).
After adding just the command line asked by Boulder, i've hit the C1061 error issue with MS compiler... :(
Why have they done this...????
As it's specific to MS compiler, i still can make the clang version, but for now, it's the only build i can do.
To do the others builds, i have to see how i can build x265 with gcc, maybe i'll do that, but not the time for now...

jpsdr

12th February 2023, 12:06

Finaly i made some tests for gcc build, but when things want to get in your way, they are very efficients... :(

I've tested msys script build, but what i'm looking for is where i can find the parameter list you see in command like this:cmake -G "MSYS Makefiles" ../../../source -DHIGH_BIT_DEPTH=ON -DEXPORT_C_API=OFF -DENABLE_SHARED=OFF -DENABLE_CLI=OFF -DMAIN12=ON
I want to make a static standalone .exe, not a one when you start it asked a lot of gcc dll...!
This is possible with cmake-gui (i have trouble to make it start also), but cmake-gui doesn't give all the options (you don't have 12bits for exemple).
Also, i would like to add "-mavx2" in the gcc command, no idea how to do that's kind of stuff (this is why i choose the Visual Studio solution, where i have access to all the compiler options in the ide).

Edit
Even with "static link CRT" checked in cmake-gui, the .exe from gcc build is still asking for dll... Not the same than my first attempt without, but still...

So, gcc builders, can you provide my your whole complete build script ?
As i said, when things want to get in your way, they are very efficients !!

Boulder

12th February 2023, 17:04

For GCC builds, I use Media Autobuild Suite to take care of the cumbersome scripting. You can use custom patches, for example with x265, create a file named x265_git_extra.sh and add the patch in the main 'build' folder and it will get applied during the compile operation. This is what I have in the file.

#!/bin/bash

_pre_ninja(){

patch -p 1 -i "$LOCALBUILDDIR/x265-jpsdr-aqauto-v5.patch"

}

qyot27

12th February 2023, 18:35

I've tested msys script build, but what i'm looking for is where i can find the parameter list you see in command like this:cmake -G "MSYS Makefiles" ../../../source -DHIGH_BIT_DEPTH=ON -DEXPORT_C_API=OFF -DENABLE_SHARED=OFF -DENABLE_CLI=OFF -DMAIN12=ON
cmake . -LAH will emit the full list of variables that get set with their values, although it isn't all that nicely formatted. In many cases it's just easier to open CMakeLists.txt and see what things are either listed as option() or being toggled as an environment variable with set() or some such.

I want to make a static standalone .exe, not a one when you start it asked a lot of gcc dll...!
This is possible with cmake-gui (i have trouble to make it start also), but cmake-gui doesn't give all the options (you don't have 12bits for exemple).
Also, i would like to add "-mavx2" in the gcc command, no idea how to do that's kind of stuff (this is why i choose the Visual Studio solution, where i have access to all the compiler options in the ide).

Edit
Even with "static link CRT" checked in cmake-gui, the .exe from gcc build is still asking for dll... Not the same than my first attempt without, but still...

So, gcc builders, can you provide my your whole complete build script ?
As i said, when things want to get in your way, they are very efficients !!
See example here (yes, x265 is in the guide, but for clarity's sake use a smaller example):
https://github.com/qyot27/mpv/blob/18aa1fd40f234931d13d3512dfd835275add7dd2/DOCS/crosscompile-mingw-tedious.txt#L475

Most GCC toolchains that are widely distributed are built assuming shared runtime by default, you have to either pass something like -static-libgcc -static-libstdc++ to LDFLAGS (in CMake, -DCMAKE_SHARED_LINKER_FLAGS for libraries and -DCMAKE_EXE_LINKER_FLAGS for executables) or you can just turn this off entirely by building GCC with static runtime only (which is what I do when building the MinGW/GCC toolchain, hence why the guide doesn't use those flags).

I need to do a good refresh on that guide because while I typically do a once-through every six months after moving to the current release of Ubuntu, I was so stressed out last year and now spend >95% of my time in Linux that I still haven't done so since last May or something like that.

jpsdr

12th February 2023, 20:26

Ok, translation issue from my part, my question is not properly made.

I was asking, from people using gcc, their whole complete build script for building static x265 (not building gcc...).

LigH

13th February 2023, 19:21

Mine (https://www.ligh.de/software/MABS/makehdr10_w64-git.sh) (depends on the media-autobuild suite directory tree and its MSYS2/MinGW64 shell).

jpsdr

14th February 2023, 19:43

@LigH
Is in your script the following line:export CXXFLAGS= allow to add compiler options (like -mavx2) that will be used afterward ?

LigH

14th February 2023, 23:10

No, just basic adaptions of the Multicoreware templates.

jpsdr

14th February 2023, 23:27

LeXXuz

15th February 2023, 10:21

@LeXXuz
For the same command line effect, you have to replace --sbrc --sbrc-aq5 --sbrc-hyst with --aq-auto 10 assuming you don't want the new sbrc (wich has totaly nothing to do with the first patch) and just keep the auto AQ feature.
(And still check with the log file ;))

jpsdr, is it normal that Mediainfo shows all kinds of aq-mode when encoding with --aq-mode 5? Some files show aq-mode=2, others 3, 4 or 5. All have been encoded with --aq-mode 5 and --aq-auto 10.

filler56789

15th February 2023, 13:04

No luck, i've tried several options, but i always have an error message telling me that libstdc++6.dll is missing when i try to run the x265.exe builded... :(
Don't know what option i have to use to make a standalone version.

As qyot27 already said, your "best" (ooops :o) option is to use a "static" build of GCC for Windows...
so you won't have to mess around with the CMake flags;
(just as an example...) ↓↓↓
-DCMAKE_EXE_LINKER_FLAGS="-static-libstdc++ -static -pthread" when configuring resolves it with vanilla MSys2 with default repo packages.

When I still had the "energy" to make builds of x265.exe, and when I did not use the GCC binaries compiled by myself, I used the GCC packages built by nevcairiel or by www.msystem.waw.pl/x265.

H.T.H. :-|

jpsdr

15th February 2023, 14:34

@LeXXuz
I don't know how Mediainfo get his information, neither how x265 stores them. As in auto-aq the aq-mode changes on the fly, according how/when x265 stores the informations retrievied by Mediainfo, it would not be surprising to see differents values. But... for the same file with the exact same parameters, you should always have the same result.
@filler56789
Ok, thanks, i'll try with the gcc versions on www.msystem.waw.pl/x265, i allready use this for one of my x264 build.
So it means that the gcc version provided by default with msys2 isn't working with the standard scripts provided in the build/msys directory of x265 git... (unless tweak them).

Edit:
According CMakeList.txt, CMAKE_CXX_FLAGS is not used unless you set CMAKE_BUILD_TYPE to None.
Using CMAKE_CXX_FLAGS seems to me the only way to add tune build options (like -mavx2).
But, i've not been able to figure out what are the CXX_FLAGS options in Release build.

FranceBB

15th February 2023, 17:11

I used the GCC packages built by nevcairiel or by www.msystem.waw.pl/x265.

I used them too, but the readme should really be updated as they're no longer Windows XP compatible and have not been for a while now (I think pre-pandemic).

@jpsdr... I'll bother you soon-ish via email, as soon as I have a bit of spare time at work (which is unlikely given the stuff I have to do right now). Anyway, to summarize, although I was totally fine with your x264 mod for x264, the fact that the company I work for is part of the partner program with Multicoreware makes me a bit "sad" to have to use modded version of x265 as well, so I'm determined to gather a list of the changes and come up with a big proposal to MCW to eventually merge them back into x265 and we can do this together given that we already had a chat with them about that other thing you asked last time. TL;DR given that we actually have a say on this, I'd like to avoid having an x265 mod that ends up becoming yet another x264 mod-like kind of situation where changes/fixes/improvements are never merged back and no one cares.

StvG

15th February 2023, 17:39

@filler56789
Ok, thanks, i'll try with the gcc versions on www.msystem.waw.pl/x265, i allready use this for one of my x264 build.
So it means that the gcc version provided by default with msys2 isn't working with the standard scripts provided in the build/msys directory of x265 git... (unless tweak them).

Edit:
According CMakeList.txt, CMAKE_CXX_FLAGS is not used unless you set CMAKE_BUILD_TYPE to None.
Using CMAKE_CXX_FLAGS seems to me the only way to add tune build options (like -mavx2).
But, i've not been able to figure out what are the CXX_FLAGS options in Release build.

For CMAKE_BUILD_TYPE=Release use CMAKE_CXX_FLAGS_RELEASE / CMAKE_C_FLAGS_RELEASE

In order to use provided msys2 gcc and have statically linked internal libs use -DCMAKE_CXX_FLAGS_RELEASE="-static-libgcc -static-libstdc++ -static".

jpsdr

15th February 2023, 19:41

@StvG
Yes !! Finaly, it's working with the msys2 gcc. Thanks, after several people tall me the same thing, it finaly enters in my head... :D

Now... The last step.

I have several mingw64 directories in my msys directory for switching gcc versions. By default, my mingw64 directory is the one provided by gcc. I have two others: mingw64_win32 (the www.msystem.waw.pl/x265 version) and another mingw64_mcf, le LH Mouse version. To switch, i just rename, for exemple: "mingw64 -> mingw64_posix" and "mingw64_win32 -> mingw64", and then, i start the "MSYS2 MinGW 64-bit" icon of MSYS2 and check with a "gcc -v". This works perfectly fine for doing my several x264 builds.
And when there is a new release of the Win32/MCF version, i delete the mingw64_xx directory and extract/replace by the new version.
So, i just tried to do the same thing for x265, and... It didn't work, the script told me "cmake: command unknow"... :(
It seems that the mcf, and "worse", the one provided by www.msystem.waw.pl/x265 don't have cmake.
I didn't try yet to "re-install" cmake on the renamed directory, is it the only way ?

Edit:
As i was afraid, as i've already installed cmake, even if i put a mingw64 directory without it, msys said that's it's already installed and has nothing to do.
I can't switch to another version... :(
Edit2:
I was using pacman -S --needed, i changed to pacman -S, it said it's allready installed but re-install anyway, but... only the package, not the necessary dependancies, msys thinks they are all installed.
cmake installed but still not working with alternative gcc version...

@FranceBB
99% of the patches are not mine. They came either from the Patman version or others, i just gathered them.

filler56789

15th February 2023, 20:53

@jpsdr && to anyone interested as well...

«User Guide: How to setup an MSYS2 building environment»

https://forum.videohelp.com/threads/394148-How-to-setup-an-MSYS2-building-environment

12) Time to edit the file home\YourUsername\.bash_profile.
Add the lines below:

export PS1='[\w]\n<$MSYSTEM> '

PATH=/commons:/CMAKE/bin:/GIT/bin:/HG:$PATH
export PATH

jpsdr

15th February 2023, 21:25

For now, i've been able in my mcf version, to "re-install" cmake, after compare the dependencies with the list of installed packages in mcf version, and after a dozen of missing dll and a dozen of more packages, to have the whole list of things to install... And build success with mcf.

StvG

15th February 2023, 22:19

I have several mingw64 directories in my msys directory for switching gcc versions. By default, my mingw64 directory is the one provided by gcc. I have two others: mingw64_win32 (the www.msystem.waw.pl/x265 version) and another mingw64_mcf, le LH Mouse version. To switch, i just rename, for exemple: "mingw64 -> mingw64_posix" and "mingw64_win32 -> mingw64", and then, i start the "MSYS2 MinGW 64-bit" icon of MSYS2 and check with a "gcc -v". This works perfectly fine for doing my several x264 builds.
And when there is a new release of the Win32/MCF version, i delete the mingw64_xx directory and extract/replace by the new version.
So, i just tried to do the same thing for x265, and... It didn't work, the script told me "cmake: command unknow"... :(
It seems that the mcf, and "worse", the one provided by www.msystem.waw.pl/x265 don't have cmake.
I didn't try yet to "re-install" cmake on the renamed directory, is it the only way ?

You can download cmake (https://cmake.org/download/). Then you can use the absolute path. For example /d/cmake/bin/cmake.exe -G Ninja ... Or you can add it PATH.

jpsdr

16th February 2023, 01:26

Thanks.

Any idea why this works:
mkdir -p 10bit

cd 10bit
cmake -G "MSYS Makefiles" ../../../source -DENABLE_SHARED=OFF -DHIGH_BIT_DEPTH=ON -DENABLE_CLI=ON -DSTATIC_LINK_CRT=ON -DCMAKE_CXX_FLAGS_RELEASE="-static-libgcc -static-libstdc++ -static"
make ${MAKEFLAGS}

and this doesn't:mkdir -p 8bit_x64 10bit_x64 12bit_x64

cd 12bit_x64
cmake -G "MSYS Makefiles" ../../../source -DHIGH_BIT_DEPTH=ON -DEXPORT_C_API=OFF -DENABLE_SHARED=OFF -DENABLE_CLI=OFF -DMAIN12=ON -DCMAKE_CXX_FLAGS_RELEASE="-static-libgcc -static-libstdc++ -static"
make ${MAKEFLAGS}
cp libx265.a ../8bit_x64/libx265_main12.a

cd ../10bit_x64
cmake -G "MSYS Makefiles" ../../../source -DHIGH_BIT_DEPTH=ON -DEXPORT_C_API=OFF -DENABLE_SHARED=OFF -DENABLE_CLI=OFF -DCMAKE_CXX_FLAGS_RELEASE="-static-libgcc -static-libstdc++ -static"
make ${MAKEFLAGS}
cp libx265.a ../8bit_x64/libx265_main10.a

cd ../8bit_x64
cmake -G "MSYS Makefiles" ../../../source -DEXTRA_LIB="x265_main10.a;x265_main12.a" -DEXTRA_LINK_FLAGS=-L. -DENABLE_CLI=ON -DENABLE_SHARED=OFF -DLINKED_10BIT=ON -DLINKED_12BIT=ON -DSTATIC_LINK_CRT=ON -DCMAKE_CXX_FLAGS_RELEASE="-static-libgcc -static-libstdc++ -static"
make ${MAKEFLAGS}

# rename the 8bit library, then combine all three into libx265.a using GNU ar
mv libx265.a libx265_main.a

ar -M <<EOF
CREATE libx265.a
ADDLIB libx265_main.a
ADDLIB libx265_main10.a
ADDLIB libx265_main12.a
SAVE
END
EOF

With LH Mouse mcf gcc version, the multilib version is asking for libmcfgthread-1.dll but the just 10bit is not...

Edit:
Didn't check yet (lack of time, ended things at 1am) with msys2 gcc version instead of mcf.

LeXXuz

16th February 2023, 12:37

@LeXXuz
I don't know how Mediainfo get his information, neither how x265 stores them. As in auto-aq the aq-mode changes on the fly, according how/when x265 stores the informations retrievied by Mediainfo, it would not be surprising to see differents values. But... for the same file with the exact same parameters, you should always have the same result.

Thanks for that explanation. :) I was just wondering because sometimes it shows aq-mode=5 which would be correct and sometimes all the others.

Guess I had the wrong understanding of mode 5. I thought it is a mode of its own which behaves like one of the other modes depending on scenery, therefore constantly flagged as mode 5 throughout the entire stream.

So it is more like a 'fictitious' mode which switches through the other modes depending on scenery and changing the mode flags accordingly? That still confuses me a little. Because in that case Mediainfo should never display aq-mode=5 for some files, no?