Log in

View Full Version : x265 HEVC Encoder


Pages : 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 [193] 194 195 196 197

benwaggoner
7th November 2024, 20:57
But at what cost ($$$:sly:)
US$1599 for the 14 core M4 Pro with 24 GB RAM (should be enough for 4K; the M-series is really memory efficient).

For a cluster, one could upgrade to 10 Gbps ethernet for another $100. Extra RAM and internal storage are pricey, but likely not needed for many applications.

FranceBB
7th November 2024, 21:25
At this point, I wouldn't be surprised if Apple's new M4 Max turns out to be the fastest x265 encoding platform on the planet.

If that ever happens, I'm gonna go in the server room and yell to each one of my 56c/112th AVX512 x86_64 Intel Xeon "you're pathetic!".

https://y.yarn.co/e058b4b3-41d8-4850-8e34-89f1b791526a_text.gif

benwaggoner
7th November 2024, 23:22
If that ever happens, I'm gonna go in the server room and yell to each one of my 56c/112th AVX512 x86_64 Intel Xeon "you're pathetic!".

https://y.yarn.co/e058b4b3-41d8-4850-8e34-89f1b791526a_text.gif
Yeah, past a certain resolution and frame threads, more cores will start winning over better cores. M4 Max likely wins for 1080p or lower, and possibly 4K if using only 1 frame thread.

Unfortunately the M4 Max is only in MacBook Pro, which is a whole lot more expensive and bigger for headless work. Mac Mini tops out with the M4 Pro currently.

LigH
7th November 2024, 23:43
New upload: x265 4.0+42-5203eb5 (https://www.mediafire.com/file/yoww4r7t1qovqet/x265_4.0+42-5203eb5.7z/file)

[Windows][GCC 14.2.0][32b/32b~XP/64b/64b+MV] 8bit+10bit+12bit

Z2697
8th November 2024, 14:44
They probably opened release 4.1 branch too early and now they have to push to two branches simultaneously

LigH
12th November 2024, 18:45
New upload: x265 4.0+60-e012793 (https://www.mediafire.com/file/xors33eth0xriax/x265_4.0+60-e012793.7z/file)

[Windows][GCC 14.2.0][32b/32b~XP/64b/64b+MV] 8bit+10bit+12bit

contains a lot of important fixes (e.g. of memory leaks) and improvements

Barough
13th November 2024, 22:10
x265 v4.0+61-7cc403076
Built on November 13 2024, GCC 14.2.0
Win32/64 / 8bit+10bit+12bit

https://www.mediafire.com/file/i1iuybxy5w3w6g6

asarian
13th November 2024, 23:15
x265 v4.0+61-7cc403076
Built on November 13 2024, GCC 14.2.0
Win32/64 / 8bit+10bit+12bit

https://www.mediafire.com/file/i1iuybxy5w3w6g6


A bit confusing. Is your x265 version in competition with that of LigH? And yours is .61 I see.

LigH
14th November 2024, 00:06
The difference between v4.0+60 and v4.0+61 is probably:

A few tabs converted to spaces to equalize indentation.

Nothing that would affect the generated code.

asarian
14th November 2024, 09:36
The difference between v4.0+60 and v4.0+61 is probably:

A few tabs converted to spaces to equalize indentation.

Nothing that would affect the generated code.


Thanks for the clarification.

LigH
21st November 2024, 11:20
More quality related patches (addressing e.g. memory leaks / double frees, crashes) are being submitted. Also pmode and pme will deprecate. Expect commits soonish...

LigH
22nd November 2024, 16:39
Version 4.1 has been released.

The v4.1 release of x265 is out now. Below is the list of new features, optimization, and bug fixes in this version.

Version 4.1
===========

Release date - 22nd November, 2024.

New feature
-----------
1. Aom Film-Grain characteristics as a SEI message to support Film Grain Synthesis.

API changes
-----------
1. API Support to enable VBV end feature.
2. Reverted the 4.0 api changes.
3. Added command line parameters for Aom film grain characteristics as a SEI message to the bitstream (--aom-film-grain).
4. Added support to configure Bitrate, CRF, and QP at frame level, along with CLI support for frame-level RC configuration (--frame-rc)
5. Declared few params(numapools, csvfn, scalingLists etc) with fixed size to fix memory leaks

Optimizations
-------------
1. Eliminated redundant pointer copies in lowresMC and lowresQPelCost, improving encoding performance by up to 0.9% on modern CPUs.
2. Mcstf optimization - Improved mcstf performance by moving mcstf computation to lookahead and search range optimization.

Bug fixes
---------
1. mcstf crash when using multiple frame threads.
2. CLI parsing for MV-HEVC Encode.
3. segfault and decoder crash in scc.
4. compilation issue in clang.
5. potential stack buffer overflow issues.
6. documentation for b-intra, max-merge, limit-refs and qg-size
7. memory leak fixes for sei, zones, sao, hme, vbv, sbrc, mcstf, alpha, multiview, scc, two pass, rskip, film grain, aq-mode, scaling_list, fades, analysis save and load etc

Known issues
------------
1. Inconsistent output with recon option
2. Output mismatch between analysis save & load with cutree for reuse-levels < 10

Thanks,
Mahesh

jpsdr
22nd November 2024, 19:41
A lot of commits, but the version is 4.0.0.077, so still a 4.0.0, not 4.1.

LigH
22nd November 2024, 19:45
You say? ... My MinGW+GCC workflow calculates version 4.1+54-fa27709.

jpsdr
22nd November 2024, 20:09
Ah... As i have a custom version, maybe something went wrong, even with pushing the tags.
The fact is the exe i built is tagged 4.0.0.77.
Well... i don't realy mind, the more important is to have the commits.

Z2697
22nd November 2024, 20:52
You say? ... My MinGW+GCC workflow calculates version 4.1+54-fa27709.

4.0+77 and 4.1+54 are both "wrong" (but technically correct ;))
4.0+77 is calculated before the merge of stable to the master (https://bitbucket.org/multicoreware/x265_git/commits/fa2770934b8f3d88aa866c77f27cb63f69a9ed39), tag is not 4.1 because the tagged commit hasn't been merged to "current branch" where he builds the binary (which probably is master).
4.1+54 is calculated after the merge, tag is correct but the distance is greater than is should be, because release branch and master branch are diverged due to the previously mentioned mistake.
By the way, it seems they are committing new changes only to a "Release_4.1" branch, and forgetting about the master branch.
They probably opened release 4.1 branch too early and now they have to push to two branches simultaneously

LigH
22nd November 2024, 23:09
Their recent code development is remarkable; but their repo mess seriously flaws it.

MaheshPittala
25th November 2024, 11:49
The x265 release 4.1 branch was started slightly earlier, requiring patches to be pushed to both the master and release_4.1 branches. As a result, the commit IDs differed between the two branches. When the release/stable branch was merged into master, the commits were treated as new. Using cherry-pick would have been the better solution.

Z2697
25th November 2024, 13:44
If you cherry-pick the tagged commit, the new commit hash will also be different, the versioniong cmake code (using git describe) probably won't pick the 4.1 tag.
I think that's just how it goes, no better way to do it.
Just... don't create release branch too early next time, do it right before the real release.

Joey BoomBats
2nd December 2024, 11:07
Aom Film-Grain characteristics as a SEI message to support Film Grain Synthesis.

This looks like a very useful feature for bit hungry grainy videos.


Has anyone already tested the efficiency improvements of this feature and the visual implications?


Or does anyone know what the command line parameters are?


I see that Handbrake has now included x265 version 4.1 but unfortunately no specific changes have been made to the UI (yet).

excellentswordfight
3rd December 2024, 11:10
This looks like a very useful feature for bit hungry grainy videos.


Has anyone already tested the efficiency improvements of this feature and the visual implications?


Or does anyone know what the command line parameters are?


I see that Handbrake has now included x265 version 4.1 but unfortunately no specific changes have been made to the UI (yet).
As far as I can see, this is not a complete built-in feature. The parameter added is "--aom-film-grain", so you need to denoise and create a grain parameter file externally, you then point that parameter to that file so it can be injected in to the SEI.

tormento
5th December 2024, 16:22
Has anyone tried --frame-dup with Patman (https://github.com/Patman86/x265-Mod-by-Patman/releases) or jpsdr (https://github.com/jpsdr/x265/releases) builds?

It doesn't work on my AVX only CPU.

I get:

Video encoding returned exit code: -1073741819 (0xC0000005)

It's unclear what this exit code means, in case it's a Windows system error then it possibly means:

# for decimal -1073741819 / hex 0xc0000005
STATUS_ACCESS_VIOLATION ntstatus.h
# The instruction at 0x%p referenced memory at 0x%p. The
# memory could not be %s.
# as an HRESULT: Severity: FAILURE (1), FACILITY_NULL (0x0), Code 0x5
# for decimal 5 / hex 0x5
ERROR_ACCESS_DENIED winerror.h
# Access is denied.

jpsdr
5th December 2024, 18:54
Does the issue also happen with a not modified build ?

Z2697
5th December 2024, 20:02
Who uses frame-dup anyways?

It's better to give more details such as the whole command line and the "variant" of the build used to produce that error, and the input spec, etc.

rwill
5th December 2024, 20:20
The interesting thing is that if x265 can frame dupe, the way it is implemented anyway, one could also encode the picture at 1 bit or below per CU and not make the stream variable framerate, cause problems with certain muxers and rely on a correct decoder implementation of picture timing SEIs.

So a very useful and bit saving feature they have there, if only it worked.

Z2697
5th December 2024, 20:40
The interesting thing is that if x265 can frame dupe, the way it is implemented anyway, one could also encode the picture at 1 bit or below per CU and not make the stream variable framerate, cause problems with certain muxers and rely on a correct decoder implementation of picture timing SEIs.

So a very useful and bit saving feature they have there, if only it worked.

Using container output (ffmpeg libx265, or supported in some mods) can have "slightly more usable" result, but yeah, currently the feature is working, I guess, but no software can recognize it. Even the "slightly more usable" container output is not utilizing timing SEIs.
The container output is using timestamps support of the container (VFR), while the picture timing SEIs don't contain actual timestamp, but only a flag indicates the corresponding frame should be doubled or tripled, if the decoder is able to support it the decoded video should still be CFR.

Even if it's working properly, both encode and decode side, the use case is still extremely limited.

tormento
5th December 2024, 20:53
Who uses frame-dup anyways?
Anime. Mostly old ones.
Does the issue also happen with a not modified build ?
Tried this (http://msystem.waw.pl/x265/x265-4.1+54-fa27709_vs2022.7z) and it works but I don't know if it honors --frame-dup flag.

Z2697
5th December 2024, 21:07
Anime. Mostly old ones.

You shouldn't.

To have a better understanding of what this feature means: this feature removes frames based on PSNR thresholding, and signal picture timing SEIs to keep the correct... picture timing... yeah... which no commonly available decoder can recognize.

It also requires VBV and HRD. Since no everyday decoder can recognize the timing SEIs the picture timing will be incorrect if any frame(s) are removed.
If you don't see any incorrect timing,
1) You didn't notice it.
2) You didn't enable the required VBV and HRD.
3) Your video didn't have any frames above the threshold.

Even if it works as it should, the PSNR thresholding is not ideal to begin with, and the bits saved with removing near identical frames are, well, did you know picture timing SEIs cost bits?

jpsdr
6th December 2024, 19:30
Does it work on CPU with more than AVX only ?

Patman
7th December 2024, 14:25
The reason for the error is the direct support of AVS and VPY scripts (MOD). However, if you use the pipe methods, the encode works. It seems that x265 requires the Y4M or YUV formatfor this case.

jpsdr
7th December 2024, 23:38
That's odd, i'm almost sure i used frame-dup on avs scripts with a moded version, during my first attempts of trying to do x265 encodes, before giving up using it because of all the issues i encountered, reasons Z2697 described.
My guess was it could be because of the multiview add, but it was wrong it seems.

Leo 69
9th December 2024, 18:21
Does anyone know how to use the new --aom-film-grain feature in the latest x265 encoders (I'm using 4.1+54)? I tried it and it fails each time with either .tbl or even .txt for film grain files. Here's the command:

--crf 18.5 --preset veryslow --output-depth 10 --aom-film-grain grain.tbl

And I always get this error:

x265 [error]: Failed to open Aom film grain characteristics binary file grain.tbl


For grain.txt file the error is the same. Does anyone know how to fix this? Or rather, did anyone manage to make this new feature work at all?

Z2697
9th December 2024, 20:57
Does anyone know how to use the new --aom-film-grain feature in the latest x265 encoders (I'm using 4.1+54)? I tried it and it fails each time with either .tbl or even .txt for film grain files. Here's the command:

--crf 18.5 --preset veryslow --output-depth 10 --aom-film-grain grain.tbl

And I always get this error:

x265 [error]: Failed to open Aom film grain characteristics binary file grain.tbl


For grain.txt file the error is the same. Does anyone know how to fix this? Or rather, did anyone manage to make this new feature work at all?

I think it expects "raw" SEI payload, just guessing, haven't tried to understand it, and probably never will because I don't think it's (I'm referring to all the current implementations of FGS features) a useful feature, until it's much more improved.

benwaggoner
11th December 2024, 18:32
The interesting thing is that if x265 can frame dupe, the way it is implemented anyway, one could also encode the picture at 1 bit or below per CU and not make the stream variable framerate, cause problems with certain muxers and rely on a correct decoder implementation of picture timing SEIs.

So a very useful and bit saving feature they have there, if only it worked.
Yeah, if only it worked! I've never been able to get a single frame to duplicate as documented, even with some pretty extreme settings. I've not tried in a 4.x build, but don't recall any patches for that part.

Maybe it only works in 2-pass or something?

benwaggoner
11th December 2024, 18:35
I think it expects "raw" SEI payload, just guessing, haven't tried to understand it, and probably never will because I don't think it's (I'm referring to all the current implementations of FGS features) a useful feature, until it's much more improved.
The tool is reasonably useful with the proper tools; there just aren't any good open source ones currently. The hardest part is the degrain-and-parameterize before encoding.

The rendering has some challenges with repeated patterns, due to the "drunken walk" nature of selecting a random 32x32 block out of a 64x64 block: pixels near the middle are much more heavily sampled than those near the edge. I so wish computer science degrees required some basic statistics! I work with a lot of engineers who are whizzes at linear algebra, but go blank when I ask "is that statistically significant?"

Z2697
12th December 2024, 06:17
Yeah, if only it worked! I've never been able to get a single frame to duplicate as documented, even with some pretty extreme settings. I've not tried in a 4.x build, but don't recall any patches for that part.

Maybe it only works in 2-pass or something?

I'm not sure "which" duplication you are referring to, the first case is the encoding side:

What it does is actually removing frames and using SEI to tell decoder to duplicate existing frames, which actually should be called de-duplication I think.

Maybe you are not looking at the right place (i.e. you think it's duplicating frames while it actually deletes frames), but assuming you are not able to get frames to trigger the de-duplication during encoding:

There's PSNR thresholding, can be configured via --dup-threshold parameter.
If you use low enough threshold, eventually some frames will be de-duped, but of course this should only be used in experiments, low threshold will just destroy the video.

The second case: the decoding side...

You need a decoder that's able to recognize and utilize the picture timing SEI (which only tells decoder to double or triple the frame, no actual timestamp is stored) and I guess things will... "just work"... yeah, who knows, the most common decoder (avcodec) doesn't support it so I can't test.


The tool is reasonably useful with the proper tools; there just aren't any good open source ones currently. The hardest part is the degrain-and-parameterize before encoding.

The rendering has some challenges with repeated patterns, due to the "drunken walk" nature of selecting a random 32x32 block out of a 64x64 block: pixels near the middle are much more heavily sampled than those near the edge. I so wish computer science degrees required some basic statistics! I work with a lot of engineers who are whizzes at linear algebra, but go blank when I ask "is that statistically significant?"

Maybe I just hate noises in general :o
But I agree, it (FGS) has potential... just it still has a long way to go.

LunaRabbit
15th December 2024, 19:43
Hello Doom9! It's been nearly 20 years since I last posted here although I've been lurking on and off all these years. I recently got back into this old hobby and have greatly appreciated the information ITT. Since I'd been away since the early x264 days and it took several weeks to get up to speed with x265. If it wasn't for the information ITT I would have spent many more weeks aimlessly testing stuff I'm sure. I have a bunch of questions. I hope you don't mind if I ask them. :)

1) --frame-rc / https://bitbucket.org/multicoreware/x265_git/commits/66352d83ae10330f9528b08238396de62dc618c1

Can someone please explain how to work with this recent commit? Can I just define this stuff in my qpfile? If so, what syntax should I use? There is no documentation about this that I can find and it isn't really explained on the git repo from what I've seen. I've been slowly trying to work my way through ratecontrol.cpp to understand it but I haven't had a chance to throw random stuff in a qpfile and check yet (I will soon when I'm back at home). Has anyone used this feature yet?

I am very interested in this since I already manually create a qpfile anyway to set I-frames at particular frames for various reasons (seeking speed and ensuring frames with overlays get an I-frame when needed). I'm already creating qpfiles by hand for each source I work with. So the ability to manually set CRF/QP/Bitrate would be a much welcomed addition. Up until now I've been having to manually make zones and multiply the bitrate up/down which is less than ideal.

2) Misc. cutree stuff and mods

I did a lot of testing with cutree on/off last year and visually I frankly couldn't see much difference but turning off cutree would sometimes greatly increase bitrates for little gain. The extra bits didn't seem worth what I was getting. I currently "reign in" cutree by tweaking --qcomp typically at something like 0.7 or 0.8 at most. Boulder posted a diff file ITT several pages ago and the repo linked in this post (https://forum.doom9.org/showpost.php?p=2005461&postcount=2) is supposed to allow tweaking cutree directly instead of in the round about way with qcomp. But I've also heard from many people over the last few years that I shouldn't use cutree at all because it's a

Bad implementation of mbtree from x264

I do not believe this because in my experience cutree hasn't degraded visual quality at the CRFs I typically use (15-18 or so). I suspect this is one of those hold over old wives tales from the early days of mbtree in x264 when it wasn't fleshed out yet. I'll be the first to admit I don't fully understand how cutree works in x265. I've been reading the source code in an attempt to better understand it. I've also spent many hours reading various posts here. But there really isn't a lot of good technical discussion about it that I can find. Perhaps it's happening on the mailing list somewhere that I haven't seen yet?

I would greatly appreciate hearing your opinions on cutree and if using the above modifications (along with the hours of testing involved) would be worth my time. I've been building x265 from source for the last year or so. But I only use a handful of modifications and mainly for the AQ modes that aren't in mainline. Which brings me to my next and last question for now;

3) Auto-aq

Earlier this year I built a modded version of x265 from I believe version 3.6 or thereabouts. It included the auto-aq mode which I quickly found to be a huge benefit for the sources I work with. In the version I had it was invoked as so;

--aq-auto 10

Today I got a chance to catch up on things and noticed x265 had developed a lot since I was away. I also quickly found out that this mode no longer worked as it did just a few months ago. Through trail and error and some lurking on the git where it's hosted I found out I can now invoke it like this

--auto-aq

But I didn't see any discussion for why it changed and if you can still supply it with an option (e.g. 6 vs. 10 for HDR or SDR content). Does it automatically determine what kind of source you're working with now? Please advise. I spent a few hours this morning trying to find the exact commit where it was changed and why along with trying to grok posts from here. But there were over 20 pages of new discussion ITT and my searches of the forum's database proved futile.

I do have more questions about how x265 works and some of the settings I've been using for the last 2 years or so. I've gotten really good results at acceptable bitrates so I'm happy with them. But improvements are always welcomed of course. My workstation is getting a bit long in the tooth these days and I'm due for an upgrade. So I'm not able to do as much testing as I'd like. My usual config is already lucky to obtain 1fps. Which means a typical project for me takes several days and sometimes weeks to finish. Although the whole manually creating qpfiles and the type of editing/filtering I do takes up far more time. As I'm usually stepping through things frame-by-frame and splicing together sometimes 4 different sources to restore the content I work with.

This is getting verbose so I'll hold off on asking more questions even though I'm full of them. It's good to be back. I'm happy to see this community is still very active. I prefer using forums to these modern locked down social media networks or God forbid Discord.

Also, thanks again to the kind admin that allowed me to post this today instead of a week from now.

Z2697
15th December 2024, 21:10
Some (paranoid) people have been disabling mbtree since x264, no wonder they'll disable cutree as well.
The real problem is not even whether cutree is a bad mbtree mimic or not, they don't even use mbtree to begin with.

LunaRabbit
15th December 2024, 21:36
Some (paranoid) people have been disabling mbtree since x264, no wonder they'll disable cutree as well.
The real problem is not even whether cutree is a bad mbtree mimic or not, they don't even use mbtree to begin with.

Yeah I understand that part of it. In their defense early mbtree wasn't that great from what I remember. But I'm sure it improved some after I stopped paying attention and got busy with life.

My limited understanding of cutree is that it isn't the same as mbtree at all. From what I understand cutree is more about looking ahead and attempting to either insert more I-frames or shift data in a GOP towards I-frames. With the hope that the following P/B frames will benefit and thus be smaller. I might be wrong but I've always understood it as being more like lookahead than what mbtree was. But I don't pretend to understand the inner workings of mbtree either.

It's just frustrating that the only discussion I see about cutree is either "turn it off it'll destroy quality" or "leave it on but I don't really know what it's doing. It just saves bits". I want to have a better understanding of how it works.

I do know for my uses/config it does save bitrate without affecting quality much that I can see. I suppose if I pixel peep I could spot very mild drop in quality here and there. But over the course of an entire episode or movie it seems to save a lot of bitrate and produce something of equal quality if nothing else is changed.

But perhaps that has more to do with the fact that I used closed GOPs and usually VBV. I do understand GOPs in x265 fine. I think I understand VBV as well and I don't think it really hurts quality that much the way I use it. I use it to impose an upper limit on the buffer so my encodes play nicer with some set-top devices and over the network. As I want to stream my stuff over my LAN and sometimes over WAN when I'm away from home. I was going to ask about VBV another time. But I only bring it up because maybe that's why I'm not seeing any improvement with cutree off.

The bitrate savings for me with cutree are very good. In a clip of say 100 frames with cutree on and closed_GOP set to 24 frames I'll produce a video stream that's about 1.5-2.0MB. The --no-cutree version would be 3.5-4MB with no gain in quality that I can see. Although I do "reel it in" with qcomp. But perhaps that's placebo. I'm mainly doing that in an attempt to prevent much variation between keyframes and non-keyframes.

Some guidance concerning cutree, how it relates to qcomp and if these modifications to tweak cu-tree are worth exploring would be very helpful and appreciated. Since I was surprised to find so little discussion about it outside of; "turn it off" or "if you're leaving it on make sure to raise qcomp a bit".

I feel like there are a lot of things like this in x265 that is just advice being repeated for the last 10+ years from people assuming it works just like early x264. Some time way back when everyone just decided to do things a certain way and never bothered to do tests as the underlying code and the codec itself changed.

I do know that people claim cutree can hurt backgrounds and scenes with a lot of movement. But in my experience it doesn't. But again. Perhaps I'm not seeing these issues because 1) I use a low CRF for 720p-1080p content anyway and 2) I regularly boost bitrate through the use of zones.

Do you use cutree?

GeoffreyA
16th December 2024, 07:11
As you point out, cutree saves a lot of bitrate, sometimes close to -50% in anime, and the same goes for mbtree. Those savings can go to smaller CRFs for example. So, I do not understand what people gain by disabling these tools. There is a small increase in quality on certain parts of the frame, but you've got to look for it, and I think one would better spend bits by lowering the CRF.

Recently, encoding a set of anime at reference quality, I settled on leaving cutree on and using a low CRF (14). I also set the rc-lookahead to 240 because a longer lookahead is said to be more beneficial for cutree.

I think if one is encoding anime in particular, disabling mbtree or cutree is a massive waste of bits, and a 2-pass encode would likely suffer more for it. As for how it works, my limited understanding is that it increases the quantiser for parts of a frame that are less referenced temporally. Dark Shikari wrote a paper about mbtree back in the day but it was a bit technical.

Boulder
16th December 2024, 07:32
I recently requested jpsdr to include the cutree strength mod (in addition to those couple of other interesting AQ related tweaks) and it might happen if he just has the time for that. Then it would be easy to control cutree if it seems too strong, I also don't believe that it should be disabled completely. Raising qcomp by 0.1 makes cutree much less effective so that's probably why it has been mentioned that qcomp should be raised a bit.

Z2697
16th December 2024, 08:47
I recently requested jpsdr to include the cutree strength mod (in addition to those couple of other interesting AQ related tweaks) and it might happen if he just has the time for that. Then it would be easy to control cutree if it seems too strong, I also don't believe that it should be disabled completely. Raising qcomp by 0.1 makes cutree much less effective so that's probably why it has been mentioned that qcomp should be raised a bit.

I've tested the "cutree-strength" mod and found that it has basically the same effect as qcomp. Not really useful IMO.
Note that I used 2-pass bitrate control in the testing because the "cutree-strength" strongly impacted the CRF rate control (iirc).

Boulder
16th December 2024, 08:51
I've tested the "cutree-strength" mod and found that it has basically the same effect as qcomp. Not really useful IMO.

Do you mean increasing qcomp and increasing CRF as well to match bitrate?

Z2697
16th December 2024, 09:37
Do you mean increasing qcomp and increasing CRF as well to match bitrate?

I use 2-pass, as I edited in the previous post.
Because without mod, the "cutree-strength" is internally calculated from qcomp, changing the "cutree-strength" without touching qcomp seems to just confuses the CRF rate control so much that iirc, makes them off-chart.
Edit: I just did a quick encode with that mod, obviously I didn't remember correctly... not off-chart.

LunaRabbit
16th December 2024, 10:19
I recently requested jpsdr to include the cutree strength mod (in addition to those couple of other interesting AQ related tweaks)

Has anything interesting been added to those mods for AQ in the last 4 months or so? Why did the flag to enable it change? Do you still need to set it manually for SDR (e.g. --aq-auto 10 in the older version).

I was getting very good results with --aq-auto 10 for SDR content over the last several months. But obviously now my copy/pasted general settings no longer work. I did some testing with --auto-aq but I'm not sure how I'm supposed to be using it. Is it just the same thing with a different flag?

Edit: I just did a quick encode with that mod, obviously I didn't remember correctly... not off-chart.

What kind of differences are you seeing as opposed to just raising qcomp a bit? I mainly use CRF/1-pass mode. I haven't used 2-pass in a long time (since the xvid days) but if there is an appreciable gain in quality for the same file size I'd be willing to go back. I'll do some testing when I get some time after the holidays are over.

Thanks for the explanation everyone.

Edit: My hope was I could leave qcomp near the default of 0.6 and tweak cutree directly with the goal of maybe shaving off some file size. My CRF is already set so low that I don't want to reduce it any further. Since I try to keep half hour content under 1GB for HD content if possible (lower is obviously better and it varies from episode to episode).

Leo 69
16th December 2024, 10:21
I think it expects "raw" SEI payload, just guessing, haven't tried to understand it, and probably never will because I don't think it's (I'm referring to all the current implementations of FGS features) a useful feature, until it's much more improved.

You know, no matter what I tried, I still haven't been able to get it working. This feature is a nice bonus to x265 and I'd say very much needed for my particular use case.

I'm not a developer but I can partly understand and navigate the source code. So I headed to:

https://bitbucket.org/multicoreware/x265_git/src/master/

I found all portions of code related to --aom-film-grain and found that the encoder actually expects .bin files as input for grain parameters. I also found specifications as to what parameters that SEI payload should include. So I grabbed all of that code and fed it to o1-preview and Claude Sonnet 3.5 and asked them to provide me with compatible .bin files.

Both of them, based on those source code snippets, were able to give me a python script to produce a (supposedly) compatible grain.bin file and that's what I did, eventually. I got those .bin files. However, no matter what I did for about 3 hours, I still couldn't get this feature to work. x265 always throws error that it cannot read characteristics of the grain file.

I'm baffled. I tried to register on bitbucket, but it requires a corporate account to do so, so mere mortals cannot register so they're effectively blocked out from asking questions to developers on that platform. And nobody provided any examples of those .bin files and in what way they can actually be acquired.

The 4.1 release notes are vague, the new commandline parameter is vague. There are no examples anywhere and there's no one that I can ask. I have no idea how to solve this and make it work. Apparently, a simple function of a modern video encoder requires brains of Einstein to even get started with it. Very frustrating to be honest. If anyone can provide any insights, that would be awesome.

Boulder
16th December 2024, 11:51
You know, no matter what I tried, I still haven't been able to get it working. This feature is a nice bonus to x265 and I'd say very much needed for my particular use case.

I'm not a developer but I can partly understand and navigate the source code. So I headed to:

https://bitbucket.org/multicoreware/x265_git/src/master/

I found all portions of code related to --aom-film-grain and found that the encoder actually expects .bin files as input for grain parameters. I also found specifications as to what parameters that SEI payload should include. So I grabbed all of that code and fed it to o1-preview and Claude Sonnet 3.5 and asked them to provide me with compatible .bin files.

Both of them, based on those source code snippets, were able to give me a python script to produce a (supposedly) compatible grain.bin file and that's what I did, eventually. I got those .bin files. However, no matter what I did for about 3 hours, I still couldn't get this feature to work. x265 always throws error that it cannot read characteristics of the grain file.

I'm baffled. I tried to register on bitbucket, but it requires a corporate account to do so, so mere mortals cannot register so they're effectively blocked out from asking questions to developers on that platform. And nobody provided any examples of those .bin files and in what way they can actually be acquired.

The 4.1 release notes are vague, the new commandline parameter is vague. There are no examples anywhere and there's no one that I can ask. I have no idea how to solve this and make it work. Apparently, a simple function of a modern video encoder requires brains of Einstein to even get started with it. Very frustrating to be honest. If anyone can provide any insights, that would be awesome.

I think the mailing list is your best bet: x265-devel@videolan.org

It's plain silly that they didn't choose the simple grain table file method which is already used by at least aomenc and SVT-AV1.

Boulder
16th December 2024, 11:57
Has anything interesting been added to those mods for AQ in the last 4 months or so? Why did the flag to enable it change? Do you still need to set it manually for SDR (e.g. --aq-auto 10 in the older version).

I was getting very good results with --aq-auto 10 for SDR content over the last several months. But obviously now my copy/pasted general settings no longer work. I did some testing with --auto-aq but I'm not sure how I'm supposed to be using it. Is it just the same thing with a different flag?

The parameter should still be --aq-auto x. Nothing has changed there, so the same parameters will apply (10 for SDR, 6 for HDR if you want to enable all the tweaks it has).

Z2697
16th December 2024, 12:15
So, auto-aq is a resurrected form of "SBRC" feature that was "deprecated" from official x265, but then in release note 3.6 the first one is "SBRC", huh... I guess they "repurposed" that name?

Boulder
16th December 2024, 12:17
So, auto-aq is a resurrected form of "SBRC" feature that was "deprecated" from official x265, but then in release note 3.6 the first one is "SBRC", huh... I guess they "repurposed" that name?

I think someone mentioned that they committed a wrong set of patches under SBRC at first :D