x265 HEVC Encoder

[James_b]

Hi Guys!
Does anyone know if i get better picture quality whith HEVC than HuffYUV? I think about from a video editor, both set at 100% (lowest compression). This is for max out before Youtube conversion sets in.

Some say VP9 gives better dynamic and colors than AVC but is there something like this in above?

Thank's for any advice!

[xxxxx]

Is there a program I can see the size of the frames in a video?

[excellentswordfight]

Hi Guys!
Does anyone know if i get better picture quality whith HEVC than HuffYUV? I think about from a video editor, both set at 100% (lowest compression). This is for max out before Youtube conversion sets in.

Some say VP9 gives better dynamic and colors than AVC but is there something like this in above?

Thank's for any advice!
HuffYUV is a lossless codec, so nothing will get you better picture quality. HEVC in lossless mode can although give you the same at a lower filesize but will be alot slower, but I doubt any NLE have a lossless mode for HEVC implemented.

[jpsdr]

Does anyone know if i get better picture quality whith HEVC than HuffYUV?
If you're looking for a lossless codec, HuffYUV is a little old and not always supported, and slow.
Choose UTVideo or MagicYUV, they're the best lossless codec i know, fast and efficient (doesn't mean it doesn't exist others lossless codec even better).

[LigH]

Is there a program I can see the size of the frames in a video?

Such tools, in general, are called "Bitrate Viewer". Not all of them support every source format, though. And please better learn what a GOP is and why the size of I, P, and B frames differs much more than the quality.

[LigH]

Just committed:

Pushed BBAQ 3 patches to master branch

[LigH]

I need advice ... I have a shell script which is supposed to update my local copy of x265 when an update is available. This used to work flawlessly before. But now a vi opens, trying to create a text file explaining the reason for a merge. So I guess there is a flaw in my git command, and I do not understand the concept of git good enough to tell apart the meaning of many similar terms in the heap of options... How do I avoid asking back? I do not even want to commit anything on the git server, only update my local copy.

#!/bin/sh
git -C /build/x265_git-git fetch -v --progress origin
git -C /build/x265_git-git pull -v --no-rebase --progress origin "${1:-master}"

The desired result is: If I submit a commit hash as parameter to this script, it shall be synced with that specific commit, else it shall be updated to the latest commit of the master branch.
_

PS: Maybe I should try to source /build/media-suite_helper.sh and then try to use do_vcs "https://bitbucket.org/multicoreware/x265_git.git"; but there is also a patch related to cmake library paths with L vs. l, maybe that one interfers.

[LigH]

New upload: x265 3.5+74-1ec26b9f6 (https://www.mediafire.com/file/kh7loyrqrgv3zx7/x265_3.5+74-1ec26b9f6.7z/file)

[Windows][GCC 12.2.0][32/32XP/64 bit] 8bit+10bit+12bit

News since v3.5+70:
--scenecut-qp-config <file> File containing scenecut-aware-qp mode, window duration and offsets settings required for the masking. Works only with --pass 2

I guess we need a sample file to know how it works...

[Boulder]

New upload: x265 3.5+74-1ec26b9f6 (https://www.mediafire.com/file/kh7loyrqrgv3zx7/x265_3.5+74-1ec26b9f6.7z/file)

[Windows][GCC 12.2.0][32/32XP/64 bit] 8bit+10bit+12bit

News since v3.5+70:
--scenecut-qp-config <file> File containing scenecut-aware-qp mode, window duration and offsets settings required for the masking. Works only with --pass 2

I guess we need a sample file to know how it works...

Yes, the functionality looks a bit confusing to me. Hopefully someone could provide a simple use case for this.

[LigH]

Unfortunately, readthedocs (https://x265.readthedocs.io/en/master/cli.html) is not yet up to date.

[jpsdr]

I was using --scenecut-aware-qp 3 in my scripts, and now with the new version i have a FLAW error, "invalid argument", but according the --fullhelp it's stil a valid argument...

Edit :
Has anyone else tested to check if --scenecut-aware-qp 3 is still working after these commits ?
For me this release is broken somewhere, all my encoding scripts are not running anymore.

[benwaggoner]

Unfortunately, readthedocs (https://x265.readthedocs.io/en/master/cli.html) is not yet up to date.
FWIW, there is an "Edit on Bitbucket" button at the top of all those pages. We can submit edits and additions ourselves for things that MCW hasn't documented yet or needs tweaking. I just submitted a commit to edit the weird discontinuity in the options list for --force-flush as a test.

[ShortKatz]

New upload: x265 3.5+74-1ec26b9f6 (https://www.mediafire.com/file/kh7loyrqrgv3zx7/x265_3.5+74-1ec26b9f6.7z/file)

[Windows][GCC 12.2.0][32/32XP/64 bit] 8bit+10bit+12bit

News since v3.5+70:
--scenecut-qp-config <file> File containing scenecut-aware-qp mode, window duration and offsets settings required for the masking. Works only with --pass 2

I guess we need a sample file to know how it works...


There is now a sample file here: https://bitbucket.org/multicoreware/x265_git/downloads/scenecut_qp_config.txt

Its from https://mailman.videolan.org/pipermail/x265-devel/2022-December/013560.html

[benwaggoner]

There is now a sample file here: https://bitbucket.org/multicoreware/x265_git/downloads/scenecut_qp_config.txt

Its from https://mailman.videolan.org/pipermail/x265-devel/2022-December/013560.html
Huh. It seems odd to use an external file for just a few static parameters. Normally x265 uses external files (multi-pass-opt-analysis, qpfile) for data that can vary per-frame.

Perhaps there will be a way to specify different parameters for different frame ranges?

[Greenhorn]

Looking at the docs for the new file on BitBucket, it can get a lot more complicated that sample, so I can see why they used a file for it even if it's still static values.

(docs/rEST/cli.rst, and you have to click "Open Raw" in the little menu next to the "Edit" button since the fancy markdown version you see by default is ancient. Search for masking-strength.)

[quietvoid]

You can specify the different QP adjustments per window, and there are 6 windows now.
So the file is useful for that, but hardly IMO.

My CLI for example is already 1000 characters, so it wouldn't bother me.

[LigH]

We can submit edits ...

... but better do so only when we actually have a clue. I do not, in this case.

But I noticed a mailing list entry so it will happen.

[benwaggoner]

... but better do so only when we actually have a clue. I do not, in this case.

But I noticed a mailing list entry so it will happen.
It still goes through the normal commit process, so MCW has a chance to review any edits. I focused on some formatting deficiencies that I'd noticed had been a common source of confusion.

[ghostshadow]

I thought there was an option to activate an automatic filling of this file with different options for each scene but the need to put them by hand from what I understand
ex : --scenecut-aware-qp 3 --masking-strength 500,2,0,200,-1,-1 in the file scenecut_qp_config.txt.
so we have to manually enter all the options per scene?

[jpsdr]

I have, just for testing, replaced :
OPT("scenecut-aware-qp") p->bEnableSceneCutAwareQp = x265_atoi(value, bError);
with
OPT("scenecut-aware-qp") p->bEnableSceneCutAwareQp = atoi(value);
No change, my log file has only one line :
x265 [FLAW]: invalid argument: scenecut-aware-qp = 3

[Boulder]

Do we still need quietvoid's patch (probably a new version of it) to make the functionality work in CRF mode?

[benwaggoner]

I thought there was an option to activate an automatic filling of this file with different options for each scene but the need to put them by hand from what I understand
ex : --scenecut-aware-qp 3 --masking-strength 500,2,0,200,-1,-1 in the file scenecut_qp_config.txt.
so we have to manually enter all the options per scene?
While it can be hand-edited, which is very useful for testing and early days evaluations like we are today. Production implementations would automatically generate the file based on preanalysis, like how qpfiles are most commonly used. Perhaps a future x265 will generate the file, which could be analyzed and edited as needed.

[jpsdr]

Ok, i think i understand my mistake.
I thought the new patch offered the possibility of having an external file to configure the scenecut-qp stuff.
But not, it's a necessity !
For just using --scenecut-aware-qp 3 i have to create an external file, acces it with --scenecut-qp-config and put --scenecut-aware-qp 3 inside it... :eek:
I will in my mod version restore the possibility of being able to just put --scenecut-aware-qp directly in command inside a script without the necessity to have an external file just for it (and make all my encoding scripts working again...:D)

[LigH]

:confused: That's all? It sounded like there is a zone control.

[jpsdr]

What i ment, it's that now you can't anymore just put --scenecut-aware-qp 3 directly in the command line, you have to put it in an external file. There is probably a lot of others possibilities with this new patch (i didn't dig to see what it's doing because i'm not using these stuff), and having the possibility of using an external file for a lot of zone description is a good thing of course, but for someone like me who was just using --scenecut-aware-qp 3, it breaks all my encode script, and to have to add an extra file just to put one command inside... :(

[ShortKatz]

Ok, i think i understand my mistake.
I thought the new patch offered the possibility of having an external file to configure the scenecut-qp stuff.
But not, it's a necessity !
For just using --scenecut-aware-qp 3 i have to create an external file, acces it with --scenecut-qp-config and put --scenecut-aware-qp 3 inside it... :eek:


Oh, really? :eek: That's stupid, who came up with that idea?

[jpsdr]

I've made a new build on my mod version (check my github).
As i have en encode running, i couldn't test it.

[madey83]

Ok, i think i understand my mistake.
I thought the new patch offered the possibility of having an external file to configure the scenecut-qp stuff.
But not, it's a necessity !
For just using --scenecut-aware-qp 3 i have to create an external file, acces it with --scenecut-qp-config and put --scenecut-aware-qp 3 inside it... :eek:
I will in my mod version restore the possibility of being able to just put --scenecut-aware-qp directly in command inside a script without the necessity to have an external file just for it (and make all my encoding scripts working again...:D)

hi,

is this work for CRF encodes?

[LigH]

The documentation says:

Works only with --pass 2

So according to that one: No. It will most probably require a statistics file gathered in a first pass, to tune scenecuts in a second/last pass of two/multiple passes.

[Boulder]

If you use a config file with only --scenecut-aware-qp 3 in it, it changes the output in CRF mode even though there's a warning that it only works with --pass 2 :scared:

[ShortKatz]

A bunch of new patches have been uploaded to the mailing list. I quite don't understand what this patches are doing, its something about temporal layers, miniGOP and NAL type.

[LigH]

Temporal sublayer for unreferenced B frames: Decoders may skip them if decoding time is tight, without introducing decoding artifacts.

A "miniGOP" is only the P(B...) portion of a GOP. I frames are essential as reference to many following P frames (and maybe some B frames too). P frames are often referenced by following P and intermediate B frames, so they are semi-essential too, missing one of these in the decoding sequence usually means skipping to the next IDR frame to recover the video stream decoding.

[ShortKatz]

Ah, thanks for your explanation. :-)

[ksec]

https://rigaya.github.io/vq_results/

Some H.264 / H.265 and AV1 encoding results.

[Fishman0919]

Commit: Niranjan Bala

Niranjan Bala committed 65eaa5a 2022-12-15

View source

Add BBAQ to CRF and fix qgSize 0 issue.

Works with CRF now

[Boulder]

Still the same error with a new build :D

if (param->bEnableSceneCutAwareQp)
{
if (!param->rc.bStatRead)
{
param->bEnableSceneCutAwareQp = 0;
x265_log(param, X265_LOG_WARNING, "Disabling Scenecut Aware Frame Quantizer Selection since it works only in pass 2\n");
}

[Boulder]

I retested the feature by bypassing that check. With default settings (i.e. setting only --scenecut-aware-qp 1/2/3 in the config file), the filesize difference for a 3000-frame sample was very small, much less than 1%. Does anyone else have any results? I didn't check the csv log yet for changes in QPs.

[LigH]

Some clean-ups have been committed.

[jpsdr]

Euh... Does it mean SBRC is removed ? It didn't live for long...:confused:

[quietvoid]

No, it's reimplemented in another patch.

[ShortKatz]

I also don't get it, why did they remove the current SBRC implementation and then try to reimplement it with another patch? What different will this make? What was wrong with the first SBRC implementation?

[quietvoid]

The new SBRC patch is totally different.
The first one didn't really make sense.

[Boulder]

The new SBRC patch is totally different.
The first one didn't really make sense.

I suppose the new patch is still pending a push to the master branch. Looking at the proposed patch, I find it a bit odd that a closed GOP is now required and that there seems to be a fixed GOP length :confused:

[jpsdr]

Ok, will wait for this new SBRC patch...

[ShortKatz]

I was wondering what the HISTOGRAM_BINS variable is used for, which was introduced in the latest SBRC removal patch. It does not occur in the new SBRC patch. Maybe there is more to come?

[LeXXuz]

I have a question regarding ctu sizes in CRF mode.

I did some testing with SD and HD content to compare differences between encodes with --ctu 64 and --ctu 32 setting in crf mode.
I read in the docs and on Wikipedia that larger coding tree units usually increase encoding efficiency, therfore should decrease filesize in crf-mode, right?

But I noticed quite the contrary where ctu=32 results in noticeabley smaller file sizes. See screenshot:
https://abload.de/thumb/vergleich64-1vs64-2vsxcemz.png (https://abload.de/image.php?img=vergleich64-1vs64-2vsxcemz.png)
top-left: ctu=64, framethreads=1
top-right: ctu=64, framethreads=2
bottom-left: ctu=32, framethreads=1
bottom-right: ctu=32, framethreads=2
(This is an example with SD video. The behaviour is similar with 1080p content. I also repeated this with different content with less or more complexity to rule out this was just coincidence.)

Okay, as mentioned in the docs, additional frame-threads reduce efficiency and slightly increase filesize. Regarding the noticeable speed increase I think the additional thread is well spend.
And I understand that ctu=32 gives a big speed increase on lower resolutions because there is more parallelism.

But why does the filesize decrease that much which clearly should have an impact on visual quality, right?

The only reason I can think of is that crf values scale differently with different ctu sizes. If so, could I just decrease crf values to get similar filesizes and, more importantly, similar visual results?

Or will ctu=64 still give better visual quality and should always be prefered?

[Boulder]

I have a question regarding ctu sizes in CRF mode.

I did some testing with SD and HD content to compare differences between encodes with --ctu 64 and --ctu 32 setting in crf mode.
I read in the docs and on Wikipedia that larger coding tree units usually increase encoding efficiency, therfore should decrease filesize in crf-mode, right?

But I noticed quite the contrary where ctu=32 results in noticeabley smaller file sizes. See screenshot:
https://abload.de/thumb/vergleich64-1vs64-2vsxcemz.png (https://abload.de/image.php?img=vergleich64-1vs64-2vsxcemz.png)
top-left: ctu=64, framethreads=1
top-right: ctu=64, framethreads=2
bottom-left: ctu=32, framethreads=1
bottom-right: ctu=32, framethreads=2
(This is an example with SD video. The behaviour is similar with 1080p content. I also repeated this with different content with less or more complexity to rule out this was just coincidence.)

Okay, as mentioned in the docs, additional frame-threads reduce efficiency and slightly increase filesize. Regarding the noticeable speed increase I think the additional thread is well spend.
And I understand that ctu=32 gives a big speed increase on lower resolutions because there is more parallelism.

But why does the filesize decrease that much which clearly should have an impact on visual quality, right?

The only reason I can think of is that crf values scale differently with different ctu sizes. If so, could I just decrease crf values to get similar filesizes and, more importantly, similar visual results?

Or will ctu=64 still give better visual quality and should always be prefered?
I've noticed this too. I believe there's more to it than just a simple difference in CTU size but that will also affect many other decisions. If you enable the csv logging, it might show some fundamental differences in skips and merges. I think the rskip mode also affects it greatly.

[benwaggoner]

For those comparing --ctu 32 versus 64, have you tried using --ctu 64 with --rdpenalty 1? That raises the cost of a 32x32 TU by 4x, theoretically meaning they'll be selected a lot less often, and only when benefits are unambiguous.

However, it's a really old parameter from x265 circa 2014, and I don't know that anyone has tested with it in ages.

--rdpenalty <0..2>
When set to 1, transform units of size 32x32 are given a 4x bit cost penalty compared to smaller transform units, in intra coded CUs in P or B slices.

When set to 2, transform units of size 32x32 are not even attempted, unless otherwise required by the maximum recursion depth. For this option to be effective with 32x32 intra CUs, --tu-intra-depth must be at least 2. For it to be effective with 64x64 intra CUs, --tu-intra-depth must be at least 3.

Note that in HEVC an intra transform unit (a block of the residual quad-tree) is also a prediction unit, meaning that the intra prediction signal is generated for each TU block, the residual subtracted and then coded. The coding unit simply provides the prediction modes that will be used when predicting all of the transform units within the CU. This means that when you prevent 32x32 intra transform units, you are preventing 32x32 intra predictions.

Default 0, disabled.

Values: 0:disabled 1:4x cost penalty 2:force splits

[LeXXuz]

For those comparing --ctu 32 versus 64, have you tried using --ctu 64 with --rdpenalty 1?

Thanks Ben. I never heard or noticed that parameter before. So no, I haven't. :o
I'll give it a try and see how this may affect speed and file size.

Still, it would be great to find an explanation to those differences in size between 32 and 64. Like Boulder said, there must be more to it.

[vpupkind]

Thanks Ben. I never heard or noticed that parameter before. So no, I haven't. :o
I'll give it a try and see how this may affect speed and file size.

Still, it would be great to find an explanation to those differences in size between 32 and 64. Like Boulder said, there must be more to it.
The lower the resolution the less probability of 64x64 being selected. It's pointless at HD resolutions. Even 32x32 is selected in a tiny % of cases in 1080p and nearly never with lower resolutions.