x265 HEVC Encoder

[microchip8]

Originally Posted by Boulder

I just noticed that --limit-tu 0 seems to be a potentially dangerous setting coupled with --rskip 2. I've encoded episodes of 'Chernobyl' and saw that the average bitrate is quite low for 1080p material. I found one scene where the encode looked really ugly and with some testing, I checked that changing from --limit-tu 4 to --limit-tu 0 made it happen. --ctu 32 also fixed the issue.

I've encoded quite a few flicks since then with limit-tu 0 and rskip 2 and have yet to see some noticable (major) distortions. Maybe your issue is sample specific

[Boulder]

I made some tests to show the issue. The basic x265 settings were "--preset slower --no-sao --aq-mode 1 --aq-strength 1.0 --qcomp 0.7 --crf 20". The source was unaltered, I only cropped the black borders off in the script.

--ctu 64 --limit-tu 4 --rskip 1 : 4730.32 kbps
--ctu 64 --limit-tu 0 --rskip 1 : 4477.60 kbps
--ctu 32 --limit-tu 4 --rskip 1 : 4735.29 kbps
--ctu 32 --limit-tu 0 --rskip 1 : 4733.48 kbps

--ctu 64 --limit-tu 4 --rskip 2 : 4863.81 kbps
--ctu 64 --limit-tu 0 --rskip 2 : 2877.67 kbps
--ctu 32 --limit-tu 4 --rskip 2 : 5090.29 kbps
--ctu 32 --limit-tu 0 --rskip 2 : 5142.87 kbps

I put the original source clip here:

https://drive.google.com/file/d/1nEa...ew?usp=sharing

The problem is very visible around the middle of the clip in the scene where the woman on the phone is in the distant background. The noise turns into very ugly shapes there.

I don't know if I should use --ctu 32 also with 1080p sources with --rskip 2 just to be safe. I need to test 1440p as well.

[Boulder]

A similar test with 1440p (2560x1048):

--limit-tu 4 : 6963.05 kbps
--limit-tu 0 : 6825.00 kbps
--limit-tu 4 --ctu 32 : 6905.23 kbps
--limit-tu 0 --ctu 32 : 6906.79 kbps
--limit-tu 4 --rskip 2 : 7563.58 kbps
--limit-tu 0 --rskip 2 : 5934.17 kbps
--limit-tu 4 --ctu 32 --rskip 2 : 7045.77 kbps
--limit-tu 0 --ctu 32 --rskip 2 : 7099.02 kbps

The large difference can also be seen there. I didn't check how the encode looked but probably there would be scenes which would look ugly. This means it's either --limit-tu 4 or --limit-tu 0 --ctu 32 to avoid the issue. I'm just not sure which one is the better choice quality-wise.

[LigH]

New upload: x265 3.4+14-gd419c7152

abr-ladder: fix memleak in copyInfo()
Fix cost estimation error for hist-based scenecut
Fix libvmaf integration issues

[Boulder]

I tested the histogram based scenecut detection yesterday quite a lot. I still couldn't get it to detect proper scenecuts -- it would either miss too many, at least one very obvious one, and lowering the threshold would cause it to find scenecuts in quite strange places. My Hot Fuzz testclip triggered scenecuts in scenes where there is some motion blur and then single frames which are just slightly less blurry. Probably the sudden sharp edged text on shirts etc. are enough to tell that it's a scenecut.

If you lower the threshold enough, it will start reducing the number of keyframes which was really strange. In the csv log, you can see that a scenecut is detected but they are often coded as B/b-frames.

[jlpsvk]

Originally Posted by LigH

New upload: x265 3.4+14-gd419c7152

abr-ladder: fix memleak in copyInfo()
Fix cost estimation error for hist-based scenecut
Fix libvmaf integration issues

testing to encode 1917 HDR10+ again (testing with every new build)...

[quietvoid]

Originally Posted by Boulder

I tested the histogram based scenecut detection yesterday quite a lot. I still couldn't get it to detect proper scenecuts -- it would either miss too many, at least one very obvious one, and lowering the threshold would cause it to find scenecuts in quite strange places. My Hot Fuzz testclip triggered scenecuts in scenes where there is some motion blur and then single frames which are just slightly less blurry. Probably the sudden sharp edged text on shirts etc. are enough to tell that it's a scenecut.

If you lower the threshold enough, it will start reducing the number of keyframes which was really strange. In the csv log, you can see that a scenecut is detected but they are often coded as B/b-frames.

I've posted about this before:

Originally Posted by quietvoid

So x265 3.4+9, they "fixed" --hist-threshold to avoid false positives. However it seems to do much worse than before (as well as 3.4+7, where they fixed the false positives).
I'm getting less I frames with --hist-threshold 0.005 (default 0.03) than the default scenecut algorithm..

Maybe my sample clip is not colorful enough..

Since then I've just given up on hist-scenecut. As well as scenecut-aware-qp, because they introduced QP changes before scene cuts...

[Boulder]

Originally Posted by quietvoid

I've posted about this before

Actually that's why I tested it now, since there was that one patch related to the feature.

Too bad they don't seem to care for the original method anymore, my bug report (https://bitbucket.org/multicoreware/...sses-two-clear) has not had any activity.

[vpupkind]

Originally Posted by quietvoid

I've posted about this before:

Since then I've just given up on hist-scenecut. As well as scenecut-aware-qp, because they introduced QP changes before scene cuts...

Have you tried scenecut-aware-qp? Unless you pause at this frame, a human observer is typically unable to notice quality degradation within ~100ms from a strong scene cut.

[Boulder]

Originally Posted by vpupkind

Have you tried scenecut-aware-qp? Unless you pause at this frame, a human observer is typically unable to notice quality degradation within ~100ms from a strong scene cut.

Why does that option only work with a multipass encode? You would expect that the lookahead could take care of the work also in CRF mode.

[quietvoid]

Originally Posted by vpupkind

Have you tried scenecut-aware-qp? Unless you pause at this frame, a human observer is typically unable to notice quality degradation within ~100ms from a strong scene cut.

I have tried it, it was fine before they changed it.
The quality change AFTER a scene cut is not visible, but the idea of lowering the quality BEFORE (which is what they did) does not sound good to me.
It makes no sense.

Originally Posted by Boulder

Why does that option only work with a multipass encode? You would expect that the lookahead could take care of the work also in CRF mode.

This is another good point, I don't know how these patches get through quality checks.
It used to work just fine for CRF, and they added a condition that makes it multipass exclusive.

[foxyshadis]

Originally Posted by quietvoid

This is another good point, I don't know how these patches get through quality checks.
It used to work just fine for CRF, and they added a condition that makes it multipass exclusive.

Since Steve Borho and chen moved on, the main quality check is "does it compile". Almost every submitted patch is immediately committed without discussion now, and reverted later if need be.

[LeXXuz]

Quick question: should SMT/Hyperthreading be enabled on a Ryzen 9 machine or not? With 2x logical cpus I had about 66% average cpu usage and lower fps than with logical cpus = cpus @ 95-100%. Same settings of course.

[Boulder]

HT on works fine. If the CPU usage tends to stay low, it could a parallelization issue like CTU 64 with less than FullHD resolution source.

[Atak_Snajpera]

Originally Posted by LeXXuz

Quick question: should SMT/Hyperthreading be enabled on a Ryzen 9 machine or not? With 2x logical cpus I had about 66% average cpu usage and lower fps than with logical cpus = cpus @ 95-100%. Same settings of course.

Always on.

[LigH]

HEVC has more dependencies than AVC. More parts of the calculation in x265 need to wait for intermediate results of previous calculations. Not approaching 100% CPU usage is not concerning. You may be able to run two separate encoding jobs (provided you have enough RAM and the cooling can handle the heat).

[LeXXuz]

Thanks for the replies guys.

Yes, I've noticed a temperature drop of about ten degrees with SMT off. But that is of no concern anyway since the cpus are watercooled.

I always use --ctu 32 for 1080p resolutions and below. Frame threads is set to auto. x.265 picks 4 threads on 3900x (12C/24T) and 5 threads on 3950x (16C/32T).

I did an encode with frame threads set to 1 but the file size was just negligible smaller than with the auto setting.

So SMT on it is. Maybe I just do 2 parallel jobs instead.

[microchip8]

Originally Posted by LeXXuz

Thanks for the replies guys.

Yes, I've noticed a temperature drop of about ten degrees with SMT off. But that is of no concern anyway since the cpus are watercooled.

I always use --ctu 32 for 1080p resolutions and below. Frame threads is set to auto. x.265 picks 4 threads on 3900x (12C/24T) and 5 threads on 3950x (16C/32T).

I did an encode with frame threads set to 1 but the file size was just negligible smaller than with the auto setting.

So SMT on it is. Maybe I just do 2 parallel jobs instead.

Try with --pmode

[Boulder]

I run my 3900X with two framethreads (-F 2). I don't know if it will increase quality (in theory it should) but the CPU usage difference to -F 4 is around 10% so I consider it more like headroom for doing all the other stuff at the same time. The encoder uses a 24-thread pool, I also assign 24 threads to Avisynth processing. The Avisynth part takes about 20-30% of total CPU time when encoding.

[LeXXuz]

Originally Posted by Boulder

I also assign 24 threads to Avisynth processing. The Avisynth part takes about 20-30% of total CPU time when encoding.

Wait. You mean you set prefetch() to 24 in Avisynth?
Or are we talking about different things here?