Which processor to encode x265 4K ?

[HD MOVIE SOURCE]

Originally Posted by benwaggoner

Yeah, it's non-intuitive for sure! I presume that some of the extra stuff in other tools that get activated by rd=6 are causing some suboptimal behaviors. I'm also sure that some of the things in rd=6 are beneficial too, so it should be better than 4 if the pathological behaviors could be disabled somehow.


Please do report back!


It almost certainly can be fixed. If the tools that are causing problems in rd=6. can be identified, hopefully they can be tweaked to not respond badly to grain. Or at least just deactivated with heavy grain. --tune grain is ancient and terrible, and a refactor that worked well with the modern x265 infrastructure would be welcome, even if it had to be activated manually.

Since the problems are by far the worst at 4K, I suspect there's some threshold of noise detail versus content detail where things go wonky if crossed.

Tested some rd=3 vs rd=4 on film grain material, and rd3 and rd4 gave exactly the same results right down to the QP values. I then realized my testing method is definitely flawed. I tested on Medium, and many of the good motion and quality settings don't kick in until you passed medium. This is probably why I got exactly the same numbers.
Now, as soon as I went to rd=2 then the quality dropped.

So, once I get a chance I'm going to retest on very slow, then just switch the rd setting from 6 to 4 and see what happens. That way most settings are turned on regardless of rd setting, and that way we can see how rd scales properly.

However, when encoding Big Buck Bunny even on Medium, resulted in better quality using rd=6 vs rd=4. Which I found interesting.

[Boulder]

Rd 3 and 4 are the exact same, as are 5 and 6. This is stated in the docs.

[benwaggoner]

Yeah. They left some extra room in case they'd want to add intermediate steps in the future.

I'd love to see an --rd 5 which would be "all the stuff from --rd 6 that doesn't mess up with high resolution film grain."

[A1]

Originally Posted by benwaggoner

Good questions. After some rumination, I have a new working theory. The big problem with noisy --rd 6 is increasingly visible differences between CUs in QP and shape. It may not be --rd 6 itself that is doing it, but features of other modes that kick in at --rd >4.

--amp, --no-amp
Enable analysis of asymmetric motion partitions (75/25 splits, four directions). At RD levels 0 through 4, AMP partitions are only considered at TU sizes 32x32 and below. At RD levels 5 and 6, it will only consider AMP partitions as merge candidates (no motion search) at 64x64, and as merge or inter candidates below 64x64.

According to your inference, as long as the maximum size of ctu is limited, the quality of noise and other high-frequency details can be guaranteed when rd6 is turned on.

[DMD]

Originally Posted by DMD

Good morning.
To understand and have a reference with the hardware, I did a test with sample video @ 23.976fps with the current PC (Intel Core i5 9600K \ GeForce RTX 2070).
I set the StaxRip preset to Slower, the process is 0.1 FPS.
MISSION IMPOSSIBLE!!!



Making a prediction of around 2 FPS (I hope) with a Ryzen 9 5900X, I made this reasoning, if that's correct.
In the duration of a second of movie there are about 24fps and the processor processes 2 FPS, it means that the ratio is 1:12, so a movie with a duration of 2 hours needs 24 hours of processing, is that correct?
If necessary, it is possible to divide the source file into several parts, in order to do the processing at different times and then carry out the final union?
Thank you

Good evening.
This is the first test with Ryzen 9 7950X preset "Slow", I hope that with this upgrade the coding time will be more acceptable.

[benwaggoner]

Those are apples and kumquats comparisons; different thread counts, presets (slower v. slow), etc.

[DMD]

Yes. You are right, I did not use the same preset.
With "Slower" it is 0.88 fps with the same processor.
At this point I think using the "Slow" preset is a good compromise between performance and final quality, I don't nthink "Slower" significantly affects the quality.
If I haven't miscalculated, an estimated process difference of about 37h40min between the two presets, is that much time difference worth it?
Example: 2 hours of movie @24fps
0.88fps>55h
2.75fps>17h 45min

[benwaggoner]

Originally Posted by DMD

Yes. You are right, I did not use the same preset.
With "Slower" it is 0.88 fps with the same processor.
At this point I think using the "Slow" preset is a good compromise between performance and final quality, I don't nthink "Slower" significantly affects the quality.
If I haven't miscalculated, an estimated process difference of about 37h40min between the two presets, is that much time difference worth it?
Example: 2 hours of movie @24fps
0.88fps>55h
2.75fps>17h 45min

Slower is the fastest preset where some of of HEVC's more modern features kick in, like relatively deep TU recursion, weighted b-frame prediction, and B-intra encoding. It's the setting I start with by default, and iterate from. It has somewhat reduced parallelism (lookahead-slices 1 instead of 4), so might not be as optimal for benchmarking with many cores available.

Apples-to-apples comparisons can't rely on just presets, however. The number of frame threads can have a big impact on perf and a smaller impact on quality, and the default number of frame threads is based on how many cores are available. Thus comparing two processors with different core counts can see the processor with more cores running with more frame threads, improving encoding speed but potentially reducing quality. So not quite apples-to-apples.

Benchmarking is hard to do in a broadly applicable way, because there are so many encoding scenarios that can impact relative performance. Comparing at slow with default frame threads is certainly a scenario that will matter to plenty of people. For me, comparing with --preset slower --frame-threads 1 would have the most relevance. Benchmarking for realtime encoding would be very different, as predictable worst-case encoding time becomes essential. Plenty of benchmarks just compare with stock default settings.

I see you are comparing with --pmode (makes good sense if you have a lot of cores relative to frame size, but can slow things down if there aren't enough cores) and --pme (which is a net negative unless you have a whole lot of cores encoding sub-HD resolutions).

I consider it "fair" to use --pmode if it is only used when it increases throughput in a given configuration, and turned off when it doesn't. As --pmode doesn't decrease quality (and can theoretically increase it a bit).

The same can apply to using --pme selectively, although the cores needed to make it a net positive are a lot higher. But for 480p with 64 cores or something, it probably would help. I personally rarely test with more than 18/36 available for any given encoder instance. Although with all the ARM patches, Graviton2/3 with 64 cores deserves some benchmarking as well.

[DMD]

Originally Posted by benwaggoner

Slower is the fastest preset where some of of HEVC's more modern features kick in, like relatively deep TU recursion, weighted b-frame prediction, and B-intra encoding. .......

Thanks for the explanation, I still have a lot to learn from this technique.

[benwaggoner]

Originally Posted by DMD

Thanks for the explanation, I still have a lot to learn from this technique.

Trust me, everyone has a lot to learn when it comes to benchmarking!

First rule is to define as specific a question as possible, and then figure out the optimal benchmarking approach for that.