Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > High Efficiency Video Coding (HEVC)

Reply
 
Thread Tools Search this Thread Display Modes
Old 17th December 2022, 14:58   #341  |  Link
HD MOVIE SOURCE
Registered User
 
HD MOVIE SOURCE's Avatar
 
Join Date: Mar 2021
Location: North Carolina
Posts: 78
Quote:
Originally Posted by benwaggoner View Post
Yeah, it's non-intuitive for sure! I presume that some of the extra stuff in other tools that get activated by rd=6 are causing some suboptimal behaviors. I'm also sure that some of the things in rd=6 are beneficial too, so it should be better than 4 if the pathological behaviors could be disabled somehow.


Please do report back!


It almost certainly can be fixed. If the tools that are causing problems in rd=6. can be identified, hopefully they can be tweaked to not respond badly to grain. Or at least just deactivated with heavy grain. --tune grain is ancient and terrible, and a refactor that worked well with the modern x265 infrastructure would be welcome, even if it had to be activated manually.

Since the problems are by far the worst at 4K, I suspect there's some threshold of noise detail versus content detail where things go wonky if crossed.
Tested some rd=3 vs rd=4 on film grain material, and rd3 and rd4 gave exactly the same results right down to the QP values. I then realized my testing method is definitely flawed. I tested on Medium, and many of the good motion and quality settings don't kick in until you passed medium. This is probably why I got exactly the same numbers.
Now, as soon as I went to rd=2 then the quality dropped.

So, once I get a chance I'm going to retest on very slow, then just switch the rd setting from 6 to 4 and see what happens. That way most settings are turned on regardless of rd setting, and that way we can see how rd scales properly.

However, when encoding Big Buck Bunny even on Medium, resulted in better quality using rd=6 vs rd=4. Which I found interesting.

Last edited by HD MOVIE SOURCE; 17th December 2022 at 15:01.
HD MOVIE SOURCE is offline   Reply With Quote
Old 17th December 2022, 16:55   #342  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Finland
Posts: 5,555
Rd 3 and 4 are the exact same, as are 5 and 6. This is stated in the docs.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 19th December 2022, 02:15   #343  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,369
Yeah. They left some extra room in case they'd want to add intermediate steps in the future.

I'd love to see an --rd 5 which would be "all the stuff from --rd 6 that doesn't mess up with high resolution film grain."
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 22nd December 2022, 08:35   #344  |  Link
A1
shortest name
 
Join Date: Sep 2022
Posts: 10
Quote:
Originally Posted by benwaggoner View Post

Good questions. After some rumination, I have a new working theory. The big problem with noisy --rd 6 is increasingly visible differences between CUs in QP and shape. It may not be --rd 6 itself that is doing it, but features of other modes that kick in at --rd >4.

--amp, --no-amp
Enable analysis of asymmetric motion partitions (75/25 splits, four directions). At RD levels 0 through 4, AMP partitions are only considered at TU sizes 32x32 and below. At RD levels 5 and 6, it will only consider AMP partitions as merge candidates (no motion search) at 64x64, and as merge or inter candidates below 64x64.
According to your inference, as long as the maximum size of ctu is limited, the quality of noise and other high-frequency details can be guaranteed when rd6 is turned on.

Last edited by A1; 22nd December 2022 at 09:03.
A1 is offline   Reply With Quote
Old 9th January 2023, 22:01   #345  |  Link
DMD
Registered User
 
DMD's Avatar
 
Join Date: Jan 2006
Location: Italy
Posts: 232
Quote:
Originally Posted by DMD View Post
Good morning.
To understand and have a reference with the hardware, I did a test with sample video @ 23.976fps with the current PC (Intel Core i5 9600K \ GeForce RTX 2070).
I set the StaxRip preset to Slower, the process is 0.1 FPS.
MISSION IMPOSSIBLE!!!



Making a prediction of around 2 FPS (I hope) with a Ryzen 9 5900X, I made this reasoning, if that's correct.
In the duration of a second of movie there are about 24fps and the processor processes 2 FPS, it means that the ratio is 1:12, so a movie with a duration of 2 hours needs 24 hours of processing, is that correct?
If necessary, it is possible to divide the source file into several parts, in order to do the processing at different times and then carry out the final union?
Thank you
Good evening.
This is the first test with Ryzen 9 7950X preset "Slow", I hope that with this upgrade the coding time will be more acceptable.


Last edited by DMD; 9th January 2023 at 22:08.
DMD is offline   Reply With Quote
Old 9th January 2023, 22:11   #346  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,369
Those are apples and kumquats comparisons; different thread counts, presets (slower v. slow), etc.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 10th January 2023, 09:09   #347  |  Link
DMD
Registered User
 
DMD's Avatar
 
Join Date: Jan 2006
Location: Italy
Posts: 232
Yes. You are right, I did not use the same preset.
With "Slower" it is 0.88 fps with the same processor.
At this point I think using the "Slow" preset is a good compromise between performance and final quality, I don't nthink "Slower" significantly affects the quality.
If I haven't miscalculated, an estimated process difference of about 37h40min between the two presets, is that much time difference worth it?
Example: 2 hours of movie @24fps
0.88fps>55h
2.75fps>17h 45min



Last edited by DMD; 10th January 2023 at 10:04.
DMD is offline   Reply With Quote
Old 10th January 2023, 16:43   #348  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,369
Quote:
Originally Posted by DMD View Post
Yes. You are right, I did not use the same preset.
With "Slower" it is 0.88 fps with the same processor.
At this point I think using the "Slow" preset is a good compromise between performance and final quality, I don't nthink "Slower" significantly affects the quality.
If I haven't miscalculated, an estimated process difference of about 37h40min between the two presets, is that much time difference worth it?
Example: 2 hours of movie @24fps
0.88fps>55h
2.75fps>17h 45min
Slower is the fastest preset where some of of HEVC's more modern features kick in, like relatively deep TU recursion, weighted b-frame prediction, and B-intra encoding. It's the setting I start with by default, and iterate from. It has somewhat reduced parallelism (lookahead-slices 1 instead of 4), so might not be as optimal for benchmarking with many cores available.

Apples-to-apples comparisons can't rely on just presets, however. The number of frame threads can have a big impact on perf and a smaller impact on quality, and the default number of frame threads is based on how many cores are available. Thus comparing two processors with different core counts can see the processor with more cores running with more frame threads, improving encoding speed but potentially reducing quality. So not quite apples-to-apples.

Benchmarking is hard to do in a broadly applicable way, because there are so many encoding scenarios that can impact relative performance. Comparing at slow with default frame threads is certainly a scenario that will matter to plenty of people. For me, comparing with --preset slower --frame-threads 1 would have the most relevance. Benchmarking for realtime encoding would be very different, as predictable worst-case encoding time becomes essential. Plenty of benchmarks just compare with stock default settings.

I see you are comparing with --pmode (makes good sense if you have a lot of cores relative to frame size, but can slow things down if there aren't enough cores) and --pme (which is a net negative unless you have a whole lot of cores encoding sub-HD resolutions).

I consider it "fair" to use --pmode if it is only used when it increases throughput in a given configuration, and turned off when it doesn't. As --pmode doesn't decrease quality (and can theoretically increase it a bit).

The same can apply to using --pme selectively, although the cores needed to make it a net positive are a lot higher. But for 480p with 64 cores or something, it probably would help. I personally rarely test with more than 18/36 available for any given encoder instance. Although with all the ARM patches, Graviton2/3 with 64 cores deserves some benchmarking as well.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 11th January 2023, 12:43   #349  |  Link
DMD
Registered User
 
DMD's Avatar
 
Join Date: Jan 2006
Location: Italy
Posts: 232
Quote:
Originally Posted by benwaggoner View Post
Slower is the fastest preset where some of of HEVC's more modern features kick in, like relatively deep TU recursion, weighted b-frame prediction, and B-intra encoding. .......
Thanks for the explanation, I still have a lot to learn from this technique.
DMD is offline   Reply With Quote
Old 12th January 2023, 00:37   #350  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,369
Quote:
Originally Posted by DMD View Post
Thanks for the explanation, I still have a lot to learn from this technique.
Trust me, everyone has a lot to learn when it comes to benchmarking!

First rule is to define as specific a question as possible, and then figure out the optimal benchmarking approach for that.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 01:40.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2023, vBulletin Solutions Inc.