Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > High Efficiency Video Coding (HEVC)

Reply
 
Thread Tools Search this Thread Display Modes
Old 24th August 2024, 02:15   #1  |  Link
SquallMX
Special SeeD
 
Join Date: Nov 2002
Location: Mexico
Posts: 340
Corrupted playback when using Hardware decoding on a file created using x265

Hi. I'm having a issue when decoding a file created by x265 3.6:

https://1drv.ms/v/s!AiCgDguh7E_Ogc4L...SHuxg?e=lgoOQf

When playing with HW acceleration of any kind there is a corrupted set of frames at 1818-1822, when decoding using LWLibavVideoSource or FFMPEG it plays perfectly.

HW Decoding:


SW Decoding


The file was encoded using x265 3.6+13+1-726aec126:[Windows][GCC 14.1.0][64 bit] using the following settings:
Quote:
--colorprim bt2020 --transfer smpte2084 --colormatrix bt2020nc --master-display "G(8500,39850)B(6550,2300)R(35400,14600)WP(15635,16450)L(40000000,50)" --crf 19 --fps 24000/1001 --sar 1:1 --profile main10 --output-depth 10 --preset slow --no-strong-intra-smoothing --no-rect --aud --limit-modes --aq-mode 2 --hdr10 --no-opt-ref-list-length-pps --hdr10-opt --no-opt-qp-pps --limit-sao --selective-sao 2 --sao-non-deblock --chromaloc 2 --keyint 60 --min-keyint 6 --log2-max-poc-lsb 8 --dynamic-refine --rskip 2 --rskip-edge-threshold 3 --rdoq-level 2 --rc-lookahead 24 --level-idc 5 --psy-rd 2.00 --psy-rdoq 1.00 --vbv-maxrate 54000 --vbv-bufsize 46000 --deblock -1 --nr-intra 0 --nr-inter 0 --no-info --ctu 64 --merange 57 --y4m
If I use a older 3.5 build the resulting file plays perfectly on both HW and SW decoding:

https://1drv.ms/v/s!AiCgDguh7E_Ogc4N...FjxTg?e=5OSEVM
SquallMX is offline   Reply With Quote
Old 27th August 2024, 22:51   #2  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,878
Huh.

I'd start by stripping out a lot of the less essential parameters and see if that changes anything. If so, you can mix and match to see what is doing it.

There's a lot of oddball stuff in there that could cause an edge case but isn't going to make any material difference at real-world bitrates. Like --no-opt-ref-list-length-pps, -log2-max-poc-lsb 8. And stuff like --no-strong-intra-smoothing, --deblock -1, -limit-sao, --sao-non-deblock I'm unaware of ever helping real-world content that I recall, while degrading compression efficiency in general. --deblock also gets two parameters; I'm not sure what happens if only one is used as in here. Lowering it was a good idea in x264, but HEVC's design is different, and the default is generally quite optimal.

--chromaloc 2 is only appropriate if your source and/or output is Blu-ray, or is otherwise known to use --chromaloc 2.

That said, I've seen some odd stuff with --rskip 2 lately; you might try with --rskip 0 or 1 and see if that helps.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 28th August 2024, 15:31   #3  |  Link
SquallMX
Special SeeD
 
Join Date: Nov 2002
Location: Mexico
Posts: 340
Quote:
Originally Posted by benwaggoner View Post
That said, I've seen some odd stuff with --rskip 2 lately; you might try with --rskip 0 or 1 and see if that helps.
You were right, culprit was --rskip 2, using 1 or 0 creates a file that plays properly on HW decoding, unfortunately this decreases encoding speed by 50%, so I will continue to use the old 3.5 build for the time being.

Quote:
-limit-sao, --sao-non-deblock
Unfortunately disabling SAO is the only way to get decent quality on some grainy content.
SquallMX is offline   Reply With Quote
Old 28th August 2024, 21:10   #4  |  Link
Hellboy.
Registered User
 
Join Date: Mar 2020
Posts: 53
Maybe you can try with the official 3.6
I use Handbrake so i don't know much about x265 but i think this is the official page
https://bitbucket.org/multicoreware/x265_git/downloads/
Hellboy. is offline   Reply With Quote
Old 29th August 2024, 09:46   #5  |  Link
excellentswordfight
Lost my old account :(
 
Join Date: Jul 2017
Posts: 339
Quote:
Originally Posted by SquallMX View Post
Unfortunately disabling SAO is the only way to get decent quality on some grainy content.
Well, just use --no-sao instead of --limit-sao --selective-sao 2 --sao-non-deblock --no-strong-intra-smoothing

And do not disable strong-intra-smoothing, it might sound bad, but I have never seen it affect any perceivable difference in details or sharpness, but disabling it creates issues with blocking. As can be seen here it can even result in more bluring with disabling it. And why limit-sao? If you wanna keep some sao (selective-sao 2), doesnt that just make the SAO filter worse (I thought this limits the analysis of the filter, and not the "effect" per se of SAO)?

And just a note on --rskip 2, I was going a to create a thread on this a long time ago, but when I found this setting I was also impressed with the speed increase. But I found that it had an odd side-effect were it would create pixel-shifting. So if you toggled stills back and forth (input->output) you could see that some "pixels" was not in the same positions as the input, so it "moved" things, like strands of hair etc. So I stopped using it. Anyone else noticed this?

Last edited by excellentswordfight; 29th August 2024 at 10:03.
excellentswordfight is offline   Reply With Quote
Old 29th August 2024, 14:37   #6  |  Link
SquallMX
Special SeeD
 
Join Date: Nov 2002
Location: Mexico
Posts: 340
Quote:
Originally Posted by Hellboy. View Post
Maybe you can try with the official 3.6
I use Handbrake so i don't know much about x265 but i think this is the official page
https://bitbucket.org/multicoreware/x265_git/downloads/
Thanks, I will try the official build.

Quote:
Originally Posted by excellentswordfight View Post
Well, just use --no-sao instead of --limit-sao --selective-sao 2 --sao-non-deblock --no-strong-intra-smoothing

And do not disable strong-intra-smoothing, it might sound bad, but I have never seen it affect any perceivable difference in details or sharpness, but disabling it creates issues with blocking. As can be seen here it can even result in more bluring with disabling it. And why limit-sao? If you wanna keep some sao (selective-sao 2), doesnt that just make the SAO filter worse (I thought this limits the analysis of the filter, and not the "effect" per se of SAO)?
Yes, for grainy content I use --no-sao and --aq 3, some movies like Bill and Ted 2 looks like sh** otherwise. But for more mainstream movies --limit-sao looks better to me while keeping bitrates low, obviously YMMV.
SquallMX is offline   Reply With Quote
Old 29th August 2024, 20:24   #7  |  Link
Asmodian
Registered User
 
Join Date: Feb 2002
Location: San Jose, California
Posts: 4,425
Quote:
Originally Posted by SquallMX View Post
But for more mainstream movies --limit-sao looks better to me while keeping bitrates low, obviously YMMV.
But --limit-sao is a speed v.s. quality tradeoff, it is not limiting the effect of SAO (it is faster, but lower quality).

Do you really like the look of --limit-sao more than the default SAO analysis?
__________________
madVR options explained
Asmodian is offline   Reply With Quote
Old 30th August 2024, 14:52   #8  |  Link
SquallMX
Special SeeD
 
Join Date: Nov 2002
Location: Mexico
Posts: 340
Quote:
Originally Posted by Asmodian View Post
But --limit-sao is a speed v.s. quality tradeoff, it is not limiting the effect of SAO (it is faster, but lower quality).

Do you really like the look of --limit-sao more than the default SAO analysis?
Thats a good question, last time I made a comparative --limit-sao vs --sao, I preferred the --limit-sao encoding, but I guess it can vary from video to video.
SquallMX is offline   Reply With Quote
Old 1st September 2024, 00:07   #9  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,878
Quote:
Originally Posted by SquallMX View Post
Thats a good question, last time I made a comparative --limit-sao vs --sao, I preferred the --limit-sao encoding, but I guess it can vary from video to video.
--limit-sao is just a speed-over-quality optimization for --sao. Are you sure you weren't comparing to --no-sao?
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 2nd September 2024, 21:07   #10  |  Link
hello_hello
Registered User
 
Join Date: Mar 2011
Posts: 4,902
Quote:
Originally Posted by excellentswordfight View Post
And do not disable strong-intra-smoothing, it might sound bad, but I have never seen it affect any perceivable difference in details or sharpness, but disabling it creates issues with blocking?
I do so like to argue.

I rarely encode anything that hasn't been run through GradFun3 or one of it's variants at the end of a script, so maybe that's why I'm not having banding issues, but --no-strong-intra-smoothing seemed to make a difference. Unless for the example below --no-sao made the difference and I'm mistaken, but I'm pretty sure I tested those two options independently at some stage.
Maybe liking --no-strong-intra-smoothing comes down to personal taste, and maybe it depends on other settings, but here's the only results I still have from my initial testing. I've probably fiddled with some of the settings for my own encodes since (I no longer include --no-rect), but....

The source is included for completeness even though the video was lightly denoised and dithered on the way through, so none of the encodes would be expected to look exactly like the original. I also ran the video by x264, just because....

Source:
https://gifyu.com/image/SiQqW

Encode A: 1756 kb/s
https://gifyu.com/image/SiQqg
x265 --crf 20.0 --profile main10 --preset slow --deblock=-1:-1 --no-open-gop --rc-lookahead 80 --lookahead-slices 0 --psy-rdoq 1.7 --aq-strength 1.3 --no-strong-intra-smoothing --no-rect --no-sao --selective-sao 0

Encode B: 2516 kb/s
https://gifyu.com/image/SiQqu
x264 --crf 20.0 --level 4.1 --preset slow --tune film --b-adapt 2 --me umh --subme 9 --partitions all --no-fast-pskip --stitchable

Encode C: 1605 kb/s
https://gifyu.com/image/SiQqc
x265 --crf 20.0 --profile main10 --preset slow --deblock=-1:-1 --no-open-gop --rc-lookahead 80 --lookahead-slices 0 --psy-rdoq 1.7 --aq-strength 1.3

Encode D: 1753 kb/s
https://gifyu.com/image/SiQqQ
x265 --crf 18.0 --profile main10 --preset slow

The encodes above are ranked according to how closely they resembled the source to me, obsessing over the background in this case, as impossible as it was for them to match the source.... my logic at the time was to look at the results based on the filtering I'd normally use rather than something different.

Last edited by hello_hello; 2nd September 2024 at 21:36.
hello_hello is offline   Reply With Quote
Old 5th September 2024, 08:38   #11  |  Link
excellentswordfight
Lost my old account :(
 
Join Date: Jul 2017
Posts: 339
Quote:
Originally Posted by hello_hello View Post
I do so like to argue.

I rarely encode anything that hasn't been run through GradFun3 or one of it's variants at the end of a script, so maybe that's why I'm not having banding issues, but --no-strong-intra-smoothing seemed to make a difference. Unless for the example below --no-sao made the difference and I'm mistaken, but I'm pretty sure I tested those two options independently at some stage.
It would be better, that instead of arguing, show examples were the only difference in settings is turning off strong intra smoothing, and thereby also validating your claim. Cause that you maybe tried it isnt really much of an argument I think no one that has used x265 and used no-sao can argue against that there are plenty of cases were its beneficial to turn off SAO, and that turning it off definitely has a impact on the image.

Personally I use no-sao for most of my encodes, selective-sao 1 or 2 for cartoon and anime, and full sao for low-bitrate/trytocompressasmuchaspossible stuff.

Last edited by excellentswordfight; 5th September 2024 at 08:47.
excellentswordfight is offline   Reply With Quote
Old 6th September 2024, 20:33   #12  |  Link
tormento
Acid fr0g
 
tormento's Avatar
 
Join Date: May 2002
Location: Italy
Posts: 2,721
Give a try to VMAF too.

Of course, you don't have to apply filters to compare apples to apples.
__________________
@turment on Telegram
tormento is offline   Reply With Quote
Old 9th September 2024, 17:02   #13  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,878
Quote:
Originally Posted by excellentswordfight View Post
Personally I use no-sao for most of my encodes, selective-sao 1 or 2 for cartoon and anime, and full sao for low-bitrate/trytocompressasmuchaspossible stuff.
If you are using SAO, --selective-sao 2 is pretty much identical to full SAO in quality, but is a bit faster (more of a benefit with faster presets). SAO is really designed for forward prediction only, so it really only does anything with I and P frames. --selective-sao 2 really should be the default, and I imagine it would be if MCW ever updates the presets.

That said, x265 3.6 has support for more layers of B-frames, and I've not experimented to see if that makes --selective-sao 3 offer any marginal benefit now. SAO won't ever do anything for non-reference b frames, which are the majority of frames in most distribution encodes.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 9th September 2024, 17:30   #14  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,878
Quote:
Originally Posted by tormento View Post
Give a try to VMAF too.

Of course, you don't have to apply filters to compare apples to apples.
VMAF is definitely our least-bad generally available full reference metric.

Still, VMAF is luma only, so won't catch chroma issues and is not great at comparing different adaptive quantization methods

And as a per-frame metric, it is up to people to figure out how to aggregate those metrics to estimate overall quality. Saying "mean VMAF of 80" for a two hour movie could indicate a movie that is consistently mediocre, or one that oscillates between pristine and terrible quality.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 9th September 2024, 18:21   #15  |  Link
tormento
Acid fr0g
 
tormento's Avatar
 
Join Date: May 2002
Location: Italy
Posts: 2,721
Quote:
Originally Posted by benwaggoner View Post
VMAF is definitely our least-bad generally available full reference metric.

Still, VMAF is luma only, so won't catch chroma issues and is not great at comparing different adaptive quantization methods

And as a per-frame metric, it is up to people to figure out how to aggregate those metrics to estimate overall quality. Saying "mean VMAF of 80" for a two hour movie could indicate a movie that is consistently mediocre, or one that oscillates between pristine and terrible quality.

That’s why I use the output graphs, feeding source as primary stream, not numbers.
__________________
@turment on Telegram
tormento is offline   Reply With Quote
Old 18th September 2024, 10:13   #16  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,878
Quote:
Originally Posted by tormento View Post
That’s why I use the output graphs, feeding source as primary stream, not numbers.
Yeah, that is a much better approach. I've yet to find one I find truly satisfying. More R&D is needed.

Harmonic mean of scores in a GOP seems less bad than others.

Choice of a metric is also a challenge. VMAF was a big step forward, but still doesn't have that high a correlation with subjective quality ratings.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 18th September 2024, 13:08   #17  |  Link
excellentswordfight
Lost my old account :(
 
Join Date: Jul 2017
Posts: 339
Quote:
Originally Posted by benwaggoner View Post
And as a per-frame metric, it is up to people to figure out how to aggregate those metrics to estimate overall quality. Saying "mean VMAF of 80" for a two hour movie could indicate a movie that is consistently mediocre, or one that oscillates between pristine and terrible quality.
Quote:
Originally Posted by tormento View Post
That’s why I use the output graphs, feeding source as primary stream, not numbers.
Yes, I agree, output it as a graph is necessity!

For people who dont really understand how big of an issue this is here is an example:

In a test AMD:s VCN encoder recieved a VMAF of 95.39 and Nvidias Nvenc 96.13, so one would think that they are pretty much the same, right? Well the graph tells a completely different story:



That an encoder can dip that much on big bunny is one thing , but the dips really demonstrate the flaw of trying to simple down quality to one number. Its a bit like the percentiles discussion that happened in gaming benchmark a few years ago, were now it standard to look at 1% and 5% lows aswell as the avrage.

Last edited by excellentswordfight; 18th September 2024 at 13:14.
excellentswordfight is offline   Reply With Quote
Old 23rd September 2024, 19:27   #18  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,878
Quote:
Originally Posted by excellentswordfight View Post
Yes, I agree, output it as a graph is necessity!

For people who dont really understand how big of an issue this is here is an example:
Wow, that is a great example. Our sense of how good a video looks is much more about how good the worst parts of the video looked than the average quality.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 06:36.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.