Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > High Efficiency Video Coding (HEVC)

Reply
 
Thread Tools Search this Thread Display Modes
Old 9th December 2023, 06:43   #1  |  Link
aletaziar
Registered User
 
Join Date: Dec 2023
Posts: 4
Advice on optimizing encode for grainy animation [sample provided]

Looking for some help optimizing an encode with ffmpeg/x265. It's a 4k animation that has grain and HDR. I'm trying to find the right flags for a transparent encode at a reasonable size but it's proving difficult.

The test encodes I've made have *mostly* worked, but at 00:10.427 (frame 249, 23.976) in the sample scene provided, the left figure has pink blotches in the waist sash in the encodes. The right figure has blockier skin in the encodes, especially in the chest/shoulder and around the nose.

These issues don't seem noticeable when played in MPC-BE via MadVR, but they do show up when I compare the encode with the original using video-compare, which uses ffplay without HDR. Is MadVR hiding these issues with the HDR tone-mapping? Maybe ffplay is kind of "inventing" issues because of the lack of HDR tone-mapping? I would think the blockiness I see is an issue regardless of whether the colors are accurate in ffplay.

I'm trying to reduce both those issues but I don't want to drop my crf anymore since I'm already at 12. Any suggestions would be helpful, this is my first foray into HEVC encoding. I've been doing a lot of reading but figuring this out is proving difficult. The three encode variations I tried all look pretty much the same to me.

samples:
Code:
one 15s original quality sample
three 15s encode variations
https://app.box.com/s/6atm9dfchj1nzjzhqaxjusaxchqcqz34
video-compare command, with padding for encode:
Code:
video-compare -w x980 -r pad=iw:ih+80:0:80/2 test.mkv test.x265_crf12-nosao-aqmode4.mkv
video-compare command, with cropping for original:
Code:
video-compare -w x980 -l crop=iw:ih-80 test.mkv test.x265_crf12-nosao-aqmode4.mkv
encode base:
Code:
ff -v level+panic -i test.mkv -max_muxing_queue_size 8192 \
-map 0:v -vf "crop=iw:ih-80" -pix_fmt yuv420p10le -f yuv4mpegpipe \
-strict -1 - | x265 - --y4m --profile main10 --preset slower --crf 12 \
--vbv-bufsize=160000 --vbv-maxrate=160000 --colorprim "bt2020" \
--transfer "smpte2084" --colormatrix "bt2020nc" --range "limited" \
--chromaloc=2 --master-display \
"G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(10000000,50)" \
--max-cll "1000,140" --hdr10 --hdr10-opt --repeat-headers --aud \
--hrd --no-strong-intra-smoothing --limit-refs 3 --rd 4 --dynamic-rd 3
crf12:
Code:
--selective-sao 2 test.x265_crf12.hevc
crf12, aqmode=4:
Code:
--selective-sao 2 --aq-mode 4 test.x265_crf12-aqmode4.hevc
crf12, aqmode=4, no-sao:
Code:
--no-sao --aq-mode 4 test.x265_crf12-aqmode4-nosao.hevc
test.mkv mediainfo:
Code:
General
Unique ID                                : 333647730702598027554471434130870826641 (0xFB02372DDF2EA7B8816759A9A0D4F691)
Complete name                            : R:\test.mkv
Format                                   : Matroska
Format version                           : Version 4
File size                                : 110 MiB
Duration                                 : 15 s 683 ms
Overall bit rate                         : 58.6 Mb/s
Frame rate                               : 23.720 FPS
Encoded date                             : 2023-12-09 05:01:16 UTC
Writing application                      : mkvmerge v81.0 ('Milliontown') 64-bit
Writing library                          : libebml v1.4.4 + libmatroska v1.7.1

Video
ID                                       : 1
Format                                   : HEVC
Format/Info                              : High Efficiency Video Coding
Format profile                           : Main 10@L5.1@High
HDR format                               : SMPTE ST 2086, HDR10 compatible
Codec ID                                 : V_MPEGH/ISO/HEVC
Duration                                 : 15 s 683 ms
Bit rate                                 : 58.6 Mb/s
Width                                    : 3 840 pixels
Height                                   : 2 160 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Variable
Frame rate                               : 23.720 FPS
Original frame rate                      : 23.976 (24000/1001) FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0 (Type 2)
Bit depth                                : 10 bits
Bits/(Pixel*Frame)                       : 0.298
Time code of first frame                 : 01:13:14:22
Stream size                              : 110 MiB (100%)
Title                                    : MPEG-H HEVC Video / 63607 kbps / 2160p / 23.976 fps / 16:9 / Main 10 @ Level 5.1 @ High / 10 bits / HDR10 / BT.2020
Writing library                          : ATEME Titan File 3.9.6 (4.9.6.2)        
Language                                 : zxx
Default                                  : Yes
Forced                                   : No
Color range                              : Limited
Color primaries                          : BT.2020
Transfer characteristics                 : PQ
Matrix coefficients                      : BT.2020 non-constant
Mastering display color primaries        : Display P3
Mastering display luminance              : min: 0.0050 cd/m2, max: 1000 cd/m2
Maximum Content Light Level              : 1000 cd/m2
Maximum Frame-Average Light Level        : 140 cd/m2
aletaziar is offline   Reply With Quote
Old 12th December 2023, 01:55   #2  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,824
Reducing your chroma qp offsets to -1 can help with the sort of color blotching you describe.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 5th January 2024, 21:07   #3  |  Link
aletaziar
Registered User
 
Join Date: Dec 2023
Posts: 4
Anyone? Here are a couple of screenshots to show the pink blotches in the waist sash I'm seeing:

slowpoke compare: https://slow.pics/c/g4qzZQm2

original:


encode:
aletaziar is offline   Reply With Quote
Old 5th January 2024, 21:34   #4  |  Link
rwill
Registered User
 
Join Date: Dec 2013
Location: Berlin, Germany
Posts: 378
Quote:
Originally Posted by aletaziar View Post
Anyone?
All I can suggest is to raise CRF a little to lower rate first and then use --cbqpoffs and --crqpoffs with both set to -4 or -6. Some BT.2020 HDR sequences need that if the encoder has no concept of chroma complexity.
rwill is offline   Reply With Quote
Old 9th January 2024, 20:07   #5  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,824
I'd remove --chromaloc=2 unless you know your source is that.

Also remove --no-strong-intra-smoothing unless you have some evidence it is helpful for this content. It generally isn't

Remove --dynamic-rd until you have quality tuned in. Changing --rd during the encode could cause some weird results, and it probably isn't doing anything given your very high VBV

Are you seeing quality degradations at full-speed playback? Those subtle differences may not be perceptible. Presenting HDR as non-HDR can also show "problems" not perceivable in HDR. Playing back at speed on a HDR display is really the only way to know what you're looking at.

As rwill said, try reducing --cbqpoffs and --crqpoffs. I'd start with -1 and incrementally increase. Too negative a value can starve luma of bitrate, and I'm not sure how well tuned and tested high values are.

Some --nr can help a lot with fine grain. --nr-inter low-pass filters out high temporal and spatial frequency noise. Since it doesn't impact intra frames/blocks, the texture of grain is generally preserved but it doesn't swirl around as much and is easier to encode. Something like --nr-intra 100 --nr-inter 400 could be a good starting point to tweak from.

I don't think --no-sao would help with content like this where the fine detail is almost entirely noise.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 2nd February 2024, 06:32   #6  |  Link
aletaziar
Registered User
 
Join Date: Dec 2023
Posts: 4
Quote:
Originally Posted by benwaggoner View Post
I'd remove --chromaloc=2 unless you know your source is that.

Also remove --no-strong-intra-smoothing unless you have some evidence it is helpful for this content. It generally isn't

Remove --dynamic-rd until you have quality tuned in. Changing --rd during the encode could cause some weird results, and it probably isn't doing anything given your very high VBV

Are you seeing quality degradations at full-speed playback? Those subtle differences may not be perceptible. Presenting HDR as non-HDR can also show "problems" not perceivable in HDR. Playing back at speed on a HDR display is really the only way to know what you're looking at.

As rwill said, try reducing --cbqpoffs and --crqpoffs. I'd start with -1 and incrementally increase. Too negative a value can starve luma of bitrate, and I'm not sure how well tuned and tested high values are.

Some --nr can help a lot with fine grain. --nr-inter low-pass filters out high temporal and spatial frequency noise. Since it doesn't impact intra frames/blocks, the texture of grain is generally preserved but it doesn't swirl around as much and is easier to encode. Something like --nr-intra 100 --nr-inter 400 could be a good starting point to tweak from.

I don't think --no-sao would help with content like this where the fine detail is almost entirely noise.
The source in MediaInfo shows Chroma subsampling : 4:2:0 (Type 2) so I think I do need chromaloc for this particular file. I will check out the other settings you and rwill mentioned, and avoid --no-sao and --no-strong-intra-smoothing.

Is --aq-mode 4 something I should leave in as well?
aletaziar is offline   Reply With Quote
Old 3rd February 2024, 01:29   #7  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,824
Quote:
Originally Posted by aletaziar View Post
The source in MediaInfo shows Chroma subsampling : 4:2:0 (Type 2) so I think I do need chromaloc for this particular file.
Yeah, if the source has it at 2, you'll want to keep that going forward.

Quote:
I will check out the other settings you and rwill mentioned, and avoid --no-sao and --no-strong-intra-smoothing.

Is --aq-mode 4 something I should leave in as well?
--aq-mode 4 is generally fine. However, grainy animation is an unusual optimization target. I imagine you'll have to play around some with --psy-rd, --psy-rdoq, and --aq-strength to find optimal values. Generally they go higher with grain and lower with animation, so I don't have clear intuition about what might work. I suggest --rskip 2 and maybe --rskip-threshold 3 to start with. The default --rskip can have issues with grain, particularly in HDR.

You could also try --aq-mode 2 if 4 isn't working. 1 and 3 are unlikely to be helpful.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 00:40.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.