Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > High Efficiency Video Coding (HEVC)

Reply
 
Thread Tools Search this Thread Display Modes
Old 4th February 2019, 09:21   #6701  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 5,929
Quote:
Should I use --dhdr10-opt too???
should be optional and do no harm, so you can probably use it without running into issues.

Quote:
can i crop?
I guess the HDR-10+ data would need to be adjusted, since at least the 'AverageRGB' should change when applying most filters.
But since you got the equipment, try and report back.
__________________
Hybrid here in the forum, homepage
Notice: Since email notifications do not work here any more, it might take me quite some time to notice a reply to a thread,..
Selur is offline   Reply With Quote
Old 4th February 2019, 17:57   #6702  |  Link
Barough
Registered User
 
Barough's Avatar
 
Join Date: Feb 2007
Location: Sweden
Posts: 341
x265 v3.0_RC+14-46b84ff665fd (32 & 64-bit 8/10/12bit Multilib Windows Binaries) (32bit : GCC 7.4.0 / 64bit : GCC 8.2.1)

Code:
https://bitbucket.org/multicoreware/x265/commits/branch/default
NOTE :
Checked with Pradeep (@MulticoreWare) about why the Default Branch haven't been pushed to v3.0 'Stable' and this is the reply/info i got

"
Our plan is to continue to use 3.0_RC on the default branch and have completed tags only on the stable branch. So we don't intend to merge back.
"
Barough is offline   Reply With Quote
Old 6th February 2019, 12:43   #6703  |  Link
sonnati
Registered User
 
Join Date: Jun 2008
Posts: 19
Quote:
Originally Posted by katzenjoghurt View Post
Oh my.
Am I the only one having such a hard time encoding scenes with red light / red backgrounds?

E.g. if a character turns on a red light, his face would turn suddenly totally blocky.
Blue light is fine, green light also seems to be a bit bad, but red light is the devil.
Looks like x265 (also x264 I think) detects the scene as super-dark and reduces
the bitrate like crazy.

I doubt that it's just a display thing as I can see the problem on my Dell display,
my Benq display and my Samsung TV.

By now I fix it by scanning every source for red scenes before encoding and setting
zones like crazy via --zones startframe,endframe,b=1.5/startframe,endfr....
Super-tedious.

I was shocked again today after I checked my encoding of Disney's Aladdin...
Red sand with black dots -> blurred to unshaded flat areas.
Red stone wall backgrounds -> bluuuurr.

Looks like I need to double or triple the bitrate manually in these scenes just to keep
the subjectively visible detail level compared to the non-reddish scenes.

AQ3 won't help all too much either. The overall bitrate would get just too high if
I want to retain the details that way. *sigh*
HVS is more sensible to yellow-red chrominance (>>550nm)
than violet-blue (<<500nm). Red has always been a mess in encoding, especially when encoding in yuv420 where chroma is subsampled and red pictures appear in their glorious blockiness.

There's still a lot of work to be done to optimize encoders to accomodate for perception: we should improve quantization both in dark picture and red-dominant picture, but this depends also on the viewing conditions (i.e. not so important in small mobile screens)

Not to mention higher order of complexity optimizations like saliency based optimizations or similar.

I have developed many times models that change params using zones-like params to improve the performance of a given encoder. This is one of those cases.

Last edited by sonnati; 6th February 2019 at 12:55.
sonnati is offline   Reply With Quote
Old 6th February 2019, 18:16   #6704  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 5,983
x265 3.0 RC+14-46b84ff665fd (MSYS2, MinGW32 + GCC 7.4.0 / MinGW64 + GCC 8.2.1)

__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid

Last edited by LigH; 14th February 2019 at 14:20.
LigH is offline   Reply With Quote
Old 6th February 2019, 18:35   #6705  |  Link
birdie
.
 
birdie's Avatar
 
Join Date: Dec 2006
Posts: 147
Version 3.0
===========

Release date - 23/01/2019

New features
-------------
1. option:: '--dolby-vision-profile <integer|float>' generates bitstreams confirming to the specified Dolby Vision profile. Currently profile 5, profile 8.1 and profile 8.2 enabled, Default 0 (disabled)

2. option:: '--dolby-vision-rpu' File containing Dolby Vision RPU metadata. If given, x265's Dolby Vision metadata parser will fill the RPU field of input pictures with the metadata
read from the file. The library will interleave access units with RPUs in the bitstream. Default NULL (disabled).

3. option:: '--zonefile <filename>' specifies a text file which contains the boundaries of the zones where each of zones are configurable.

4. option:: '--qp-adaptation-range' Delta-QP range by QP adaptation based on a psycho-visual model. Default 1.0.

5. option:: '--refine-ctu-distortion <0/1>' store/normalize ctu distortion in analysis-save/load. Default 0.

6. Experimental feature option:: '--hevc-aq' enables adaptive quantization
It scales the quantization step size according to the spatial activity of one coding unit relative to frame average spatial activity. This AQ method utilizes
the minimum variance of sub-unit in each coding unit to represent the coding unit’s spatial complexity.

Encoder enhancements
--------------------
1. Preset: change param defaults for veryslow and slower preset. Replace slower preset with defaults used in veryslow preset and change param defaults in veryslow preset as per experimental results.
2. AQ: change default AQ mode to auto-variance
3. Cutree offset reuse: restricted to analysis reuse-level 10 for analysis-save -> analysis-load
4. Tune: introduce --tune animation option which improves encode quality for animated content
5. Reuse CU depth for B frame and allow I, P frame to follow x265 depth decision

Bug fixes
---------
1. RC: fix rowStat computation in const-vbv
2. Dynamic-refine: fix memory reset size.
3. Fix Issue #442: linking issue on non x86 platform
4. Encoder: Do not include CLL SEI message if empty
5. Fix issue #441 build error in VMAF lib
birdie is offline   Reply With Quote
Old 7th February 2019, 08:47   #6706  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 5,983
Hooray, I should have waited one more day...
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Old 7th February 2019, 08:52   #6707  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 5,929
@LigH: Why? Last code change was from 2019-01-25,... 3.0 release was out there since 2019-01-23 -> https://bitbucket.org/multicoreware/...f10364bb232c2c
__________________
Hybrid here in the forum, homepage
Notice: Since email notifications do not work here any more, it might take me quite some time to notice a reply to a thread,..
Selur is offline   Reply With Quote
Old 7th February 2019, 09:08   #6708  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 5,983
So if there is a v3.0 release since 2 weeks, why do I still receive an RC version ... did I not update to the "tip"? Usually I do...

Ah, there was no back merge. Stable and default branch are still distinct.
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Old 7th February 2019, 10:28   #6709  |  Link
jlpsvk
Registered User
 
Join Date: Dec 2014
Posts: 202
Quote:
Originally Posted by LigH View Post
So if there is a v3.0 release since 2 weeks, why do I still receive an RC version ... did I not update to the "tip"? Usually I do...

Ah, there was no back merge. Stable and default branch are still distinct.
Barought wrote that few days ago:

Code:
Our plan is to continue to use 3.0_RC on the default branch and have completed tags only on the stable branch. So we don't intend to merge back.
So RC14 is newer than 3.0+1
__________________
AMD Ryzen 9 3950X, 32GB DDR4-3200 CL16, RTX 2070, 1TB NVMe SSD, 56TB NAS
jlpsvk is offline   Reply With Quote
Old 7th February 2019, 12:02   #6710  |  Link
pradeeprama
Registered User
 
Join Date: Sep 2015
Posts: 48
Quote:
Originally Posted by jlpsvk View Post
Barought wrote that few days ago:

Code:
Our plan is to continue to use 3.0_RC on the default branch and have completed tags only on the stable branch. So we don't intend to merge back.
So RC14 is newer than 3.0+1
We are starting to use 3.0_Au (Gold) to signal that the default branch has moved to gold state. We are just trying to avoid merging back from stable to default as it is good practice to avoid merges. Hope this isn't too confusing!
pradeeprama is offline   Reply With Quote
Old 7th February 2019, 19:38   #6711  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 3,042
Quote:
Originally Posted by sonnati View Post
HVS is more sensible to yellow-red chrominance (>>550nm)
than violet-blue (<<500nm). Red has always been a mess in encoding, especially when encoding in yuv420 where chroma is subsampled and red pictures appear in their glorious blockiness.

There's still a lot of work to be done to optimize encoders to accomodate for perception: we should improve quantization both in dark picture and red-dominant picture, but this depends also on the viewing conditions (i.e. not so important in small mobile screens)

Not to mention higher order of complexity optimizations like saliency based optimizations or similar.

I have developed many times models that change params using zones-like params to improve the performance of a given encoder. This is one of those cases.
Yeah, very flat gradients in red and blue have always been hard (since luma is mainly green, green is pretty much non subsampled).

Using --crqpoffs -* would help red without spending more bits on blue. It makes a certain logical sense that, when decreasing chroma QP, it would make sense to reduce 2 red for every blue given the relatively higher sensitivity to red. However, red also accounts for a greater portion of luma than blue which likely would offset that some.

Encoding in 10-bit instead of 8-bit should help as well. Sometimes these problems can be as much in the 10-bit Y'CbCr -> 8-bit RGB conversion on a device than in the encoding itself. Display controllers that just truncate the 8-bit instead of dithering can cause all kinds of issues even in SDR. Dithering should be done even when doing a 16-235 Y'CbCr to 0-255 RGB conversion.


To humorously paraphrase Gen. Robert H. Barrow, USMC

"Hobbyists talk about average bitrates and per-title metrics, but experts study color volume algorithms and rate control."
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 7th February 2019, 19:42   #6712  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 3,042
Quote:
Originally Posted by Selur View Post
should be optional and do no harm, so you can probably use it without running into issues.
All it does is remove duplicates of identical metadata; the majority of frames in a given title will be identical to the prior frame in display order. It'll still ensure that every IDR has its metadata.

This can save some bits (only material at very low bitrates), and potentially some work by the tone mapper. Since display<>decode order, you'll get cases where the tone mapper could find out in advance that several frames in a row have identical color characteristics.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 7th February 2019, 21:43   #6713  |  Link
katzenjoghurt
Registered User
 
Join Date: Feb 2007
Posts: 86
Quote:
Originally Posted by sonnati View Post
HVS is more sensible to yellow-red chrominance (>>550nm)
than violet-blue (<<500nm). Red has always been a mess in encoding, especially when encoding in yuv420 where chroma is subsampled and red pictures appear in their glorious blockiness.

There's still a lot of work to be done to optimize encoders to accomodate for perception: we should improve quantization both in dark picture and red-dominant picture, but this depends also on the viewing conditions (i.e. not so important in small mobile screens)

Not to mention higher order of complexity optimizations like saliency based optimizations or similar.

I have developed many times models that change params using zones-like params to improve the performance of a given encoder. This is one of those cases.
Thanks for the confirmation, sonnati.
I'll continue sticking to zones then or live with rising the overall bitrate beyond general need if the pain becomes too much.
katzenjoghurt is offline   Reply With Quote
Old 8th February 2019, 00:51   #6714  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 3,042
Quote:
Originally Posted by katzenjoghurt View Post
Thanks for the confirmation, sonnati.
I'll continue sticking to zones then or live with rising the overall bitrate beyond general need if the pain becomes too much.
And when you find examples that need zones, grab that snippet and open an Issue with MCW. An encoder should be able to detect when this would be an issue and then compensate for it.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 12th February 2019, 20:50   #6715  |  Link
redbtn
Registered User
 
redbtn's Avatar
 
Join Date: Jan 2019
Location: Russia
Posts: 135
--ctu --max-tu-size --qg-size tests

I've made some tests for 1080p with --tune-psnr, --tune-ssim, and no-tune for different variants --ctu --max-tu-size --qg-size

Does this mean that for 1080p encoding default settings (--ctu 64 --max-tu-size 32 --qg-size 32) is the best option?

Results:
Code:
no-tune
===
--ctu 64 --max-tu-size 32 --qg-size 32 Global PSNR: 55.009, SSIM Mean Y: 0.9965665 (24.643 dB) higher value
--ctu 32 --max-tu-size 32 --qg-size 16 Global PSNR: 54.982, SSIM Mean Y: 0.9965609 (24.636 dB)
--ctu 32 --max-tu-size 32 --qg-size 8  Global PSNR: 54.866, SSIM Mean Y: 0.9964265 (24.469 dB)
--ctu 32 --max-tu-size 16 --qg-size 16 Global PSNR: 54.908, SSIM Mean Y: 0.9964738 (24.527 dB)
--ctu 32 --max-tu-size 16 --qg-size 8  Global PSNR: 54.791, SSIM Mean Y: 0.9963344 (24.359 dB)
===

tune-psnr
===
--ctu 64 --max-tu-size 32 --qg-size 32 Global PSNR: 54.936 higher value
--ctu 32 --max-tu-size 32 --qg-size 16 Global PSNR: 54.910
--ctu 32 --max-tu-size 32 --qg-size 8  Global PSNR: 54.806
--ctu 32 --max-tu-size 16 --qg-size 16 Global PSNR: 54.869
--ctu 32 --max-tu-size 16 --qg-size 8  Global PSNR: 54.765
===

tune-ssim
===
--ctu 64 --max-tu-size 32 --qg-size 32 SSIM Mean Y: 0.9964914 (24.549 dB) higher value
--ctu 32 --max-tu-size 32 --qg-size 16 SSIM Mean Y: 0.9964839 (24.539 dB)
--ctu 32 --max-tu-size 32 --qg-size 8  SSIM Mean Y: 0.9963653 (24.395 dB)
--ctu 32 --max-tu-size 16 --qg-size 16 SSIM Mean Y: 0.9964291 (24.472 dB)
--ctu 32 --max-tu-size 16 --qg-size 8  SSIM Mean Y: 0.9963079 (24.327 dB)
Attached Files
File Type: txt ctu-logs-no-tune.txt (20.0 KB, 2 views)
File Type: txt ctu-logs-tune-psnr.txt (19.2 KB, 3 views)
File Type: txt ctu-logs-tune-ssim.txt (18.3 KB, 1 views)
redbtn is offline   Reply With Quote
Old 12th February 2019, 20:52   #6716  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 5,929
Quote:
Does this mean that for 1080p encoding default settings (--ctu 64 --max-tu-size 32 --qg-size 32) is the best option?
to state the obvious: only if your quality perception aligns with PSM and SSIM Mean,..
__________________
Hybrid here in the forum, homepage
Notice: Since email notifications do not work here any more, it might take me quite some time to notice a reply to a thread,..
Selur is offline   Reply With Quote
Old 12th February 2019, 21:05   #6717  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 3,042
Quote:
Originally Posted by Selur View Post
to state the obvious: only if your quality perception aligns with PSM and SSIM Mean,..
The modern codec design process biases heavily for fixed QP maximizing mean PSNR, so adaptive quant and psychovisual optimizations are generally only helpful when targeting subjective quality. Which is all viewers care about. Iíve seen lots of beautiful PSNR plots turn into lousy looking video.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 12th February 2019, 21:15   #6718  |  Link
redbtn
Registered User
 
redbtn's Avatar
 
Join Date: Jan 2019
Location: Russia
Posts: 135
Quote:
Originally Posted by Selur View Post
to state the obvious: only if your quality perception aligns with PSM and SSIM Mean,..
Quote:
Originally Posted by benwaggoner View Post
The modern codec design process biases heavily for fixed QP maximizing mean PSNR, so adaptive quant and psychovisual optimizations are generally only helpful when targeting subjective quality. Which is all viewers care about. I’ve seen lots of beautiful PSNR plots turn into lousy looking video.
I do not have enough knowledge yet to make difficult conclusions. I'm just trying to find the optimal settings for 4K HDR -> 1080p HDR.
Which option can you advise?

And one more question. Can I somehow optimize the settings for quality improvement without significantly reducing speed? Or will these settings be enough? My traget bitrate 20-25mbs

--preset placebo --crf 12 --profile main10 --output-depth 10 --level-idc 51 --sar 1:1 --colorprim 9 --colormatrix 9 --transfer 16 --range limited --master-display "G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(10000000,1)" --hdr --hdr-opt --chromaloc 2 --repeat-headers --hrd --max-cll "0,0" --min-luma 0 --max-luma 1023 --subme 7 --me star --merange 48 --limit-tu 4 --max-merge 4 --limit-modes --limit-refs 3 --rd 6 --rd-refine --no-tskip --rskip --no-cutree --no-sao --no-open-gop --no-b-pyramid --no-strong-intra-smoothing --vbv-bufsize 160000 --vbv-maxrate 160000 --cbqpoffs -2 --crqpoffs -2 --min-keyint 5 --keyint 240 --ipratio 1.30 --pbratio 1.20 --deblock -3:-3 --qcomp 0.65 --aq-mode 2 --aq-strength 0.8 --psy-rd 1.5 --psy-rdoq 5

Last edited by redbtn; 12th February 2019 at 21:32.
redbtn is offline   Reply With Quote
Old 13th February 2019, 16:08   #6719  |  Link
excellentswordfight
Lost my old account :(
 
Join Date: Jul 2017
Posts: 125
Quote:
Originally Posted by redbtn View Post
I do not have enough knowledge yet to make difficult conclusions. I'm just trying to find the optimal settings for 4K HDR -> 1080p HDR.
Which option can you advise?

And one more question. Can I somehow optimize the settings for quality improvement without significantly reducing speed? Or will these settings be enough? My traget bitrate 20-25mbs

--preset placebo --crf 12 --profile main10 --output-depth 10 --level-idc 51 --sar 1:1 --colorprim 9 --colormatrix 9 --transfer 16 --range limited --master-display "G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(10000000,1)" --hdr --hdr-opt --chromaloc 2 --repeat-headers --hrd --max-cll "0,0" --min-luma 0 --max-luma 1023 --subme 7 --me star --merange 48 --limit-tu 4 --max-merge 4 --limit-modes --limit-refs 3 --rd 6 --rd-refine --no-tskip --rskip --no-cutree --no-sao --no-open-gop --no-b-pyramid --no-strong-intra-smoothing --vbv-bufsize 160000 --vbv-maxrate 160000 --cbqpoffs -2 --crqpoffs -2 --min-keyint 5 --keyint 240 --ipratio 1.30 --pbratio 1.20 --deblock -3:-3 --qcomp 0.65 --aq-mode 2 --aq-strength 0.8 --psy-rd 1.5 --psy-rdoq 5
Imo, the bitrate you are targeting is so high that most of your settings are way overkill. You will spend alot of cpu cycles for virtually no quality gain, you will be well beyond the dimishing return point.

Have you actually tried those settings? It wouldn't surprise me that placebeo crf12 could render a file bigger then the original 4k one, and you will be lucky if you do it above 1fps. I'm sure you have your reasons for this scenario, but if quality was that important to me, I would just go with the original.

Last edited by excellentswordfight; 13th February 2019 at 16:22.
excellentswordfight is offline   Reply With Quote
Old 13th February 2019, 16:54   #6720  |  Link
redbtn
Registered User
 
redbtn's Avatar
 
Join Date: Jan 2019
Location: Russia
Posts: 135
Quote:
Originally Posted by excellentswordfight View Post
Imo, the bitrate you are targeting is so high that most of your settings are way overkill. You will spend alot of cpu cycles for virtually no quality gain, you will be well beyond the dimishing return point.

Have you actually tried those settings? It wouldn't surprise me that placebeo crf12 could render a file bigger then the original 4k one, and you will be lucky if you do it above 1fps. I'm sure you have your reasons for this scenario, but if quality was that important to me, I would just go with the original.
I tried this settings only without (--rd 6 --rd-refine --opt-cu-delta-qp), with (--rd 3 --no-rskip). For example for file bitrate 57 Mbs size 45 GB output bitrate 26 Mb/s size 21 GB with 1.4fps encode speed. I think with new settings it's be around 0.9-1fps on my 6 core i5 8400.
I view at 4K UHD TV, but it is connected to a computer as duplicated with a monitor with a resolution of 1080p because windows 10 cannot normally scale many applications in 4K (it's sadly). Therefore, I decided that there is no point in reducing the image via MadVR in real time. Although I have a GTX 1080 which does it without problems. And also save about 50 percent of the HDD space.
redbtn is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 22:37.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, vBulletin Solutions Inc.