Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > High Efficiency Video Coding (HEVC)

Reply
 
Thread Tools Search this Thread Display Modes
Old 7th February 2019, 09:08   #6701  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 6,753
So if there is a v3.0 release since 2 weeks, why do I still receive an RC version ... did I not update to the "tip"? Usually I do...

Ah, there was no back merge. Stable and default branch are still distinct.
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Old 7th February 2019, 10:28   #6702  |  Link
jlpsvk
Registered User
 
Join Date: Dec 2014
Posts: 240
Quote:
Originally Posted by LigH View Post
So if there is a v3.0 release since 2 weeks, why do I still receive an RC version ... did I not update to the "tip"? Usually I do...

Ah, there was no back merge. Stable and default branch are still distinct.
Barought wrote that few days ago:

Code:
Our plan is to continue to use 3.0_RC on the default branch and have completed tags only on the stable branch. So we don't intend to merge back.
So RC14 is newer than 3.0+1
__________________
AMD Ryzen 9 5950X, 32GB DDR4-3200 CL16, RTX 3060, 2TB NVMe PCIE4.0, NAS with 8x16TB HDD
jlpsvk is offline   Reply With Quote
Old 7th February 2019, 12:02   #6703  |  Link
pradeeprama
Registered User
 
Join Date: Sep 2015
Posts: 48
Quote:
Originally Posted by jlpsvk View Post
Barought wrote that few days ago:

Code:
Our plan is to continue to use 3.0_RC on the default branch and have completed tags only on the stable branch. So we don't intend to merge back.
So RC14 is newer than 3.0+1
We are starting to use 3.0_Au (Gold) to signal that the default branch has moved to gold state. We are just trying to avoid merging back from stable to default as it is good practice to avoid merges. Hope this isn't too confusing!
pradeeprama is offline   Reply With Quote
Old 7th February 2019, 19:38   #6704  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,750
Quote:
Originally Posted by sonnati View Post
HVS is more sensible to yellow-red chrominance (>>550nm)
than violet-blue (<<500nm). Red has always been a mess in encoding, especially when encoding in yuv420 where chroma is subsampled and red pictures appear in their glorious blockiness.

There's still a lot of work to be done to optimize encoders to accomodate for perception: we should improve quantization both in dark picture and red-dominant picture, but this depends also on the viewing conditions (i.e. not so important in small mobile screens)

Not to mention higher order of complexity optimizations like saliency based optimizations or similar.

I have developed many times models that change params using zones-like params to improve the performance of a given encoder. This is one of those cases.
Yeah, very flat gradients in red and blue have always been hard (since luma is mainly green, green is pretty much non subsampled).

Using --crqpoffs -* would help red without spending more bits on blue. It makes a certain logical sense that, when decreasing chroma QP, it would make sense to reduce 2 red for every blue given the relatively higher sensitivity to red. However, red also accounts for a greater portion of luma than blue which likely would offset that some.

Encoding in 10-bit instead of 8-bit should help as well. Sometimes these problems can be as much in the 10-bit Y'CbCr -> 8-bit RGB conversion on a device than in the encoding itself. Display controllers that just truncate the 8-bit instead of dithering can cause all kinds of issues even in SDR. Dithering should be done even when doing a 16-235 Y'CbCr to 0-255 RGB conversion.


To humorously paraphrase Gen. Robert H. Barrow, USMC

"Hobbyists talk about average bitrates and per-title metrics, but experts study color volume algorithms and rate control."
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 7th February 2019, 19:42   #6705  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,750
Quote:
Originally Posted by Selur View Post
should be optional and do no harm, so you can probably use it without running into issues.
All it does is remove duplicates of identical metadata; the majority of frames in a given title will be identical to the prior frame in display order. It'll still ensure that every IDR has its metadata.

This can save some bits (only material at very low bitrates), and potentially some work by the tone mapper. Since display<>decode order, you'll get cases where the tone mapper could find out in advance that several frames in a row have identical color characteristics.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 7th February 2019, 21:43   #6706  |  Link
katzenjoghurt
Registered User
 
Join Date: Feb 2007
Posts: 128
Quote:
Originally Posted by sonnati View Post
HVS is more sensible to yellow-red chrominance (>>550nm)
than violet-blue (<<500nm). Red has always been a mess in encoding, especially when encoding in yuv420 where chroma is subsampled and red pictures appear in their glorious blockiness.

There's still a lot of work to be done to optimize encoders to accomodate for perception: we should improve quantization both in dark picture and red-dominant picture, but this depends also on the viewing conditions (i.e. not so important in small mobile screens)

Not to mention higher order of complexity optimizations like saliency based optimizations or similar.

I have developed many times models that change params using zones-like params to improve the performance of a given encoder. This is one of those cases.
Thanks for the confirmation, sonnati.
I'll continue sticking to zones then or live with rising the overall bitrate beyond general need if the pain becomes too much.
katzenjoghurt is offline   Reply With Quote
Old 8th February 2019, 00:51   #6707  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,750
Quote:
Originally Posted by katzenjoghurt View Post
Thanks for the confirmation, sonnati.
I'll continue sticking to zones then or live with rising the overall bitrate beyond general need if the pain becomes too much.
And when you find examples that need zones, grab that snippet and open an Issue with MCW. An encoder should be able to detect when this would be an issue and then compensate for it.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 12th February 2019, 20:50   #6708  |  Link
redbtn
Registered User
 
redbtn's Avatar
 
Join Date: Jan 2019
Location: Russia
Posts: 105
--ctu --max-tu-size --qg-size tests

I've made some tests for 1080p with --tune-psnr, --tune-ssim, and no-tune for different variants --ctu --max-tu-size --qg-size

Does this mean that for 1080p encoding default settings (--ctu 64 --max-tu-size 32 --qg-size 32) is the best option?

Results:
Code:
no-tune
===
--ctu 64 --max-tu-size 32 --qg-size 32 Global PSNR: 55.009, SSIM Mean Y: 0.9965665 (24.643 dB) higher value
--ctu 32 --max-tu-size 32 --qg-size 16 Global PSNR: 54.982, SSIM Mean Y: 0.9965609 (24.636 dB)
--ctu 32 --max-tu-size 32 --qg-size 8  Global PSNR: 54.866, SSIM Mean Y: 0.9964265 (24.469 dB)
--ctu 32 --max-tu-size 16 --qg-size 16 Global PSNR: 54.908, SSIM Mean Y: 0.9964738 (24.527 dB)
--ctu 32 --max-tu-size 16 --qg-size 8  Global PSNR: 54.791, SSIM Mean Y: 0.9963344 (24.359 dB)
===

tune-psnr
===
--ctu 64 --max-tu-size 32 --qg-size 32 Global PSNR: 54.936 higher value
--ctu 32 --max-tu-size 32 --qg-size 16 Global PSNR: 54.910
--ctu 32 --max-tu-size 32 --qg-size 8  Global PSNR: 54.806
--ctu 32 --max-tu-size 16 --qg-size 16 Global PSNR: 54.869
--ctu 32 --max-tu-size 16 --qg-size 8  Global PSNR: 54.765
===

tune-ssim
===
--ctu 64 --max-tu-size 32 --qg-size 32 SSIM Mean Y: 0.9964914 (24.549 dB) higher value
--ctu 32 --max-tu-size 32 --qg-size 16 SSIM Mean Y: 0.9964839 (24.539 dB)
--ctu 32 --max-tu-size 32 --qg-size 8  SSIM Mean Y: 0.9963653 (24.395 dB)
--ctu 32 --max-tu-size 16 --qg-size 16 SSIM Mean Y: 0.9964291 (24.472 dB)
--ctu 32 --max-tu-size 16 --qg-size 8  SSIM Mean Y: 0.9963079 (24.327 dB)
Attached Files
File Type: txt ctu-logs-no-tune.txt (20.0 KB, 37 views)
File Type: txt ctu-logs-tune-psnr.txt (19.2 KB, 35 views)
File Type: txt ctu-logs-tune-ssim.txt (18.3 KB, 36 views)
redbtn is offline   Reply With Quote
Old 12th February 2019, 20:52   #6709  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 7,259
Quote:
Does this mean that for 1080p encoding default settings (--ctu 64 --max-tu-size 32 --qg-size 32) is the best option?
to state the obvious: only if your quality perception aligns with PSM and SSIM Mean,..
__________________
Hybrid here in the forum, homepage
Selur is offline   Reply With Quote
Old 12th February 2019, 21:05   #6710  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,750
Quote:
Originally Posted by Selur View Post
to state the obvious: only if your quality perception aligns with PSM and SSIM Mean,..
The modern codec design process biases heavily for fixed QP maximizing mean PSNR, so adaptive quant and psychovisual optimizations are generally only helpful when targeting subjective quality. Which is all viewers care about. I’ve seen lots of beautiful PSNR plots turn into lousy looking video.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 12th February 2019, 21:15   #6711  |  Link
redbtn
Registered User
 
redbtn's Avatar
 
Join Date: Jan 2019
Location: Russia
Posts: 105
Quote:
Originally Posted by Selur View Post
to state the obvious: only if your quality perception aligns with PSM and SSIM Mean,..
Quote:
Originally Posted by benwaggoner View Post
The modern codec design process biases heavily for fixed QP maximizing mean PSNR, so adaptive quant and psychovisual optimizations are generally only helpful when targeting subjective quality. Which is all viewers care about. I’ve seen lots of beautiful PSNR plots turn into lousy looking video.
I do not have enough knowledge yet to make difficult conclusions. I'm just trying to find the optimal settings for 4K HDR -> 1080p HDR.
Which option can you advise?

And one more question. Can I somehow optimize the settings for quality improvement without significantly reducing speed? Or will these settings be enough? My traget bitrate 20-25mbs

--preset placebo --crf 12 --profile main10 --output-depth 10 --level-idc 51 --sar 1:1 --colorprim 9 --colormatrix 9 --transfer 16 --range limited --master-display "G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(10000000,1)" --hdr --hdr-opt --chromaloc 2 --repeat-headers --hrd --max-cll "0,0" --min-luma 0 --max-luma 1023 --subme 7 --me star --merange 48 --limit-tu 4 --max-merge 4 --limit-modes --limit-refs 3 --rd 6 --rd-refine --no-tskip --rskip --no-cutree --no-sao --no-open-gop --no-b-pyramid --no-strong-intra-smoothing --vbv-bufsize 160000 --vbv-maxrate 160000 --cbqpoffs -2 --crqpoffs -2 --min-keyint 5 --keyint 240 --ipratio 1.30 --pbratio 1.20 --deblock -3:-3 --qcomp 0.65 --aq-mode 2 --aq-strength 0.8 --psy-rd 1.5 --psy-rdoq 5

Last edited by redbtn; 12th February 2019 at 21:32.
redbtn is offline   Reply With Quote
Old 13th February 2019, 16:08   #6712  |  Link
excellentswordfight
Lost my old account :(
 
Join Date: Jul 2017
Posts: 322
Quote:
Originally Posted by redbtn View Post
I do not have enough knowledge yet to make difficult conclusions. I'm just trying to find the optimal settings for 4K HDR -> 1080p HDR.
Which option can you advise?

And one more question. Can I somehow optimize the settings for quality improvement without significantly reducing speed? Or will these settings be enough? My traget bitrate 20-25mbs

--preset placebo --crf 12 --profile main10 --output-depth 10 --level-idc 51 --sar 1:1 --colorprim 9 --colormatrix 9 --transfer 16 --range limited --master-display "G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(10000000,1)" --hdr --hdr-opt --chromaloc 2 --repeat-headers --hrd --max-cll "0,0" --min-luma 0 --max-luma 1023 --subme 7 --me star --merange 48 --limit-tu 4 --max-merge 4 --limit-modes --limit-refs 3 --rd 6 --rd-refine --no-tskip --rskip --no-cutree --no-sao --no-open-gop --no-b-pyramid --no-strong-intra-smoothing --vbv-bufsize 160000 --vbv-maxrate 160000 --cbqpoffs -2 --crqpoffs -2 --min-keyint 5 --keyint 240 --ipratio 1.30 --pbratio 1.20 --deblock -3:-3 --qcomp 0.65 --aq-mode 2 --aq-strength 0.8 --psy-rd 1.5 --psy-rdoq 5
Imo, the bitrate you are targeting is so high that most of your settings are way overkill. You will spend alot of cpu cycles for virtually no quality gain, you will be well beyond the dimishing return point.

Have you actually tried those settings? It wouldn't surprise me that placebeo crf12 could render a file bigger then the original 4k one, and you will be lucky if you do it above 1fps. I'm sure you have your reasons for this scenario, but if quality was that important to me, I would just go with the original.

Last edited by excellentswordfight; 13th February 2019 at 16:22.
excellentswordfight is offline   Reply With Quote
Old 13th February 2019, 16:54   #6713  |  Link
redbtn
Registered User
 
redbtn's Avatar
 
Join Date: Jan 2019
Location: Russia
Posts: 105
Quote:
Originally Posted by excellentswordfight View Post
Imo, the bitrate you are targeting is so high that most of your settings are way overkill. You will spend alot of cpu cycles for virtually no quality gain, you will be well beyond the dimishing return point.

Have you actually tried those settings? It wouldn't surprise me that placebeo crf12 could render a file bigger then the original 4k one, and you will be lucky if you do it above 1fps. I'm sure you have your reasons for this scenario, but if quality was that important to me, I would just go with the original.
I tried this settings only without (--rd 6 --rd-refine --opt-cu-delta-qp), with (--rd 3 --no-rskip). For example for file bitrate 57 Mbs size 45 GB output bitrate 26 Mb/s size 21 GB with 1.4fps encode speed. I think with new settings it's be around 0.9-1fps on my 6 core i5 8400.
I view at 4K UHD TV, but it is connected to a computer as duplicated with a monitor with a resolution of 1080p because windows 10 cannot normally scale many applications in 4K (it's sadly). Therefore, I decided that there is no point in reducing the image via MadVR in real time. Although I have a GTX 1080 which does it without problems. And also save about 50 percent of the HDD space.
redbtn is offline   Reply With Quote
Old 13th February 2019, 18:30   #6714  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Finland
Posts: 5,718
You can safely use --rskip, it will speed up things a lot and you won't notice any difference. I'd also use something like --subme 3.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 13th February 2019, 22:13   #6715  |  Link
redbtn
Registered User
 
redbtn's Avatar
 
Join Date: Jan 2019
Location: Russia
Posts: 105
Boulder thank you!

Quote:
Originally Posted by Boulder View Post
This is probably a silly question, but here goes anyway: if I use --hdr-opt, do I need to feed the encoder with 10-bit data or is 16-bit data as good if the source is a standard UHD with HDR? I always process things in 16-bit domain and let the encoder dither down to 10 bits.
Quote:
Originally Posted by FranceBB View Post
I would let x265.exe do the dithering, 'cause other dithering options like the Floyd Steinberg error diffusion may have a nicer look, but they could increase the bitrate required by x265. The built in dithering filter in x265 is supposed to dither everything down to the target bit depth without introducing banding. Blocks and macro blocks dithered by x265 are more likely to be recognised during the motocompensation by x265 than the ones dithered using a third party dithering method, therefore compression should be better.

In a nutshell, let x265 do the dithering and always pipe to it the highest bit depth you have, unless you like a specific dithering method and you have enough bitrate.
Im encode 10bit HDR, so if I want to do the same, pipe 16bit to x265 with the flag --dither, I need LWLibavSource format="YUV420P10" or format="YUV420P16" ?
And i should remove this? clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P10, range_s="limited")

Thank you!

My VS scrypt:
Quote:
clip = core.lsmas.LWLibavSource(source="video.mkv", format="YUV420P10", cache=1)
clip = core.resize.Point(clip, matrix_in_s="2020ncl",range_s="limited")
clip = core.std.AssumeFPS(clip, fpsnum=24000, fpsden=1001)
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)
clip = core.std.CropRel(clip=clip, left=0, right=0, top=276, bottom=276)
clip = core.fmtc.resample(clip=clip, kernel="spline64", w=1920, h=804, interlaced=False, interlacedd=False)
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P10, range_s="limited")
clip.set_output()

Last edited by redbtn; 13th February 2019 at 22:17.
redbtn is offline   Reply With Quote
Old 14th February 2019, 01:33   #6716  |  Link
hevc_enocder
Registered User
 
Join Date: Jan 2019
Posts: 2
advice

Code:
--crf 18 --profile main10 --level-idc 5.1 --output-depth 10 --rd 4 --ctu 32 --amp --aq-mode 2 --vbv-bufsize 160000 --vbv-maxrate 160000 --ipratio 1.3
--pbratio 1.2 --no-cutree --subme 7 --me star --merange 24 --max-merge 3 --bframes 12 --rc-lookahead 60 --lookahead-slices 4 --ref 6 --min-keyint 24 --keyint 240 --deblock -3:-3
--no-sao --no-strong-intra-smoothing --high-tier
Hi guys, I would like to know, how to improve my setting, Iam quite satified, but there is problem with red color and if there is sometig to change for encoding normal new movies.

Someone told me, that there is no reason to use subme 7, I am use to from x264 use subme 10 so I thought that it could be usefull.

Thank you for your advice and some explonation.

Last edited by hevc_enocder; 14th February 2019 at 02:24. Reason: spelling
hevc_enocder is offline   Reply With Quote
Old 14th February 2019, 06:20   #6717  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 7,259
Quote:
there is problem with red color
You probably did use the search in this thread and already tried using negative chroma offsets (--cbqpoffs and --crqpoffs), right?
__________________
Hybrid here in the forum, homepage
Selur is offline   Reply With Quote
Old 14th February 2019, 09:09   #6718  |  Link
RainyDog
Registered User
 
Join Date: May 2009
Posts: 184
Quote:
Originally Posted by hevc_enocder View Post
Code:
--crf 18 --profile main10 --level-idc 5.1 --output-depth 10 --rd 4 --ctu 32 --amp --aq-mode 2 --vbv-bufsize 160000 --vbv-maxrate 160000 --ipratio 1.3
--pbratio 1.2 --no-cutree --subme 7 --me star --merange 24 --max-merge 3 --bframes 12 --rc-lookahead 60 --lookahead-slices 4 --ref 6 --min-keyint 24 --keyint 240 --deblock -3:-3
--no-sao --no-strong-intra-smoothing --high-tier
Hi guys, I would like to know, how to improve my setting, Iam quite satified, but there is problem with red color and if there is sometig to change for encoding normal new movies.

Someone told me, that there is no reason to use subme 7, I am use to from x264 use subme 10 so I thought that it could be usefull.

Thank you for your advice and some explonation.
In x264, RDO is tied to subme. So higher levels of subme also incorporate higher levels of RDO.

Subme and RD(O) level are separate settings in x265 and I personally don't think it's worth using anything higher than subme 5.

Whenever I've tested, it's never actually been worth it to use anything higher than subme 3 really which is the lowest subme to include chroma residual cost.

I'd definitely choose --rd 5 (or 6 since they're the same) plus --rd-refine over subme 7 everytime. Though that will half your encoding time so I rarely ever bother with that either
RainyDog is offline   Reply With Quote
Old 14th February 2019, 10:35   #6719  |  Link
excellentswordfight
Lost my old account :(
 
Join Date: Jul 2017
Posts: 322
Quote:
Originally Posted by redbtn View Post
I tried this settings only without (--rd 6 --rd-refine --opt-cu-delta-qp), with (--rd 3 --no-rskip). For example for file bitrate 57 Mbs size 45 GB output bitrate 26 Mb/s size 21 GB with 1.4fps encode speed. I think with new settings it's be around 0.9-1fps on my 6 core i5 8400.
I view at 4K UHD TV, but it is connected to a computer as duplicated with a monitor with a resolution of 1080p because windows 10 cannot normally scale many applications in 4K (it's sadly). Therefore, I decided that there is no point in reducing the image via MadVR in real time. Although I have a GTX 1080 which does it without problems. And also save about 50 percent of the HDD space.
As I said, I'm sure that you have your reasons, but I dont find it that appealing to spend 48h per title to get a filesize that probably would look undisguisable at half the bitrate.

I just did a quick test on tears of steal with those settings and compared it an encode at native res with slow preset (+ --deblock -1:-1 --no-sao --no-strong-intra-smoothing)... the 1080p one ended up at 40Mbps (downscaled with spline), and the 2160 one at 25, the one at at native res was sharper and was about 3x faster to encode...

Ofc, if you find the time/compression/quality to be a good tradeoff for you go ahead with that. But when you even have a 4k TV I would jut go with a 4k workflow cause you can get away with 4k re-encodes with that bitrate target. Not gonna tell you what to do, but I would at least play with different preset levels and crf values and look at the actually video to see if you can find a better "sweetspot". Cause it sounds awful to me o spend 2days for an encode, just to get a bloated file that still has a loss of detail/sharpness (cause of downscaling).

Last edited by excellentswordfight; 14th February 2019 at 11:18.
excellentswordfight is offline   Reply With Quote
Old 14th February 2019, 14:19   #6720  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 6,753
x265 3.0 Au+4-dcbec33bfb0f (MSYS2, MinGW32 + GCC 7.4.0 / MinGW64 + GCC 8.2.1)

Gold!
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 17:48.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.