Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > High Efficiency Video Coding (HEVC)

Reply
 
Thread Tools Search this Thread Display Modes
Old 18th July 2016, 03:59   #3981  |  Link
littlepox
Registered User
 
Join Date: Nov 2012
Posts: 218
Quote:
Originally Posted by herbert View Post
Thanks again, littlepox.

Depending on the chosen speed preset the following options get enabled or disabled:

--no-rskip --limit-modes --rect

Do you have any data as to the usefulness of the above options yet?
--limit-modes is enabled by default at slower or above;
--no-rskip is intolerably slow with little improvement.

Last edited by littlepox; 18th July 2016 at 04:05.
littlepox is offline   Reply With Quote
Old 18th July 2016, 06:25   #3982  |  Link
littlepox
Registered User
 
Join Date: Nov 2012
Posts: 218
--tune film Update:

--crf 18 --ctu 32 --pbratio 1.2 --cbqpoffs -3 --crqpoffs -3 --no-sao --subme 3 --b-intra --no-amp --weightb --aq-mode 3 --aq-strength 0.9 --rd 4 --psy-rd 2.5 --psy-rdoq 4.0 --rdoq-level 2 --rc-lookahead 80 --qcomp 0.65 --no-strong-intra-smoothing --limit-modes

Reintroduce --ctu 32 since further tests suggest the buggy term is --max-tu-size 16, while --ctu 32 is innocent.
Increase psy 2.0:3.0 to 2.5:4.0 which should further reduce blurriness, enhancing visual quality. (bit-rate should increase as well, but worthy)
Add --limit-modes in case someone enables --rect at preset medium (which I strongly DISRECOMMEND; --rect should only be used at preset slower or above)

At the same time, here we go for a --tune animation:

--crf 18 --ctu 32 --ref 4 --bframes 6 --pbratio 1.2 --cbqpoffs -3 --crqpoffs -3 --no-sao --subme 3 --b-intra --no-amp --weightb --aq-mode 3 --aq-strength 0.8 --rd 4 --psy-rd 1.8 --psy-rdoq 1.5 --rdoq-level 2 --rc-lookahead 80 --qcomp 0.65 --no-strong-intra-smoothing --limit-modes

Last edited by littlepox; 18th July 2016 at 06:33.
littlepox is offline   Reply With Quote
Old 18th July 2016, 12:39   #3983  |  Link
mandarinka
Registered User
 
mandarinka's Avatar
 
Join Date: Jan 2007
Posts: 734
What sorts of artifacts do you get from --max-tu-size 16? I encoded some stuff with may/june builds so maybe I could look for them.
mandarinka is offline   Reply With Quote
Old 18th July 2016, 16:26   #3984  |  Link
Barough
Registered User
 
Barough's Avatar
 
Join Date: Feb 2007
Location: Sweden
Posts: 333
x265-v2.0+5-98a948623fdc (MSYS/MinGW, GCC 5.4.0, 32 & 64bit 8/10/12bit multilib EXEs)

Last edited by Barough; 18th July 2016 at 17:07.
Barough is offline   Reply With Quote
Old 19th July 2016, 00:05   #3985  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 2,951
Quote:
Originally Posted by littlepox View Post
--tune film Update:

--crf 18 --ctu 32 --pbratio 1.2 --cbqpoffs -3 --crqpoffs -3 --no-sao --subme 3 --b-intra --no-amp --weightb --aq-mode 3 --aq-strength 0.9 --rd 4 --psy-rd 2.5 --psy-rdoq 4.0 --rdoq-level 2 --rc-lookahead 80 --qcomp 0.65 --no-strong-intra-smoothing --limit-modes

Reintroduce --ctu 32 since further tests suggest the buggy term is --max-tu-size 16, while --ctu 32 is innocent.
Increase psy 2.0:3.0 to 2.5:4.0 which should further reduce blurriness, enhancing visual quality. (bit-rate should increase as well, but worthy)
Add --limit-modes in case someone enables --rect at preset medium (which I strongly DISRECOMMEND; --rect should only be used at preset slower or above)
For film, what about --ctu 64 --rdpenalty 1 --qg-size 32? That'll still allow those big 32x32 intra blocks, but bias strongly against them so they only get used when they really pay off, and still give you adaptive quant at the 32x32 level.

As for --limit-modes, that only does something when --amp or --rect are on anyway, which is make them a lot faster with a tiny reduction in efficiency.

Also, note there's an issue in --aq-mode 3 currently which causes it to use WAY more bits in CRF mode than --aq-mode 2.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 19th July 2016, 00:06   #3986  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 2,951
Quote:
Originally Posted by mandarinka View Post
What sorts of artifacts do you get from --max-tu-size 16? I encoded some stuff with may/june builds so maybe I could look for them.
I wouldn't expect artifacts so much as reduced compression efficiency.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 19th July 2016, 08:55   #3987  |  Link
K.i.N.G
Registered User
 
Join Date: Aug 2009
Posts: 70
Wow, that '--no-sao' switch drasticly increased detail (grain) retention for me!

Anyone more knowledgeable on here can tell me what it does... Google results get kinda too technical for me (i dont understand the terminologie)
K.i.N.G is offline   Reply With Quote
Old 19th July 2016, 08:58   #3988  |  Link
RainyDog
Registered User
 
Join Date: May 2009
Posts: 168
Quote:
Originally Posted by benwaggoner View Post
Also, note there's an issue in --aq-mode 3 currently which causes it to use WAY more bits in CRF mode than --aq-mode 2.
Hasn't --aq-mode 3 always been like this even with x264?
RainyDog is offline   Reply With Quote
Old 19th July 2016, 09:04   #3989  |  Link
RainyDog
Registered User
 
Join Date: May 2009
Posts: 168
Quote:
Originally Posted by benwaggoner View Post
For film, what about --ctu 64 --rdpenalty 1 --qg-size 32? That'll still allow those big 32x32 intra blocks, but bias strongly against them so they only get used when they really pay off, and still give you adaptive quant at the 32x32 level.
Also note that littlepox's --tune-film is geared for 1080p.
RainyDog is offline   Reply With Quote
Old 19th July 2016, 10:35   #3990  |  Link
gamebox
Registered User
 
Join Date: Nov 2011
Posts: 66
@K.i.N.G: SAO is an encoding step that tries to compensate against some typical encoding artifacts seen in most MPEG compression technologies to date (h265 as well), by altering the image after it is compressed. It changes the way mostly edges look, trying to "iron" (smooth) some defects introduced by compression - the ones that are "expected" to be present in those areas. It does a very good job, but sadly seems to eliminate a lot of details by error too, as it can not define compression defects in an image "precisely" (it would take a lot of data to describe them), so it only has some "general impression" or "expectation" about the way they should look.

A small question about x265 "blurriness" most people complain about - has anyone tried pre-filtering the video before sending it to x265, by applying some sharpening to counter-effect the issue? To me, problem seems motion estimation related - defining further steps in sub-pixel motion precision over previous technologies increased efficiency but introduced many calculation steps that ended up smoothing the pixels too much. That's just an amateur guess, I might be wrong. The issue of x265 sharpness seems to influence "detail clarity" as well as (or even more than) "edge contrast" - it seems to need something like HDR filter in preprocessing as well to improve detail "amplitudes" by actually "over-accenting" them before compression takes place.

Last edited by gamebox; 19th July 2016 at 11:01.
gamebox is offline   Reply With Quote
Old 19th July 2016, 10:43   #3991  |  Link
Barough
Registered User
 
Barough's Avatar
 
Join Date: Feb 2007
Location: Sweden
Posts: 333
x265-v2.0+8-669dc9bfe7eb (MSYS/MinGW, GCC 5.4.0, 32 & 64bit 8/10/12bit multilib EXEs)
Barough is offline   Reply With Quote
Old 19th July 2016, 12:30   #3992  |  Link
K.i.N.G
Registered User
 
Join Date: Aug 2009
Posts: 70
Quote:
Originally Posted by gamebox View Post
@K.i.N.G: SAO is an encoding step that tries to compensate against some typical encoding artifacts seen in most MPEG compression technologies to date (h265 as well), by altering the image after it is compressed. It changes the way mostly edges look, trying to "iron" (smooth) some defects introduced by compression - the ones that are "expected" to be present in those areas. It does a very good job, but sadly seems to eliminate a lot of details by error too, as it can not define compression defects in an image "precisely" (it would take a lot of data to describe them), so it only has some "general impression" or "expectation" about the way they should look.

A small question about x265 "blurriness" most people complain about - has anyone tried pre-filtering the video before sending it to x265, by applying some sharpening to counter-effect the issue? To me, problem seems motion estimation related - defining further steps in sub-pixel motion precision over previous technologies increased efficiency but introduced many calculation steps that ended up smoothing the pixels too much. That's just an amateur guess, I might be wrong. The issue of x265 sharpness seems to influence "detail clarity" as well as (or even more than) "edge contrast" - it seems to need something like HDR filter in preprocessing as well to improve detail "amplitudes" by actually "over-accenting" them before compression takes place.
Hey thanks for the very clear explanation.

I dont know about other ppl, but to my eyes, without SAO it still looks allot better than x264 at the same bitrate (arround 4500kb/s) and I dont notice any prominent artifacts (much less than the x264 encode for sure). At least, on my laptop screen it does... I'll be testing it later on my TV.

To me it looks like the SAO switch could use some more dev-tweaking and/or should only be used with very low bitrates... It is way to aggressive right now, imho.

Im using a slightly tuned 'slower' preset so it is allot slower than x264 (it's encoding 1080p at 1.34fps right now).

I'll be testing some more and try to get it faster. If things keep looking this promessing then this will probably make me switch over to x265.

Im using version 1.9+200-6098ba3e0cf16b11 (with staxrip).
Id like to test v2.0 but im still waiting for staxrip to update and have optimized UI for v2.0

Last edited by K.i.N.G; 19th July 2016 at 13:14.
K.i.N.G is offline   Reply With Quote
Old 19th July 2016, 13:44   #3993  |  Link
Barough
Registered User
 
Barough's Avatar
 
Join Date: Feb 2007
Location: Sweden
Posts: 333
Regarding --ag-mode 3 so do i have to say that it can do more 'harm' then good some times. Have noticed on and off on some of my darker test videos that at the edges of dark & very dark areas there can be some kind of 'ghosting'. With -aq-mode 2 so have there been no 'ghosting' like 60 % of the time.

Preset : Medium
CRF : 19-22
Custom command lines used :
--me 3 --ref 4 --aq-mode 2 --aq-strength 2 --rdoq-level 1 --psy-rdoq 4 --rc-lookahead 80
--me 3 --ref 4 --ctu 32 --no-sao --aq-mode 3 --aq-strength 2 --rdoq-level 1 --psy-rdoq 4 --rc-lookahead 80

Last edited by Barough; 19th July 2016 at 15:43.
Barough is offline   Reply With Quote
Old 19th July 2016, 15:37   #3994  |  Link
gamebox
Registered User
 
Join Date: Nov 2011
Posts: 66
@ K.I.N.G. You're welcome

You will not notice compression artifacts in x265 as often and as easily as in x264 partially because x265 encodes video differently. Artifacts in 64x64 and 32x32 image blocks are different than those in x264, slightly "dissolved" and sometimes covering greater areas.

Apart from that, your bitrate can be considered "generous", and not many artifacts should appear. I typically encode at bitrates closer to limits of "acceptable" image quality and "sane" quants - namely 3 Mbps for 1080p - and I could clearly see benefits from SAO. SAO will probably be very useful for broadcasting and streaming as it will "mask" disturbing artifacting during high motion scenes, when limits of data throughput force encoder to temporarily use extreme levels of compression.
gamebox is offline   Reply With Quote
Old 19th July 2016, 16:08   #3995  |  Link
littlepox
Registered User
 
Join Date: Nov 2012
Posts: 218
OK here is a good demo about x265's max-tu-size 16 bug:

https://www.dropbox.com/sh/slftjn7me...EE-V6P3ba?dl=0

source.mkv is the source with 99 frames; denoted as source.
dft_20.hevc is encoded with --preset slow --no-sao --crf 20; denoted as A
dft_19.5_maxtu16.hevc is encoded with --preset slow --no-sao --crf 19.5 --max-tu-size 16; denoted as B

The size difference of them is 0.6%.

In many frames (e.g, frame 17,21,63,83...) You see unpleasant patterns in the flat area of B, like worms on the wall, especially noticable when the grain is lost and the area is clean. In A the situation is much better.

This is A(part of frame 83):


This is B(part of frame 83):


Asking my friend to drop an issue to the https://bitbucket.org/multicoreware/...ew&status=open (done)

Last edited by littlepox; 19th July 2016 at 16:28.
littlepox is offline   Reply With Quote
Old 19th July 2016, 16:39   #3996  |  Link
littlepox
Registered User
 
Join Date: Nov 2012
Posts: 218
Quote:
For film, what about --ctu 64 --rdpenalty 1 --qg-size 32? That'll still allow those big 32x32 intra blocks, but bias strongly against them so they only get used when they really pay off, and still give you adaptive quant at the 32x32 level.
After clearing the --max-tu-size bug, we tested various possibilities like --ctu 32, --ctu 64 + --rdpenalty 0/1/2
They are indifferent both quality-wise and size-wise, at least with our test settings.
The only difference is --ctu 32 is faster and better paralleled; that's why it is reintroduced.


Quote:
Also, note there's an issue in --aq-mode 3 currently which causes it to use WAY more bits in CRF mode than --aq-mode 2.
That's why we suggest to tweak the aq-strength -0.2 for aqmode=3 and +0.2 for aqmode=2. With reduced strength, the bit-rate increase is acceptable and worthy.

Quote:
Regarding --ag-mode 3 so do i have to say that it can do more 'harm' then good some times. Have noticed on and off on some of my darker test videos that at the edges of dark & very dark areas there can be some kind of 'ghosting'. With -aq-mode 2 so have there been no 'ghosting' like 60 % of the time.

Preset : Medium
CRF : 19-22
Custom command lines used :
--me 3 --ref 4 --aq-mode 2 --aq-strength 2 --rdoq-level 1 --psy-rdoq 4 --rc-lookahead 80
--me 3 --ref 4 --ctu 32 --no-sao --aq-mode 3 --aq-strength 2 --rdoq-level 1 --psy-rdoq 4 --rc-lookahead 80
having aq-strength outside [0.5,1.5] is probably a bad idea, aq-strength=2 for mode=3 is WAY too large.
Also, you need to control your variables. I don't see it reasonable to use different sao settings for the test.
littlepox is offline   Reply With Quote
Old 19th July 2016, 17:42   #3997  |  Link
Barough
Registered User
 
Barough's Avatar
 
Join Date: Feb 2007
Location: Sweden
Posts: 333
Quote:
Originally Posted by littlepox View Post
having aq-strength outside [0.5,1.5] is probably a bad idea, aq-strength=2 for mode=3 is WAY too large.
Also, you need to control your variables. I don't see it reasonable to use different sao settings for the test.
Still doing tests here. Have lowered the aq-strength to between 1.2 and 1.5. Will do a full movie encode tonight of a dark movie. Will see what my bookshelf have 2 offer.

Also waiting for a m8 that work with video to get back to me. He's not into HEVC so much yet as i understand but his knowledge in the world of video encoding is good.
Barough is offline   Reply With Quote
Old 19th July 2016, 17:46   #3998  |  Link
littlepox
Registered User
 
Join Date: Nov 2012
Posts: 218
Quote:
Originally Posted by Barough View Post
Still doing tests here. Have lowered the aq-strength to between 1.2 and 1.5. Will do a full movie encode tonight of a dark movie. Will see what my bookshelf have 2 offer.

Also waiting for a m8 that work with video to get back to me. He's not into HEVC so much yet as i understand but his knowledge in the world of video encoding is good.
I think 1.2~1.5 is still too high for aqmode 3. For me, I'd use it as 0.9.
littlepox is offline   Reply With Quote
Old 19th July 2016, 23:11   #3999  |  Link
divxmaster
Registered User
 
Join Date: Mar 2015
Location: New Zealand
Posts: 45
Quote:
Originally Posted by gamebox View Post
@K.i.N.G: SAO is an encoding step that tries to compensate against some typical encoding artifacts seen in most MPEG compression technologies to date (h265 as well), by altering the image after it is compressed. It changes the way mostly edges look, trying to "iron" (smooth) some defects introduced by compression - the ones that are "expected" to be present in those areas. It does a very good job, but sadly seems to eliminate a lot of details by error too, as it can not define compression defects in an image "precisely" (it would take a lot of data to describe them), so it only has some "general impression" or "expectation" about the way they should look.

A small question about x265 "blurriness" most people complain about - has anyone tried pre-filtering the video before sending it to x265, by applying some sharpening to counter-effect the issue? To me, problem seems motion estimation related - defining further steps in sub-pixel motion precision over previous technologies increased efficiency but introduced many calculation steps that ended up smoothing the pixels too much. That's just an amateur guess, I might be wrong. The issue of x265 sharpness seems to influence "detail clarity" as well as (or even more than) "edge contrast" - it seems to need something like HDR filter in preprocessing as well to improve detail "amplitudes" by actually "over-accenting" them before compression takes place.
@gamebox, I have tested sharpening before processing, and it makes a huge difference. Ironically it is through QTGMC in vapoursynth (I have just noticed that a few select cgi scenes in later seasons of ds9 are interlaced...ick!)
The side effect of the qtgmc deinterlacing is a strong sharpen and also seems to be colour enhancement. This does increase bitrate +25%, so I increased CRF from 21 to 22, but it still looks way better than no qtgmc/sharpen and is slightly lower bitrate (5%) as a bonus.
Specifically ds9 really benefits from degrain also 'vid = haf.SMDegrain(vid, tr=3,thSAD=300,RefineMotion=True,contrasharp=True,pel=2)'
params are --preset slow --rskip --crf 22 --no-sao --aq-mode 2 --rdoq-level 2 --psy-rd 2 --psy-rdoq 1.1 --qg-size 32 --tu-intra-depth 3 --merange 27 --weightb --early-skip --fast-intra --b-intra --tskip --tskip-fast --ref 6 --bframes 6 --max-merge 5 --min-keyint 23 --keyint 288 --deblock -1:-1 --no-open-gop

Last edited by divxmaster; 19th July 2016 at 23:15.
divxmaster is offline   Reply With Quote
Old 20th July 2016, 02:50   #4000  |  Link
nakTT
Registered User
 
Join Date: Dec 2008
Posts: 386
Quote:
Originally Posted by Barough View Post
x265-v2.0+8-669dc9bfe7eb (MSYS/MinGW, GCC 5.4.0, 32 & 64bit 8/10/12bit multilib EXEs)
Hi,

Is this the same as the one that can be found in the webpage below?

https://builds.x265.eu
nakTT is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 09:50.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.