Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > High Efficiency Video Coding (HEVC)

Reply
 
Thread Tools Search this Thread Display Modes
Old 19th February 2019, 16:53   #1  |  Link
JELSTUDIO
Registered User
 
Join Date: Dec 2011
Posts: 27
How to focus bitrate on video's POI?

I don't know if this is the place to ask, of if it really pertains to some other codec, but I will try.

Say we have a video of an interview.
2 people and an almost static background (perhaps some trees in gentle wind or a road with a car from time to time or a house with some smoke coming up the chimney)

How do I get the codec to focus its bitrate on the 'talking heads' and not share it equally between the less interesting background and the point-of-interest (the 'talking heads' in this case)?

Or, as another example, a static non-moving camera showing a video of a lawn with a cat walking through the frame.
Which settings would leave the frame mostly a still-image and use most bits on the moving cat?

Objective: to limit the needed bits to the minimum (I guess that's obvious ) required to make the POI look fair in quality (the cat or the 'talking head', while letting the background 'suffer' from lower bitrate)

Maybe it's simple and I just don't know the correct term, but I'm hoping somebody knows what I'm talking about (and perhaps how to achieve it or where to find further info on how to achieve it)

I'm testing using handbrake (perhaps not the optimal tool for this?), but the 'extra options' (what, and how to syntax it, in that box) is a bit over my head still.

Thanks
jacob.
JELSTUDIO is offline   Reply With Quote
Old 19th February 2019, 20:49   #2  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,752
What's the command line you're using now? By default the higher presets do a lot of adaptive quantization that can help this use case. The --aq-motion experimental mode might help (or hurt).

What you are really asking for ROI-specific adaptive quantization based on face detection and tracking, which is very common in videoconferencing. A very basic implementation would lower the QP of the ROI by 2, and then raise the background as needed to maintain bitrate.Patching the x265 library to allow ROI is probably not a huge task for a skilled engineer familiar with the code base.

We had ROI feature back in the PEP VC-1 encoder and it could make a huge difference in encoding some kinds of content, like mixed natural/synthetic.

Of course, you'd also need to know the coordinates of the face per frame, which is out of scope for x265. Bear in mind people move around, so the ROI would be dynamic. Youd'd also need a strategy to blend the ROI region with the surrounding region so you don't get a blocky seam (like tapering from -3 to -1 QP at the edge). Lowering the qg-size to get finer granularity would help.

If you have a way to define the ROI at least, blurring and applying noise reduction to the background area would reduce bits required there and thus increase quality in the face region, without having to patch x265. Bumping up --psy-rd in that case could also help.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 20th February 2019, 00:01   #3  |  Link
Motenai Yoda
Registered User
 
Motenai Yoda's Avatar
 
Join Date: Jan 2010
Posts: 709
IIRC x265 got a ROI patch some time ago
__________________
powered by Google Translator
Motenai Yoda is offline   Reply With Quote
Old 20th February 2019, 00:07   #4  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,752
Quote:
Originally Posted by Motenai Yoda View Post
IIRC x265 got a ROI patch some time ago
Oh, good.

Of course, determining what the ROI IS is the hard part.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 20th February 2019, 06:05   #5  |  Link
JELSTUDIO
Registered User
 
Join Date: Dec 2011
Posts: 27
Quote:
Originally Posted by benwaggoner View Post
What's the command line you're using now? By default the higher presets do a lot of adaptive quantization that can help this use case. The --aq-motion experimental mode might help (or hurt).
I am using handbrake and it doesn't list what options are available to input in the 'extra options' box for the various codecs.

It also doesn't list how to syntax them so the chosen encoder can understand them.

However, your mention of 'higher presets' and '--aq-motion' helped me along with both handbrake and google

I found this web-page: https://x265.readthedocs.io/en/default/cli.html where there appears to be a list of all the available options (for x265 at least) and a description of what they do.

And in handbrake I discovered, by selecting a 'higher preset' (I hadn't really tried the presets before, but only been using the manual options), that some of them actually fills in information into the 'extra options' box. So that at least gives a hint on how handbrake expects the syntax in that box to be.

So with that I can begin some experimentation

Thanks a lot




Quote:
Originally Posted by benwaggoner View Post
What you are really asking for ROI-specific adaptive quantization based on face detection and tracking, which is very common in videoconferencing.
No doubt face-detection, or perhaps skin-tone detection as a more simplistic method, would be a great option to have.



Quote:
Originally Posted by benwaggoner View Post
If you have a way to define the ROI at least, blurring and applying noise reduction to the background area would reduce bits required there and thus increase quality in the face region, without having to patch x265. Bumping up --psy-rd in that case could also help.
Currently the codec (when using the manual options in handbrake, such as speed and bitrate) seems to give more weight to 'background' than 'foreground'.

When I lower the bitrate, the first thing that get 'ugly' is the faces and then the background.

I can use a lower bitrate with the background than the foreground and since I only have one bitrate setting this means I end up with more bitrate used on the background than necessary (per my own subjective quality needs of course )

So the ROI is more or less already defined by the codec. I just need to find a way to have the codec use less of the bits on the background and spend them on the foreground instead.

Background being large flat areas (such as walls and relatively static content), and foreground being all the moving things (such as facial expressions and people moving around)

A frequency separated weighting perhaps (Which I assume is what the codec already uses in some form. I just want to modify the weights given to low vs high frequencies, such that the single bitrate I end up using makes both foreground and background 'ugly' at the same setting, so to speak )



I will experiment a bit with this now that I have some info to go on, and see if anything comes off it
Thanks again for the help.


/////////////////////////
EDIT:

First few tests done.
Turning "--aq-mode" OFF (entering "aq-mode=0" in handbrake's 'extra options' box) appears to shift weight slightly more from static objects to moving objects.
I get a sharper face and more artifacts on walls. It's a beginning

Last edited by JELSTUDIO; 20th February 2019 at 08:23. Reason: added test-result
JELSTUDIO is offline   Reply With Quote
Old 20th February 2019, 17:57   #6  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Finland
Posts: 5,718
I think that cutree does shift bits towards flat areas because they are referenced more often in other frames. You might want to try disabling that as well.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 21st February 2019, 02:35   #7  |  Link
Motenai Yoda
Registered User
 
Motenai Yoda's Avatar
 
Join Date: Jan 2010
Posts: 709
Code:
https://bitbucket.org/multicoreware/x265/commits/a3a99fa18cbca8e00647a63ad6f498a828f78366
__________________
powered by Google Translator
Motenai Yoda is offline   Reply With Quote
Old 21st February 2019, 19:25   #8  |  Link
JELSTUDIO
Registered User
 
Join Date: Dec 2011
Posts: 27
I did an extensive set of tests (including the cutree option, and many others) and compared until I found the settings I preferred.

This is the line I entered into the 'extra options' in Handbrake:

aq-mode=0:rd=6:rect:amp:cu-lossless:rd-refine:strong-intra-smoothing:rdoq-level=1sy-rdoq=50:deblock=6:6
(it says ' psy-rdoq=50 ', not a smiley of course)

(Using X265 set to 'medium' and with fast-decode unchecked)

This gave me an image quality at around 2000 kbps that worked for the video I worked with (faces were sharp and walls and doors and floors were blurred a bit)

And then I discovered that the Chrome-browser can not play x265 videos

I should have checked that first, but oh well lesson learned.

So I googled around and found out which video-types the various web-browsers support, and are now focussing on WebM-VP9 instead.

Sometimes video-encoding is just SO much fun...

Anyway, thanks all for taking time to reply to this thread

Last edited by JELSTUDIO; 21st February 2019 at 19:28. Reason: unwanted smiley
JELSTUDIO is offline   Reply With Quote
Old 21st February 2019, 21:36   #9  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,752
Quote:
Originally Posted by JELSTUDIO View Post
I did an extensive set of tests (including the cutree option, and many others) and compared until I found the settings I preferred.

This is the line I entered into the 'extra options' in Handbrake:

aq-mode=0:rd=6:rect:amp:cu-lossless:rd-refine:strong-intra-smoothing:rdoq-level=1sy-rdoq=50:deblock=6:6
(it says ' psy-rdoq=50 ', not a smiley of course)

(Using X265 set to 'medium' and with fast-decode unchecked)

This gave me an image quality at around 2000 kbps that worked for the video I worked with (faces were sharp and walls and doors and floors were blurred a bit)
I would be stunned if --cu-lossless made that much of a difference unless you had static synthetic elements on the screen. If --cu-lossless actually helped, you could probably get even better results from --tskip.

The other options seem pretty nuts too, but possibly in interesting ways! Can you share your source? deblock 6:6 will reduce detail AND artifacts, certainly.

Did you compare just using --preset slower?

Quote:
And then I discovered that the Chrome-browser can not play x265 videos

I should have checked that first, but oh well lesson learned.
Yeah, Firefox and Chrome don't support HEVC. They used to support passing on to a HW decoder, but then explicitly blocked that from working for political reasons.

Quote:
So I googled around and found out which video-types the various web-browsers support, and are now focussing on WebM-VP9 instead.
You might find x264 works better than VP9 for your use case. x264 is MUCH more mature and flexible than libvpx.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 21st February 2019, 22:31   #10  |  Link
SeeMoreDigital
Life's clearer in 4K UHD
 
SeeMoreDigital's Avatar
 
Join Date: Jun 2003
Location: Notts, UK
Posts: 12,219
Quote:
Originally Posted by benwaggoner View Post
...Yeah, Firefox and Chrome don't support HEVC. They used to support passing on to a HW decoder, but then explicitly blocked that from working for political reasons.
Bonkers isn't it.

I've not checked recently but does anyone know if Youtube is HEVC friendly?
__________________
| I've been testing hardware media playback devices and software A/V encoders and decoders since 2001 | My Network Layout & A/V Gear |
SeeMoreDigital is offline   Reply With Quote
Old 21st February 2019, 22:33   #11  |  Link
nevcairiel
Registered Developer
 
Join Date: Mar 2010
Location: Hamburg/Germany
Posts: 10,344
Quote:
Originally Posted by SeeMoreDigital View Post
I've not checked recently but does anyone know if Youtube is HEVC friendly?
If by friendly you mean that you can upload it, sure. On intake they take almost anything.
__________________
LAV Filters - open source ffmpeg based media splitter and decoders
nevcairiel is offline   Reply With Quote
Old 21st February 2019, 22:58   #12  |  Link
SeeMoreDigital
Life's clearer in 4K UHD
 
SeeMoreDigital's Avatar
 
Join Date: Jun 2003
Location: Notts, UK
Posts: 12,219
Quote:
Originally Posted by nevcairiel View Post
If by friendly you mean that you can upload it, sure. On intake they take almost anything.
How about Youtube HEVC playback via various browsers?
__________________
| I've been testing hardware media playback devices and software A/V encoders and decoders since 2001 | My Network Layout & A/V Gear |
SeeMoreDigital is offline   Reply With Quote
Old 21st February 2019, 23:29   #13  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,346
Quote:
Originally Posted by SeeMoreDigital View Post
How about Youtube HEVC playback via various browsers?
Youtube does not stream HEVC .

You can upload HEVC, but their re-encoded versions that end users get are never HEVC
poisondeathray is offline   Reply With Quote
Old 22nd February 2019, 14:23   #14  |  Link
JELSTUDIO
Registered User
 
Join Date: Dec 2011
Posts: 27
Quote:
Originally Posted by benwaggoner View Post
I would be stunned if --cu-lossless made that much of a difference unless you had static synthetic elements on the screen. If --cu-lossless actually helped, you could probably get even better results from --tskip.

The other options seem pretty nuts too, but possibly in interesting ways! Can you share your source? deblock 6:6 will reduce detail AND artifacts, certainly.

Did you compare just using --preset slower?


Yeah, Firefox and Chrome don't support HEVC. They used to support passing on to a HW decoder, but then explicitly blocked that from working for political reasons.


You might find x264 works better than VP9 for your use case. x264 is MUCH more mature and flexible than libvpx.
The source-video "retest.7z" (28 MegaBytes): https://1drv.ms/f/s!Ap4PGUC6dXtHgRUAbIFdK8aGuGbW

In there are 3 videos:

The source "SpejderSourceTestBest.mov" is a 10 second clip exported from Davinci at the setting called 'best'. The footage is originally from an old VHS after it has been denoised. This is the master-material.

And then 2 of the test-renders:

"01_x265_a200_v35_medium,53s.mp4" is with 'my' settings:
speed: medium
tune: none
fast decode: OFF
profile: auto
extra options (labeled 'v35' as it was my 35th test, and entered as this in handbrake): aq-mode=0:rd=6:rect:amp:cu-lossless:rd-refine:strong-intra-smoothing:rdoq-level=1sy-rdoq=50:frame-threads=1:deblock=6:6
render-time: 53 seconds

"01_x265_a200_v0_slower,54s.mp4" is with blank settings in the 'extra options' box and just using the 'slower' preset instead of medium.
speed: slower
tune: none
fast decode: OFF
profile: auto
extra options (labeled 'v0' as it was my 1st test): none
render-time: 54 seconds

For both test-renders:
average bitrate used was 200 kbps single-pass.
constant framerate of 60 fps
audio-track not included


The differences are not huge (between using 'slower' preset with no options and 'my' preset with options), and on some frames 'my' settings produce a worse result. But overall, on the majority of frames, I prefer what I get with 'my' settings (for this video at least, and for what I choose as ROI of course )
A different combination of settings might very possibly be better, but testing all these variations obviously takes a lot of time and I only just figured out how to even input all these options into handbrake


I did not include the x264 render, because that one was just a block-fest at this bit-rate. It only took a few seconds to render though, even though I set it to 'slower', so the fastest, but ugliest, of all options.
In Handbrake I left the 'extra options' blank for the x264 render.

I did not include the WebM-VP9 either (although I'm pretty sure already this is the codec I will use from now on), but in my opinion it lands somewhere between the X265-slower and X265-JEL (the render with 'my' settings), and took 15 seconds to render, so quite a lot faster and with roughly similar overall quality (at my chosen ROIs) to the two x265 renders.
In Handbrake I used this line in the 'extra options' box for VP9, but I'm not sure all of them actually get parsed (the speed does for sure, but the weights appeared to make no difference at all. I need to test it further to be sure though, but I can't see any visual difference in the few tests I have done with VP9) : good:cpu-used=5:minsection-pct=0:maxsection-pct=1000:bias-pct=100:min-q=0:max-q=63:threads=1:static-thresh=10000


I included a picture here of the 4 test-renders (all done at 200 kbps-average single-pass). I upscaled it 200x (using no interpolation) just to give a quick crude idea of how each of them look.

And then a frame-grab (shown at 200% in Vdub) between x265-slower and x265-'my' settings, where I point out 1 ROI, the faces, and 1 'RONI' (region of non-interest ), the dot on the wall.

They both were too big for the forum though, so it converted them to jpg, but they still show the general idea.

And finally; the full movie (both the x265 and vp9 render) is uploaded here (just in case somebody is curious ) : https://archive.org/details/JELSTUDI...enModEnSpejder
Attached Images
  

Last edited by JELSTUDIO; 22nd February 2019 at 14:39. Reason: typos
JELSTUDIO is offline   Reply With Quote
Old 22nd February 2019, 21:32   #15  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,752
Net, net, HEVC for the win .

As you increase in-loop deblocking strength, the filtering reduces frequencies and QP will drop. So you get a softer-looking image with fewer artifacts. The combination of really high RDOQ and really high deblocking is a novel one to me, and looks to defeat the purpose of --deblock 6:6. v0 certainly looks superior to v35 to me. The RONI is distractingly bad in v35, and the ROI shows a lot more ringing.

x265 has most of its tuning done around typical parameters, and they are typical because the HM and other experimentation have shown them to be generally appropriate. Going out to the edges means that you're far away from what's generally worked before, AND far away from what's gotten detailed psychovisual tuning.

Did you try --aq-mode 2 as well? I would think that might have been helpful.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 24th February 2019, 12:03   #16  |  Link
JELSTUDIO
Registered User
 
Join Date: Dec 2011
Posts: 27
I tested aq-mode zero, 1, 2 and 3 and all except zero softened the image to my eyes.
Aq-mode at zero was the sharpest looking to me (for my chosen ROIs of course AND my subjective opinion )

Anyway, with Chrome not playing x265 videos my options with it are limited.
JELSTUDIO is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 06:04.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.