madVR - high quality video renderer (GPU assisted) [Archive] - Page 773

e-t172

10th July 2016, 19:04

But CAM02-UCS is not hue linear, or is it?

I don't know. I'm also not convinced that it matters as much as you think it does. What you want is the closest color that is contained in the destination gamut, so that the color shift is minimized. One could imagine a scenario in which the destination gamut is constrained in such a way that the best color to map to happens to have slight hue shift in order to avoid large swings in saturation or brightness. It would depend on the situation and the specific source and destination gamuts, I guess. The closest color is the closest color, and this is what traditional gamut mapping methods (based, for example, on CAM02-SCD, which is basically DeltaE but based on CAM02-UCS instead of CIELAB) give you. I'm not sure I see the point in obsessing about whether hue is preserved or not; CIECAM02 is smarter than this, and already takes this into account.

Why did Dolby invent ICtCp if CAM02-UCS would be better? ICtCp is brand new and recommended by Dolby for HDR processing.

I just read the whitepaper. I suspect Dolby invented ICtCp for two reasons:

First, it seems to me that performance mattered in the design of ICtCp:

Performing these tasks in an efficient essence representation reduces complexity and increases speed. ICtCp is designed for this purpose.

In contrast, I don't think "traditional" gamut mapping done by CMS is optimized for speed at all. That might mean that "traditional" methods such as CAM02-UCS could give better results because they don't make this tradeoff.

Specifically, ICtCp is meant to be a simple transformation on top of RGB (i.e. matrix + EOTF/gamma curve/tone mapping curve, a.k.a. "matrix + shaper"). Gamut mapping in CMSes, including gamut mapping using CAM02-UCS (which Argyll can do), is way more costly from a performance perspective. First you need to generate a transform (e.g. in Argyll you would run collink, which can take some time to do the computation), and then the output of that is a full-blown 3DLUT, not a simple matrix and EOTF.

Second reason (and probably the most critical one): from the tone of the paper it seems that ICtCp is meant to be an encoding format, which is a very different set of constraints compared to an intermediate color space such as CIECAM02. The former is meant to be used as a storage or transmission format; the latter is meant to be used as a color space inside some color management pipeline/workflow, such as madVR or other color-critical applications. For example, in the ICtCp whitepaper (page 1) they explain that while designing the color space they made sure that the encoded color information could withstand operations such as chroma subsampling or fading. These are reasonable constraints for an encoding format, but they make no sense for an intermediate, transient representation such as CIEXYZ, CIELAB or CIECAM02, which are meant to be used as "bridges" between a source gamut and a destination gamut, not for encoding.

In light of the above, I suspect that using ICtCp as an intermediate color space to do gamut conversions is not the best choice. From what I can tell, it's simply not designed to do that - it's designed to encode video, not do heavy color-critical processing such as gamut mapping. If you want to use a perceptually linear color space that is designed for such heavyweight transforms, then use CIELAB, or better yet, CIECAM02 - that's what they're for. (Conversely, you wouldn't encode video in CIELAB or CIECAM02 - though that didn't stop the DCI people from inventing a format that uses CIEXYZ…) Or, you could delegate all this work to a CMS that will generate a high-quality gamut-mapping 3DLUT for you using the best known perceptually linear color spaces and tricks that are known to color scientists.

There might be vendors out there that use ICtCp as an intermediate color space for actual processing. If that's the case, I suspect that they do it for simplicity/performance reasons, not because it is the most perceptually accurate.

madshi

10th July 2016, 19:52

I just read the whitepaper. I suspect Dolby invented ICtCp for two reasons: [...]
The Dolby ICtCp White Paper is mostly an advertising brochure to tout the benefits of using ICtCp instead of YCbCr for encoding. So in order to judge the suitability of using IPT or ICtCp for gamut mapping, this Dolby White Paper is pretty much useless.

To get the full picture you should start with the IPT science paper, which contains all the real meat. Here's the conclusion of the IPT science paper:

A color space named IPT has been developed that is more uniform in perceived hue than existing popular color spaces. It is simple to implement and invertable, thus lending itself to imaging applications. The IPT color space is similar in model to CIELAB color space, although the coefficients are different. Visualization of several psychophysical data sets were shown for IPT, CIELAB, and CIECAM97s. In all cases shown, IPT appears to perform as well as (and in some cases performs better than) either CIELAB or CIECAM97s. A verification experiment was performed the results of which show that IPT is judged to be more uniform than two constant hue data sets.
So basically IPT beats CIECAM97s (at least in the confines of those tests/data sets). CIECAM02 seems to be an evolution of CIECAM97s. I'm not sure how much it is improved over CIECAM97s in quality. Websites mostly state performance and simplicity improvements over CIECAM97s, so it's hard to say.

Next, see here:

http://ieeexplore.ieee.org/xpl/login.jsp?tp=&arnumber=5170710&url=http%3A%2F%2Fieeexplore.ieee.org%2Fxpls%2Fabs_all.jsp%3Farnumber%3D5170710

The uniformity of color appearance space is a important yardstick for evaluating color appearance model. This paper is focused on the test of three color appearance spaces CIELAB, IPT and CIECAM02. Several psychophysical data sets were used to evaluate three types of scales and color difference formula in these color spaces. The results show that IPT performs as well as (in some cases performs better than) either CIELAB or CIECAM02.
I see no reason to bother with a very complicated CIECAM02, if IPT seems to perform just as well. And ICtCp is a further improvement over IPT.

More information:

http://www.hpl.hp.com/news/2005/oct-dec/CIC05_CIECAMICC_final.pdf

CIECAM02 does not invert for a range of colors, real and impossible, inside the encoding range of the ICC PCS. Therefore, any processing needs to consider careful handling of these colors. This implementation does two geometric mappings – one to the extended spectrum locus and another to the device gamut. It is important to ensure that artifacts do not result from these distinct mappings.

Another point worth considering is whether a color appearance model like CIECAM02 is the best possible choice if one wants to attack the problem of hue constancy and perceptual uniformity. Alternatives would be to perform the gamut mapping in a space like IPT.
I originally used IPT in the madVR HDR mapping algorithms, but had some problems with that. So the SpectraCal/Calman devs suggested that I'd look into ICtCp. Which is what I'm using now and it solved all the problems I had with IPT.

e-t172

10th July 2016, 20:07

Oh well, if there is evidence that ICtCp is actually better than CIECAM02 at being perceptually uniform, then by all means, ignore everything I said :) I was working on the assumption that ICtCp was a performance tradeoff between YCbCr and CIECAM02. If it's not actually a tradeoff, then there's no reason to use CIECAM02 instead of ICtCp. I'm quite impressed that people managed to come up with a superior alternative to CAM02-UCS that also has other nice characteristics. We live in exciting times!

For completeness, I should probably also mention that, sometimes, some standard will mandate some method to do the display adjustment (i.e. gamut mapping by another name). For example, BT.2390 indicates that for a PQ HDR system, the recommended method to adjust for display luminance constraints is to apply some function to the I channel of ICtCp (§5.4.1). In that case, it makes sense to use ICtCp for processing, since the standard defines the transform in ICtCp terms.

That doesn't answer the question as to how exactly gamut mapping should be done in ICtCp, though - specifically, which color difference function to use (DeltaE and CAM02-SCD work in CIELAB and CAM02-UCS respectively, not in ICtCp). From what you wrote previously, currently you're using an ad-hoc color difference function that considers hue changes to be infinitely more important than luminance changes. It would be interesting to find out if there's a better way.

nevcairiel

10th July 2016, 20:18

But CAM02-UCS is not hue linear, or is it? Why did Dolby invent ICtCp if CAM02-UCS would be better? ICtCp is brand new and recommended by Dolby for HDR processing.

The usual reasons, slap a patent on it and make money?
Who says everything is ruled by technical reasons that would make sense? :)

jerryleungwh

10th July 2016, 20:26

I have this laptop PC which has both Intel HD 530 GPU and a Nvidia 960M GPU. I use MPC-HC with madVR but when I'm using the intel GPU, the video stutters and freezes my computer eventually, and I've tried copying the MGC-HC exe file and renaming it so I can use the Nvidia GPU, but it fails to enter D3D11 FSE mode(it says "exclusive mode failed"). How can I fix this problem?

madshi

10th July 2016, 20:48

I uploaded a sample. On the 90.19 version this artifact happened a lot of time during all this sample but now (90.20 version) it happens at only one moment so it's already a big improvement. You can find this scene easily, I attached some pictures with the sample, just look at the timer.
Which artifacts do you mean exactly? Earlier you mentioned teeth, but I can't see any artifacts there. I do see some artifacting under the glasses. Is that what you mean? Or something else?

Mixed scaling is broken again :(
This issue still exists. Seems to me that madVR calculates scaling factor = (window height) / (source height), which can activate image doubling even when the image is actually downscaled quite a lot as this example shows (http://imgur.com/a/8EBcV). With doubling set to activate when the scaling factor is 1.2 or bigger, it actually activates when the window height is increased from 839 pixels (first picture) to 840=1.2*700 (second picture) pixels regardless of the window width which in this case actually defines the scaling factor.
These two problems should hopefully be fixed in the next build. If not, please let me know.

Oh well, if there is evidence that ICtCp is actually better than CIECAM02 at being perceptually uniform, then by all means, ignore everything I said :) I was working on the assumption that ICtCp was a performance tradeoff between YCbCr and CIECAM02. If it's not actually a tradeoff, then there's no reason to use CIECAM02 instead of ICtCp. I'm quite impressed that people managed to come up with a superior alternative to CAM02-UCS that also has other nice characteristics. We live in exciting times!

For completeness, I should probably also mention that, sometimes, some standard will mandate some method to do the display adjustment (i.e. gamut mapping by another name). For example, BT.2390 indicates that for a PQ HDR system, the recommended method to adjust for display luminance constraints is to apply some function to the I channel of ICtCp (§5.4.1). In that case, it makes sense to use ICtCp for processing, since the standard defines the transform in ICtCp terms.
FWIW, I'm not sure if IPT/ICtCp are always and for all purposes equal to or better than CIECAM02. E.g. iCam06 uses CIECAM02 for some processing steps and IPT for some other processing steps. So I suppose both may have their strengths and weaknesses. But from what I've seen IPT/ICtCp should be well suited for what I'm doing.

That doesn't answer the question as to how exactly gamut mapping should be done in ICtCp, though - specifically, which color difference function to use (DeltaE and CAM02-SCD work in CIELAB and CAM02-UCS respectively, not in ICtCp). From what you wrote previously, currently you're using an ad-hoc color difference function that considers hue changes to be infinitely more important than luminance changes. It would be interesting to find out if there's a better way.
I'm considering hue changes and luminance changes to be very bad and I try to avoid both at all costs. I'm happy enough to desaturate colors to keep (perceived) hue and luminance as intended. I'm not sure if this is the best approach, but considering that the human eye is most sensitive to brightness variations, I thought that reproducing luminance correctly should have the highest priority.

Anyway, I've emailed Graeme about this.

The usual reasons, slap a patent on it and make money?
Who says everything is ruled by technical reasons that would make sense? :)
True enough. :devil:

I have this laptop PC which has both Intel HD 530 GPU and a Nvidia 960M GPU. I use MPC-HC with madVR but when I'm using the intel GPU, the video stutters and freezes my computer eventually, and I've tried copying the MGC-HC exe file and renaming it so I can use the Nvidia GPU, but it fails to enter D3D11 FSE mode(it says "exclusive mode failed"). How can I fix this problem?
Try searching this thread for "Optimus". You may find some hints about that. Personally, I can't help because I don't have such hardware to test with.

zoyd

10th July 2016, 21:03

I don't think that matters. Tone mapping by itself does not affect gamut, it affects how colors are encoded within the gamut. Arbitrary tone mapping curves can be represented in an ICC profile. If you want to generate a source profile with an ST.2084 curve, nothing is stopping you (I think). At worst, maybe you could run into numerical precision issues with some CMSes due to the huge dynamic range.

yes I think constructing a source profile would work that way but it's not a desirable solution. IMO it's better to decouple it and use an ideal source to destination link switch (like BT.1886 is handled) and also to incorporate destination black and peak white ala BT.2390. :)

madshi

10th July 2016, 21:37

madVR v0.90.22 released

http://madshi.net/madVR.zip

* added Shiandow's new (Cross) Bilateral chroma upscaler version
* optimized black bar detection performance, should now work for 4K 60p
* removed Anti-Bloating for SuperRes, because it didn't really help
* fixed: mixed x/y SSIM2D downscaling produced green screen
* fixed: doubling decision was based on window size instead of video size
* fixed: HDR peak luminance of 120/180 nits negatively affected SDR playback
* fixed: HDR "measure each frame's peak luminance" froze with Intel/NVidia
* fixed: HDR compressed/clipped highlights were ever so slightly too saturated
* HDR: disabled HDR processing now disables 3dlut, color and gamma tweaks
* HDR: improved quality for 120 and 180 nits
* HDR: file "DisableHdrBrightnessTweak" disables 1xx nits special handling
You can now choose between the "old" Bilateral chroma upscaler (from v0.90.21 and older) and two new versions, "soft" vs "sharp". Please let me know which of the 3 algos you like best. They all have advantages and disadvantages, depending on which images you test with. So don't limit your testing to just one test image, please.

har3inger

10th July 2016, 22:25

Just tested out the new filters and have some quick impressions.

The old bilateral scaler is the fastest by far of the three. It sometimes struggles with a noisy/irregular look for edges and lines in the chroma layer, but these tend to be hidden by the same line in the luma layer, mitigating its visual impact. This scaler (and all the other bilateral ones) suffers problems with desaturating thin lines of color, and occasionally completely erases edge lines (for cartoons). Both these problems are almost completely solved with superres2 in chroma, at the cost of slightly increased edge irregularity and ringing. In my experience, the old bilateral plus sr2 is better than nnedi3 with or without SR in almost all scenarios.

Soft: in a lot of ways looks like the old bilateral algorithm, but with much smoother and cleaner edges. The downside is performance and a tendency to randomly emphasize the 2x2 pixelation in the chroma layer as edges in certain scenes, while no other algorithm in madvr does this. I'll post up some screenshots at some point. Erases edge lines less than the old algorithm, but is still a long ways away from nnedi64 in preserving the original intent of those edge lines. I'd switch to using this plus SR1 or SR2 in a heartbeat if the pixelation issues are solved and performance is increased. Otherwise, doesn't really justify use over the old one.

Sharp: Damn those edges are clean. Very sharp overall, and haven't seen any bad artifacts resulting from that yet. Suffers the same pixelation issue as the soft variant, except they look even more obvious. Sometimes the details in the chroma end up sharper than other (non-overlapping) details in the luma, which can look a little strange.

For now, mainly sticking to the old algorithm because the new ones don't improve enough on it for their performance costs.

Images demonstrating pixelation: (look at blood spatter on left half of image, particularly over the guy's right ankle)

nnedi64: https://i.imgsafe.org/2c02181112.png
old with SR2: https://i.imgsafe.org/2c024d841f.png
old: https://i.imgsafe.org/2c0247a68b.png
soft: https://i.imgsafe.org/2c025200c0.png
sharp: https://i.imgsafe.org/2c025c831f.png

Shiandow: PM me if you want the source where those screenshots came from.

madshi

10th July 2016, 22:39

Thanks for the feedback! Shiandow would probably appreciate some samples (+ screenshots) which demonstrate those "pixelation" issues.

leeperry

10th July 2016, 22:47

zoyd

10th July 2016, 22:49

madVR v0.90.22 released

* fixed: HDR "measure each frame's peak luminance" froze with Intel/NVidia

Thanks, that fixed the crash/freeze I was seeing.

Warner306

10th July 2016, 23:08

Thanks a lot for the new build but I really liked SR's AB, what could ever bring it back please? Screenshots comparisons of some sort? Was the "it didn't really help" reason purely subjective on your end and if so, is it prone to reevaluation ? :o

My screenshot comparisons showed SuperRes anti-bloat suppressed valid sharpening because SuperRes doesn't really bloat if it isn't oversharpening. You are probably doing more harm than good.

Warner306

10th July 2016, 23:09

SuperRes anti-ringing could also be removed. That's just my opinion.

Shiandow

10th July 2016, 23:15

Just tested out the new filters and have some quick impressions.

[...]

For now, mainly sticking to the old algorithm because the new ones don't improve enough on it for their performance costs.

It should be possible to improve performance somewhat (especially the 'sharp' version could be made faster), but for now I'm focusing on improving the quality.

leeperry

11th July 2016, 01:45

SuperRes anti-ringing could also be removed.
Fully agreed, SR doesn't ring to begin with.

My screenshot comparisons showed SuperRes anti-bloat suppressed valid sharpening because SuperRes doesn't really bloat if it isn't oversharpening. You are probably doing more harm than good.
That's the thing, IIRC you like a very sharp picture but I don't, there's no such thing as "valid sharpening" to me and my personal goal is to reach the sharpest/most natural compromise and 75% AB for SR@2 did just that with sxbr75 to my eyes, by removing some of the harsh digital look(especially on 360x288@1080p refined after every 2x step).....I just tried to find another way to do what I want but I very seriously miss SR AB(especially with the new deringer code that's drastically sharper than the very first build), hopefully I can change madshi's mind somehow =/

huhn

11th July 2016, 02:08

than post picture that show the benefit.

wouldn't be the first time that this would change his mind.

leeperry

11th July 2016, 02:11

sure thing, reason why I asked madshi whether it was all subjective and/or what screenshots would change his mind but I really don't see why AR stayed and AB left :(

I also can't find any practical use for the built-in AB versions of sxbr for that matter but you can leave them there, I don't mind ^^

James Freeman

11th July 2016, 04:45

madVR v0.90.22 released

Thank you, all is well.

QBhd

11th July 2016, 05:13

Thanks for the mixed scaling fix. All seems to be good. I can't thank you enough for your time and skills!

QB

AngelGraves13

11th July 2016, 05:19

SR AB was nice especially for upscaled MPEG-2 DVDs. It really smoothed out the rough edges in the transfers and made SR look smoother.

SR AR on the other hand does practically nothing and should be removed.

madshi

11th July 2016, 07:52

You guys know the drill: Please provide me with samples + screenshots which show that SuperRes Anti-Bloating helps in some situations, and I will reconsider. I've removed it because I couldn't really see any benefit in my own tests, and the feedback from the majority of users seemed to confirm that.

Anti-ringing for sharpening and SuperRes is not finished yet, it will be improved in a future version. My SuperRes shader has had some fast and effective anti-ringing code built into it for a long time, which is really useful and will definitely stay. However, the extra SuperRes anti-ringing option adds another anti-ringing shader pass on top of that. In most cases this extra option does nothing, so maybe I'll remove it, but I wanted to wait until I've implemented the "final" version and then do some tests with real world images and test patterns to see if it might not sometimes have a positive effect on SuperRes. I do think it might, because the built in anti-ringing has some limitations and isn't always perfect.

Betroz

11th July 2016, 08:54

@madshi

When your changelog states that you have improved performance with some of your algos, is it posible to say by how much? Like 10ms lower rendering time or something like that.

madshi

11th July 2016, 09:34

Betroz

11th July 2016, 09:40

Usually I can't say in ms because it depends on the GPU. Could maybe say in percent, but even that might be GPU dependent. The only performance improvement in 0.90.22 is black bar detection, which in this rare case is actually CPU dependent. Black bar detection CPU cost should be about 3x lower now than it was before.

Yes people have different hardware in their systems, so that makes sense of course. It's just that madvr 0.90.17 workes well on my HTPC now, so I'm wondering if upgrading is worth it for me, and hense the performance question. Thanks anyway :)

madshi

11th July 2016, 09:42

Well, if you don't need deringing, the new HDR algos, or the new Bilateral algos, and if you haven't run into any of the 0.90.17 bugs then there's no problem staying with 0.90.17.

Betroz

11th July 2016, 12:56

Well the deringing feature is one that I need, so I will upgrade some day. Thank you for your work and support madshi :)

foozoor

11th July 2016, 13:00

Could you re-add the one-pass adaptive sharpen?

A guy (igv) ported and improved this shader for the mpv hook system:
https://gist.github.com/igv/8a77e4eb8276753b54bb94c1c50c317e

I don't really know what he improved but it looks amazingly on my sister's laptop with a weak intel hd3000 iGPU. The two-passes version is too intensive!

I prefer adaptive-sharpen than crispen-edge/finesharp.

Thanks again for this build! ;)

madshi

11th July 2016, 13:05

Could you re-add the one-pass adaptive sharpen?

A guy (igv) ported and improved this shader for the mpv hook system:
https://gist.github.com/igv/8a77e4eb8276753b54bb94c1c50c317e

I don't really know what he improved but it looks amazingly on my sister's laptop with a weak intel hd3000 iGPU. The two-passes version is too intensive!
He modified the 2-pass version into 1-pass, by removing some features. So speed is better, but quality is slightly lower. I'm not sure if that is what many users want?

chros

11th July 2016, 13:23

The only performance improvement in 0.90.22 is black bar detection, which in this rare case is actually CPU dependent. Black bar detection CPU cost should be about 3x lower now than it was before.
1. Thank You so much for making this multithreading! I CAN use this now on my old rig, and it can do wonders with 4K content! :) (Now I can remove my processing profiles, that were created for only this reason.)
2. All the HDR related bugs are fixed (that I know of) that were present in 0.9.21. Thanks!
Yes people have different hardware in their systems, so that makes sense of course. It's just that madvr 0.90.17 workes well on my HTPC now, so I'm wondering if upgrading is worth it for me, and hense the performance question. Thanks anyway :)
Go ahead to 0.9.22, it seems to work fine.

Sarasa

11th July 2016, 13:30

...
Try reinstalling your GPU drivers, or try a different GPU driver version.
...
After some more test I found my problem, it's not a driver one, it's my alim that starting to show sign of "to old" (7 Years), when the GPU kick to high gear it ask more that the alim can still give and bang :(
Oh well, was going to change it next month, seem I need to advance the schedule

James Freeman

11th July 2016, 15:07

I have noticed that the HDR algorithm clips the highlights even when compress highlights is selected.
Is it how it should be? I thought compress = NOT Clip.

madshi

11th July 2016, 15:24

It should not clip, unless the frame peak luminance is higher than the metadata claims.

James Freeman

11th July 2016, 15:40

Ah, okay.
I also noticed that "preserve Hue" will clip the colors if the color is too saturated.

Isn't the point of "measure peak luminance" is to prevent clipping, or it works only when the measured luminance is lower than the metadata peak?
This does not take into account that the mastering monitor may have ABL and the peak luminance actually below the metadata value.

I think the image quality would benifit from measuring the peak luminance even if it slightly higher that the metadata value to prevent clipping.

madshi

11th July 2016, 15:43

"Preserve hue" does not clip, it desaturates in a hue linear color space. Please re-read the tone mapping discussions on the previous 5 pages or so of this thread. This topic has been discussed extensively there.

James Freeman

11th July 2016, 16:00

Perhaps it would be better if I post images of what I see and then ask questions.

Compress only:
http://www.mediafire.com/convkey/8dce/f9v3yqnlt2q1452zg.jpg

Compress and Hue (HQ):
http://www.mediafire.com/convkey/710b/ghslorwws4ba66lzg.jpg

At 400nit.
It looks like the brightest areas of the headlights and tent are clipped when I enable Hue correction.
Or maybe they are so desaturated that the gradations are completely lost?

Did you consider my previous post that ABL might effect the true peak luminance and in reality it may be lower than the metadata value but higher in code value?
I heard that at least few movies were mastered on the Samsung HDR TV that has obvious ABL behavior with anything above 2% window.

I believe many (if not all) of the professional studio HDR monitors also have ABL to some extent, so it will not be wise to clip AT the metadata value.

madshi

11th July 2016, 16:16

As I've explained a couple of pages ago, after tone mapping (compression) there may be colored pixels (e.g. red/orage in this case) which have a luminance value assigned to it which is near to the peak luminance of the video. E.g. I suspect those pixels which are red/orage without hue correction, are near to the peak luminance of the video. The problem with that is that your display reaches its peak luminance with *white* pixels, not with red pixels. So if your display can only do 400 nits, and the video asks for a 400 nits red pixel, what can madVR do? Without hue correction, I simply do nothing. Which means the RGB pixel will go out of bounds (outside of 0-255 for PC levels), which without hue correction will then simply be clipped to the legal RGB range. This will modify hue, but it might preserve some color differences. If you enable hue correction, madVR detects this situation and it will then desaturate such 400 nits red pixels just as much as necessary to make them fit into the legal RGB range. In this case they get almost or even completely desaturated/white. Yellow is less effected because it has more green in it than red, and green produces much more luminance than red. So in this situation hue correction may lower perceived detail because the difference between white and bright yellow is much smaller than the difference between red and bright yellow. However, without hue correction, the exact hue isn't reproduced correctly, and the luminance is WAY too dark.

I don't think this is a case of incorrect metadata or ABL related problems.

P.S: You can see the same effect in the spheric tent in the background. Without hue correction it's WAY too dark, but is nicely red. This is likely again an area which is near to the video's peak luminance, but with a highly saturated red color. Turning hue correction on will detect this and desaturate the pixel accordingly. So the pixel goes towards white, which is not nice, but which helps reproducing the correct luminance.

James Freeman

11th July 2016, 16:22

Now I understand , thanks.

The ABL is a different issue, I just posted it in the same go; I see clipped highlights where they should not be, the reason I assume is ABL when mastering.
Not to be repetitive but: true peak luminance in reality may be lower than the metadata value but higher in code value.
Therefor it is not wise to clip at the metadata value in software.
Leave some "headroom" to compensate for the ABL behavior of the mastering monitor.

Or even better, measure the peak white code value even if it's above the metadata peak value.

I hope this makes sense.

madshi

11th July 2016, 16:27

Leaving headroom also means highlights getting duller. There's no free lunch here.

James Freeman

11th July 2016, 16:31

That's where your "measure peak white" comes in, but it should do its job with code values above metadata values also, not only below like it does now (I assume).

The colorist is not limited to the code values of the peak luminance of his monitor, he might use higher code values while pushing the ABL of his monitor.
We must remember that the code values control the LED backlight, and they do NOT actually clip the code values if the the backlight can't go brighter.

James Freeman

11th July 2016, 17:23

To solidify my statements about ABL and code values, here is an image from the LG_Chess_HDR video:
With HDR OFF, his nose has values of 255 which equate to 10,000 nit but the metadata peaks at 1000 nit.
http://www.mediafire.com/convkey/05f5/3bi36d5zz0cydwjzg.jpg

May I suggest a solution, not only complaint.

When the measured peak white is below the peak metadata value, the measured luminance value should be stretched like it does now (if I'm not mistaken).
But when the measured peak value is higher than the metadata (like this example) the luminance should be given headroom in % (not a free lunch).

In other words, give 105% as the new luminance, and super compress anything above metadata into these 5%.
Example: If metadata is 1000nit and there is code values above that, compress them to 1050 nit.
We lose 5% in average luminance but we do not lose picture information which is way more important.

EDIT:
If we actually measure the code/bit values, many of the HDR test videos have code values above the peak metadata value, which currently madvr clips above.

leeperry

11th July 2016, 20:22

You guys know the drill: Please provide me with samples + screenshots which show that SuperRes Anti-Bloating helps in some situations, and I will reconsider. I've removed it because I couldn't really see any benefit in my own tests, and the feedback from the majority of users seemed to confirm that.

Fair enough, in order to make my screenshots comparison valid I went back to your original description of AB that went like "The new anti-bloating tries to concentrate sharpening on higher frequencies and to remove lower frequencies. I like the look that produces, but your mileage may vary", that's basically what it's boiling down to my eyes with the lighthouse picture I regularly use downscaled to 640*360 and then reupscaled to 1920*180: it looks less edgy, obviously softer and I really really dig the look on moving objects and especially on double SR refined after every 2X step in combination with 60Hz FRC and noisy low bit-rate low-resolution footage....so at the end of the day it's still entirely subjective I'm afraid? I'm just not sure what my screenshots would have to prove? Should I use test patterns in order to make it less subjective? :o

leeperry

11th July 2016, 21:17

OK here goes, please don't shoot, 1080p original Vs SR@2 LL Vs SR@2 LL+AB75%, dithering was disabled:

http://screenshotcomparison.com/comparison/178286

http://screenshotcomparison.com/comparison/178284

http://screenshotcomparison.com/comparison/178285

All files can be found here (https://mega.nz/#!vsJSSDxY!v4cIHS0_3P3BNdMJ8UaYKo448tcIvISnAyigts73l0U).

Long story short edges are less noisy and it looks wonderful with FRC, double SR@2 after every 2X upscale looks seriously edgy otherwise IME. It might also be closer to the ground truth, at least edges would appear to be.

madshi

11th July 2016, 21:49

May I suggest a solution, not only complaint.

When the measured peak white is below the peak metadata value, the measured luminance value should be stretched like it does now (if I'm not mistaken).
But when the measured peak value is higher than the metadata (like this example) the luminance should be given headroom in % (not a free lunch).

In other words, give 105% as the new luminance, and super compress anything above metadata into these 5%.
Example: If metadata is 1000nit and there is code values above that, compress them to 1050 nit.
We lose 5% in average luminance but we do not lose picture information which is way more important.
But what should I do if the measured peak value is only 4% higher than the metadata? Should I still increase headroom by 5%? What if it's 7% higher? Or 170% higher? A proper solution would probably have to change the added headroom "size" depending on the amount of overshoot.

And then there's the problem that this kind of thing would require me to piece 3 line/curve segments together, which makes the math quite a bit more complicated.

Furthermore, compression already gets much stronger the higher the peak is. In order to compress a large overshoot into just 5% more room would require even stronger compression ratios which I'm not sure would look much different to clipping.

Overall I'm not sure it's worth spending lots and lots of development time and effort into a probably very small improvement like this. I'm not even sure there would be a visible benefit at all. I think the main visual problems don't come from luminance overshoots, but from highly saturated pure color highlights, as explained before.

OK here goes, please don't shoot, 1080p original Vs SR@2 LL Vs SR@2 LL+AB75%, dithering was disabled:

http://screenshotcomparison.com/comparison/178286

http://screenshotcomparison.com/comparison/178284

http://screenshotcomparison.com/comparison/178285

All files can be found here (https://mega.nz/#!vsJSSDxY!v4cIHS0_3P3BNdMJ8UaYKo448tcIvISnAyigts73l0U).

Long story short edges are less noisy and it looks wonderful with FRC and double SR@2 after every 2X upscale looks seriously edgy otherwise IME. It might also be closer to the ground truth, at least edges would appear to be.
Just to complete the picture, could you please add screenshots for the following 3 configuration variants?

1) SR@1 with AB25.
2) SR@1 with AB disabled.
3) SR disabled.

The purpose is to find out whether maybe activating AB maybe simply just decreases the effect/strength of SR without having any other benefit. Thanks.

leeperry

11th July 2016, 22:20

Just to complete the picture, could you please add screenshots for the following 3 configuration variants?

1) SR@1 with AB25.
2) SR@1 with AB disabled.
3) SR disabled.

The purpose is to find out whether maybe activating AB maybe simply just decreases the effect/strength of SR without having any other benefit.
OK, here goes:

http://screenshotcomparison.com/comparison/178293

http://screenshotcomparison.com/comparison/178295

http://screenshotcomparison.com/comparison/178298

files on mega (https://mega.nz/#!q8ZD1a5b!y0GwLF6MjqVZSK8-1ggtrNsWcEiTIIhf--8oCj48-Ig)

I got a good bunch of other features enabled in the upscaling refinement window, my point was in my real-world use AB75 does wonders. Of course SR looks sharper than without but I believe the top left of the picture in my first link with SR1 looks better with AB25 than without. The whole point of AB to me is that motion looks smoother because edges aren't nearly as noisy anymore. Anyway, please bring it back :scared:

zoyd

11th July 2016, 22:41

But what should I do if the measured peak value is only 4% higher than the metadata? Should I still increase headroom by 5%? What if it's 7% higher? Or 170% higher? A proper solution would probably have to change the added headroom "size" depending on the amount of overshoot.

I don't know what you do in a case like this. There is no part of the PQ encoding standard that allows for overshoot so I would assume it's either a bad encode or the metadata is wrong. The chess clip specifies a max luminance of 1000 nits but clearly has codes above that, especially in the chess piece highlights. I've checked both the Sony sample (bright setting sun) as well as a reference clip I have, and all codes are within their specified max luminance of 4000 nits.

no HDR processing
https://dl.dropboxusercontent.com/u/2621383/20160711172425_HDMI1.jpg

Warner306

12th July 2016, 04:22

This is my previously posted comparison of SuperRes with and without anti-bloating:

I have a couple of examples of where anti-bloating removes detail that would be considered valid sharpening.

SD -> FHD
super-xbr100 + SuperRes (3)

In the first example, search for the headline "Rick Nielsen: no cheap collector" in the middle of the image. SuperRes does a good job of making the original letters more clear and easy to read. While anti-bloating returns these letters to a blurred smear.

The rest of the lettering, especially on the lower part of the guitar shows the same improvement and then degradation when anti-bloating is enabled.

Guitar (https://drive.google.com/open?id=0B2LAr9InPMjta3NmYTdrSXNaUEE)
Guitar SuperRes (3) (https://drive.google.com/file/d/0B2LAr9InPMjtRkZYLVZVNEM0TU0/view?usp=sharing)
Guitar SuperRes (3) + Anti-Bloating 100% (https://drive.google.com/open?id=0B2LAr9InPMjtVGxNa3IzS2tNcUU)

In the second example, look at the leather patches on Kata Mara's arm and the black guy.

Also, the objects in the shadows (top-right) become obscured when anti-bloating is enabled. SuperRes again slightly enhances the low-res original and anti-bloating effectively cancels this out.

Fantastic Four (https://drive.google.com/file/d/0B2LAr9InPMjtQURBVmF6R0ZJSGM/view?usp=sharing)
Fantastic Four SuperRes (3) (https://drive.google.com/file/d/0B2LAr9InPMjtMXhaeGk5WXNSaU0/view?usp=sharing)
Fantastic Four SuperRes (3) + Anti-Bloating 100% (https://drive.google.com/file/d/0B2LAr9InPMjtMm9YZVpqNnFTaVk/view?usp=sharing)

So far, only my original test where SuperRes (3) was combined with crispen edges (1.0) did anti-bloating prove to be an effective countermeasure for SuperRes. I am thankful to see that SuperRes, at the right strength, does not oversharpen the image. This has completely changed my settings preferences.

James Freeman

12th July 2016, 04:58

Overall I'm not sure it's worth spending lots and lots of development time and effort into a probably very small improvement like this. I'm not even sure there would be a visible benefit at all.

I don't know what you do in a case like this. There is no part of the PQ encoding standard that allows for overshoot so I would assume it's either a bad encode or the metadata is wrong.

I agree, it is not worth fixing a bad encoding in madVR.

Yet again, we must remember that the code values control the LED backlight, and the code value do NOT actually clip if the the backlight can't go brighter.
Lets just hope no colorist relies on that trait of current HDR TVs/mastering Monitors because in the future a TV might just show these 10,000 nit peaks and look blindingly bright in the wrong spots.

In my example with the nose and zoyds example with the chess piece the peak brightness is 10,000 nit in some places while the peak metadata is 1000 nit.
We should never see a chess piece or a human nose radiating as bright as the sun... so this LG_Chess video has some "noob" flaws in terms of HDR encoding.

I may be completely wrong and the camera actually captured 10,000 nit from the boom lights illuminating the scene and reflecting from shiny objects.
So the colorist actually did an excellent job preserving that in the encoding even though the final video was mastered for/on a 1000nit peak display.