madVR - high quality video renderer (GPU assisted) [Archive] - Page 490

leeperry

7th March 2014, 23:47

For those of you guys who already have signs of withdrawal from the lack of dither testing
I guess I'd very much fancy the impression of depth of A4 with the low-noise of ED11.....you've gotten us to expect the impossible, so that's kinda your fault :p

The algorithms are basically the same as in v0.87.6, however, I've added Shiandow's linear light processing idea.
My initial impression with mono-static A4 is that killed the impression of depth on 720p content so I just spent quite some time comparing both builds in mono-static mode, with A4 on the bluray of Captain Harlock and ED11 on very crisp 30fps 1080p video content and in both cases the new build kills the sense of depth to my eyes...the wow factor is gone, PQ is pretty dull.

I guess an extra "use LL" checkbox would be great if it does anything good to anyone in dynamic/color mode but that's a no-go to my eyes in mono-static.

Qaq

7th March 2014, 23:49

..here's one more dither test build:

http://madshi.net/madVRlinearLightDithering.rar

The algorithms are basically the same as in v0.87.6, however, I've added Shiandow's linear light processing idea.
Call me a videophile but this one seems to me a huge step in right direction. MadVR's picture never looked THAT transparent. I wonder if its even possible to make it better (should be!).
No way its just Catalyst (stripped to the bone) does tricks on me :D
I use ED1 with both options disabled.

MistahBonzai

7th March 2014, 23:57

1600х1200@100MHz
Error Diffusion - option 2 enabled

https://mega.co.nz/#!JF92jR6K!PlNXU3xJ01jO3D4cJBSFSn61aYl3zB-t3eZJdj9c0oo (the BEST test video)

OK..Now edit madVR settings:

Goto "scaling algorithms"
Drop down and select "image doubling"
Select "use NNEDI3 to double Luma resolution"
Drop down and select "only if scaling factor is 1.2 (or bigger)"
Select "128 neurons"

Play you test 640x480/30 test video (the BEST test video) and play full screen (Alt + Enter)

Still no frame drops?
Yes - how much GPU are you using?
No? Good!
Repeat setting procedure for "Use NNEDI3 to double Chroma resolution" but select "32 neurons"
Still no frame drops?
Yes? That's normal with your GPU
No? Ditch that Zoom Player ;)

At that point (after selecting NNEDI3 Luma and Chrome up-scaling) my HD-7850 is maxed out. How are you doing?

TheShadowRunner

8th March 2014, 00:03

cyberbeing

8th March 2014, 00:11

madshi, is madVR still using the "settings.bin" file or is it now working exclusively with the registry?
When installing the latest version, i forgot to copy over the "settings.bin" file and all my settings were there so I'm wondering.

The "settings.bin" is only loaded by madVR if the "Settings" registry key is not present. Whenever settings are changed, they are written to both settings.bin and the registry key.

turbojet

8th March 2014, 00:22

Speaking of settings.bin, I don't think in doubt deinterlacing and debanding settings are stored in it, otherwise they are getting reset by new versions.

secvensor

8th March 2014, 00:25

OK..Now edit madVR settings:

Goto "scaling algorithms"
Drop down and select "image doubling"
Select "use NNEDI3 to double Luma resolution"
Drop down and select "only if scaling factor is 1.2 (or bigger)"
Select "128 neurons"

Play you test 640x480/30 test video (the BEST test video) and play full screen (Alt + Enter)

Still no frame drops?
Yes - how much GPU are you using?
No? Good!
Repeat setting procedure for "Use NNEDI3 to double Chroma resolution" but select "32 neurons"
Still no frame drops?
Yes? That's normal with your GPU
No? Ditch that Zoom Player ;)

At that point (after selecting NNEDI3 Luma and Chrome up-scaling) my HD-7850 is maxed out. How are you doing?

MPC-HC last ver.
128 no drop GPU use 0~41~91%
32 no drop GPU use 0~41~91%
Zoom Player:
no drop GPU use 0~41~91%
:devil::devil::devil:
256х4
no drop GPU use 0~41~91%
---------------------------
Jink 8/Jink 8 / Catroom:
drope 50-100 frames per sec. on 1600x1200@100MHz (1080p10bit60fps).

bacondither

8th March 2014, 00:40

At least switching between 8bit and 3bit doesn't brighten up the image, anymore. So it does exactly what it's supposed to do. Let me know what you think! FWIW, it will probably cost a bit of extra performance, but I hope it will not be too much. The test build always applies linear light processing for dithering, for both ordered dithering and the 2 error diffusion algorithms.
Well, the image is not brighter anymore but darker areas are darker then they should be.
I suppose you are using Shiandow code with a decode gamma of 1/0.45. Use a decode gamma of 1/0.53, works much better with BT.709 content which most content is.

And yes, even with 1/0.53 decoding gamma a tiny amout of dark parts will be darker then then the original. But a larger part of the image will be closer to linear.

Example images(ordered dither):

3-bit linear light dither (http://i.imgur.com/ZP9K0Mi.png)
8-bit linear light dither (http://i.imgur.com/ObFfYJB.jpg)

1/0.45 and 1/0.53 decoding gamma applied to a BT.709 encoded gamma.
Values normalised so that y values of 1 is 100% of linear light output, y at 0.6 is 60% bright as linear light output and so on...
0 on the x axis is black, 1 is white.

http://i.imgur.com/Y60zAlS.png

cyberbeing

8th March 2014, 00:46

Speaking of settings.bin, I don't think in doubt deinterlacing and debanding settings are stored in it, otherwise they are getting reset by new versions.

Both are stored in settings bin, the only things which are not stored are things like "custom" debanding parameters, which is rather annoying. Though as mentioned, the settings.bin is ignored unless HKEY_CURRENT_USER\Software\madshi\madVR\Settings does not exist. That registry key always takes priority, if it exists and is newer.

Shiandow

8th March 2014, 00:46

I guess an extra "use LL" checkbox would be great if it does anything good to anyone in dynamic/color mode but that's a no-go to my eyes in mono-static.

Well, assuming madshi just used the code I posted you could also invert the LL processing using the following shader:

#define depth 256
sampler s0 : register(s0);

float4 f(float4 x) { return pow(x,1/0.45); }
float4 finv(float4 x) { return pow(x,0.45); }

float4 main(float2 tex : TEXCOORD0) : COLOR
{
float4 c0 = tex2D(s0, tex);
float4 low = floor(depth*c0)/depth;
float4 hi = ceil (depth*c0)/depth;
float4 c1 = (c0-low)*depth;
return finv(c1*(f(hi)-f(low))+f(low));
}

Bit of a waste of GPU power :D, but it should effectively remove the LL processing. This code assumes you're using 8bit output, but otherwise just change 'depth' to 2^k where k is the bitdepth you're using.

Shiandow

8th March 2014, 00:56

Well, the image is not brighter anymore but darker areas are darker then they should be.
I suppose you are using Shiandow code with a decode gamma of 1/0.45. Use a decode gamma of 1/0.53, works much better with BT.709 content which most content is.

In theory the code should use the gamma curve of the monitor, not the gamma curve of the encoding. But 1/0.45 was just a guess, if I recall correctly madshi used that gamma in one of the first Linear Light test builds, so I figured that was the default. Anyway you shouldn't base your choice for the gamma on the content but on the monitors.

Edit: Oh and you probably shouldn't base your choice on the absolute/relative difference, but rather the difference in (local) gamma, or some other measure of 'curvature'.

MistahBonzai

8th March 2014, 01:02

Madshi, While attempting to help out secvensor I reconfigured MPC-HC,MadVR,AC3filter,LAV to test some recommendations I was making...

Disconnected ffdshow/avisynth processing.

MadVR:

Selected Luma and Chrome upscaling (128 and 32 neurons)
Selected ED Option 2 (both options selected).
Chroma and image upscaling (Bicubic,sharpness 75, AR).
Image downscaling (Bicubic, sharpness 100, AR and LL)
Banding (low,med)
Trade quality... All clear.

Test video:

bbb_sunflower_1080p_60fps_normal.mp4 (http://ftp.dk.debian.org/blender/blender-demo/movies/BBB/bbb_sunflower_1080p_60fps_normal.mp4)

Shaders:

Vibrance Anleck and LumaSharpen 1.4.1 (sharp values).

Display:

Playback with FSE on well calibrated 40" 1080P/60hz display.

Holee $h!t. Sharp? Hell yeah! But I'm not seeing any real artifacts. I have grown complacent with trade-offs trying to balance CPU & GPU usage while image doubling and the like. After cutting it all free I gotta say this was a real 'eye opener'. Thanks - UR the Bomb :cool:

BTW:I'm not ready to give up 'soap opera video' yet. As far as I'm concerned they should have ditched 24FPS and interlace long ago. Both are hold overs from a bygone world :D (ducking)

bacondither

8th March 2014, 01:09

In theory the code should use the gamma curve of the monitor, not the gamma curve of the encoding. But 1/0.45 was just a guess, if I recall correctly madshi used that gamma in one of the first Linear Light test builds, so I figured that was the default. Anyway you shouldn't base your choice for the gamma on the content but on the monitors.

Edit: Oh and you probably shouldn't base your choice on the absolute difference, but rather the difference in (local) gamma, or some other measure of 'curvature'.

No, when converting a gamma encoded signal to linear light you should NOT use the gamma of the display. The signal should be decoded with the decode gamma or invese transform of the encode gamma.

1/0.53 is a pure power aproximation of bt709 and will work better with every display then 1/0.45 which undercompensates.

Shiandow

8th March 2014, 01:13

No, when converting a gamma encoded signal to linear light you should NOT use the gamma of the display. The signal should be decoded with the decode gamma or invese transform of the encode gamma.

The problem is that the algorithm isn't exactly converting the image to linear light, it is just estimating what percentage of pixels should be rounded up/down under the assumption that the screen has a certain gamma curve.

cyberbeing

8th March 2014, 01:16

secvensor

8th March 2014, 01:17

MistahBonzai
?:
http://i33.fastpic.ru/big/2014/0308/ba/92e4e4a8a220edf29eee86c974ec9cba.jpg

http://i33.fastpic.ru/big/2014/0308/62/da30c91fc2a3ea43bcc76c249d5b2462.jpg

bacondither

8th March 2014, 01:26

The problem is that the algorithm isn't exactly converting the image to linear light, it is just estimating what percentage of pixels should be rounded up/down under the assumption that the screen has a certain gamma curve.

Yes, real convertion to linear light would make the whole image very dark.

The algorithm corrects the gamma between output integer steps?.

If i use you code with a gamma of 1000 i get almost the same result as with no dithering. :D

6233638

8th March 2014, 01:30

1/0.53 is a pure power aproximation of bt709 and will work better with every display then 1/0.45 which undercompensates.Isn't 0.51 normally used?

And why not just use the exact inverse of the BT.709 gamma?

cyberbeing

8th March 2014, 01:30

?

This isn't a bug. Disabling "present several frame in advance" enables the FSE 'Old Path' which uses similar display logic to Windowed mode. If you find performace is better with the less optimal 'Old Path' and you are using refresh rates higher than the video framerate, then your GPU must be present bottlenecked. In general, AMD GPUs have this issue, so you need to use low GPU queue and Presentation queue values like 6.

Shiandow

8th March 2014, 01:35

Either way, it's problematic if madVR linear light dithering really is still causing any kind of output gamma shift (Input != Output) like bacondither seems to suggest.

In madVR, it was an intentional design decision to have unmodified output gamma by default.

Well, this method of correcting gamma might indeed be wrong, and using a gamma of 1/0.53 might also be a better choice. But not for the reasons bacondither mentioned. The algorithm also isn't supposed to change the gamma in any way, since it doesn't change values that don't need dithering. It only change the way dithering interpolates between two adjacent colours, slightly. It does this in such a way that for a monitor with a pure power gamma curve and a gamma of 1/0.45, the 'effective' gamma curve (i.e. one where you include dithered values) is a pure power gamma curve as well.

bacondither

8th March 2014, 01:48

Kalanoch

8th March 2014, 01:52

Oh man, that's bad. I think this is probably cause by the cosmetical improvements I made in 0.87.5. That seems to work fine for most users, but seemingly it crashes on your PC. Don't know why, though.

How do those crashes look like? Does MPC-HC automatically disappear without any trace left? Or is there still some sort of crash complaint window or something? And even if you wait a couple of seconds the madVR crash assistant doesn't appear?

Anyway, here's a test build:

http://madshi.net/madVRoverlayFix.rar

Does it fix the problem for you? If so, please also try resizing the window multiple times, just to make sure that's stable for you, too.

When it crashes, MPC-HC will open up, initialize a black screen with the correct video size, then madVR pops up the "Please wait a moment..." box and starts working, then everything stops and gets grayed out as windows gives me the standard "MPC-HC has stopped working" crash dialog. The audio starts playing for a split second before it crashes, also. Sometimes I can very briefly see the OSD before the crash, but usually not.

In any case, the test build you provided fixed the issue, including resizing/fullscreen windowed overlay. Also, the problem is being caused by my dual monitor setup, because I did some more testing with 0.87.6 and found that if I remove my second monitor I have no issues. Since I'm surely not the only person running madVR and dual monitors, perhaps it has something to do with my weird resolutions? (2560x1600 on primary and 1920x1200 on secondary)

seiyafan

8th March 2014, 01:53

More test result:

(doubling neurons/quadrupling neurons)

http://s12.postimg.org/5h25q5qij/madvr6.png

Shiandow

8th March 2014, 02:04

So we should then use the "the displays is calibrated to the following transfer function / gamma:" to set the pseudo linear light target gamma?.

My main display has a gamma approx to sRGB, which has a lower gamma near black then a pure power curve would have.
Maybe it could explain the slightly darker dark shades with pseudo linear light dithering i am experiencing.

It would indeed be best to use the settings for that, but if we can get close enough (on average) then that shouldn't be necessary.

Also you shouldn't forget that the error diffusion algorithms have a tendency to make dark colours darker (partially by design). So maybe if you think that dark shades are too dark then perhaps you should check if this is also true when you use ordered dithering. If that doesn't help then perhaps you could try using the shader I posted but with a different gamma curve. You should also keep in mind that the difference between the 'assumed' gamma curve and the actual gamma curve of the monitor become far less significant if you raise the bitdepth.

cyberbeing

8th March 2014, 02:05

The algorithm also isn't supposed to change the gamma in any way, since it doesn't change values that don't need dithering.

When using a 3DLUT for gamut correction, it's safe to assume the majority of pixels end up dithered. Same with 10bit video to some extent. Add debanding or other image processing on top and even more dithering will be needed. The goal should be for the dithering algorithm to maintain identical gamma (near-black, mid-tones, near-white, overall average) even when nearly 100% of pixels are being dithered in 8-bit mode.

More test result:

(doubling neurons/quadrupling neurons)

http://s12.postimg.org/5h25q5qij/madvr6.png
Just a heads-up that the direct links for most all of these images you are uploading to postimg do not work with IMG tags on the forum. For example, with that image (http://s12.postimg.org/5h25q5qij/madvr6.png) if I open it as a url instead, I do not get an direct image but am instead re-directed to this url (http://postimg.org/image/cx1fbye7t/) with a direct image link of this (http://s12.postimg.org/cx1fbye7x/madvr6.png).

Shiandow

8th March 2014, 02:40

When using a 3DLUT for gamut correction, it's safe to assume the majority of pixels end up dithered. Same with 10bit video to some extent. Add debanding or other image processing on top and even more dithering will be needed. The goal should be for the dithering algorithm to maintain identical gamma even when nearly 100% of pixels are being dithered in 8-bit mode.

Well, you might detect some very small differences in gamma, but it really shouldn't affect the gamma curve that much. What I was trying to say is that since there are a lot of places where the 'effective' gamma curve is going to be identical to the gamma curve of the monitor, it shouldn't noticeably change the gamma. It can affect the 'local' gamma a bit, so we should check that this doesn't have weird consequences, but the original algorithm was worse in that respect since it's 'local' gamma was 1 almost everywhere.

cyberbeing

8th March 2014, 02:50

At the end of the day, this just means that fractional 8-bit colors need to always average as close as possible to those optimal fractional values over a decent sized surface area. You'll only have a gamma shift (localized or otherwise) if the dithering algorithm fails to simulate the fractional 8bit color values of the correct target. If you have a 16-bit color half-way between 8-bit 1 & 2, the only correct answer is an average value of 1.5 after dithering. This is not something open to interpretation between gamma or linear light. There is always only a single correct answer, based on the source RGB values prior to dithering. The only difference should be how the dithering algorithm chooses to shuffle pixels to arrive at the target value.

turbojet

8th March 2014, 02:52

Originally Posted by turbojet View Post
Speaking of settings.bin, I don't think in doubt deinterlacing and debanding settings are stored in it, otherwise they are getting reset by new versions.

Both are stored in settings bin, the only things which are not stored are things like "custom" debanding parameters, which is rather annoying. Though as mentioned, the settings.bin is ignored unless HKEY_CURRENT_USER\Software\madshi\madVR\Settings does not exist. That registry key always takes priority, if it exists and is newer.

On source computer madvr has been set for months to deinterlace when in doubt and debanding was enabled at medium/medium. Copying the madvr folder with settings.bin inside over to 2 new windows installs, deinterlacing was deactivated when in doubt, debanding was disabled and set to low/high. The rest of the settings stayed the same.

I think I've seen deinterlacing switching to disabled when updating to one of the more recent builds, either an 83.x or 84.x

cyberbeing

8th March 2014, 02:55

On source computer madvr has been set for months to deinterlace when in doubt and debanding was enabled at medium/medium. Copying the madvr folder with settings.bin inside over to 2 new windows installs, deinterlacing was deactivated when in doubt, debanding was disabled and set to low/high.

Have you double-checked to ensure that madVR has read/write permissions to settings.bin in your install folder, and it's actually being updated when you change settings? Do you always run "restore default settings.bat" to delete the registry key before you copy over your backed up settings.bin?

turbojet

8th March 2014, 03:03

madVR writes to settings.bin, no I never used restore default settings, why would that be wanted in this case?

cyberbeing

8th March 2014, 03:08

madVR writes to settings.bin, no I never used restore default settings, why would that be wanted in this case?
settings.bin will not be used unless you run "restore default settings.bat" or otherwise delete the registry key before you copy over the setting.bin you want used to the madVR dir.

Otherwise, if you are sure you're doing everything correctly, maybe it is a bug. I cannot reproduce the behavior you describe though. All my profiles and settings are restored as expected when I backup my settings.bin, run "restore default settings.bat", copy settings.bin to madVR's directory, and start madVR.

Ver Greeneyes

8th March 2014, 03:25

Use a decode gamma of 1/0.53, works much better with BT.709 content which most content is.

And yes, even with 1/0.53 decoding gamma a tiny amout of dark parts will be darker then then the original. But a larger part of the image will be closer to linear.Agreed. As I posted here (http://forum.doom9.org/showthread.php?p=1668400#post1668400) back when the first linear light build was introduced, the best fit pure power function gamma for BT.709 is 1.950477 (rounded for single precision floating point as used by shaders). That works out to about 1.0 / 0.512695, pretty close to your suggested value of 0.53 (and the best fit for the inverse transform, 1.924805, is even closer at 1.0 / 0.519533).

iSunrise

8th March 2014, 03:26

For those of you guys who already have signs of withdrawal from the lack of dither testing, here's one more dither test build:

http://madshi.net/madVRlinearLightDithering.rar

The algorithms are basically the same as in v0.87.6, however, I've added Shiandow's linear light processing idea. The results look quite promising to me. At least switching between 8bit and 3bit doesn't brighten up the image, anymore. So it does exactly what it's supposed to do. Let me know what you think! FWIW, it will probably cost a bit of extra performance, but I hope it will not be too much. The test build always applies linear light processing for dithering, for both ordered dithering and the 2 error diffusion algorithms.
Haha, "signs of withdrawal", you´re funny. I think my eyes enjoyed the rest period quite a bit :D

Seriously, though:

At least from some of the short tests I did with actual movies, brightness-wise, 3bit linear dithered now matches 8bit gamma dithered, while 3bit gamma dithered just looks totally wrong in comparison (completely washed-out blacks). Also, again, brightness-wise, 3bit linear dithered now looks like 8bit linear dithered, so there´s also no brightness-change, anymore.

Here´s a comparison (try an image viewer in slideshow-mode, as it´s really damn obvious):

(Ordered dithering with all options unticked)

linear 3bit vs. linear 8bit (no brightness-change)
http://abload.de/thumb/conan_madrvr_linear_3l3prb.png (http://abload.de/image.php?img=conan_madrvr_linear_3l3prb.png)http://abload.de/thumb/conan_madrvr_linear_8owr2s.png (http://abload.de/image.php?img=conan_madrvr_linear_8owr2s.png)

gamma 3bit vs. gamma 8bit (quite obvious brightness-change)
http://abload.de/thumb/conan_madvr_gamma_3bicorvh.png (http://abload.de/image.php?img=conan_madvr_gamma_3bicorvh.png)http://abload.de/thumb/conan_madvr_gamma_8bic4r3i.png (http://abload.de/image.php?img=conan_madvr_gamma_8bic4r3i.png)

linear 3bit vs. gamma 8bit (again, no brightness-change)
http://abload.de/thumb/conan_madrvr_linear_3l3prb.png (http://abload.de/image.php?img=conan_madrvr_linear_3l3prb.png)http://abload.de/thumb/conan_madvr_gamma_8bic4r3i.png (http://abload.de/image.php?img=conan_madvr_gamma_8bic4r3i.png)

linear 3bit vs. gamma 3bit: (again, quite obvious brightness-change)
http://abload.de/thumb/conan_madrvr_linear_3l3prb.png (http://abload.de/image.php?img=conan_madrvr_linear_3l3prb.png)http://abload.de/thumb/conan_madvr_gamma_3bicorvh.png (http://abload.de/image.php?img=conan_madvr_gamma_3bicorvh.png)

At least from looking at these, LL dithered is a lot more correct than gamma light ever will be at these low bit depths. When looking at actual movies, the dithering results at low bit depths are even more impressive than before, since the only thing you will really notice now, is the added dithering dots, nothing more, nothing less, which IMHO is exactly what we want. I don´t think there´s anything wrong with Shiandow's algorithm as it is, by looking at these images. It just looks way more correct.

Question (maybe a silly one, but I´m curious):
Is this build expected to change anything, when we´re doing any gamma processing? Let´say, we have a calibrated display target, which uses a BT.709/601 curve at 1.90 gamma (recording gamma, for monitoring purposes) and we´re converting that to a pure power curve of, say 2.35 or 2.40 (for watching movies). Is that still done in gamma light/does that interfere in any way with LL dithering? Or is that still a completely separate conversion stage (because it happens WAY before, dithering at the very end)?

turbojet

8th March 2014, 03:28

I can't reproduce now either, although I had 87.5 when copied over and now 87.6 before playing anything but doing that again didn't show any problems either. Nevermind about it, I'll keep an eye on it in the future and see if it happens between versions again.

MistahBonzai

8th March 2014, 03:33

MPC-HC last ver.
128 no drop GPU use 0~41~91%
32 no drop GPU use 0~41~91%
Zoom Player:
no drop GPU use 0~41~91%
:devil::devil::devil:
256х4
no drop GPU use 0~41~91%

Did you actually look at the madVR status update (ctrl + J) to verify that frames weren't being dropped? The GPU running flat at 91% doesn't necessarily mean that 9% is left. My HD-7850 hits the wall at ~93% - at least with "GPU Observer". Watch the MadVR status and note the dropped frame tally for a more realistic summary.

I noted your post concerning queues. I have fiddled with mine time and time again. A lot seems to depend on the nature of the file rendering. Currently I use CPU/12, GPU/6, BB/8, Frames in advance/8 but it's open to change.

BTW: Frame rate has a lot to do with performance so I recommend that you display the 60fps test file at 60fps. That way we are on common ground when comparing data.

TheShadowRunner

8th March 2014, 05:57

The "settings.bin" is only loaded by madVR if the "Settings" registry key is not present. Whenever settings are changed, they are written to both settings.bin and the registry key.
Got you, thank you.

James Freeman

8th March 2014, 06:39

@cyberbeing
@bacondither
@Ver Greeneyes

You don't expect 3-bit to look exactly like 8-bit in Linear Light do you? Its impossible considering 3-bit is only 8 shades.
For example the lower 64 shades in 8-bit will be only 2 shades in 3-bit (Black and dithering dots of the next shade).
Depending on the image, the larger contrast ratio the image has (an extreme would be from Black to White pattern), the less transitional shades it has to work with.

I don't understand why are you thinking about "fixing" this with altered/customized BT.709 curve?
From what I know BT.709 is only opto-electrical (Camera) curve to keep the sensor SNR down, its not supposed to be used as a calibration curve for displays (or as an option in madVR),
nor it should be used as a quick fix to make dithered lower bit depth gamma to look like the original image for that matter.

Use a decode gamma of 1/0.53, works much better with BT.709 content which most content is.
I'm pretty sure you're wrong.
BT.709 should ONLY be used for encoding, it's then meant to be played on a 2.20 (or 2.35) Power Curve or BT.1886 calibrated displays.
Using a BT.709 calibration curve for displays, defies its purpose (lowering the camera noise).
I still don't understand why this option is available in madVR as an output gamma curve.

Personally,
I don't want any BT.709 curve on my dithering or anywhere else for any reason.
Linear Light (inverse power) is how the original content meant to be played (or images to be shown), and that's the way it should be.
What a person does with his display calibration after that is his own business, whether its right, wrong, pretty or ugly.

madshi, hopefully you're reading this and will not make the mistake of enforcing a BT.709 curve on the output.

EDIT:
This post is irrelevant as I misunderstood what the discussion was about before Asmodian cleared that up for me.
Although, I'll leave my ignorance posted...

cyberbeing

8th March 2014, 07:24

Linear Light (power inverse) is an exact representation of the original content, and that's the way it should be.

The question isn't about the virtues of linear light processing, it's about how to implement it properly for dithering. madshi's first attempt with linear light dithering resulted in an inaccurate representation of original content near-black and near-white. I've personally not tested this recent build, but from what bacondither stated, Shiandow's idea was only a partial-fix.

All in all, it should send up red flags if linear light processing being used with dithering isn't giving a linear result, while gamma light processing supposedly is giving a linear result. Maybe Ver Greeneyes could make some 16bit fullscreen color patches (with a key for the source 16bit RGB values encoded), which all fall on fractional random 8bit color values near-black, near-white, and midtones. It would be very simple at that point to see what's going on objectively. Take screenshot with gamma dither and linear dither in 8bit mode, open in Photoshop, take an average sample over a large surface area, and see which ends up closer to the expected 16bit color value from the key.

James Freeman

8th March 2014, 07:36

madshi's first attempt with linear light dithering resulted in an inaccurate representation of original content near-black and near-white.
As expected in LL with very low bit depth.
I've already said that, In 4/3-bit Linear or Gamma will not look correct, one will crush the black shades, the other will elevate them.
LL should NOT be tested in low bit depth, because you WILL get crushed blacks in 3-bit.
The custom BT.709 curve the guys are talking about might get us closer (in between too dark/bright) in lower bit depth, but what I fear is it'll have negative effect on the 8-bit picture.

Our goal is to find a "close enough"/"sweet spot" gamma curve.
For me, the latest LL build does just that.
Or can it be further improved?

Maybe Ver Greeneyes could make some 16bit fullscreen color patches....
This is a great idea, and probably the only way to find a true matching curve.

Its important that this experiment will be done in 8-bit (6-bit minimum), NOT lower.
Fixing a lower bit depth (3/4-bit) to look like 16-bit will result in a wrong curve at the Default 8-bit (IMO).
This test should be done in a floating point precision, to really determine what curve should be used to represent the real 16-bit image at every one of the 255 dithered patches.

I'm quite sure that the results will be very close to Linear Light in 8-bit, but the lower we got in bit depth the more we have to customize (brighten the dark shades) the gamma curve for each bit depth and that is completely unnecessary.

I think our goal should be to find a "close enough"/"sweet spot" gamma curve between 6-bit to 8-bit that on average looks (rather measured) to be like 16-bit.
I repeat, NOT in 3/4-Bit.

EDIT:
This post is irrelevant as I misunderstood what the discussion was about before Asmodian cleared that up for me.
Although, I'll leave my ignorance posted...

Asmodian

8th March 2014, 09:26

Personally,
I don't want any BT.709 curve on my dithering or anywhere else for any reason.
Linear Light (inverse power) is how the original content meant to be played (or images to be shown), and that's the way it should be.

madshi, hopefully you're reading this and will not make the mistake of enforcing a BT.709 curve on the output.

It sounds like you are missunderstanding what is being discussed with "linear light dithering gamma curve" (1/0.53 is a pure power curve btw, not actually BT.709). This is the curve used to calculate the amount of error a pixel has (pixel's float value - output int value) not the actual display gamma. Also you do not want to display in linear light, that looks very bad.

I believe the error should be calculated in linear light.

An example: a bright pixel wants a value of 230.4 in gamma light and the error is moved to a dark pixel with a value of 18.9 in gamma light.

If everything is done in gamma light the 18.9 becomes 19.3
If the error is calculated and added in linear light the 18.9 becomes 19.02055 in gamma light.
This is why gamma light dithering looks brighter; this is effect is bit depth independent, it is only more obvious at lower bit depths because the relative errors are higher.

The math:
230.4 / 235 = 0.980426, ^0.519533 = 0.989782
230 / 235 = 0.978723, ^0.519533 = 0.988889
18.9 / 235 = 0.080426, ^0.519533 = 0.269970

gamma 0.080426 + (0.980426 - 0.978723) = 0.082128; 0.082128 * 235 = 19.3
linear 0.269970 + (0.989782 - 0.988889) = 0.270863; (0.270863^1.92805) * 235 = 19.02055

So with gamma light we have 0.3 = error left to diffuse and with linear light we have 0.02055 error left to diffuse.

This is an oversimplification to explain the problem but think of this example as step one of a more complex algorithm.

cyberbeing

8th March 2014, 10:03

madshi's first attempt with linear light dithering resulted in an inaccurate representation of original content near-black and near-white.As expected in LL with very low bit depth.
What are you talking about? The initial linear light dithering builds only supported 8-bit error diffusion (as did all the dithering builds prior to Feb 20th), and it had gamma error near-black and near-white... And really, don't respond to me with such nonsense. I personally only test with 8bit and think the low bitdepth modes are useless for making a determination about anything.

James Freeman

8th March 2014, 10:09

It sounds like you are missunderstanding what is being discussed with "linear light dithering gamma curve" (1/0.53 is a pure power curve btw, not actually BT.709). This is the curve used to calculate the amount of error a pixel has (pixel's float value - output int value) not the actual display gamma.
Yes, I missunderstood, thank you for clarifying that.
I almost got a heart attack seeing Gamma+BT.709 in the same sentence... :D

Also you do not want to display in linear light, that looks very bad.
True.
I completely missunderstood what the discussion was all about, and pushed myself in talking nonsense.

I believe the error should be calculated in linear light.
This is exactly what I was backing up, before you explained that the discussed power curve (1/0.52) is for error calculation.

An example: a bright pixel wants a value of 230.4 in gamma light and the error is moved to a dark pixel with a value of 18.9 in gamma light.

If everything is done in gamma light the 18.9 becomes 19.3
If the error is calculated and added in linear light the 18.9 becomes 19.02055 in gamma light.
This is why gamma light dithering looks brighter; this is effect is bit depth independent, it is only more obvious at lower bit depths because the relative errors are higher.

The math:
230.4 / 235 = 0.980426, ^0.519533 = 0.989782
230 / 235 = 0.978723, ^0.519533 = 0.988889
18.9 / 235 = 0.080426, ^0.519533 = 0.269970

gamma 0.080426 + (0.980426 - 0.978723) = 0.082128; 0.082128 * 235 = 19.3
linear 0.269970 + (0.989782 - 0.988889) = 0.270863; (0.270863^1.92805) * 235 = 19.02055
So with gamma light we have 0.3 = error left to diffuse and with linear light we have 0.02055 error left to diffuse.

Thanks for that,
It does makes it a lot clearer to understand.

What are you talking about?
Excuse my ignorance.
It's clear to me now (after Asmodian's post), that we are discussing the error itself not the output curve.
Yes, my two raving posts are nonsense.
I made a fool of myself again... meh...that's life...