madVR - high quality video renderer (GPU assisted)

[Plutotype]

Originally Posted by seriouser

It's an European VT50, specifically TX-P50VT50Y and yes it looks a bit like that, not as bad though. Maybe I spend too much looking for possible errors than enjoying the content.

The image I have attached is a shot of a VT30, which had these bad..VT50 improved a lot.

[Niyawa]

I have a question. Not too long ago a friend here gave me some tips about calibration options in madVR but there's still one thing I'm curious about. When I mentioned my display was calibrated with sRGB preset, he mentioned that the primaries to follow would be BT.709. Following that, in what occasion do we need the SMPTE-C or EBU/PAL? How do I know which one is/would be the one for my monitor?

[e-t172]

If you mainly watch HD content and don't care about SD, you don't need to concern yourself with SMPTE-C or EBU/PAL primaries. They are obsolete, basically.

[mindbomb]

is it a good idea to use smooth motion on a 1080i29 projector?
It looks as if madvr treats it as a 59.94 display and then the gpu creates 29.97i from that.

I have no idea how the whole interlacing process factors into this.

[Niyawa]

Originally Posted by e-t172

If you mainly watch HD content and don't care about SD, you don't need to concern yourself with SMPTE-C or EBU/PAL primaries. They are obsolete, basically.

I see, thanks.

[turbojet]

When converting ntsc rec.709 hd to sd using colormatrix("Rec.709->Rec.601") which x264 primary, transfer, matrix should be used to get the most accurate colors in madvr? Which to use for pal hd sources?

from x264 --fullhelp
--colorprim <string> Specify color primaries ["undef"]
- undef, bt709, bt470m, bt470bg
smpte170m, smpte240m, film
--transfer <string> Specify transfer characteristics ["undef"]
- undef, bt709, bt470m, bt470bg, linear,
log100, log316, smpte170m, smpte240m
--colormatrix <string> Specify color matrix setting ["???"]
- undef, bt709, fcc, bt470bg
smpte170m, smpte240m, GBR, YCgCo

[Graeme Gill]

After a lot of testing, tweaking and fixing bugs, I think I'm done for the moment, and the visual results seem quite promising.
To try it for yourself you need ArgyllCMS V1.5.2 from here http://www.argyllcms.com/downloadwin.html
and then this set of extra & replacement files from here http://www.argyllcms.com/Win32_collink_3dlut.zip installed over the top of it, plus a color instrument and some patience.
The main guide is at the bottom of doc/Scenarios.html, & updated documentation for collink in doc/collink.html. Video colorspace profiles are in ref.

[madshi]

@Graeme, thanks a lot! Two questions, if you don't mind:

(1) Are you ok now with the current way madVR supports only one external 3dlut for all source media types? Or do you think results would be better using one separate 3dlut per source media type?
(2) Does ArgyllCMS take color measurements in one IRE (e.g. 75 or 100) and base its calibration on that? Or do you measure and correct a raster like 5x5x5, similar to how professional image editor 3dlut correction works?

@madVR users, feedback (and comparison to yCMS) very welcome!

[Graeme Gill]

Originally Posted by madshi

Having 3dluts be a 2^x size makes things easier for Direct3D and my shader code. Also it allows me to feed BTB and WTW into the 3dlut, which may or may not be useful, depending on the software which creates the 3dlut.

The disadvantage of feeding Video encoded RGB values through the 3DLut is that it is highly desirable that black lands on a grid point, and 64 isn't as good as 65. You could pull the same trick as the eeColor and fake a 65 res. grid out of a 64 by hard coding the last rows at full scale out, since they are reserved for sync. anyway.

If you were to feed full range RGB into & out of the 3dLUT, then the grid resolution is not so critical for RGB. You do need to consider where neutral lands if you were doing YCbCr in though (ie. Cb and Cr = 0 need to land on a grid point).

How about the following workaround: Instead of allowing users to specify only one external 3dlut I could allow them to specify one external 3dlut per each input media type. Wouldn't that do the trick for now, without me having to totally change the way madVR calibration works right now?

I think in practice it works quite well at the moment - the different video colorspaces are not huge, so a single .3dlut will suffice. But a no compromise purist would want the option of having 3 different 3DLuts for the 3 different Video input spaces. So it's worth thinking about providing that, if it is not too hard.

I have zero knowledge about how ICC works. Currently madVR is pretty "stupid" in that it simply feeds data into the 3dlut and that's it. Ok, I'm doing some gamut and gamma processing, but that's just relatively simple math. I think good quality calibration is probably much more complicated. So basically I don't know how to convert an ICC device link into a proper 3dlut.

It's not that complicated to use ICC device profiles or links. A library like LittleCMS or ArgyllCMS/icclib makes it fairly straightforward - it takes care of all the details of loading the profile and then making the transformations it contains available. For instance, it takes about a page or so of C code to open an ICC device link and create a 3DLut. Using a single ICC device profile is a very similar piece of code. Of course it can take some effort understand the terminology and required technical detail.

I also don't know the extent of what ICC supports. I've been told that photo editors measure their displays at e.g. a 5x5x5 raster and then calculate a full 3dlut based on those measurements to make the display response perfect at all IRE levels. AFAIK this is also what the latest Lumagen Radiance 5x5x5 calibration does.

I'm afraid that's rather crude by my standards :-) A regular grid of test values is about the least efficient way of doing it - it's only advantage is simplicity.

"Conventional" calibration usually only corrects everything at one IRE, and then you have to hope that your display behaves linearly, so that the other IREs may look more or less right, too. Does ICC support such 5x5x5 measurements and the corresponding complex corrections to make a display virtually perfect?

ICC has nothing to say on the matter - it's a file format for storing color models, a super-set of something like a 3DLut. There are two basic models - 1D curves/Luts + a matrix, and cLUT. You can make the cLUT any res. you want up to 256^input channels, but the way those grid points are set is up to the profile builder. ArgyllCMS uses a scattered data point interpolation/extrapolation algorithm (regular splines), so the measurement data can be anything. By default it's a set of points that aims to optimally sample the colorspace. On top of the cLUT are per channel input and output curves. They play an important role in characterising some devices, but are probably not so important for Video, since RGB video colorspaces and displays are reasonable perceptually uniform due to the gamma encoding. I don't know how comprehensible it is, but I wrote an introduction here http://www.argyllcms.com/doc/ColorManagement.html that may be of interest.

From my naive point of view, for ideal results I'd imagine to measure maybe 5x5x5 and in addition to that the gray scale in maybe 32 or 64 steps. Throwing all those measurements together and turning it into a monstruous 3dlut should allow to get a perfect color and gamma response, I'd imagine. Does ICC support that concept?

Yes. That's what ArgyllCMS lets you do.

Just double checked my code. You're right. The OSD doesn't show the source transfer function, nor do I even care what it is. The reason for that is that BT.601 and BT.709 use the same transfer function and for PAL (AFAIK) it is usually recommend to use the same once again.

Yes, I agree. BT1886 does use a power of 2.4 though. There is some slight disagreement amongst various authorities as to whether "typical" CRT's have a gamma of 2.2 or 2.4. I suspect it actually comes down to the viewing conditions, 2.4 for quite dark conditions, and 2.2 for not so dark conditions.

madVR is using a pure power 2.2 conversion to get to linear light, then it performs gamut, contrast, saturation and hue changes (if necessary), then it converts back to gamma corrected light by using a pure power 2.2 conversion again. So the transfer function stays untouched by madVR. Unless you activate gamma processing in the madVR settings. Then madVR might actually modify the transfer function, depending on the settings...

OK - I can work with that. The fact that the transfer curve isn't touched when no contrast/sat/hue changes are present makes it transparent enough.

[Graeme Gill]

Originally Posted by cyberbeing

You can think of it as current versions of madVR treating the input transfer function of all videos is a 2.2 power curve for the purpose of converting to and from linear light.

Yes, that seems to be the conclusion.

Alternative 1: madshi's current implementation of optional shader-based gamma correction implementation expects that you have the 3DLUT change Display's response curve to output an actual 2.2 power-curve. This allow expected results, if for example, you enable gamma processing in madVR, and change the gamma to a 2.4 power-curve, the display response should actually end up as a 2.4 power-curve when measured. 3DLUT Gamma in -> 3DLUT 2.2 Power Curve out (gamut changed, gamma changed).

OK, I think I understand. It does provide a level of transparency at the moment, but I guess I'm still worried that it's heading in a direction that doesn't guarantee that a video in to display out Device Link/3dLut will continue to be viable.

Alternative 2: You can ignore madshi's intentions, and use a 3DLUT to change the display's gamma from the video's input transfer function to anything you want. Gamma in -> Custom Gamma out (gamut changed, gamma changed).

Yes, that's what I'm doing. It's not so much "Custom Gamma out" as "display color space out".

nand-chan made ti3 parser which allows merging a ICC profile gamma ramp into a 3DLUT, but the results were always somewhat poor, occasionally resulting in low-light banding and clipping. If this a flaw how ti3 parser modifies 3DLUT files, or a limitation of doing gamma correction in a 3DLUT, I don't know.

I'm afraid I've always been puzzled as to why he uses the .ti3 file, when a matrix ICC profile made from that .ti3 file would be so much more solid and easier to deal with. The low-light behaviour of my .3dluts seems pretty reasonable as far as I can tell. If I do a screen scrape and exaggerate the steps I can see each step from black up in MadVR dithered pixels.

[madshi]

Originally Posted by Graeme Gill

But a no compromise purist would want the option of having 3 different 3DLuts for the 3 different Video input spaces. So it's worth thinking about providing that, if it is not too hard.

It wouldn't be hard at all. I'll put it on my to do list.

Anything else coming to your mind which I could do to improve madVR <-> ArgyllCMS? Should I use a different way to do contrast, saturation and hue changes (e.g. a different way to convert to/from linear light, instead of a pure power 2.2 curve)? Or would it help if I allowed ArgyllCMS to remote control madVR to show color test patterns for measurement? The code for that (including network access etc) has been mostly ready for years now, but has never been put to use yet...

Originally Posted by Graeme Gill

The disadvantage of feeding Video encoded RGB values through the 3DLut is that it is highly desirable that black lands on a grid point, and 64 isn't as good as 65. You could pull the same trick as the eeColor and fake a 65 res. grid out of a 64 by hard coding the last rows at full scale out, since they are reserved for sync. anyway.

If you were to feed full range RGB into & out of the 3dLUT, then the grid resolution is not so critical for RGB.

Gotta admint, it never crossed my mind that black not landing on a grid point could be a problem. That's probably because I was always thinking of 256x256x256 3dluts and didn't spend too much time thinking about smaller sized 3dluts.

Originally Posted by Graeme Gill

It's not that complicated to use ICC device profiles or links. A library like LittleCMS or ArgyllCMS/icclib makes it fairly straightforward - it takes care of all the details of loading the profile and then making the transformations it contains available. For instance, it takes about a page or so of C code to open an ICC device link and create a 3DLut. Using a single ICC device profile is a very similar piece of code. Of course it can take some effort understand the terminology and required technical detail.

Maybe I'll look into this in the future. For now I'm happy that the latest ArgyllCMS now can create good 3dluts for madVR!

Originally Posted by Graeme Gill

I'm afraid that's rather crude by my standards :-) A regular grid of test values is about the least efficient way of doing it - it's only advantage is simplicity.

ICC has nothing to say on the matter - it's a file format for storing color models, a super-set of something like a 3DLut. There are two basic models - 1D curves/Luts + a matrix, and cLUT. You can make the cLUT any res. you want up to 256^input channels, but the way those grid points are set is up to the profile builder. ArgyllCMS uses a scattered data point interpolation/extrapolation algorithm (regular splines), so the measurement data can be anything. By default it's a set of points that aims to optimally sample the colorspace.

Cool, that's much better than I expected!

So do you think madVR + ArgyllCMS can now compete in calibration quality with e.g. Lumagen + Calman's 5x5x5 calibration? That would be very very nice!!

[n3w813]

Originally Posted by Graeme Gill

After a lot of testing, tweaking and fixing bugs, I think I'm done for the moment, and the visual results seem quite promising.
To try it for yourself you need ArgyllCMS V1.5.2 from here http://www.argyllcms.com/downloadwin.html
and then this set of extra & replacement files from here http://www.argyllcms.com/Win32_collink_3dlut.zip installed over the top of it, plus a color instrument and some patience.
The main guide is at the bottom of doc/Scenarios.html, & updated documentation for collink in doc/collink.html. Video colorspace profiles are in ref.

Awesome! Testing this now. Question, can ArcgyllCMS be configured to profile/calibrate a display outputting 16-236? I have 2 TVs that does not support 0-255 from a PC.

[fairchild]

I'd be willing to test the Argyll usage if someone can walk me through it somewhat. I am not new to doing calibrations but have never done it with Argyll and especially not in a non-gui version. I currently have access to a Colormunki spectrometer which I've already used to perform a pretty good calibration at the display level using ColorHCFR fork which implements Argyll code in it. From briefly reading the Scenarios.html section of MadVR. I would enter the following:

collink -v -3 m -e t -E t -I b -I 2.2 -G -i r Rec709.icm TV.icm HD.icm

And this would perform a calibration for me? Or is there much more to it.

This is different than my past experience with MadVR and 3dluts, where I'd just manually enter in the grayscale and gamut information into it and it would spit out a 3dlut which would perform additional conversions.

For example this is what I had entered into ycms using calibration data that I had obtained from colorHCFR:

red, Yxy, 21.018, 0.641, 0.330
green, Yxy, 72.897, 0.301, 0.601
blue, Yxy, 8.324, 0.148, 0.060
white, Yxy, 103.463, 0.314, 0.332

30, Yxy, 6.689, 0.312, 0.329
40, Yxy, 12.683, 0.314, 0.329
50, Yxy, 21.290, 0.313, 0.328
60, Yxy, 31.541, 0.315, 0.327
70, Yxy, 46.163, 0.312, 0.329
80, Yxy, 61.423, 0.315, 0.331
90, Yxy, 79.445, 0.314, 0.329
100, Yxy, 101.703, 0.314, 0.332

[n3w813]

Originally Posted by fairchild

I'd be willing to test the Argyll usage if someone can walk me through it somewhat. I am not new to doing calibrations but have never done it with Argyll and especially not in a non-gui version.

Use the following guide to profile/calibrate your display using ArgyllCMS and DispcalGUI but skip the ti3parser section. Instead, use the commands in the Scenarios.html to generate the MadVR 3dlut using the icm and cal files from the profile/calibration.

http://www.hometheatershack.com/forums/video-calibration/65669-htpc-color-calibration-3d-lut.html

[fairchild]

Originally Posted by n3w813

Use the following guide to profile/calibrate your display using ArgyllCMS and DispcalGUI but skip the ti3parser section. Instead, use the commands in the Scenarios.html to generate the MadVR 3dlut using the icm and cal files from the profile/calibration.

http://www.hometheatershack.com/forums/video-calibration/65669-htpc-color-calibration-3d-lut.html

I followed that guide, but once I get to the point where I have to hit calibrate + profile it's greyed out.



I'm guessing this is happening because I don't have my spectrometer connected and ready to get readings or is it something else? Either way, I'll check back later, gotta head off to work now. Thanks for the link.

[n3w813]

Originally Posted by n3w813

Awesome! Testing this now. Question, can ArcgyllCMS be configured to profile/calibrate a display outputting 16-236? I have 2 TVs that does not support 0-255 from a PC.

Got quite good results on my laptop! Attached are 25% saturation charts for before and after.

Originally Posted by fairchild

I'm guessing this is happening because I don't have my spectrometer connected and ready to get readings or is it something else? Either way, I'll check back later, gotta head off to work now. Thanks for the link.

Yes, you will need to have a meter connected....how else are you going to measure the patterns???


Madshi,
Confirming Graeme's observation, the "disable GPU gamma ramps" had no effect on the image in both Windowed or FSE modes during my testing. Is this a bug or does the option only apply with specific settings in the OS or MadVR?

[e-t172]

Originally Posted by Graeme Gill

Yes, I agree. BT1886 does use a power of 2.4 though. There is some slight disagreement amongst various authorities as to whether "typical" CRT's have a gamma of 2.2 or 2.4. I suspect it actually comes down to the viewing conditions, 2.4 for quite dark conditions, and 2.2 for not so dark conditions.

Well, actually, BT.1886 translates to an average gamma of 2.4 when the display is "perfect" (i.e. infinite contrast, think CRT), and translates to an average gamma of 2.2 when you feed it with black/white luminance values for typical LCDs (which typically have a CR of approx. 1000:1). You might be interested in this calculator which illustrates it quite well.

[Graeme Gill]

Originally Posted by madshi

@Graeme, thanks a lot! Two questions, if you don't mind:

(1) Are you ok now with the current way madVR supports only one external 3dlut for all source media types? Or do you think results would be better using one separate 3dlut per source media type?
(2) Does ArgyllCMS take color measurements in one IRE (e.g. 75 or 100) and base its calibration on that? Or do you measure and correct a raster like 5x5x5, similar to how professional image editor 3dlut correction works?

1) Yes, the current support is workable, and should give satisfactory reslt. Being able to specify a 3dlut for each meadia source type would be a potential quality improvement.
2) I'm not really sure what you mean. It's pretty typical device value color management. I have RGB values in one colorspace (the Video media) and I know what device independent color they should be (From Rec709/Pal/NTSC + BT.1886 + viewing conditions adjustment). I have a display and know what device independent colors it produces when I feed RGB into it (the display ICC profile). I link the two together to create a Video RGB to display RGB transformation, and store it in the 3dlut.

[Graeme Gill]

Originally Posted by madshi

IShould I use a different way to do contrast, saturation and hue changes (e.g. a different way to convert to/from linear light, instead of a pure power 2.2 curve)? Or would it help if I allowed ArgyllCMS to remote control madVR to show color test patterns for measurement? The code for that (including network access etc) has been mostly ready for years now, but has never been put to use yet...

I think your current color adjustment code will interact OK with the ArgyllCMS 3dluts, since (when used in the recommended fashion) it emulates a device with a pure power curve. A purist won't use such controls anyway.
As for test patterns - it would be interesting for verification through the whole video rendering system, but certainly isn't needed (or probably useful) for the display characterisation.
One way of hooking things up for video verification would be to have MadVR respond to a command to set a color. Argyll's tools have a call-out option - see http://www.argyllcms.com/doc/dispread.html#C - so having something that works with that would be good.

Gotta admint, it never crossed my mind that black not landing on a grid point could be a problem. That's probably because I was always thinking of 256x256x256 3dluts and didn't spend too much time thinking about smaller sized 3dluts.

In ICC profiles, the per channel input curves are used to finesse things so that certain critical values land on grid points.

So do you think madVR + ArgyllCMS can now compete in calibration quality with e.g. Lumagen + Calman's 5x5x5 calibration? That would be very very nice!!

Should be at least as good, and is much more sophisticated and mature in most ways, even if it's not so tailored to Video work currently (they're still obsessed with regular grid measurement, and aren't even using something like body centred cubic, etc.).

If someone really gets keen, they could also consider throwing http://www.argyllcms.com/doc/refine.html at a display as well, although some tweaks may be needed to target Video.

[Graeme Gill]

Originally Posted by n3w813

Awesome! Testing this now. Question, can ArcgyllCMS be configured to profile/calibrate a display outputting 16-236? I have 2 TVs that does not support 0-255 from a PC.

Not currently. My understanding was that most graphics cards either automatically, or as an option do this scaling. If you tell me they do not, I think I can add an option to dispcal & dispread to do this (I am a little worried about how to detect how many bits are being sent to the display though, since Video encoding uses a shift rather than a scale when operating at higher bit depths.)