.ti3 Parser / .3dlut Generator for madVR + .3dlut toolset

[Thunderbolt8]

at which position to add this line?

I keep getting error: parsing failure: unexpected '!'; expected end of input (position XX) messages when trying to use the command line

edit: now haali does start up properly with every (valid) 3dlut file I feed to it. I uninstalled LAVsplitter this time, before that I might just have reinstalled haali after having installed LAV instead of proper uninstallaling LAV first. maybe that caused some problems.

[nand chan]

Oh, totally my bad - I forgot that you have to set the bit depth for each input channel, so it's !Input_Depth(7, 7, 7)

And good to hear about the Haali thing.

[nand chan]

In case anybody's interested, I'm working on a system to perceptually expand the gamut of a source file to the maximum representable gamut in your monitor space that won't clip any levels (or clip some of the blue, how far you expand it depends on your preferences).

In other words, you will be able to take a dull and gray source image and expand the gamut to something bright and exciting, depending on how much your monitor can represent.

This is basically like using an uncalibrated wide gamut monitor, but with one very important difference: the hue will stay the same, just the chromaticity increases or decreases. So basically, the red will be the same shade of red - just more vivid.

This solves the “plastic greens” problem of wide gamut monitors while giving you some of their benefits.

The main drawback? On the average wide gamut monitor, you can't get a significant increase since the blue primaries are often very close to the originals. For example on my U2410, I can increase the chromaticity by around 1.5% before I start clipping the blue channel. This is a barely detectable increase, only very slight at best. But if you have something like a projector with a much wider blue primary, you can benefit more from the increase.

Alternatively, you can choose to clip the blue channel slightly, which will begin shifting the hue of colors with a lot of blue saturation.

[fairchild]

Hey nand, I have a question regarding these tools which you are providing. Normally what I had been using was use the following coordinates which had been generated by my calibration software + meter (I used colorHCFR + i1DisplayLT) and entered these values into the MadVR yCMS configuration tab:

Gamut_Measurements 1
8.522508 0.659738 0.332989
27.843536 0.297610 0.595287
3.192925 0.149621 0.066393
46.652584 0.311805 0.327390

Grayscale_Measurements
20 1 3.572751 0.309652 0.327431
30 1 7.791107 0.312203 0.327093
40 1 13.703165 0.312105 0.327861
50 1 21.226063 0.312431 0.329232
60 1 31.068672 0.310517 0.327090
70 1 42.082134 0.312013 0.327941
80 1 53.747086 0.312122 0.327018
90 1 68.256332 0.311853 0.328607
100 1 81.266403 0.312070 0.327252

Is there an easy/quick way to just enter the above values into Agryll/your software and have it+your software generate a 3dlut to be used by MadVR or is there something else that needs to be done? (I'd rather not re-calibrate using Agryll since I like the way my set is already configured for both my HTPC usage + Blu-ray player + Cable TV.

If that's out of the scope of your tools, then that's understandable. I'd like to try an alternative to MadVR + yCMS.

Thanks!

[nand chan]

If that's out of the scope of your tools, then that's understandable. I'd like to try an alternative to MadVR + yCMS.

Thanks!

My tools use yCMS as a back-end for those kind of measurements. The only gamut mapping I've implemented myself is currently just a linear transition, it isn't capable of calibrating the gamma curve like that or anything.

[nand chan]

Previews of the technology I mentioned earlier:

80% source gamut (http://img9.imageshack.us/img9/6438/23008245.jpg)
90% source gamut (http://img52.imageshack.us/img52/2797/15644120.jpg)
100% source gamut (original image) (http://img853.imageshack.us/img853/6200/41836425.jpg)
110% source gamut (http://img215.imageshack.us/img215/5918/93329443.jpg)
120% source gamut (http://img685.imageshack.us/img685/5355/68964914.jpg)
130% source gamut (http://img820.imageshack.us/img820/2463/56361002.jpg)
140% source gamut (http://img560.imageshack.us/img560/5020/20171732.jpg)
150% source gamut (http://img703.imageshack.us/img703/8674/58569394.jpg)

Do note that these images are for the standard mode of my U2410, calibrated to D65 and colors will appear inaccurate on your monitor. It is just to demonstrate how the technology works. At 100%, the colors (as shown on my monitors) are exactly BT.709.

It is important to note, that (on my monitor):


Above 1.5% the blue channel starts clipping
Above 15.2% the red channel starts clipping
Above 41.8% the green channel starts clipping

[nand chan]

Version 0.14 release

Download link: http://www.mediafire.com/?m7nr5qgkdso6bdh

This build mainly highlights some significant back-end changes in terms of color space support. If I keep going at the current rate, I'll be able to replace yCMS and LittleCMS soon. The library is currently already capable of gamma encoding/decoding, color space conversion, chromatic adaption, chromatic desaturation etc.

In addition, I'm working on the proper GUI front-end which will replace the command line utilities / lutscript. (Apart from that, I need to re-work the way color spaces work in LutScript significantly. I may just scrap LutScript completely and switch to an existing scripting language like ruby or haskell and provide bindings instead)

Change log:
Version 0.14:
* LinkICCGUI now automatically updates yCMS on first run as well
* imagecal.exe clamps the levels to 0-255 properly now, before it would overflow
* Another attempt at fixing the locale bug, this time by forcing en-US explicitly instead of the invariant culture
* Gamma encoding for values under 0 clamps the values now (instead of resulting in NaN or Infinity)
* Fixed a very slight error in the predefined BT709 primaries
+ LutScript scripts can now be directly dragged onto lutscript.exe instead of having to run it from the console
You can also assign the program as default program for the .3ls file extension, so you just need to double click it
+ Added new gamuts (usable in LutScript):
BT709 (aliases: HDTV, sRGB, scRGB)
ROMM (aliases: ProPhotoRGB)
AdobeRGB (aliases: AdobeRGB98)
AppleRGB
NTSC (aliases: NTSC53, FCC1953, BT470M)
NTSC87 (aliases: SMPTEC, SMPTE_C, SMPTE_RP_145, SMPTE_170M)
SECAM (aliases: PAL, EBU_Tech_3213, BT470_B)
AdobeWide
CIE1931
ACES
+ Chromaticity expanding in the L*Ch(ab) color space is now supported
+ Chromatic adaptation between two different white points is now supported
+ Separated the white point component from the color space, added more default white points:
A, B, C, D50, D55, D65, D75, E, F1, F2, F3, F4, F5, F6, F7, F8, F9, F10, F11, F12
+ Added a function to calculate the white point from a color temperature
? Note: The above four features are not yet available in LutScript or any program

[nand chan]

Going for a double release since I finished this version up much, much faster than anticipated:

Version 0.15 release

Download link: http://www.mediafire.com/?rowlp5bf44y2205

This build mainly implements the features added in 0.14 into LutScript, doing so by completely rewriting the LutScript engine. Basically, LutScript is now a derivative of C# (some things are modified, for example ! in function names gets changed to _, and a few global constants have been added eg. 3DLUT for !Filetype). What this means is that you can basically extend its capabilities with standard C#. The plaintext of a LutScript file (.3ls2) is stuck inside a code provider's constructor and executed directly, so any sort of variables or other processing logic you declare in it get executed as well. The main syntactic change is that top-level functions (starting with a !) need a semicolon (;) after them, eg. !Filetype(3DLUT); and !Pixel()'s functions need to be separated by a regular comma, eg. !Pixel( GammaDec(2.4), GammaEnc(2.2) );

To complement this, a new filter has been added named Code(Func<ColorTriple, ColorTriple>) which basically provides you with the possibility of writing your own transformation logic in C# source. The !Pixel function has been rewritten and is now !Pixel(params ITransformationProvider<ColorTriple>[] filters), which is a fancy way of saying you can give it a bunch of filters as arguments.

Other than that, the main new filters this release highlights are ChromaticAdapt(ColorTriple source, ColorTriple dest, [ScalingMode mode]) and ChromaticScale(double factor, ColorTriple illuminant).

Using these filters, it's possible to, for example, desaturate or oversaturate colors, which is useful when moving into a large color space (where saturation wouldn't cause clipping).

Here's a new example of what a LutScript file looks like:

!Filetype(3DLUT);

!Pixel(
GammaDec(2.2),
ColorDec(ColorSpace.BT709),
ChromaticScale(1.3),
IccSingle("monitor.icm")
);

This creates a madVR-compatible .3dlut that decodes from BT.709, oversaturates by 130% (increases the chromaticity) and decodes back into the color space defined by my monitor's ICC profile.

Full changelog:

Version 0.15:
! LutScript completely re-worked, instead of creating a DSL from scratch it is now (nearly) syntax-conform C#.NET
This, in practice, doesn't really change much (as far as the scripting goes), but there are a few syntax differences
Most notably, semicolons are required after every command, the { } blocks have been removed from !Pixel, and
individual filters are separated by commas instead, eg. !Pixel( Grayscale(), Inverse(), Scale(0.1, 0.9) );
The new preferred extension is .3ls2
+ The “old” LutScript behavior can still be obtained by appending the --legacy flag to lutscript.exe. If you open
a file ending with .3ls instead of .3ls2, this flag is appended automatically.
+ New LutScript function: ChromaticAdapt(ColorTriple source, ColorTriple dest, [ScalingMode mode])
mode can either be ScalingMode.Bradford, ScalingMode.VonKries or ScalingMode.XYZ - Bradford is the best/default
+ New LutScript function: ChromaticScale(double factor, ColorTriple illuminant)
This will scale the chromaticity in L*Ch(ab) space by a constant factor
+ New LutScript functions: sRGBGammaEncode() and sRGBGammaDecode()
+ New LutScript function: Code(Func<ColorTriple, ColorTriple>), can be used to code your own transform filters, eg:
Code((input) => input * 0.7)
~ The LutScript functions dealing with primaries have changed their signature to (ColorSpace c) instead of a bunch of
double values. If you want to create a color space from the x/y chromaticity values, you can do so using the class's
default constructors, for example: new ColorSpace(0.64, 0.33, 0.30, 0.60, 0.15, 0.06, Illuminant.D65)
+ Added “smart” versions of ChromaticX / ColorX which attempt to guess the required color space based on the output
of the previous filter, the primaries of the 3dlut, or failing that, BT.709
For example, !Input_Primaries(ColorSpace.BT709); !Pixel( GammaDec(2.2), ColorDec(), ColorEnc(), GammaEnc(2.2) );
+ All standard colorspaces and illuminants are now readily accessible from within LutScript, as ColorSpace.<name> or
Illuminant.<name>, eg. ColorSpace.sRGB or Illuminant.D50
+ A white point can be calculated from the temperature using Illuminant.FromTemperature(double temp)
* Exceptions are a bit more descriptive

[Thunderbolt8]

is the input line you gave me for colour correction

# Set the properties properly
!Input_Range(Limited)
!Output_Range(Limited)
!Filetype(3DLUT)
!Input_Primaries(BT709)

!Pixel( {

Scale(-0.07305936073059360730593607305936, 1.0913242009132420091324200913242)
Clamp(0, 1)

} )

still valid with the new version or has anything changed?

[nand chan]

is the input line you gave me for colour correction

<snip>

still valid with the new version or has anything changed?

If you keep the extension as .3ls, it's still valid. However, to take advantage of the new features (eg. chromatic expansion), you can change the extension to .3ls2 and modify the script slightly:

!Filetype(3DLUT);
!Input_Primaries(ColorSpace.BT709);

!Pixel(

Scale(-16.0 / 219.0, 239.0 / 219.0),
Clamp(0, 1)

);

Firstly, instead of just “BT709” it's changed to “ColorSpace.BT709”.

Take note of the semicolons (;) and commas (,) added. I'm trying to figure out how I can add these automatically so the user doesn't have to, but for now, that's the only significant change.

I've also removed the !Input_Range and !Output_Range tags because those are limited range by default either way. And since this version supports mathematical operators, I've replaced the lengthy constants by their mathematical representations instead. (The .0 is important though, otherwise it won't be done with floating point precision math but instead with integer precision)

Once I'm happy with LutScript I'll write up a real documentation / user guide so all of these questions will be unnecessary, but for now I'm keeping it open so I can still make changes before it's too late.

Once I bump the version to 1.0 I'm not going to make any more drastic changes like that.

[alph@]

Once I'm happy with LutScript I'll write up a real documentation / user guide so all of these questions will be unnecessary.
I wait patiently on documentation for benefit from the library of software that you share with us:thanks:

[vega22]

First of all, thank warmly Nand all these tools and all their efforts.
Now, a whimsical (and silly) question: I would like to manually alter some calibration data (namely the primary coordinates) before creating the .3dlut files through your tools. What is the best way to do it? Manipulate data in the .cal and/o ti3 files seems complicated. Maybe I should create a .3dlut for only gamut mapping using yCMS and my custon script, then merge this .3dlut with one creating from LinkICCGUI with only grayscale and gamma corrections? It´s possible? How?
The issue is to get a .3dlut to correct grayscale and gamma, but only correct the gamut interpolating one or two primaries, leaving the other untouched.

[Wile-E-Coyote]

Thank you for keeping your tools updated nand. Any chance you could also update your guides on haruhichan.com ? You said yourself there they were outdated and it's getting hard to sort the right settings in the comments.

[nand chan]

First of all, thank warmly Nand all these tools and all their efforts.
Now, a whimsical (and silly) question: I would like to manually alter some calibration data (namely the primary coordinates) before creating the .3dlut files through your tools. What is the best way to do it? Manipulate data in the .cal and/o ti3 files seems complicated. Maybe I should create a .3dlut for only gamut mapping using yCMS and my custon script, then merge this .3dlut with one creating from LinkICCGUI with only grayscale and gamma corrections? It´s possible? How?
The issue is to get a .3dlut to correct grayscale and gamma, but only correct the gamut interpolating one or two primaries, leaving the other untouched.

I don't quite understand why you want to alter the primaries of your calibration data. If you have an exact measurement available, use those, if you don't, just use a regular transform (without calibration).

If you just want the gamma curves, you could calculate those independently and export them to a 3x1DLUT file like a .cal file or #GammaCurve text file. Then you can do the primary mapping using standard color space logic, and perform the .cal correction on top of that (using LutScript).

Thank you for keeping your tools updated nand. Any chance you could also update your guides on haruhichan.com ? You said yourself there they were outdated and it's getting hard to sort the right settings in the comments.

Sooner or later, maybe

[vega22]

I don't quite understand why you want to alter the primaries of your calibration data


First, my LCD with only hardware calibrations (i use a little boost in "color" setting looking the best ratio for the Y between all colors):
http://img521.imageshack.us/img521/4061/precalibrado.jpg
http://img716.imageshack.us/img716/8002/precalibrado2.jpg
The gamut is unacceptable, highly saturated. Now, automatically corrected by yCMS:
http://img23.imageshack.us/img23/6008/calibradoycms.jpg
http://img846.imageshack.us/img846/7093/calibradoycms2.jpg
The gamut is unacceptable, highly unsaturated. And now, the yCMS gamut corrected manually:
http://img198.imageshack.us/img198/5706/calibradoycmsymanual.jpg
http://img210.imageshack.us/img210/6961/calibradoycmsymanual2.jpg
Although still far from optimal, this gamut is the only decent for watching movies.
The process of finding the right spots manually is slow and cumbersome. It seems that the algorithm implemented in the "Upsilon mixer" tries to find the sweet spots interacting with the probe. This seems a good direction. However, i think it's even better to implement a GUI with "XY" sliders or something, for manually locate the points in the most comfortable way.

[6233638]

So I finally had a chance to try this out. I followed your guide, let dispcalGUI create an "XYZ LUT + matrix" profile using the "Large testchart for LUT profiles" (396 patches) and left it alone for a couple of hours.

When trying to use LinkICCGUI I get this error:

http://i.imgur.com/DXaiI.png

Any ideas?

[nand chan]

So I finally had a chance to try this out. I followed your guide, let dispcalGUI create an "XYZ LUT + matrix" profile using the "Large testchart for LUT profiles" (396 patches) and left it alone for a couple of hours.

When trying to use LinkICCGUI I get this error:

Any ideas?

Post the .ti3 file in question. Judging by the error message, it seems to be in an incorrect format (eg. not an XYZ->RGB LUT).

[6233638]

Post the .ti3 file in question. Judging by the error message, it seems to be in an incorrect format (eg. not an XYZ->RGB LUT).

http://www.mediafire.com/?glwa374o1czy7z7

[nand chan]

http://www.mediafire.com/?glwa374o1czy7z7

Ah, I see what the issue is. Since you are using a spectrophotometer and not a colorimeter, it doesn't store the individual XYZ entries but the actual values at every wavelength. Understandably, yCMS can't work with these values at all.

What you can do is skip LinkICCGUI (which uses yCMS as well as LittleCMS) and use a LutScript:

!Filetype(3DLUT);

!Pixel(
ColorDec(ColorSpace.madVR),
IccSingle("your_monitor_profile.icm")
);

save as output.3ls2 and drag that onto lutscript.exe, the resulting test.3dlut can be used with madVR.

[6233638]

Thanks. I'm now getting:Error: System.Exception: [3] 'TI3ParserLib.ColorSpace' does not contain a definition for 'madVR'

at TI3ParserLib.LutScript.MakeLutFromString(String s)

at TI3ParserLib.LutScript.MakeLutFromStream(Stream s)

at TI3ParserLib.LutScript.MakeLutFromFile(String file)

at lutscript.lutscript.Main(String[] args)

I guess I'm probably supposed to be defining something for "ColorSpace.madVR" but I'm not sure what?

http://www.mediafire.com/?yj5gblrd677j3w1

Here's the ICC profile as well, if that helps. Once I get this working, I'll have to see what the results are like, and then possibly do a longer profile, and make one at 2.40 gamma for film.

Using the ICC profile it generated, I'm getting banding and some discolouration in the greyscale, but I'm hoping that's a result of using the (presumably 8-bit) video card LUT, and that madVR with yCMS will be better.

Another issue I've run into is that when using the ICC profile, it's raising the black level of my display, as if it expected to read something from it. I have an LED backlit LCD that turns the backlight off for black, so that's obviously less than optimal.

I have to change brightness from 50 to 48 to get black to appear black again, and that's obviously going to affect the whole range from black to white. Don't know if the same will also apply when using a 3DLUT file in madVR, but I assume so.

Black point correction was set to 0% in dispcalGUI, which I assumed would take care of this. I guess I need to have a look over the dispcalGUI documentation to see what all the black level options do. (I just followed your guide more-or-less to the letter)

[nand chan]

Oh wow, I totally forgot to upload version 0.16. This was done ages ago.

Download link: http://www.mediafire.com/?d8wmeqlx7dd378y

Changelog:
Version 0.16:
! Color spaces are now aware of their gamma functions by default
~ ColorEnc() and ColorDec() renamed to XYZEnc() and XYZDec(). The new ColorEnc() and ColorDec() functions apply gamma
encoding/decoding by default as well, for color spaces that support it
? Note: This does not apply to all color spaces, eg. ones which define no reference gamma curves, the only gamuts that
automatically apply gamma en/decoding are:
BT.709 (PPC 2.35), Apple RGB (PPC 1.8), sRGB/scRGB (sRGB curve), ROMM/ProPhotoRGB (ROMM curve),
HDTV (709 curve), NTSC87/SMPTE "C" (PPC 2.2), PAL (PCC 2.8), madVR (PPC 2.2) and SECAM/EBU Tech 3213 (PCC 2.8)
+ New gamut: madVR, this gamut uses BT.709 primaries with a 2.2 PPC, to match with what madVR inputs to the .3dlut
~ GammaDec() and GammaEnc() changed to accept an ICurveProvider argument.
+ New functions sRGB() and PPC(double gamma) give you an sRGB or Pure Power Curve. So, for example, GammaEnc(PPC(2.2))
would replace what was previously just GammaEnc(2.2), and GammaEnc(sRGB()) would replace sRGBGammaEncode()
+ New function L_(L_Mode standard) gives you an L* curve, L_Mode can either be Actual or Intent
- Removed sRGBGammaEncode() and sRGBGammaDecode()

Truth be told, I don't exactly know much about the black point compensation options in either LittleCMS or dispcalGUI. If you want to do some more research, maybe in the LittleCMS manual or mailing list, and maybe test some options, it would be a great improvement.

[6233638]

So, I think it just didn't like the ICM filename. Installed 0.16, renamed the LUT from the dispcalGUI generated one, and it seems to have worked. I'm betting it was the "cdm²" that was causing problems.

Still seems to be raising the black level considerably and introducing some discolouration near black. Will give it another try this weekend.

Looks good otherwise though, will need to verify with test patterns and see how it actually measures.

[nand chan]

Still seems to be raising the black level considerably and introducing some discolouration near black. Will give it another try this weekend.

That seems to be an effect done by LittleCMS, even with “black point compensation” disabled (it's disabled in 0.16. I tried enabling it, but it didn't seem to have any effect). I believe it's linked to the fact that the device doesn't have a perfect black level, so something that should be perfectly black in theory will get cut off to the minimum that the display supports.

What I tried doing to work around the issue, at the cost of accuracy near the blacks, is “warping” the curve near the bottom, bending it to fit within the monitor's envelope. This is what “LinkICCGUI” does internally, but it relies on yCMS's calculations, which doesn't currently support spectrophotometer readings.

I'll see if there's another way to get the desired effect, maybe by pre-scaling values to fit into the black level.

If you can work out the ratio of your monitor's black level to your monitor's white level (ie. its contrast), and pre-scale the XYZ values by that factor, it might have a desirable effect. This would be a simple command in LutScript: Scale(contrast, 1) where contrast is the ratio of black to white (a very small figure). This would be put in between IccSingle and ColorDec.

[6233638]

If you can work out the ratio of your monitor's black level to your monitor's white level (ie. its contrast), and pre-scale the XYZ values by that factor, it might have a desirable effect. This would be a simple command in LutScript: Scale(contrast, 1) where contrast is the ratio of black to white (a very small figure). This would be put in between IccSingle and ColorDec.Contrast is effectively infinite, the backlight turns off with black, so I'm not really sure what should be done. (this is a local-dimming LED backlit screen)

[nand chan]

Contrast is effectively infinite, the backlight turns off with black, so I'm not really sure what should be done. (this is a local-dimming LED backlit screen)

You're calibrating a display with dynamic contrast enabled? Doesn't that completely screw up accuracy? Either way, it might just be an issue with your sensor at that point. Spectrophotometers are notoriously bad for very-low-light measurements, it's possible that this is what causes the high inaccuracies around black when using stock LittleCMS settings. Or maybe LittleCMS simply isn't equipped to work with such low light levels. Or maybe it's a curve inaccuracy somewhere along the line. Who knows? (Maybe it's even an issue with the source file, eg. being tagged incorrectly)

[6233638]

You're calibrating a display with dynamic contrast enabled? Doesn't that completely screw up accuracy?It's not dynamic contrast, it's local dimming.

Dynamic contrast simply dims the whole screen with low APL images, and brightens it with high APL images.

LED local-dimming combines both the LC panel and backlight zones to greatly expand the dynamic range of the display. (improving black levels and shadow detail without changing the image)

Either way, it might just be an issue with your sensor at that point. Spectrophotometers are notoriously bad for very-low-light measurements, it's possible that this is what causes the high inaccuracies around black when using stock LittleCMS settings.Spectros can have issues with low light, but I have no issues using CalMAN measuring the kind of levels that are showing big errors in the greyscale.

I didn't get a chance to try calibrating things again. I think what I'm going to do next time is use it as a contact meter (on the screen rather than on a tripod) which I would prefer not to do, and then disable backlight scanning during the calibration which increases the light output from the display significantly, without introducing any colour shifts and should help with readings at lower light levels.

[nand chan]

It's not dynamic contrast, it's local dimming.

Dynamic contrast simply dims the whole screen with low APL images, and brightens it with high APL images.

LED local-dimming combines both the LC panel and backlight zones to greatly expand the dynamic range of the display. (improving black levels and shadow detail without changing the image)

I was under the impression that “dynamic contrast” is an over-arching term for any sort of technology in which the static contrast ratio is improved by adjusting the backlight in any way.

Spectros can have issues with low light, but I have no issues using CalMAN measuring the kind of levels that are showing big errors in the greyscale.

I didn't get a chance to try calibrating things again. I think what I'm going to do next time is use it as a contact meter (on the screen rather than on a tripod) which I would prefer not to do, and then disable backlight scanning during the calibration which increases the light output from the display significantly, without introducing any colour shifts and should help with readings at lower light levels.

Well, if the measured black level is very low, then I don't see why black detail would be removed when using LittleCMS. Have you tried my suggestion regardless? How large is your measured black and white level?

[6233638]

I was under the impression that “dynamic contrast” is an over-arching term for any sort of technology in which the static contrast ratio is improved by adjusting the backlight in any way.It's not the same thing really, though I can see why you might think that. Dynamic contrast is awful.

Well, if the measured black level is very low, then I don't see why black detail would be removed when using LittleCMS. Have you tried my suggestion regardless? How large is your measured black and white level?Black detail is not being removed, but the black level is being raised. (dramatically lowering contrast)

You cannot divide by zero. Black level = 0, White level = 100.

[Wile-E-Coyote]

I have this weird issue when using 3dluts with madVR.

So I calibrated and profiled my monitor with dispcalGUI and created a 3dlut with linkICC. The profile is loaded system-wide after creation.

When I launch a video (using madVR as renderer), it all looks nice. However when I go into madVR settings to set the 3dlut file as calibration, the dark areas of the image are waaaaaay to dark. It's most obvious when using a THX Optimizer-like tool. It's like there's no dark grey anymore. It's either black or light grey. This happens both when the auto-calibrate box is checked or unchecked.

Anybody else had this before? Any idea what could be the issue?

[6233638]

Well, I liked what this seemed to be doing for colour and gamma, though it definitely needed improved somewhat, but I definitely seemed to be noticing a lot more banding, though I never got around to doing a proper test.

From the beginning of Pitch Black:

Original:
http://img818.imageshack.us/img818/8171/originalh.th.jpg (http://img818.imageshack.us/img818/8171/originalh.jpg)

yCMS:
http://img687.imageshack.us/img687/7669/ycms.th.jpg (http://img687.imageshack.us/img687/7669/ycms.jpg)

ArgyllCMS:
http://img403.imageshack.us/img403/9822/argyllcms.th.jpg (http://img403.imageshack.us/img403/9822/argyllcms.jpg)


This is exactly the sort of problem you get when using too much data to create a LUT, as I warned about before.

Any ideas on what can be done to improve things?

[nand chan]

I find it nearly impossible to believe that ArgyllCMS will produce that result, what were the exact steps you took to get it?

Even with 2400 sample points I get nowhere near that amount of banding, I see practically no difference in quality to yCMS except for a few very minor tone differences (especially around the greens).

[Wile-E-Coyote]

I just got this error message after trying to create a 3dlut with LinkICC.

http://i285.photobucket.com/albums/ll68/Petiflo/f8e68a8a.png

First time I ever had this message. Any ideas?

[6233638]

I find it nearly impossible to believe that ArgyllCMS will produce that result, what were the exact steps you took to get it?

Even with 2400 sample points I get nowhere near that amount of banding, I see practically no difference in quality to yCMS except for a few very minor tone differences (especially around the greens).It's the same 396 patch LUT I made from my initial measurements, all I did was play the video.

[rurika]

Hi, nand chan
I can't generate 3dlut with LinkICC. It crashed when it generating 3dlut.

http://i.imgur.com/pHyeU.png

My system is Phenom II x6 1055T + Windows 7 x64.
It work fine when I used other PC to generate ( core2duo E6750 + Windows XP x86 or Phenom II x4 945 + Windows 7 x64 )

I send this program to my friend that has Phenom II x6 & it seem he had a problem like me.
So, I guess the LinkICC can't work with Phenom II x6.

Hope it will work with Phenom II x6.

[markanini]

nand chan, I followed you calibration guide I & II and I'm wondering now what would good for Flash video in Firefox or Chrome.

[nand chan]

Hi, nand chan
I can't generate 3dlut with LinkICC. It crashed when it generating 3dlut.

My system is Phenom II x6 1055T + Windows 7 x64.
It work fine when I used other PC to generate ( core2duo E6750 + Windows XP x86 or Phenom II x4 945 + Windows 7 x64 )

I send this program to my friend that has Phenom II x6 & it seem he had a problem like me.
So, I guess the LinkICC can't work with Phenom II x6.

Hope it will work with Phenom II x6.

Probably an issue due to the number of cores, I'll look into it if I ever get the time. I assume you can just use your other PC for now.

[Wile-E-Coyote]

I just got this error message after trying to create a 3dlut with LinkICC.

http://i285.photobucket.com/albums/ll68/Petiflo/f8e68a8a.png

First time I ever had this message. Any ideas?
Up. Anyone got an answer for me?

[BeNooL]

Up. Anyone got an answer for me?

I'd say you are doing too many measurements (ie too close to each other) and/or your meter is not precise enough.

Try lowering the number of measurements so that each patch is out of the error range of the meter or eventually edit the file and remove the line where the results are not in increasing order.

[IceB]

Nand, thank you for your work and the great tool.
I have used your previous calibration guide with the REC.709 suggestions.
During the weekend I have tried to recalibrate my LCD TV with the updated guide - gamma 2.4 from here : http://haruhichan.com/wpblog/?p=364 .
My rig is HTPC ( ATI 6450 ) -HDMI-> Yamaha RX-V2700 -HDMI-> LCD Philips 42PFL8404.
The calibration processed with DTP-94 (Monaco Optix XR) colorimeter with dispcalGUI + ArgyllCMS (Win7 32bit).
The problem is every time I have tried to calibrate my monitor with 2.4 gamma tone curve the picture is very dark.
As a result I can not see any details on the dark patterns both on pictures in windows and in MPC-HC with madvr.
Creating 3dlut calibration file for madvr improves the colors in the movies, but still very dark with no details on near black parts of the picture.
After several unfortunate tries I had to recalibrate all back to the REC.709 tone curve. As a result - the picture is great, the colors are bit less vivid comparing to the gamma 2.4, but all the near blacks are clear and well detailed with the created 3dlut calibration file in madvr and in windows as well.

Despite my countless calibration related :readfaq: readings, I can not understand why have you changed the guide from REC.709 to gamma 2.4 ? Would you please explain ?
What might be the reason for the explained issues with my calibration with 2.4 gamma tone curve ?
Am I doing something wrong or my TV is not compatible with the gamma 2.4 colors ?

:thanks:

[nand chan]

Nand, thank you for your work and the great tool.
I have used your previous calibration guide with the REC.709 suggestions.
During the weekend I have tried to recalibrate my LCD TV with the updated guide - gamma 2.4 from here : http://haruhichan.com/wpblog/?p=364 .
My rig is HTPC ( ATI 6450 ) -HDMI-> Yamaha RX-V2700 -HDMI-> LCD Philips 42PFL8404.
The calibration processed with DTP-94 (Monaco Optix XR) colorimeter with dispcalGUI + ArgyllCMS (Win7 32bit).
The problem is every time I have tried to calibrate my monitor with 2.4 gamma tone curve the picture is very dark.
As a result I can not see any details on the dark patterns both on pictures in windows and in MPC-HC with madvr.
Creating 3dlut calibration file for madvr improves the colors in the movies, but still very dark with no details on near black parts of the picture.
After several unfortunate tries I had to recalibrate all back to the REC.709 tone curve. As a result - the picture is great, the colors are bit less vivid comparing to the gamma 2.4, but all the near blacks are clear and well detailed with the created 3dlut calibration file in madvr and in windows as well.

Despite my countless calibration related :readfaq: readings, I can not understand why have you changed the guide from REC.709 to gamma 2.4 ? Would you please explain ?
What might be the reason for the explained issues with my calibration with 2.4 gamma tone curve ?
Am I doing something wrong or my TV is not compatible with the gamma 2.4 colors ?

:thanks:

Rec709 is simply incorrect. The black levels you are seeing are unnaturally bright. It's supposed to be as dark as 2.4. If 2.4 is too dark you can try 2.2 or maybe 2.1 instead, but definitely don't use Rec709.

The reason you're experiencing the problem is probably because your TV doesn't have very good detail in the dark regions in the first place. I personally can see the difference between (0/0/0), (1/1/1) and (2/2/2) on my monitor.

[IceB]

Rec709 is simply incorrect. The black levels you are seeing are unnaturally bright. It's supposed to be as dark as 2.4. If 2.4 is too dark you can try 2.2 or maybe 2.1 instead, but definitely don't use Rec709.

The reason you're experiencing the problem is probably because your TV doesn't have very good detail in the dark regions in the first place. I personally can see the difference between (0/0/0), (1/1/1) and (2/2/2) on my monitor.

I have resolved the problem.
The "Ambient light level" measurement was still configured on dispcalGui from my previous calibrations.
Since I have updated the software, the "Ambient light level" configuration was hidden in the new version.
I have disabled it through the advanced settings and re-calibrated the TV with XYZ LUT + Matrix with Gamma 2.4.
Using the ti3 Parser the 3dlut was created for madVR calibration settings.
I have tried several profiles with different brightness/contrast/tint settings.
The best results were received with the "Relative" gamma tone curve.
The movies are much more vivid and the colors are deeper now comparing to the Rec709 profile i used before.
Yet using the gamma curve 2.4 in madVR results a bit darker picture than I would like, but the 2.3 seems to be the sweet spot for me resulting stunning and polished rendering.
:thanks:

[Mama70]

Well, I liked what this seemed to be doing for colour and gamma, though it definitely needed improved somewhat, but I definitely seemed to be noticing a lot more banding, though I never got around to doing a proper test.

From the beginning of Pitch Black:

Original:
http://img818.imageshack.us/img818/8171/originalh.th.jpg (http://img818.imageshack.us/img818/8171/originalh.jpg)

yCMS:
http://img687.imageshack.us/img687/7669/ycms.th.jpg (http://img687.imageshack.us/img687/7669/ycms.jpg)

ArgyllCMS:
http://img403.imageshack.us/img403/9822/argyllcms.th.jpg (http://img403.imageshack.us/img403/9822/argyllcms.jpg)


This is exactly the sort of problem you get when using too much data to create a LUT, as I warned about before.

Any ideas on what can be done to improve things?

The problem is you are using a spectrophotometer (i1 Pro?) and it doesn't give accurate reading in low light measurements.

I have ColorMunki and i1 Display 2 and user ColorMunki only for creating correction matrix for Display2. Then I do the actual calibration with Display2. But still, it doesn't get the low light measurents well.

At first I didn't understand ti3-file is used only for the black end. I tried using MPC-HC's own ICC-CMS and black end seemed ok. I think MPC-HC's CMS uses LittleCMS and it also creates 3dlut internally.

Easy but not perfect solution was to use yCMS and only ti3-file. You don't get perfect calibration though.

Launch TI3ParserGUI.exe and open your ti3-file and you'll see what the problem is.

My grayscale values were:
0, XYZ, 0, 0, 0
1.7857, XYZ, 0, 0, 0
3.5714, XYZ, 0, 0, 0
5.3571, XYZ, 0.027882, 0.02763, 0.042054
7.1429, XYZ, 0.10489, 0.10941, 0.1317
8.9286, XYZ, 0.2645, 0.28805, 0.2925
10.714, XYZ, 0.4104, 0.43535, 0.45801
12.5, XYZ, 0.65471, 0.69213, 0.6966
etc.

The problem in my file is values for 1.7857, 3.5714 and even 5.3571 are too low so my colorimeter doesn't get any values or they are inaccurate. Because program sees to low values, it tries to raise the values in 3dlut, and there you go, gray instead of black.

Simple but not perfect and final solution is just to drop those inaccurate values. So I copy&pasted those values to madVR's yCMS and used these values for gray scale:
0, XYZ, 0, 0, 0
7.1429, XYZ, 0.10489, 0.10941, 0.1317
8.9286, XYZ, 0.2645, 0.28805, 0.2925
10.714, XYZ, 0.4104, 0.43535, 0.45801
12.5, XYZ, 0.65471, 0.69213, 0.6966
etc.

I just wish we could have a version that has an option to only use ICC-files and LittleCMS.

[nand chan]

I just wish we could have a version that has an option to only use ICC-files and LittleCMS.

Use:

!Filetype(3DLUT);

!Pixel(
ColorDec(ColorSpace.madVR),
IccSingle("your_profile.icc")
);

Save it as anything and drag/drop onto lutscript.exe

[Mama70]

Use:

!Filetype(3DLUT);

!Pixel(
ColorDec(ColorSpace.madVR),
IccSingle("your_profile.icc")
);

Save it as anything and drag/drop onto lutscript.exe

Thanks a lot! With SVP and 3DLUT picture is so lifelike.
Why don't we have televisions&projectors that support natively 3dlut :(

[nand chan]

Thanks a lot! With SVP and 3DLUT picture is so lifelike.
Why don't we have televisions&projectors that support natively 3dlut :(

Higher end computer monitors and projectors support reprogrammable 3DLUTs.

[Mama70]

Higher end computer monitors and projectors support reprogrammable 3DLUTs.

But you cannot import 3dlut from pc, can you? Which projectors supports this?

[6233638]

The problem is you are using a spectrophotometer (i1 Pro?) and it doesn't give accurate reading in low light measurements. With the correct software, the i1Pro is accurate down to fairly low light levels—about 0.5cd/m2, which is 11% grey at 2.40 gamma with white at 100cd/m2. The banding shown here is nowhere close to that—it's in the 25–55% luminance range, far outside where meter error comes into play. (P.S. this is another reason why I dislike automated calibration—I can see when a low-light reading is clearly wrong and discard it, automated software doesn't know that)

If I remember correctly, I also disabled the backlight scanning when doing that calibration, which brings the panel brightness up significantly higher, placing the error range down around 6% grey.


I have had far greater success running test patterns in MPC-HC with the madVR yCMS options open and manually editing the xyY values (co-ordinates are far easier to manipulate by hand than XYZ) for each point, taking measurements with CalMAN running on another screen.

It's far more time consuming than it ought to be—if I could simply enter the RGB value I want out rather than tweaking xyY input, I'd arrive at the correct values a lot quicker, but it gives accurate results without the banding that this introduces.

For what it's worth, even then, I won't give it data below about 20% or so. I calibrate that range using the display's own controls, because any changes below 20% or so in yCMS seems to introduce posterisation/discolouration. (not due to meter error)

I also have the option of profiling my i1Pro to a colorimeter, but find that it's unnecessary in most cases, unless I need really low-light readings. I have yet to find a display where you have control over the very dark areas of the picture requiring readings that low, which does not introduce banding, discolouration or some other kind of issue, so while it's nice to know what the display is doing there, you can't fix it without introducing bigger errors anyway. Picture controls are very coarse down near black, and it's obvious when something is wrong.


Perhaps I'll give this another try some time with a significantly reduced dataset. As I've said from the beginning, problems like this tend to arise from using too much data when profiling a display.

[kasper93]

Rec709 is simply incorrect. The black levels you are seeing are unnaturally bright. It's supposed to be as dark as 2.4. If 2.4 is too dark you can try 2.2 or maybe 2.1 instead, but definitely don't use Rec709.

But if we adjust ambient light level (32 lux for me) in dispcal than rec709 is dark enough(little brighter than gamma 2.2), correct me if I'm wrong. I'm using 2.2 for movies since I have shitty monitor and gamma 2.4 is too dark for me.

Thanks for great tool. If I may suggest you should add information about LinkICCGUI "auto-calibration" function in yours madvr calibration guide. It's enabled by default, and in your guide you wrote to enable profile before watching what is wrong, because if auto-calibration is enabled we should disable any other color management, but yo know that :-)

P.S For rec709 default ambient light level is 1000 lux which is good for studio, but definitely not for home :)

[Thunderbolt8]

when having a lut script like this (.3ls2)

!Filetype(3DLUT);
!Input_Primaries(ColorSpace.BT709);

!Pixel(

Scale(-16.0 / 219.0, 239.0 / 219.0),
Clamp(0, 1)

);

here e.g. in order to correct wrong 16/235 BDs, how and where do I have to insert a line or formula to change the gamma value? how does the formula/line for the default gamma look like with this type of script and how do I have to modify it in order to increase or decrease gamma a little?

[tschi]

hi nand chan
I have an error with ti3parser :
System.execption : failed parsing the ti3 file : Sample_ID RGB_R .... not found
I use Argyll 1.4 and dispcalgui 1.0.7.7
here my ti3 file http://www.mediafire.com/?k847ll93lggxp3z
Thank you for your consideration