.ti3 Parser / .3dlut Generator for madVR + .3dlut toolset

[nand chan]

TI3 Parser (v0.16)

http://img705.imageshack.us/img705/8360/linkiccgui.png

Information

I hereby present my TI3 Parser toolset. This collection of programs is intended to provide a somewhat easier front-end for color management (http://en.wikipedia.org/wiki/Color_management) using ArgyllCMS¹, yCMS and madVR, as well as a small collection of command line tools. The motivation for creating this was due to the steps involved in locating, opening and extracting the information from a .ti3 file, then properly formatting them so madVR or yCMS accept them was a bit too daunting to explain well in my madVR CMS guide. This program initially started out as a script to simply format the values properly, but I decided to refactor it and enlarge the scope to include a largely automatized 3dLUT generation process. This also makes it easy to generate multiple .3dLUTs for different environments and choose a certain one easily using madVR (instead of having to copy/paste over the new primaries + re-generate it every time).

All logic is also abstracted into a managed .dll so that those who wish to use it in other projects can do so, and there is a collection of (somewhat) useful command line tools included.

1. At the moment, ArgyllCMS is still not automatized - you will have to calibrate+profile manually using dispcalGUI (http://hoech.net/dispcalGUI/)

Features:

This program will:

Read and parse a .ti3/.cal file as generated by ArgyllCMS (http://www.argyllcms.com/)
Present the gamut/grayscale measurements in madVR (http://forum.doom9.org/showthread.php?t=146228)-friendly form
Provide a yCMS (http://forum.doom9.org/showthread.php?t=154719) generation script
Create a madVR-compatible .3dlut file (http://thr3dlut.sourceforge.net/)
Modify an existing .3dlut by swapping channels, inverting it or applying calibration data from an ArgyllCMS .cal file
Create new .3dluts
Allow inspection and modification of .3dluts (to check for errors etc)
Automatically update yCMS
Scale/adapt the output bit depth or value range of .3dluts
Merge multiple .3dluts into a single one
Inspect, Reset, Load or Save the device adapter's CLUT tables
Allow application of 3dlut files to images manually
Create .3dluts from linking ICC profiles together
Perform, gamma correction, gamut mapping, chromatic saturation (eg. to increase the gamut range while preserving colors) and chromatic adaptation
Join together .3dluts using a composite algorithm (eg. to fix the black levels)
Work with both .3dlut and .3dl2 (http://pastebin.com/raw.php?i=kQ1btFtW) files
Be fully accessible from the command line console


Usage guide:

I intend for this section to be obsolete by version 1.0, but for now, a quick guide towards calibrating your monitor / profiling it properly can be found here (http://haruhichan.com/wpblog/?p=323) or here (http://haruhichan.com/wpblog/?p=364).

If you just want the GUI (shown in the screenshot), after extracting, run TI3Parser\bin\LinkICCGUI.exe.

A list of current command line utilities:


applycal: Applies an ArgyllCMS .cal/.ti3 or a CLUT file to a .3dlut, thus calibrating it for a specific display device
changedepth: Changes the output bit depth of a .3dlut or adjusts the levels (PC or TV) - note that this transformation is lossy
dispclut: Allows inspection, changing, resetting, saving or loading of a display adapter's CLUT tables (globally or per-monitor)
gen3dlut: Generates a new, linear or grayscale .3dlut
genscript: Generates a yCMS script from .ti3 source value, using the same base as madVR
imagecal: Apply a .cal file or .3dlut to a bitmap image manually, useful for proofing
inspect3dlut: Inspects a .3dlut and prints information about the meta-tags or creation script
join3dlut: Composites together two .3dluts, switching between the two based on luminance.
linkicc: Links two ICC profiles together and generates a .3dlut from the result
lutscript: Generates a 3dlut from a provided .3ls file (http://forum.doom9.org/showthread.php?p=1522819#post1522819) or .3ls2 file (https://forum.doom9.org/showthread.php?p=1529672#post1529672)
make3dlut: Generates a .3dlut from a .ti3 file, using yCMS (can also apply calibration data)
merge3dlut: Merges multiple .3dluts together into a single file
parseti3: Opens a .ti3 file and presents the primaries / grayscale information in madVR-friendly form. Also allows copying to clipboard
swapchan: Allows swapping or inverting channels of a .3dlut
tag3dlut: Modifies a 3dlut's attached parameters to change the declared levels or primaries (this does not modify the 3dlut itself, unlike changedepth)
updateycms: Updates this toolset's copy of yCMS
version: Prints the overall packaging version number


Help for each program can be obtained by running <program name> --help.

To-do (planned features list):


Find a better name for the project, since soon it won't be limited to just parsing TI3 files
Integration and updating of the binaries provided by ArgyllCMS
Automation for ArgyllCMS, similar in nature to dispcalGUI but tailored for video playback
Self-update system (will likely need a decent web host for this first)
If possible, interface with madVR and set the .3dlut files automatically (perhaps per display device)


Download and license information

All code is currently in public domain but I withhold the rights to change this license at any time (license changes will not apply retroactively).


Executables and source code: Download from mediafire (packaging v0.16) (http://www.mediafire.com/?d8wmeqlx7dd378y)


All executables can be found in the TI3Parser\bin folder. The program requires the .NET library version 4.0 or above to run.

Feedback

Please post any bugs, issues, suggestions or other sorts of feedback in this thread. If you want to submit source code patches you can do that too.

I'm also not quite satisfied with the current interface. What would you prefer - the current, a tab-based layout or a bunch of buttons like it was before? I'm also probably going to make the interface much larger, add more regions for everything, allow loading multiple files and generally expand the program into a larger, fullscreen management studio instead of a simple tool.

I'm also taking suggestions for a better name, since it is no longer a simple TI3 Parser

Changelog

Moved to second post due to length issues

[alph@]

very good,when you use 'rec709' curve, you use an adjustement of ambient light level ?

[nand chan]

very good,when you use 'rec709' curve, you use an adjustement of ambient light level ?

No, instead, I make sure my ambient light level is as close to zero as possible. In other words, I watch anime in a nearly perfectly dark room.

In fact, I've discovered that calibrating to a Rec.709 level makes madVR adjust more using its 3dlut than calibrating to an sRGB or even 2.35 tone curve, so I'm thinking of changing that. I have it calibrated to the D65 sRGB curve right now and the 3dlut doesn't make a huge difference, so I know it's only minor corrections.

I'm thinking of having the program calibrate to a set gamma instead (allowing you to choose between “bright, medium and dark” room or something)

Changelog

Version numbers refer to the packaging version, not the version numbers of the individual components.
*all new changes will be in the release post, vBulletin caps the length of posts unfortunately*

Version 0.10:
! Added basic support for ICC profiles
+ Color space information is extracted from the ICC profiles and written into the resulting .3dl2
+ New program: linkicc.exe
+ New LutScript function: IccTransform(String, String, Intent), intents: Perceptual, Relative, Saturation or Absolute
+ New LutScript function: !Filetype(Type), possible values: 3DL2 or 3DLUT
+ New LutScript function: !Parameters(String), overrides the parameters data (not recommended for 3DLUT filetype)
+ Added support for LZO compression (CompressionMethod=1). This is used by default.
? Note: Legacy LUTs do not support compression, it will always be disabled when using --legacy mode in tag3dlut
+ Added a --compression (-c) flag to tag3dlut, valid values are: None, LZO
+ New LutScript function: !Compression(Method), possible values: None, LZO
+ LutScript now computes the intermediate gamut after each step (this changes when using eg. IccTransform())
It is recommended to *NOT* set the !Input_Primaries or !Output_Primaries tags when performing gamut mapping!
LutScript will auto-detect these correctly for you, eg. when mapping together multiple ICC profiles
~ LutScript now defaults to Limited range when not specified, since it just makes more sense for video content
? Changelog v0.9 was flawed, !Input_Primaries did not accept a string but an /enum/
(eg. “!Input_Primaries(BT709)” and not “!Input_Primaries("BT709")”)

Version 0.9:
! Created and added support for the new .3dl2 format, changes include:
+ Added support for 64-bit floating point LUTs
+ Added support for color-space aware LUTs
+ Added support for RGB/YCbCr -> XYZ LUTs for profiling purposes, similar to how ICC device profiles work
+ Added support for file-encoded value ranges
* Utilities such as gen3dlut and changedepth which create new LUTs now use the 3dl2 by default,
so you no longer need to fix tagging using tag3dlut
? Note: Since programs such as madVR still only support .3dluts, you need to “convert” these files for those programs
+ --legacy (-l) tag added to tag3dlut in order to convert to a legacy .3dlut, this also overwrites the params
in order to correctly tag everything for madVR etc.
+ --color-space (-c) flag added to inspect3dlut
+ !Input_Primaries(rx, ry, gx, gy, bx, by, wx, wy) tag added to LutScript
+ !Input_Primaries(ColorSpace) overload added to LutScript, acceptable values are currently: BT709 and None
+ Same as above for !Output_Primaries() added as well
? I aim to make the input_primaries and output_primaries tags obsolete sooner or later though, when I introduce
gamut conversion functions
+ --ycbcr (-y) flag added to gen3dlut to generate a blank YCbCr LUT
* ColorTriple<T> is now aware of its encoding, in order to ensure transformations are properly chained
eg. the grayscale filter now simply sets Cr and Cb to 0 if the input is already YCbCr
~ Significantly increased performance here and there by operating on a single value in memory instead of copying it
over and over, however the old style of doing it is still available for cases where it's needed
- Removed the static methods .FromFile() and .FromStream(), moved logic into constructor instead
* !Pixel() no longer has to be the last function in a .3ls, meta-tags can appear after it
* make3dlut now automatically converts the yCMS output to a .3dl2, use tag3dlut to convert it back if desired
* Disabled the “Update program” field of the GUI, this is not supposed to do anything yet

Version 0.8:
* 3dluts now detect input and output range independently, allowing for a TV -> PC or PC -> TV 3dlut.
* Limited range floating point 3dluts work correctly now, the white level was set too high
- Removed the level override flag from applycal, it was in the wrong place - if you want to override levels, use
tag3dlut to modify the parameters data
- Removed many functions from _3dLut since transformation factories deprecate them, changes affect:
.Map(), .Assign(), .Modify(), .ChangeDepth(), .ChangeRange(), .Transform()
- Removed all of the overloaded Pulldown / Pullup functions from ColorTriple (which has been renamed Color). Instead of
using these functions, the proper method is to create a Pulldown or Pullup transformer (available in Triple variants)
+ Due to the above changes, level limits for pulldown/pullup are now precomputed on demand instead of at program start
+ LUTs are now theoretically unlimited, you could create a 32-bit LUT if you wanted to (and had 950737 YB of RAM)
- Removed the CalFileTransformer and _3dLutTransformer classes, implemented ITransformationProvider instead
To transform an arbitrary color, first get the transformation using .GetTransformer(), then .Invoke() that
* Updated changedepth to use a transformer to handle range conversions which uses the internal transformer; due
to this change, range conversions will *no* longer result in banding.
? Do note that since interpolation is still linear you will encounter macro-banding when performing extreme interpolation
If you're pulling up something like a 3-bit LUT to an 8-bit LUT, you should encounter it. With a 4-bit LUT (16x16x16)
I noticed no significant banding in a fullscreen gradient, but it might be more pronounced with less steep ramps
? Note that changedepth does not correctly tag the resulting LUT (in fact it strips all parameters). I won't change this
until the version 2 spec of .3dlut comes out, for now just use tag3dlut.
- Tags --full and --limited removed from tag3dlut
+ Tags --input-range and --output-range added to tag3dlut instead.

Version 0.7:
! Addition of LutScript, a scripting language for generating .3dluts from scratch. Documentation will follow.
? Transformation facility now also works in per-channel mode, as opposed to per-color (for the multiplexer)
* 3d LUTs now interpolate their values (linearly), this allows for full 64-bit floating point precision
Note that I have not yet done any gamma detection and curve adjustment, so this will still generate minor aliasing
+ Added logic to transform .3dluts further (that is, operate on its /output/)
+ New program: lutscript.exe
+ New program: merge3dlut.exe
~ More programs updated to use the transformation factory instead of hard logic

Version 0.6:
! All .3dlut code is now multi-threaded and will run many times faster
(do note that loading and saving a 100 MB file still takes around a second on modern hard drives)
! Transformations are now abstracted behind ITransformationProvider and can be assembled into TransformationFactories
Most program logic has been rewritten to use this functionality, and a GUI version for .3dlut creation
should be coming sometime soon
? dispclut can save and modify the CLUT at the same time now,
eg. dispclut -d 1 --save old.txt --reset before playback and dispclut -d 1 --load old.txt afterwards
+ New program: imagecal.exe
+ Beta functionality for 3dlut color transformations using 3dLutLib added, note that does not yet properly
interpolate the values. Once I've rewritten it to do so, I'll rewrite .ChangeRange to use this, thus
eliminating its current banding as well - which will affect changedepth.exe (when using the -r flag)
* make3dlut's built-in --calibrate option now works properly
* All tools now work with limited-range floating point values as well (0.0625 as black and 0.918 as white, approximately)
* 3dLutLib correctly detects full-range LUTs created with the RGB_PC format flag (instead of Output_Range 0 255)
* gen3dlut no longer prints luma information to screen with --grayscale, was left over from debugging
* The parser for .ti3/.cal files is now culture-invariant, to solve an issue where it would fail to
parse the values in some european locales which used a comma instead of a period to specify decimals
+ dispclut now has an “--reset-all” flag
* dispclut now uses the linear 0-65535 levels

Version 0.5:
! Re-worked entire .3dlut code base, much cleaner now (at the cost of a tiny speed loss)
* Detection for range (full or limited) is now handled by the 3dLUT loader itself and should be more consistent
In addition, this information will be displayed by the inspect3dlut program
+ applycal can load .ti3 files as well, and...
+ ..make3dlut can automatically apply .cal data from a .ti3 if found
+ applycal now supports Calibration Tester's #GammaRamp format
+ 32-bit floating point LUTs are now supported in all applicable programs
+ New program: changedepth.exe
Do note that while changedepth can pull values up/down to full or limited, this introduces a lot of banding
+ New program: dispclut.exe
+ New program: tag3dlut.exe

Version 0.4:
* Saving/loading to stdin or stdout now works properly
* Applycal now (attempts to) correctly detect the input value range, and also works for 8-bit LUTs (not just 16-bit)
* The TI3 parsing process now actually converts the entries to numbers,
removing the need for ugly string detection hacks - should fix the duplicate key bug also
+ It's now possible to generate new .3dluts from scratch using the TI3ParserLib
+ New program: inspect3dlut.exe
+ New program: gen3dlut.exe
? Speed improvements here and there

Version 0.3:
+ New program: applycal.exe
+ New program: swapchan.exe
+ New program: version.exe

Version 0.2:
? Now requires the .NET version 4.0 to run properly
~ Updated UI, removed ugly buttons and added a menu
+ New program: genscript.exe
+ New program: make3dlut.exe
+ New program: parseti3.exe
+ New program: updateycms.exe
* Temporary files are properly removed now
* Updating yCMS now correctly downloads the 64-bit or 32-bit versions depending on your processor architecture

Version 0.1:
? Initial release

Legend:
! Major modification
+ New feature
- Removed feature
* Fixed bug
~ Sideways change
? Information

[alph@]

ok,what values you use in ambient light level,Have you tried the 'black point correction'. I usually made ​​a calibration (d6500-120cd/m-gama 2.35- black level native-calbration quality high-testchart 124)and i check with color hcfr.

[leeperry]

looks good! if you could allow merging LUT's like one for gamut mapping and one from Argyll off a .cal file, that'd be full of win :)

and if you could allow importing the primaries/secondaries saturations from Color HCFR, then it'd be :eek:

http://thumbnails46.imagebam.com/14590/995122145896247.jpg (http://www.imagebam.com/image/995122145896247)

maybe none of this is technically possible though, and I need to document myself on the .ti3 file format :o

An option to output 8bit 3DLUT's could be useful for the happy few using rgb3dlut() within the Avisynth filter of ffdshow.

[nand chan]

looks good! if you could allow merging LUT's like one for gamut mapping and one from Argyll off a .cal file, that'd be full of win :)

and if you could allow importing the primaries/secondaries saturations from Color HCFR, then it'd be :eek:

http://thumbnails46.imagebam.com/14590/995122145896247.jpg (http://www.imagebam.com/image/995122145896247)

maybe none of this is technically possible though, and I need to document myself on the .ti3 file format :o

An option to output 8bit 3DLUT's could be useful for the happy few using rgb3dlut() within the Avisynth filter of ffdshow.

Calibration is done using the TI1 files, I could do that though - just generate two .3dluts and link them together.

Also the 8 bit 3DLUTs is also possible.

[nand chan]

looks good! if you could allow merging LUT's like one for gamut mapping and one from Argyll off a .cal file, that'd be full of win :)

and if you could allow importing the primaries/secondaries saturations from Color HCFR, then it'd be :eek:

http://thumbnails46.imagebam.com/14590/995122145896247.jpg (http://www.imagebam.com/image/995122145896247)

maybe none of this is technically possible though, and I need to document myself on the .ti3 file format :o

An option to output 8bit 3DLUT's could be useful for the happy few using rgb3dlut() within the Avisynth filter of ffdshow.

While thinking about this, I realized that it would be pretty hard without manual adjustment because ArgyllCMS works with 0-255 values, and madVR works with 16-235 levels. You can't simply generate a .3dlut off the .cal and combine it to the madVR .3dlut due to that reason.

I could possible morph the 3dlut by transforming it from 0-255 to 16-235, but that's significantly beyond the scope of this project and I don't intend to do any complex color management logic myself - just tie together existing tools (like yCMS).

Merging noncompatible 3dluts would be something yCMS would have to do, and you should request for that feature there.

ok,what values you use in ambient light level,Have you tried the 'black point correction'. I usually made ​​a calibration (d6500-120cd/m-gama 2.35- black level native-calbration quality high-testchart 124)and i check with color hcfr.

I've never used black point correction, mainly because my black point is dark enough and doesn't show any noticeable color distortion.

I usually use 6504K (I enter the X/Y points manually) - 6500K is a bit redder than D65 - and 120 cd/m², sRGB tone curve, native black level, native black point and massive LUT testchart.

[yesgrey]

Congratulations for your tool! :)

[leeperry]

While thinking about this, I realized that it would be pretty hard without manual adjustment because ArgyllCMS works with 0-255 values, and madVR works with 16-235 levels. You can't simply generate a .3dlut off the .cal and combine it to the madVR .3dlut due to that reason.

I could possible morph the 3dlut by transforming it from 0-255 to 16-235, but that's significantly beyond the scope of this project and I don't intend to do any complex color management logic myself - just tie together existing tools (like yCMS).

Merging noncompatible 3dluts would be something yCMS would have to do, and you should request for that feature there.
TBH I use 0-255 LUT's using an old version of cr3dlut, but I'm quite sure yCMS could create them as well...the only problem would be mVR that might expect 16-235 LUT's indeed, but if you want automatic gamut mapping depending on the frame rate/resolution, the only solution atm is to use rgb3dlut(8bit) or t3dlut(16bit) within ffdshow automatic profiles.

Several ppl already requested the feature of embedding together within the same 3DLUT both the Argyll calibration and the gamut mapping, so mVR's dithering wouldn't be messed around by the graphic card's CLUT...this would require an option in mVR to disable the latter, though.

Anyway this was just some plain brainstorming, and you're the author...making you the final judge in the end ;)

[nand chan]

Version 0.1.1 is out now that I have gained yesgrey's acceptance of the yCMS download method

TBH I use 0-255 LUT's using an old version of cr3dlut, but I'm quite sure yCMS could create them as well...the only problem would be mVR that might expect 16-235 LUT's indeed, but if you want automatic gamut mapping depending on the frame rate/resolution, the only solution atm is to use rgb3dlut(8bit) or t3dlut(16bit) within ffdshow automatic profiles.

True, but I dislike the usage of ffdshow because it does not yet have a native 10-bit pipeline, also that would require working with YUV LUTs instead of RGB luts unless you want to use ffdshow's piss poor RGB conversion. I'll still consider adding 3dLUT combining as a future feature but it's not high on the priority list because there are too many variables to be careful of when combining them, and it's not easily doable with madVR LUTs. (unless madVR allows working with 0-255 LUTs in the future)

It's fairly straightforward to /create/ the correct LUTs using yCMS, it's just that madVR will reject the .3dlut.

Several ppl already requested the feature of embedding together within the same 3DLUT both the Argyll calibration and the gamut mapping, so mVR's dithering wouldn't be messed around by the graphic card's CLUT...this would require an option in mVR to disable the latter, though.

This would indeed be very important since iirc the CLUTs are only one-dimensional. The main problem would be generating a proper 16-235 .3dlut from .cal data, which probably lies within the domain of a tool such as yCMS. I do not (yet) have the necessarily technical knowledge to interpolate and generate tables like that.

My first priority now though is getting a working, automatized front-end for Argyll and yCMS in place to bridge the process from profiling to .3dlut creation.

[leeperry]

Oops yeah, indeed I use YUY2 LUT's in ffdshow so I guess merging them w/ a .cal file would turn into headaches, if even possible.

And tbh I'm very happy w/ tritical's yv12toyuy2() because it allows you to choose what resizing algorithm to use for the YV12>YUY2 interpolation. I've finetuned all thoses settings to get the crispiest non-ringing PQ I've ever seen. I use SmoothL() to dither nicely and avoid banding, I'm not hung up on 10bit processing anymore...especially when dealing w/ 16-235 SMPTE-C content, because it might be a different story w/ xvYCC but we're not quite there yet.

[nand chan]

This would indeed be very important since iirc the CLUTs are only one-dimensional. The main problem would be generating a proper 16-235 .3dlut from .cal data, which probably lies within the domain of a tool such as yCMS. I do not (yet) have the necessarily technical knowledge to interpolate and generate tables like that.

My first priority now though is getting a working, automatized front-end for Argyll and yCMS in place to bridge the process from profiling to .3dlut creation.

Update: Just realized that Argyl''s .cal files are one-dimensional LUTs as well, so the practical gains of this are questionable. It would only serve to eliminate the video card's CLUTs, which are potentially sub-optimal.

Applying a .cal to a .3dlut seems like a simple process then, I'll write the code for it sooner or later.

Generating a .3dlut from a .ti1 however would be much harder since it requires significant interpolation.

Updated the UI slightly also: http://img805.imageshack.us/img805/7990/ti3parserdemo.png

[nand chan]

Bumping with a semi-big update, version 0.2 is now out. See the initial posts for a changelog.

Download link: http://www.mediafire.com/?9by785si95tu73b (source is included)

[madshi]

Thank you, nice work... :)

[ryrynz]

Thank you. just a minor issue, the about box for 0.2 shows version number 0.1.1.4186.

[Audionut]

Spent about an hour last night trying to get everything working again with madVR after not using it for awhile.

Then come across this one-click solution this morning :)

Nice work and thanks.

the about box for 0.2 shows version number 0.1.1.4186.

I see version 0.2.0.36408

[ryrynz]

Different naming (no space) for the new version, must have deleted the wrong folder. All good, thanks.

[nand chan]

HUGE SUCCESS

I've finally finished writing all of the code for loading, editing (mapping a function over each entry) and saving .3dlut files, as well as the code for loading and properly interpolating a .cal file. All of this takes place in little under a second of processing time (and they said C# was slow).

As proof of concept, I've edited my existing .3dlut and swapped the red and blue channels just for fun. Here's the somewhat amusing result: http://img5.imageshack.us/img5/5215/chanswap.jpg

Expect to see a fully working version that will apply .cal calibration data to a .3dlut sometime tomorrow, as well as a few utilities to swap channels for fun.

For now, more fun stuff:
chanswap.exe -i test.3dlut --invert produces: http://img192.imageshack.us/img192/2923/chaninvert.jpg

Thank you. just a minor issue, the about box for 0.2 shows version number 0.1.1.4186.

While this was not intentional, do note that the packaging version (0.2) is not necessarily the version of the GUI program (in this case, 0.2.0.36408) - I'll update it in the future to include both. The packaging version is merely a version number for the overall collection and will get increased with every release (similarly to madVR's current 0.74 version number). The versions of the individual programs (eg. the GUI) will be following a more traditional major.minor.iteration.build version number based on features on that specific program.

[Audionut]

Expect to see a fully working version that will apply .cal calibration data to a .3dlut sometime tomorrow

Excellent. Can't wait to try it out.

[nand chan]

Another small update, 0.3 is out.

With this release, it's now possible to directly alter .3dlut files by applying a .cal file or swapping the levels around (for fun).

Unfortunately, there is currently no GUI front-end for this functionality - I am waiting until I've come up with a better GUI design before I add this. However, it's easily done via the command line:

applycal.exe -i source.3dlut -o destination.3dlut calibrationfile.cal

Edit: I just realized this technique has another advantage. Since I'm interpolating the values to a 16-bit LUT, it transcends the precision of the graphics card's built-in 8-bit CLUTs. So if you have a 10-bit monitor (like I do), and an appropriate graphics card (like I don't), your calibration corrections will preserve that depth when merged onto a .3dlut.

[Audionut]

No problems opening the original TI3 file I originally opened, but trying to process any other TI3 file is giving me this error.

http://farm7.static.flickr.com/6084/6072851162_dec4f46ea3.jpg


Getting the same error message with,

genscript:
make3dlut:
parseti3:

[nand chan]

No problems opening the original TI3 file I originally opened, but trying to process any other TI3 file is giving me this error.

http://farm7.static.flickr.com/6084/6072851162_dec4f46ea3.jpg


Getting the same error message with,

genscript:
make3dlut:
parseti3:

Can you upload the .ti3 file somewhere so I can have a look at it?

[nand chan]

Big update, version 0.4 is out
See the original post for a full list of changes

Download link: http://www.mediafire.com/?5f7mid5m131rjxb

The main new addition is that of gen3dlut.exe, which allows the creation of custom 3dluts. Currently, only two modes are supported - flat (output=input) and grayscale (output=luminosity).

Some fun things:
gen3dlut.exe --grayscale -o test.3dlut: http://img189.imageshack.us/img189/3889/bnwp.jpg
or
gen3dlut.exe --grayscale -o - | swapchan --invert -i - -o test.3dlut: http://img641.imageshack.us/img641/5039/invertgray.jpg

Note: This utility also allows you to create generic .3dluts that just include calibration data from a .cal, eg. like so:
gen3dlut.exe -o - | applycal -i - -o output.3dlut calibration.cal

[Audionut]

Here are 5 of my results.
http://www.mediafire.com/?szmzpi50gao1mco

Version 0.4 can open the ones that weren't working before. And I can see what the problem is. Luminance values are screwed.

From what I can see, the consistancy between the ones that don't work are that they have greyscale values going up in incriments of 5 where I have created my own testchart file.
The one that works, from memory I just selected the large testchart.

[nand chan]

Here are 5 of my results.
http://www.mediafire.com/?szmzpi50gao1mco

Version 0.4 can open the ones that weren't working before. And I can see what the problem is. Luminance values are screwed.

From what I can see, the consistancy between the ones that don't work are that they have greyscale values going up in incriments of 5 where I have created my own testchart file.
The one that works, from memory I just selected the large testchart.

Which one was working fine before version 0.4?

[Audionut]

VK266H 2011-08-22 80cdm² 0.3127x 0.3291y 0.35cdm² native 1.85 LQ-HQ 1xCurve+MTX

[leeperry]

OMG, he's unstoppable! So I can take a 0-255 gamut mapping 16bit LUT from cr3dlut, merge it w/ a .cal file and output a 8bit LUT to feed rgb3dlut() in ffdshow? :eek:

I'm asking instead of trying coz my .net crapola is not up to date I'm afraid...but if that's the case I might have an urgent need for a "reset CLUT while playing" in mVR http://forum.slysoft.com/images/smilies/agreed.gif

But I guess you can't get it to output a YUY2 LUT, can you? :o

[nand chan]

OMG, he's unstoppable! So I can take a 0-255 gamut mapping 16bit LUT from cr3dlut, merge it w/ a .cal file and output a 8bit LUT to feed rgb3dlut() in ffdshow? :eek:

There is currently no utility for changing the bit-depth of a LUT (eg. from 16-bit down to 8-bit) but I can add a program for that by version 0.5.

Note: This would obviously only work for the /output/ bit-depth of a LUT. Changing the input bit depth is unfeasible because anything about 3x8-bit would take such obscene amounts of storage that you couldn't even load it. For example, a fully 10-bit .3DLUT will take 4 GB of memory, more than the average video card can handle. A 16-bit LUT will take several petabytes.

However, what you can do is transform a 8->16 bit LUT down to a 8->8 bit LUT (or the reverse), though at the cost of precision (in the former).

But yes, you can basically take a 8->16 bit LUT gamut mapping (like the type madVR uses) and apply .cal data to it.

I'm asking instead of trying coz my .net crapola is not up to date I'm afraid...but if that's the case I might have an urgent need for a "reset CLUT while playing" in mVR http://forum.slysoft.com/images/smilies/agreed.gif

Agreed on this. :) It's such a hassle to reset the CLUTs manually using dispcalGUI all the time.

But I guess you can't get it to output a YUY2 LUT, can you? :o

That wouldn't be in the realm of impossibility as far as I know how YUY2 LUTs work, then I'll also have to do the RGB->LUT conversion since .cal files work in RGB space. (You'd have to choose which reference decoding matrices to use). I wouldn't expect anything resembling YUV support for at least another version or two though, maybe 0.6 or 0.7, mainly because I'm only primarily concerned with madVR which uses RGB LUTs and this simplifies things greatly.

[nand chan]

There is currently no utility for changing the bit-depth of a LUT (eg. from 16-bit down to 8-bit) but I can add a program for that by version 0.5.

Updated, added this, in addition to re-coding most of the .3dlut code to make it much, much cleaner.

All 3dlut processing is now done using 64-bit floating point calculations at every step, and they are only converted to/from their appropriate lower-bit-depth representations when saving/loading to memory.

The .3dlut processing pipeline now only accepts functions that use normalized 0-1 doubles.

This has the advantage of: 1. Allowing for seamless interaction of .3dluts with any bit-depth and 2. level detection is now done in the .3dlut library instead of in the program, and the .3dlut lib correctly compensates for full or limited range values when pulling down the levels to 0-1 doubles.

It also removes about 70% of the code size, which was previously full of “Transform16, Transform8, Transform32” or “Pullup16, Pullup8, Pullup32” and so forth. All of these duplicate versions have now been removed and replace by generic versions that work on any-bitdepth.

Which means that I am now being bottlenecked by the limited capacity of the .3dlut file standard, even though my code is perfectly capable of processing something irregular like 10-bit LUTs (as opposed to 8, 16 or 32).

Edit: For the curious, a comparison of 16-bit LUT vs 8-bit LUT (to make madVR load it, I first converted it down to 8-bit and then converted that back up to 16-bit which is simply a bit-shift, the same way madVR converts levels). You can clearly see the advantage of greater-than-8-bit precision processing even on 8-bit devices. So if you want to use a .3dlut with avisynth be wary of this difference:

http://screenshotcomparison.com/comparison/75573

If you do want to create an 8-bit 3dLUT for avisynth etc, you can do something like this:
gen3dlut -o - -b 8 | applycal -i - -o output.3dlut calibration.cal

[madshi]

Yeah, an 8bit 3dlut is probably not good enough. yCMS also internally uses 64bit floats for calculation and then rounds that down to 16bit integer for output. I wouldn't use anything less than 16bit integer output. It's debatable, though, whether input must be 8bit. madVR/yCMS can also use 7bit or 6bit input, to save space and improve performance, on the cost of a very small (probably not visible) drop in quality.

[cyberbeing]

Can you add more format support for applycal?

CAL data from a TI3 file (Argyll Device Calibration State)?
Calibration Tester (http://www.xrite.com/product_overview.aspx?ID=795&Action=Support&SoftwareID=546)'s 0-65535 format?
An ICC profile vctg tag?
The currently loaded CLUT?

applycal from version 0.4 is now extremely slow. Is that because of working in 64-bit floating point or is it a bug? After 30 minutes went by I just gave up and killed it. If it's going to be that slow, I hope you can optimize or multi-thread it. Maybe you could port it to OpenGL GLSL shader code and run it on the GPU.

madVR/yCMS can also use 7bit or 6bit input, to save space and improve performance, on the cost of a very small (probably not visible) drop in quality.
Just don't use a 6bit 3dlut with applycal or you'll get crappy results.

[madshi]

Just don't use a 6bit 3dlut with applycal or you'll get crappy results.
Have you found the time to measure madVR/yCMS's 7bit and 6bit 3dluts yet to check if there's any difference (worth nothing) between 8bit and lower bitdepths?

[nand chan]

Can you add more format support for applycal?

CAL data from a TI3 file (Argyll Device Calibration State)?

I never realized this differed from the .cal file. I'll add support for it soon enough.

Note: As a “feature” of the current coding, all you have to do is change the first line of a .ti3 file to “CTI3” to “CAL” and applycal will accept it.

Calibration Tester (http://www.xrite.com/product_overview.aspx?ID=795&Action=Support&SoftwareID=546)'s 0-65535 format?

Would depend on the format in question. If I don't have access to a freely available documentation and a few sample cases for it, it's unlikely.

An ICC profile vctg tag?

I'm probably adding ICC profile support sometime around version 1.0.

The currently loaded CLUT?

I don't know how to access those.

applycal from version 0.4 is now extremely slow. Is that because of working in 64-bit floating point or is it a bug? After 30 minutes went by I just gave up and killed it. If it's going to be that slow, I hope you can optimize or multi-thread it. Maybe you could port it to OpenGL GLSL shader code and run it on the GPU.

That has to be a bug, it runs in a few seconds on my CPU. How many entries does your .cal file have?

Yeah, an 8bit 3dlut is probably not good enough. yCMS also internally uses 64bit floats for calculation and then rounds that down to 16bit integer for output. I wouldn't use anything less than 16bit integer output. It's debatable, though, whether input must be 8bit. madVR/yCMS can also use 7bit or 6bit input, to save space and improve performance, on the cost of a very small (probably not visible) drop in quality.

How do you properly pull down to a limited range 6-bit or 7-bit integer? I can't right shift 235 any further without cutting bits off. Is that just what I should do?

[madshi]

How do you properly pull up to a 6-bit or 7-bit integer? I can't right shift 235 any further without cutting bits off. Is that just what I should do?
I've no idea. I only know how to use 3dluts, not how to create them. You'll have to ask yesgrey about that.

[e-t172]

The currently loaded CLUT?
I don't know how to access those.

http://msdn.microsoft.com/en-us/library/dd316946%28v=VS.85%29.aspx

[cyberbeing]

Have you found the time to measure madVR/yCMS's 7bit and 6bit 3dluts yet to check if there's any difference (worth nothing) between 8bit and lower bitdepths?
It's about what you'd expect for a lut with 4x4 accuracy, interpolated to 1x1. Using a 6-bit lut results in an ever so slightly, smoother histogram. Though at least on a standard gamut monitor, it's not really visible unless you pixel peek, since the skew is often 0.1-0.5 dE. The trade-off is worth it to me, to avoid the occasional 3dlut showstopper causing massive dropped frames, but unless someone is in a similar situation, I don't know why they'd choose to use a 6-bit lut. I don't use yCMS Grayscale Measurements, a Wide Gamut monitor, or madVR gamma correction, so I have no idea if the results would be similar mine while using those things.

Anyway, not relavant to this thread, but here are some comparison screenshots for you madshi:
http://www.mediafire.com/?74lk6yli3l7n7cx
The random noise + pattern image with 6 semi-transparent colored blocks overlayed, is just something I made on-the-fly as a torture test for the 6-bit lut.

Would depend on the format in question. If I don't have access to a freely available documentation and a few sample cases for it, it's unlikely.
It's just a text file 0-255 with 16bit integer CLUT values (vs the normalized 16bit floating point value in a TI3). It's one of the few tools to load, dump, modify, an re-load the active CLUT.
The layout is always the same. "#GammaRamp" on first line identifying the format. 0-255 in first column, RED, GREEN, BLUE.

#GammaRamp
0 1599 1305 1270
1 2161 1923 1791
2 2722 2538 2310
3 3280 3150 2825
4 3835 3760 3338
5 4383 4365 3846
6 4908 4949 4342
7 5398 5494 4813
8 5847 5991 5256
9 6255 6439 5667
10 6629 6843 6047
11 6976 7211 6402
12 7300 7550 6734
13 7605 7864 7050
14 7894 8159 7350
15 8171 8440 7639
16 8438 8709 7919
17 8696 8967 8191
18 8945 9217 8454
19 9187 9459 8712
20 9423 9697 8963
21 9653 9929 9206
22 9879 10155 9448
23 10102 10377 9683
24 10319 10596 9916
25 10531 10812 10143
26 10741 11026 10368
27 10949 11237 10590
28 11155 11446 10809
29 11359 11653 11027
30 11562 11858 11243
31 11763 12063 11457
32 11962 12268 11668
33 12162 12474 11880
34 12362 12680 12093
35 12562 12887 12306
36 12762 13093 12519
37 12962 13299 12732
38 13162 13505 12945
39 13362 13711 13156
40 13561 13918 13367
41 13757 14127 13578
42 13954 14338 13786
43 14152 14548 13995
44 14351 14759 14206
45 14550 14970 14416
46 14749 15182 14627
47 14949 15392 14836
48 15149 15603 15046
49 15350 15812 15256
50 15552 16019 15466
51 15755 16226 15676
52 15958 16430 15885
53 16164 16635 16095
54 16369 16841 16303
55 16576 17050 16514
56 16784 17261 16724
57 16996 17474 16936
58 17211 17689 17152
59 17429 17904 17369
60 17651 18118 17587
61 17873 18333 17805
62 18097 18549 18023
63 18321 18765 18242
64 18546 18982 18459
65 18772 19199 18678
66 18998 19417 18900
67 19224 19636 19120
68 19451 19854 19343
69 19677 20074 19564
70 19904 20292 19787
71 20132 20508 20011
72 20358 20722 20232
73 20584 20935 20453
74 20808 21147 20675
75 21032 21360 20897
76 21256 21572 21118
77 21480 21783 21342
78 21703 21996 21565
79 21927 22209 21787
80 22151 22424 22011
81 22374 22640 22236
82 22599 22859 22462
83 22824 23079 22689
84 23049 23300 22917
85 23275 23522 23144
86 23500 23744 23370
87 23724 23965 23596
88 23947 24188 23820
89 24169 24410 24043
90 24391 24633 24268
91 24612 24855 24492
92 24834 25077 24717
93 25057 25299 24940
94 25281 25521 25164
95 25506 25746 25389
96 25732 25972 25616
97 25962 26202 25847
98 26196 26432 26080
99 26432 26665 26315
100 26672 26898 26551
101 26913 27133 26787
102 27154 27370 27026
103 27397 27608 27266
104 27639 27846 27505
105 27879 28080 27742
106 28115 28312 27976
107 28347 28540 28208
108 28577 28767 28440
109 28804 28992 28668
110 29031 29217 28899
111 29259 29444 29130
112 29487 29671 29360
113 29717 29898 29591
114 29948 30126 29823
115 30179 30355 30054
116 30411 30582 30286
117 30643 30809 30518
118 30876 31036 30752
119 31110 31264 30987
120 31344 31492 31221
121 31578 31719 31456
122 31812 31947 31692
123 32046 32175 31929
124 32280 32404 32168
125 32513 32632 32404
126 32745 32861 32643
127 32977 33091 32881
128 33208 33322 33123
129 33441 33557 33366
130 33676 33794 33610
131 33914 34032 33856
132 34151 34271 34101
133 34389 34510 34347
134 34627 34751 34593
135 34867 34991 34840
136 35108 35234 35089
137 35351 35477 35339
138 35595 35721 35590
139 35841 35964 35840
140 36087 36209 36090
141 36333 36454 36342
142 36578 36699 36593
143 36823 36943 36844
144 37069 37187 37093
145 37314 37432 37343
146 37561 37676 37593
147 37807 37921 37841
148 38053 38165 38089
149 38298 38409 38337
150 38541 38651 38580
151 38782 38891 38822
152 39021 39129 39061
153 39258 39365 39299
154 39494 39601 39538
155 39729 39835 39776
156 39967 40072 40014
157 40205 40309 40254
158 40444 40546 40494
159 40685 40784 40734
160 40926 41024 40977
161 41168 41264 41219
162 41411 41505 41464
163 41656 41749 41711
164 41903 41996 41960
165 42153 42246 42212
166 42408 42502 42468
167 42667 42764 42729
168 42931 43030 42993
169 43197 43298 43259
170 43466 43566 43526
171 43735 43833 43794
172 44005 44100 44063
173 44277 44367 44332
174 44549 44635 44602
175 44819 44901 44869
176 45087 45165 45136
177 45351 45426 45398
178 45609 45682 45657
179 45862 45932 45912
180 46110 46177 46164
181 46357 46418 46413
182 46602 46657 46661
183 46844 46894 46909
184 47086 47131 47156
185 47329 47369 47404
186 47573 47610 47655
187 47817 47852 47905
188 48062 48097 48157
189 48308 48346 48409
190 48554 48596 48663
191 48799 48847 48917
192 49045 49099 49171
193 49293 49354 49428
194 49542 49611 49685
195 49792 49871 49942
196 50041 50133 50199
197 50291 50396 50455
198 50541 50659 50710
199 50792 50921 50965
200 51043 51183 51219
201 51295 51445 51470
202 51549 51708 51722
203 51807 51973 51974
204 52068 52237 52226
205 52331 52500 52479
206 52596 52762 52730
207 52861 53024 52982
208 53125 53283 53232
209 53387 53540 53480
210 53646 53794 53727
211 53904 54044 53975
212 54159 54292 54223
213 54414 54540 54472
214 54667 54788 54720
215 54920 55036 54972
216 55175 55289 55225
217 55431 55544 55481
218 55690 55802 55738
219 55950 56065 55996
220 56211 56329 56252
221 56471 56595 56506
222 56732 56861 56760
223 56991 57128 57010
224 57249 57392 57258
225 57510 57656 57504
226 57772 57920 57751
227 58035 58187 57998
228 58297 58453 58244
229 58559 58717 58489
230 58820 58981 58734
231 59080 59243 58980
232 59339 59506 59226
233 59598 59770 59474
234 59854 60034 59722
235 60110 60299 59971
236 60364 60564 60220
237 60619 60829 60469
238 60872 61094 60719
239 61124 61357 60968
240 61377 61620 61217
241 61629 61881 61465
242 61881 62144 61716
243 62134 62407 61966
244 62386 62670 62216
245 62639 62932 62465
246 62892 63194 62713
247 63144 63455 62963
248 63397 63716 63212
249 63648 63978 63462
250 63899 64239 63712
251 64150 64500 63961
252 64399 64762 64211
253 64648 65023 64461
254 64897 65284 64712
255 65146 65535 64963



That has to be a bug, it runs in a few seconds on my CPU. How many entries does your .cal file have?
It's just a normal ArgyllCMS .cal file, so 256. Process monitor appears to show applycal apparently stuck in a loop reading the .cal file around 1,600 times per second.

[nand chan]

It's just a text file 0-255 with 16bit integer CLUT values (vs the normalized 16bit floating point value in a TI3). It's one of the few tools to load, dump, modify, an re-load the active CLUT.
Oh, that exists? I've just finished writing my own :P

Oh well, my version will work for .ti3 files, .cal files and those funky calibration tester formats.

The layout is always the same. "#GammaRamp" on first line identifying the format. 0-255 in first column, RED, GREEN, BLUE.

#GammaRamp
0 1599 1305 1270
1 2161 1923 1791
2 2722 2538 2310
3 3280 3150 2825
4 3835 3760 3338
5 4383 4365 3846
6 4908 4949 4342
7 5398 5494 4813
8 5847 5991 5256
9 6255 6439 5667
10 6629 6843 6047
11 6976 7211 6402
12 7300 7550 6734
13 7605 7864 7050
14 7894 8159 7350
15 8171 8440 7639
16 8438 8709 7919
17 8696 8967 8191
18 8945 9217 8454
19 9187 9459 8712
20 9423 9697 8963
21 9653 9929 9206
22 9879 10155 9448
23 10102 10377 9683
24 10319 10596 9916
25 10531 10812 10143
26 10741 11026 10368
27 10949 11237 10590
28 11155 11446 10809
29 11359 11653 11027
30 11562 11858 11243
31 11763 12063 11457
32 11962 12268 11668
33 12162 12474 11880
34 12362 12680 12093
35 12562 12887 12306
36 12762 13093 12519
37 12962 13299 12732
38 13162 13505 12945
39 13362 13711 13156
40 13561 13918 13367
41 13757 14127 13578
42 13954 14338 13786
43 14152 14548 13995
44 14351 14759 14206
45 14550 14970 14416
46 14749 15182 14627
47 14949 15392 14836
48 15149 15603 15046
49 15350 15812 15256
50 15552 16019 15466
51 15755 16226 15676
52 15958 16430 15885
53 16164 16635 16095
54 16369 16841 16303
55 16576 17050 16514
56 16784 17261 16724
57 16996 17474 16936
58 17211 17689 17152
59 17429 17904 17369
60 17651 18118 17587
61 17873 18333 17805
62 18097 18549 18023
63 18321 18765 18242
64 18546 18982 18459
65 18772 19199 18678
66 18998 19417 18900
67 19224 19636 19120
68 19451 19854 19343
69 19677 20074 19564
70 19904 20292 19787
71 20132 20508 20011
72 20358 20722 20232
73 20584 20935 20453
74 20808 21147 20675
75 21032 21360 20897
76 21256 21572 21118
77 21480 21783 21342
78 21703 21996 21565
79 21927 22209 21787
80 22151 22424 22011
81 22374 22640 22236
82 22599 22859 22462
83 22824 23079 22689
84 23049 23300 22917
85 23275 23522 23144
86 23500 23744 23370
87 23724 23965 23596
88 23947 24188 23820
89 24169 24410 24043
90 24391 24633 24268
91 24612 24855 24492
92 24834 25077 24717
93 25057 25299 24940
94 25281 25521 25164
95 25506 25746 25389
96 25732 25972 25616
97 25962 26202 25847
98 26196 26432 26080
99 26432 26665 26315
100 26672 26898 26551
101 26913 27133 26787
102 27154 27370 27026
103 27397 27608 27266
104 27639 27846 27505
105 27879 28080 27742
106 28115 28312 27976
107 28347 28540 28208
108 28577 28767 28440
109 28804 28992 28668
110 29031 29217 28899
111 29259 29444 29130
112 29487 29671 29360
113 29717 29898 29591
114 29948 30126 29823
115 30179 30355 30054
116 30411 30582 30286
117 30643 30809 30518
118 30876 31036 30752
119 31110 31264 30987
120 31344 31492 31221
121 31578 31719 31456
122 31812 31947 31692
123 32046 32175 31929
124 32280 32404 32168
125 32513 32632 32404
126 32745 32861 32643
127 32977 33091 32881
128 33208 33322 33123
129 33441 33557 33366
130 33676 33794 33610
131 33914 34032 33856
132 34151 34271 34101
133 34389 34510 34347
134 34627 34751 34593
135 34867 34991 34840
136 35108 35234 35089
137 35351 35477 35339
138 35595 35721 35590
139 35841 35964 35840
140 36087 36209 36090
141 36333 36454 36342
142 36578 36699 36593
143 36823 36943 36844
144 37069 37187 37093
145 37314 37432 37343
146 37561 37676 37593
147 37807 37921 37841
148 38053 38165 38089
149 38298 38409 38337
150 38541 38651 38580
151 38782 38891 38822
152 39021 39129 39061
153 39258 39365 39299
154 39494 39601 39538
155 39729 39835 39776
156 39967 40072 40014
157 40205 40309 40254
158 40444 40546 40494
159 40685 40784 40734
160 40926 41024 40977
161 41168 41264 41219
162 41411 41505 41464
163 41656 41749 41711
164 41903 41996 41960
165 42153 42246 42212
166 42408 42502 42468
167 42667 42764 42729
168 42931 43030 42993
169 43197 43298 43259
170 43466 43566 43526
171 43735 43833 43794
172 44005 44100 44063
173 44277 44367 44332
174 44549 44635 44602
175 44819 44901 44869
176 45087 45165 45136
177 45351 45426 45398
178 45609 45682 45657
179 45862 45932 45912
180 46110 46177 46164
181 46357 46418 46413
182 46602 46657 46661
183 46844 46894 46909
184 47086 47131 47156
185 47329 47369 47404
186 47573 47610 47655
187 47817 47852 47905
188 48062 48097 48157
189 48308 48346 48409
190 48554 48596 48663
191 48799 48847 48917
192 49045 49099 49171
193 49293 49354 49428
194 49542 49611 49685
195 49792 49871 49942
196 50041 50133 50199
197 50291 50396 50455
198 50541 50659 50710
199 50792 50921 50965
200 51043 51183 51219
201 51295 51445 51470
202 51549 51708 51722
203 51807 51973 51974
204 52068 52237 52226
205 52331 52500 52479
206 52596 52762 52730
207 52861 53024 52982
208 53125 53283 53232
209 53387 53540 53480
210 53646 53794 53727
211 53904 54044 53975
212 54159 54292 54223
213 54414 54540 54472
214 54667 54788 54720
215 54920 55036 54972
216 55175 55289 55225
217 55431 55544 55481
218 55690 55802 55738
219 55950 56065 55996
220 56211 56329 56252
221 56471 56595 56506
222 56732 56861 56760
223 56991 57128 57010
224 57249 57392 57258
225 57510 57656 57504
226 57772 57920 57751
227 58035 58187 57998
228 58297 58453 58244
229 58559 58717 58489
230 58820 58981 58734
231 59080 59243 58980
232 59339 59506 59226
233 59598 59770 59474
234 59854 60034 59722
235 60110 60299 59971
236 60364 60564 60220
237 60619 60829 60469
238 60872 61094 60719
239 61124 61357 60968
240 61377 61620 61217
241 61629 61881 61465
242 61881 62144 61716
243 62134 62407 61966
244 62386 62670 62216
245 62639 62932 62465
246 62892 63194 62713
247 63144 63455 62963
248 63397 63716 63212
249 63648 63978 63462
250 63899 64239 63712
251 64150 64500 63961
252 64399 64762 64211
253 64648 65023 64461
254 64897 65284 64712
255 65146 65535 64963


Seems simple enough, I'll just pull down values to 64-bit doubles and use my existing logic. Confirmed for 0.5.

It's just a normal ArgyllCMS .cal file, so 256. Process monitor appears to show applycal apparently stuck in a loop reading the .cal file around 1,600 times per second.

I'll try to figure out what's going wrong. How are you running the command, btw?

[cyberbeing]

I'll try to figure out what's going wrong. How are you running the command, btw?

Windows XP SP3 x86.
Command Prompt.

cd "C:\Program Files\TI3Parser\"
applycal -i 8bit.3dlut GDM-F520_2011-08-23_rec709_Matrix.cal

[nand chan]

Oh, that exists? I've just finished writing my own :P

Oh well, my version will work for .ti3 files, .cal files and those funky calibration tester formats.

I spoke too soon, SetDeviceGammaRamp is seemingly independent of the device-specific gamma ramps that tools such as dispcalGUI or the CalibrationTester load. Those are per-monitor, and SetDeviceGammaRamp applies globally. In fact, the tool doesn't even detect any SetDeviceGammaRamp changes.

Do you know a way around this limitation? I haven't been able to find out the correct functions for loading a monitor-specific gamma ramp other than the DirectX ones.

Either way, you can use CalibrationTester to save the CLUT to a file and applycal will be able to apply it.

[cyberbeing]

You probably forgot to specify a hardware device context. The following I assume are the relevant imported functions for what you want to do, since all lut loaders contain them.

User32.dll
GetMonitorInfoA
EnumDisplayMonitors
MonitorFromWindow

GDI32.dll
CreateDCA
GetICMProfileA
SetDeviceGammaRamp
GetDeviceGammaRamp
DeleteDC

[nand chan]

You probably forgot to specify a hardware device context. The following I assume are the relevant imported functions for what you want to do, since all lut loaders contain them.

User32.dll
GetMonitorInfoA
EnumDisplayMonitors
MonitorFromWindow

GDI32.dll
CreateDCA
GetICMProfileA
SetDeviceGammaRamp
GetDeviceGammaRamp
DeleteDC

Done.

[leeperry]

For the curious, a comparison of 16-bit LUT vs 8-bit LUT (to make madVR load it, I first converted it down to 8-bit and then converted that back up to 16-bit which is simply a bit-shift, the same way madVR converts levels). You can clearly see the advantage of greater-than-8-bit precision processing even on 8-bit devices. So if you want to use a .3dlut with avisynth be wary of this difference:

http://screenshotcomparison.com/comparison/75573
fair enough, but here's a real world banding test pattern going through LAV CUVID: Banding 720p.rec709.mkv (http://www.mediafire.com/?wbmyif3yf2mijti)

no post-processing in ffdshow, simply bypassing as 16-235 YV12, and mVR taking care of TV>PC conversion and dithering on top(no LUT enabled in there): http://thumbnails39.imagebam.com/14649/351a4a146483981.jpg (http://www.imagebam.com/image/351a4a146483981)

and now SmoothL() plus an 8bit YUY2 LUT in ffdshow, outputting 0-255 RGB32 to mVR, only taking care of dithering: http://thumbnails41.imagebam.com/14649/4ff1f3146483972.jpg (http://www.imagebam.com/image/4ff1f3146483972)

on my semi-10bit CRT, the latter looks much smoother.

[nevcairiel]

Its an artificial pattern, how is that real world? :D

Anyhow, while the first shows very minor 1px bands (no severe macro bands), the second shows very strong dithering noise.
Personally, on this LCD screen, i prefer the first. But of course, that may just be a matter of taste, and/or different screen technologys.

[Wile-E-Coyote]

Hello.

I just successfully calibrated my TV with dispcalGUI and wanted to create a 3Dlut file with it but when I open my ti3 file I get an error message saying the input string format is incorrect.

[leeperry]

Its an artificial pattern, how is that real world? :D

Anyhow, while the first shows very minor 1px bands (no severe macro bands), the second shows very strong dithering noise.
Personally, on this LCD screen, i prefer the first. But of course, that may just be a matter of taste, and/or different screen technologys.
Well, real world as in "going through the whole video pipeline".

I'll give you that I was using SuperSampling and Grainfactory4()....now w/ only SmoothL+YUY2 LUT: http://thumbnails49.imagebam.com/14649/8f5e97146487017.jpg (http://www.imagebam.com/image/8f5e97146487017)

any better on your LCD? Hopefully not a 6bit TN? :o

I could also disable dithering in mVR, but I like a grainy picture tbh: http://thumbnails42.imagebam.com/14649/99ded0146487520.jpg (http://www.imagebam.com/image/99ded0146487520)

[nand chan]

Well, real world as in "going through the whole video pipeline".

I'll give you that I was using SuperSampling and Grainfactory4()....now w/ only SmoothL+YUY2 LUT: http://thumbnails49.imagebam.com/14649/8f5e97146487017.jpg (http://www.imagebam.com/image/8f5e97146487017)

any better on your LCD? Hopefully not a 6bit TN? :o

I could also disable dithering in mVR, but I like a grainy picture tbh: http://thumbnails42.imagebam.com/14649/99ded0146487520.jpg (http://www.imagebam.com/image/99ded0146487520)

The third pattern looks identical to the first pattern on my monitor. The second pattern has awful dithering noise as mentioned.

10-bit H-IPS with 12 bit internal LUTs.

Hello.

I just successfully calibrated my TV with dispcalGUI and wanted to create a 3Dlut file with it but when I open my ti3 file I get an error message saying the input string format is incorrect.

Upload the .ti3 you get errors with.

[Wile-E-Coyote]

Here you go (http://petiflo.free.fr/profil.ti3)

[nand chan]

Going to go ahead and release version 0.5 as-is because the software is still alpha either way and I don't want to delay the current release/feedback cycle too much.
Might still be some bugs, I haven't extensively tested every single tool after the code re-write. Everything should be a lot, lot cleaner though.

In version 0.6 I am going to be adding multi-threaded .3dlut processing to speed up execution time on multi-core processors.

As usual, see the first post for a changelog: http://www.mediafire.com/?wwbuu5vhau5x6jm

I'm still trying to think of a new name for the project.

Here you go (http://petiflo.free.fr/profil.ti3)

Works fine for me in version 0.5, try it.

[nevcairiel]

The third pattern looks identical to the first pattern on my monitor. The second pattern has awful dithering noise as mentioned.

I agree, third and first look pretty much identical.
The 4th shows more banding then the others.

Also H-IPS 10bit with 12bit LUT (Dell U3011)

[leeperry]

It's not dithering at all, it's grain added on purpose via the GrainFactory4() Avisynth script(I'm a movie grain whore :p). But you can see that 10bit untouched mVR processing and SmoothL+8bit YUY2 LUT in ffdshow+mVR dithering look identical, which was my original point...so to me that still leaves the ffdshow LUT option very much open.

[Wile-E-Coyote]

Going to go ahead and release version 0.5 as-is because the software is still alpha either way and I don't want to delay the current release/feedback cycle too much.
Might still be some bugs, I haven't extensively tested every single tool after the code re-write. Everything should be a lot, lot cleaner though.

In version 0.6 I am going to be adding multi-threaded .3dlut processing to speed up execution time on multi-core processors.

As usual, see the first post for a changelog: http://www.mediafire.com/?wwbuu5vhau5x6jm

I'm still trying to think of a new name for the project.



Works fine for me in version 0.5, try it.
I did. Same error.

[nand chan]

I agree, third and first look pretty much identical.
The 4th shows more banding then the others.

Also H-IPS 10bit with 12bit LUT (Dell U3011)

Dell U2410 here :rolleyes:

I've finished writing the multi-threading code now, applycal.exe completes /calculation/ in about 0.6 seconds. Saving/loading the .3dlut still takes about 1 second overall.

CPU usage spikes nice and high at 100%. :)

I did. Same error.

I can't reproduce it. It works fine here, even using the GUI. Are you sure you're using the latest version?

[Wile-E-Coyote]

I can't reproduce it. It works fine here, even using the GUI. Are you sure you're using the latest version?
Yes, .NET framework is also up to date. I'll guess I have no choice but sit through another 2 hours of calibration to try again.

[nand chan]

Yes, .NET framework is also up to date. I'll guess I have no choice but sit through another 2 hours of calibration to try again.

Does running
genscript.exe -i profil.ti3 -o -
work?

It works here and it should output the following: http://pastebin.com/pmei93qL

You can go ahead and copy/paste that into a .txt file, run it using yCMS and it should give you a working .3dlut.
(You already have a copy of yCMS in %AppData%\TI3Parser\yCMS if you've been using this program)

It's not dithering at all, it's grain added on purpose via the GrainFactory4() Avisynth script(I'm a movie grain whore :p). But you can see that 10bit untouched mVR processing and SmoothL+8bit YUY2 LUT in ffdshow+mVR dithering look identical, which was my original point...so to me that still leaves the ffdshow LUT option very much open.

Fair enough. What is it exactly that you want to do? Apply calibration information from a .cal file to a YCbCr LUT?

If so, it should be a simple process of going through the YCbCr LUT, converting each entry to RGB, running the RGB through the Calibration LUT, then converting the result back into YCbCr.

I can have a program for this running by 0.6, but I can't test it myself. You'll have to check for bugs yourself.

[Wile-E-Coyote]

Does running
genscript.exe -i profil.ti3 -o -
work?

It works here and it should output the following: http://pastebin.com/pmei93qL

You can go ahead and copy/paste that into a .txt file, run it using yCMS and it should give you a working .3dlut.
(You already have a copy of yCMS in %AppData%\TI3Parser\yCMS if you've been using this program)

No running that command line gives me the same error message. I'll try your advice later. Thanks.

EDIT : yCMS gives me an error message as well
Error! Invalid Command: '109.8925'.

[nand chan]

No running that command line gives me the same error message. I'll try your advice later. Thanks.

EDIT : yCMS gives me an error message as well

Very odd, I get:
The file '..\test.3dlut' was successfully created.
Running time: 4449ms; Computation time: 1765ms (39.7%)

When you're saving the file do you have some sort of word wrap enabled maybe? Also what operating system are you on? Maybe it's detecting the line breaks incorrectly, which would explain both issues.

@leeperry: Can you run inspect3dlut on a YCbCr .3dlut and upload the output somewhere? (Or simply tell me what it reports for the two color encoding fields)

[Wile-E-Coyote]

Very odd, I get:
The file '..\test.3dlut' was successfully created.
Running time: 4449ms; Computation time: 1765ms (39.7%)

When you're saving the file do you have some sort of word wrap enabled maybe? Also what operating system are you on? Maybe it's detecting the line breaks incorrectly, which would explain both issues.
Word wrap? Not that I know of. I pasted the text on notepad.

I'm running Win 7 Ultimate x64

[nand chan]

Word wrap? Not that I know of. I pasted the text on notepad.

I'm running Win 7 Ultimate x64

See “Format -> Word Wrap”.

Regardless, I don't know what else could be causing the problems.

[Wile-E-Coyote]

See “Format -> Word Wrap”.

Regardless, I don't know what else could be causing the problems.
Word wrap is deactivated, but I decided to open the file in Notepad++ and re-save it, and it worked, at least for yCMS.

I'm trying to use that method for the 3ti files but without success so far.

[alph@]

Hello.

I just successfully calibrated my TV with dispcalGUI and wanted to create a 3Dlut file with it but when I open my ti3 file I get an error message saying the input string format is incorrect.
here too, same error message?i have the framwork 4,it's the last version of ti3 Parser
upload
http://www.mediafire.com/?rr2648w3ow9ftb4

[nand chan]

here too, same error message?i have the framwork 4,it's the last version of ti3 Parser
upload
http://www.mediafire.com/?rr2648w3ow9ftb4

I think I know what the issue may be, the culture information used to parse the values might be set to use a comma as decimal indicator (instead of a period) for your computer's locale.

I'll fix it in 0.6 by setting the culture used to parse to invariant.

Edit: Here's a preview release, see if that fixes it. http://www.mediafire.com/?04i9rct4be5q7pn

[Wile-E-Coyote]

I think I know what the issue may be, the culture information used to parse the values might be set to use a comma as decimal indicator (instead of a period) for your computer's locale.

I'll fix it in 0.6 by setting the culture used to parse to invariant.

Edit: Here's a preview release, see if that fixes it. http://www.mediafire.com/?04i9rct4be5q7pn

That did the trick, works fine for me now. Thanks a lot! :)

[leeperry]

Fair enough. What is it exactly that you want to do? Apply calibration information from a .cal file to a YCbCr LUT?

If so, it should be a simple process of going through the YCbCr LUT, converting each entry to RGB, running the RGB through the Calibration LUT, then converting the result back into YCbCr.

I can have a program for this running by 0.6, but I can't test it myself. You'll have to check for bugs yourself.
Glad to see that we agree, and I can certainly confirm that SmoothL() provides the most stunning TV>PC conversion I have ever seen...it's like the picture becoming even "deeper" subjectively speaking, and it kills two birds w/ one stone providing 10bit'ish performance within 8bit pipelines too.

Well, atm I use an 8bit VGA connection w/ a 10bit CLUT...so all is well, but soon or later I'll be upgrading to a 40" LCD from SONY/LG(I want at least 4K:1 native CR) so it'll be about a 8bit CLUT on top of a 8bit 3DLUT AFAIK...so ideally I would like Argyll's calibration to be embedded within the 3DLUT.

To avoid all those YCbCr>RGB>YCbCr lossy conversions, ideally I would like to input a 16bit RGB gamut mapping LUT from cr3dlut, merge it w/ a .cal file and output a BT.601/709 8bit YUY2 LUT to use w/ rgb3dlut() in ffdshow.

But this would very much require a "reset CLUT while playing" option in mVR, otherwise it'll end up useless....and maybe the flat screen I'll be buying will have serious 10/12bit calibration options so I won't have to use a .cal kludge anymore :o

So as I said earlier, it's all sheer brainstorming at this point from my side...still, I know some ppl already requested the ability to merge gamut mapping and .cal files together in order to avoid having the CLUT messing around w/ mVR's dithering, but I dunno how much of a PQ difference it would make. And at the end of the day, adding three 1D CLUT's on top of a 3DLUT is nothing more than a kludge at the price of a dispensable quality loss.

Can you run inspect3dlut on a YCbCr .3dlut and upload the output somewhere? (Or simply tell me what it reports for the two color encoding fields)
Sorry for lagging behind in the testing department, I will install a W7SP1 VM on my XPSP3 coz updating .net sounds like headaches.

Here's a 0-255 8bit YUY2 LUT from my CRT gamut to SMPTE-C: BT709_SMPTE-C.rar (http://www.mediafire.com/?ana45biwywbhej4)

[nand chan]

Well, atm I use an 8bit VGA connection w/ a 10bit CLUT...so all is well, but soon or later I'll be upgrading to a 40" LCD from SONY/LG(I want at least 4K:1 native CR) so it'll be about a 8bit CLUT on top of a 8bit 3DLUT AFAIK...so ideally I would like Argyll's calibration to be embedded within the 3DLUT.

Right, this is currently well-possible.

To avoid all those YCbCr>RGB>YCbCr lossy conversions, ideally I would like to input a 16bit RGB gamut mapping LUT from cr3dlut, merge it w/ a .cal file and output a BT.601/709 8bit YUY2 LUT to use w/ rgb3dlut() in ffdshow.

Except for the YUY2 part this is possible already: Create a 16-bit gamut mapping LUT from BT.601 to BT.709 using cr3dlut, merge it with a .cal file using applycal, then change the bit-depth down to 8-bits with changedepth.

Then you'd just have to use that 8-bit RGB lut in ffdshow. It won't be YUY2 though, the problem here would be that .cal doesn't work in YUY2 space - it works in RGB space, so you can't just “apply” a .cal to a YUY2 lut without converting all of the values back and forth. (These won't necessarily be lossy because conversions are performed with 64-bit precision)

But this would very much require a "reset CLUT while playing" option in mVR, otherwise it'll end up useless....and maybe the flat screen I'll be buying will have serious 10/12bit calibration options so I won't have to use a .cal kludge anymore :o

What you can do is set up a .bat script to run dispclut -d 1 --save file.txt --reset, run MPC-HC, then run dispclut -d 1 --load file.txt once it closes. That way, whenever you start MPC-HC, the LUTs should automatically be reset.

So as I said earlier, it's all sheer brainstorming at this point from my side...still, I know some ppl already requested the ability to merge gamut mapping and .cal files together in order to avoid having the CLUT messing around w/ mVR's dithering, but I dunno how much of a PQ difference it would make. And at the end of day, adding three 1D CLUT's on top of a 3DLUT is nothing more than a kludge at the price of a dispensable quality loss.

Mapping three 1D CLUTs onto a 16-bit 3DLUT using my methods should *always* be higher quality than loading them to the video card, because my calculations properly interpolate the 1d CLUTs to 16-bit space (they are normally 8 bit -> 16 bit on the display device and my program calculates them as 16 bit -> 16 bit). So that gives madVR 16 bits of information to work with after calibration, and it can dither this down properly.

It's certainly an advantage, but you are right, the PQ difference in real world scenarios is debatable.


Here's a 0-255 8bit YUY2 LUT from my CRT gamut to SMPTE-C: BT709_SMPTE-C.rar (http://www.mediafire.com/?ana45biwywbhej4)

I'll have a look, thanks

Edit: Alright, seems like cr3dlut ignores the Color Encoding field as well and just scans through the parameters file to figure out which value type to use.

You can take that YCbCr -> RGB .3dlut as-is and just apply a .cal onto it, should already be fully supported* because the .cal process only modifies the output value and not the input value. It's still a good reference file though, so I know what to scan for to detect ranges etc.

* Since my scripts only work with yCMS parameters at the moment, you'll have to tag your .3dluts as full range using tag3dlut --full -i BT709_SMPTE-C.3dlut, otherwise it will assume limited range and mess up the colors.

But after doing that, applycal -i BT709_SMPTE-C.3dlut your_cal_file.cal should work fine.

[tschi]

Thank you very much for your software :thanks:
it was so boring to do calibration and after take measure with colorhcfr and after export to Excel and after make data for madvr.
Thank You ! :)

edit : I use your pre 0.6 version because of decimal issue. If i do copy & paste of the result, the all the . are convert to , and I can not paste correctly into madVR (can not click on apply in madVR) Anyway, I was able to generate directly a 3dlut with ti3parser and ycms and use it in madVR

[nand chan]

Thank you very much for your software :thanks:
it was so boring to do calibration and after take measure with colorhcfr and after export to Excel and after make data for madvr.
Thank You ! :)

edit : I use your pre 0.6 version because of decimal issue. If i do copy & paste of the result, the all the . are convert to , and I can not paste correctly into madVR (can not click on apply in madVR) Anyway, I was able to generate directly a 3dlut with ti3parser and ycms and use it in madVR

Thanks for mentioning this, I have to apply the correct culture information when /outputting/ numbers as well. >.<

I think I'll just set a global culture information for the entire application, reduce the amount of hassle.

[nand chan]

I think I'll just set a global culture information for the entire application, reduce the amount of hassle.

Can you tell me whether this version works or not?

http://www.mediafire.com/?ct1xzd6h492d6e3

[leeperry]

you'd just have to use that 8-bit RGB lut in ffdshow. It won't be YUY2 though, the problem here would be that .cal doesn't work in YUY2 space - it works in RGB space, so you can't just “apply” a .cal to a YUY2 lut without converting all of the values back and forth.
The whole point of going YUY2 is that the YCbCr>RGB32 conversion through a LUT hardly requires any CPU power, and it can be MT'ed. It also allows the use of yv12toyuy2() which has a lot of nifty features(like choosing the chroma interpolating algorithm). Using RGB LUT's in ffdshow is far from ideal, and same goes for 16bit LUT's because t3dlut() is required and it's far more CPU hungry than rgb3dlut(). They also double the loading time over 8bit, even off a ramdisk.

What you can do is set up a .bat script to run dispcal -d 1 --save file.txt --reset, run MPC-HC, then run dispcal -d 1 --load file.txt once it closes. That way, whenever you start MPC-HC, the LUTs should automatically be reset.
I'm really looking for a kludge-free HTPC at this point, I've got all my gamut mapping settings within automatic ffdshow profiles, that's how cool it's gotten :D

Hopefuly, madshi is reading this: an option to bypass the graphic card's CLUT when mVR is playing would be amazing :cool:

You can take that YCbCr -> RGB .3dlut as-is and just apply a .cal onto it, should already be fully supported* because the .cal process only modifies the output value and not the input value. [..] applycal -i BT709_SMPTE-C.3dlut your_cal_file.cal should work fine.

Alright, sounds like a plan! Whenever I'll find the courage to install a W7SP1 VM, I'll get back to you. :thanks:

[nand chan]

Also keep in mind that some tools* may not work entirely correctly yet because I haven't written code for separately detecting the input depth and output depth (I'll add that by 0.7), so right now if the output range changes then the entire range changes.

I'll write code sooner or later to pulldown the input range and output range independently of each-other and detect both ranges correctly.

*This should probably just affect changedepth -r and imagecal -3

Alright, sounds like a plan! Whenever I'll find the courage to install a W7SP1 VM, I'll get back to you. :thanks:

Is it that hard to install .NET 4.0 on XP? If it's a huge problem I may be able to find some way around it, but the way I'm generating lambda functions on the fly* right now requires 4.0.

* Since .3dLUTs can be so different (bytes, ushorts, floats, PC levels, TV levels, color encoding etc) I generate the correct function for an operation during runtime - or at least, I used to, I could probably reduce these lambdas to static delegates now that I'm just about finished outlying all logic to the unified 64-bit floating point pipeline.

What I'll probably doing by version 0.7 or 0.8 is abstracting all transformation operations behind an ITransformationProvider interface and compositing these objects together on the fly and packaging them into a TransformationFactory. That way you will be able to serialize a bunch of different composite operations into a loadable file and pass them around, or store them inside a .3dlut's meta-tags, allowing for rapid prototyping, modification and large-scale distribution of environment-tailored 3dLUTs through a common interface.

Then I could probably create a drag and drop UI front-end for it that allows you to visualize composite transformations and preview them on the fly.

But for now, the delegates shall suffice. :rolleyes:

[leeperry]

It all sounds very appetizing, even though most of it flies way above my head...but I like the GUI part :p

I already tried to update .net from 2.0 to 4.0, it started downloading zillion updates, deleted all the files I had at the root of my C: drive, ended up claiming more than 1GB of hard drive space, and then whined right in the middle that some components couldn't be updated...duh. .net is great for coders, but it's a whole bunch of bloat as far as I can see.

[nand chan]

It all sounds very appetizing, even though most of it flies way above my head...but I like the GUI part :p

I already tried to update .net from 2.0 to 4.0, it started downloading zillion updates, deleted all the files I had at the root of my C: drive, ended up claiming more than 1GB of hard drive space, and then whined right in the middle that some components couldn't be updated...duh. .net is great for coders, but it's a whole bunch of bloat as far as I can see.

Well, .NET 3.5 comes pre-installed on Windows 7 afaik and updating to 4.0 on Win7 should work flawlessly.

Updating to 3.5 on Windows XP also worked fine back when I was using it.

Note: .NET is also great for end users because it allows processor-specific optimization on the target platform. It's the reason why I can distribute a single managed executable that will assemble down to the correct architecture (x86, AMD64 / Intel 64, SSE etc.) when you run it for the first time.

Here are some prerequisites:


.NET 4.0 requires XP SP3, Win2k3 SP2, Vista, 7, or 2008(R2)
.NET 3.5 requires XP SP2 or newer.
.NET 2.0 requires Win2K SP(3?) or newer.

[cyberbeing]

Quick comment on dispclut.exe. Using --reset (Windows-default 8-bit CLUT?) doesn't do the desired behavior, since most calibration applications use a 16-bit linear CLUT.

--reset results in:
#GammaRamp
0 0 0 0
1 256 256 256
2 512 512 512
3 768 768 768
4 1024 1024 1024
5 1280 1280 1280
6 1536 1536 1536
7 1792 1792 1792
8 2048 2048 2048
9 2304 2304 2304
10 2560 2560 2560
11 2816 2816 2816
12 3072 3072 3072
13 3328 3328 3328
14 3584 3584 3584
15 3840 3840 3840
16 4096 4096 4096
17 4352 4352 4352
18 4608 4608 4608
19 4864 4864 4864
20 5120 5120 5120
21 5376 5376 5376
22 5632 5632 5632
23 5888 5888 5888
24 6144 6144 6144
25 6400 6400 6400
26 6656 6656 6656
27 6912 6912 6912
28 7168 7168 7168
29 7424 7424 7424
30 7680 7680 7680
31 7936 7936 7936
32 8192 8192 8192
33 8448 8448 8448
34 8704 8704 8704
35 8960 8960 8960
36 9216 9216 9216
37 9472 9472 9472
38 9728 9728 9728
39 9984 9984 9984
40 10240 10240 10240
41 10496 10496 10496
42 10752 10752 10752
43 11008 11008 11008
44 11264 11264 11264
45 11520 11520 11520
46 11776 11776 11776
47 12032 12032 12032
48 12288 12288 12288
49 12544 12544 12544
50 12800 12800 12800
51 13056 13056 13056
52 13312 13312 13312
53 13568 13568 13568
54 13824 13824 13824
55 14080 14080 14080
56 14336 14336 14336
57 14592 14592 14592
58 14848 14848 14848
59 15104 15104 15104
60 15360 15360 15360
61 15616 15616 15616
62 15872 15872 15872
63 16128 16128 16128
64 16384 16384 16384
65 16640 16640 16640
66 16896 16896 16896
67 17152 17152 17152
68 17408 17408 17408
69 17664 17664 17664
70 17920 17920 17920
71 18176 18176 18176
72 18432 18432 18432
73 18688 18688 18688
74 18944 18944 18944
75 19200 19200 19200
76 19456 19456 19456
77 19712 19712 19712
78 19968 19968 19968
79 20224 20224 20224
80 20480 20480 20480
81 20736 20736 20736
82 20992 20992 20992
83 21248 21248 21248
84 21504 21504 21504
85 21760 21760 21760
86 22016 22016 22016
87 22272 22272 22272
88 22528 22528 22528
89 22784 22784 22784
90 23040 23040 23040
91 23296 23296 23296
92 23552 23552 23552
93 23808 23808 23808
94 24064 24064 24064
95 24320 24320 24320
96 24576 24576 24576
97 24832 24832 24832
98 25088 25088 25088
99 25344 25344 25344
100 25600 25600 25600
101 25856 25856 25856
102 26112 26112 26112
103 26368 26368 26368
104 26624 26624 26624
105 26880 26880 26880
106 27136 27136 27136
107 27392 27392 27392
108 27648 27648 27648
109 27904 27904 27904
110 28160 28160 28160
111 28416 28416 28416
112 28672 28672 28672
113 28928 28928 28928
114 29184 29184 29184
115 29440 29440 29440
116 29696 29696 29696
117 29952 29952 29952
118 30208 30208 30208
119 30464 30464 30464
120 30720 30720 30720
121 30976 30976 30976
122 31232 31232 31232
123 31488 31488 31488
124 31744 31744 31744
125 32000 32000 32000
126 32256 32256 32256
127 32512 32512 32512
128 32768 32768 32768
129 33024 33024 33024
130 33280 33280 33280
131 33536 33536 33536
132 33792 33792 33792
133 34048 34048 34048
134 34304 34304 34304
135 34560 34560 34560
136 34816 34816 34816
137 35072 35072 35072
138 35328 35328 35328
139 35584 35584 35584
140 35840 35840 35840
141 36096 36096 36096
142 36352 36352 36352
143 36608 36608 36608
144 36864 36864 36864
145 37120 37120 37120
146 37376 37376 37376
147 37632 37632 37632
148 37888 37888 37888
149 38144 38144 38144
150 38400 38400 38400
151 38656 38656 38656
152 38912 38912 38912
153 39168 39168 39168
154 39424 39424 39424
155 39680 39680 39680
156 39936 39936 39936
157 40192 40192 40192
158 40448 40448 40448
159 40704 40704 40704
160 40960 40960 40960
161 41216 41216 41216
162 41472 41472 41472
163 41728 41728 41728
164 41984 41984 41984
165 42240 42240 42240
166 42496 42496 42496
167 42752 42752 42752
168 43008 43008 43008
169 43264 43264 43264
170 43520 43520 43520
171 43776 43776 43776
172 44032 44032 44032
173 44288 44288 44288
174 44544 44544 44544
175 44800 44800 44800
176 45056 45056 45056
177 45312 45312 45312
178 45568 45568 45568
179 45824 45824 45824
180 46080 46080 46080
181 46336 46336 46336
182 46592 46592 46592
183 46848 46848 46848
184 47104 47104 47104
185 47360 47360 47360
186 47616 47616 47616
187 47872 47872 47872
188 48128 48128 48128
189 48384 48384 48384
190 48640 48640 48640
191 48896 48896 48896
192 49152 49152 49152
193 49408 49408 49408
194 49664 49664 49664
195 49920 49920 49920
196 50176 50176 50176
197 50432 50432 50432
198 50688 50688 50688
199 50944 50944 50944
200 51200 51200 51200
201 51456 51456 51456
202 51712 51712 51712
203 51968 51968 51968
204 52224 52224 52224
205 52480 52480 52480
206 52736 52736 52736
207 52992 52992 52992
208 53248 53248 53248
209 53504 53504 53504
210 53760 53760 53760
211 54016 54016 54016
212 54272 54272 54272
213 54528 54528 54528
214 54784 54784 54784
215 55040 55040 55040
216 55296 55296 55296
217 55552 55552 55552
218 55808 55808 55808
219 56064 56064 56064
220 56320 56320 56320
221 56576 56576 56576
222 56832 56832 56832
223 57088 57088 57088
224 57344 57344 57344
225 57600 57600 57600
226 57856 57856 57856
227 58112 58112 58112
228 58368 58368 58368
229 58624 58624 58624
230 58880 58880 58880
231 59136 59136 59136
232 59392 59392 59392
233 59648 59648 59648
234 59904 59904 59904
235 60160 60160 60160
236 60416 60416 60416
237 60672 60672 60672
238 60928 60928 60928
239 61184 61184 61184
240 61440 61440 61440
241 61696 61696 61696
242 61952 61952 61952
243 62208 62208 62208
244 62464 62464 62464
245 62720 62720 62720
246 62976 62976 62976
247 63232 63232 63232
248 63488 63488 63488
249 63744 63744 63744
250 64000 64000 64000
251 64256 64256 64256
252 64512 64512 64512
253 64768 64768 64768
254 65024 65024 65024
255 65280 65280 65280

The per-calibration linear CLUT of ArgyllCMS, i1Profiler, basICColor, ColorEyes:
#GammaRamp
0 0 0 0
1 257 257 257
2 514 514 514
3 771 771 771
4 1028 1028 1028
5 1285 1285 1285
6 1542 1542 1542
7 1799 1799 1799
8 2056 2056 2056
9 2313 2313 2313
10 2570 2570 2570
11 2827 2827 2827
12 3084 3084 3084
13 3341 3341 3341
14 3598 3598 3598
15 3855 3855 3855
16 4112 4112 4112
17 4369 4369 4369
18 4626 4626 4626
19 4883 4883 4883
20 5140 5140 5140
21 5397 5397 5397
22 5654 5654 5654
23 5911 5911 5911
24 6168 6168 6168
25 6425 6425 6425
26 6682 6682 6682
27 6939 6939 6939
28 7196 7196 7196
29 7453 7453 7453
30 7710 7710 7710
31 7967 7967 7967
32 8224 8224 8224
33 8481 8481 8481
34 8738 8738 8738
35 8995 8995 8995
36 9252 9252 9252
37 9509 9509 9509
38 9766 9766 9766
39 10023 10023 10023
40 10280 10280 10280
41 10537 10537 10537
42 10794 10794 10794
43 11051 11051 11051
44 11308 11308 11308
45 11565 11565 11565
46 11822 11822 11822
47 12079 12079 12079
48 12336 12336 12336
49 12593 12593 12593
50 12850 12850 12850
51 13107 13107 13107
52 13364 13364 13364
53 13621 13621 13621
54 13878 13878 13878
55 14135 14135 14135
56 14392 14392 14392
57 14649 14649 14649
58 14906 14906 14906
59 15163 15163 15163
60 15420 15420 15420
61 15677 15677 15677
62 15934 15934 15934
63 16191 16191 16191
64 16448 16448 16448
65 16705 16705 16705
66 16962 16962 16962
67 17219 17219 17219
68 17476 17476 17476
69 17733 17733 17733
70 17990 17990 17990
71 18247 18247 18247
72 18504 18504 18504
73 18761 18761 18761
74 19018 19018 19018
75 19275 19275 19275
76 19532 19532 19532
77 19789 19789 19789
78 20046 20046 20046
79 20303 20303 20303
80 20560 20560 20560
81 20817 20817 20817
82 21074 21074 21074
83 21331 21331 21331
84 21588 21588 21588
85 21845 21845 21845
86 22102 22102 22102
87 22359 22359 22359
88 22616 22616 22616
89 22873 22873 22873
90 23130 23130 23130
91 23387 23387 23387
92 23644 23644 23644
93 23901 23901 23901
94 24158 24158 24158
95 24415 24415 24415
96 24672 24672 24672
97 24929 24929 24929
98 25186 25186 25186
99 25443 25443 25443
100 25700 25700 25700
101 25957 25957 25957
102 26214 26214 26214
103 26471 26471 26471
104 26728 26728 26728
105 26985 26985 26985
106 27242 27242 27242
107 27499 27499 27499
108 27756 27756 27756
109 28013 28013 28013
110 28270 28270 28270
111 28527 28527 28527
112 28784 28784 28784
113 29041 29041 29041
114 29298 29298 29298
115 29555 29555 29555
116 29812 29812 29812
117 30069 30069 30069
118 30326 30326 30326
119 30583 30583 30583
120 30840 30840 30840
121 31097 31097 31097
122 31354 31354 31354
123 31611 31611 31611
124 31868 31868 31868
125 32125 32125 32125
126 32382 32382 32382
127 32639 32639 32639
128 32896 32896 32896
129 33153 33153 33153
130 33410 33410 33410
131 33667 33667 33667
132 33924 33924 33924
133 34181 34181 34181
134 34438 34438 34438
135 34695 34695 34695
136 34952 34952 34952
137 35209 35209 35209
138 35466 35466 35466
139 35723 35723 35723
140 35980 35980 35980
141 36237 36237 36237
142 36494 36494 36494
143 36751 36751 36751
144 37008 37008 37008
145 37265 37265 37265
146 37522 37522 37522
147 37779 37779 37779
148 38036 38036 38036
149 38293 38293 38293
150 38550 38550 38550
151 38807 38807 38807
152 39064 39064 39064
153 39321 39321 39321
154 39578 39578 39578
155 39835 39835 39835
156 40092 40092 40092
157 40349 40349 40349
158 40606 40606 40606
159 40863 40863 40863
160 41120 41120 41120
161 41377 41377 41377
162 41634 41634 41634
163 41891 41891 41891
164 42148 42148 42148
165 42405 42405 42405
166 42662 42662 42662
167 42919 42919 42919
168 43176 43176 43176
169 43433 43433 43433
170 43690 43690 43690
171 43947 43947 43947
172 44204 44204 44204
173 44461 44461 44461
174 44718 44718 44718
175 44975 44975 44975
176 45232 45232 45232
177 45489 45489 45489
178 45746 45746 45746
179 46003 46003 46003
180 46260 46260 46260
181 46517 46517 46517
182 46774 46774 46774
183 47031 47031 47031
184 47288 47288 47288
185 47545 47545 47545
186 47802 47802 47802
187 48059 48059 48059
188 48316 48316 48316
189 48573 48573 48573
190 48830 48830 48830
191 49087 49087 49087
192 49344 49344 49344
193 49601 49601 49601
194 49858 49858 49858
195 50115 50115 50115
196 50372 50372 50372
197 50629 50629 50629
198 50886 50886 50886
199 51143 51143 51143
200 51400 51400 51400
201 51657 51657 51657
202 51914 51914 51914
203 52171 52171 52171
204 52428 52428 52428
205 52685 52685 52685
206 52942 52942 52942
207 53199 53199 53199
208 53456 53456 53456
209 53713 53713 53713
210 53970 53970 53970
211 54227 54227 54227
212 54484 54484 54484
213 54741 54741 54741
214 54998 54998 54998
215 55255 55255 55255
216 55512 55512 55512
217 55769 55769 55769
218 56026 56026 56026
219 56283 56283 56283
220 56540 56540 56540
221 56797 56797 56797
222 57054 57054 57054
223 57311 57311 57311
224 57568 57568 57568
225 57825 57825 57825
226 58082 58082 58082
227 58339 58339 58339
228 58596 58596 58596
229 58853 58853 58853
230 59110 59110 59110
231 59367 59367 59367
232 59624 59624 59624
233 59881 59881 59881
234 60138 60138 60138
235 60395 60395 60395
236 60652 60652 60652
237 60909 60909 60909
238 61166 61166 61166
239 61423 61423 61423
240 61680 61680 61680
241 61937 61937 61937
242 62194 62194 62194
243 62451 62451 62451
244 62708 62708 62708
245 62965 62965 62965
246 63222 63222 63222
247 63479 63479 63479
248 63736 63736 63736
249 63993 63993 63993
250 64250 64250 64250
251 64507 64507 64507
252 64764 64764 64764
253 65021 65021 65021
254 65278 65278 65278
255 65535 65535 65535

My suggestion would be keep both, but just add a 'dispclut.exe --linear' command.

[nand chan]

Quick comment on dispclut.exe. Using --reset (Windows-default 8-bit CLUT?) doesn't do the desired behavior, since most calibration applications use a 16-bit linear CLUT.

No, I'm actually generating the 16-bit ramp you posted manually.

I don't know how Windows (or rather, the graphics card) translates a 16-bit ramp down to an 8-bit display device.

I'll see if I can find out whether 65535 or 65280 is used for maximum white (255 signal).

In a somewhat unrelated note, I've now provided a common interface for color transformations (ITransformationProvider) as well as a way to combine these together (TransformationFactory) and sequence them (PipelineTransformer).

This allows a developer to create and arrange parameterized transformation construct, then “compute” a single 64bit->64bit function using .GetTransformer(), that can be re-used to look up a bunch of values.

For example, if I want to create an 8 bit -> 16 bit grayscale 3dlut with attached calibration data, I can do this:

new TransformationFactory(
PulldownTransformer(8, Range.Full),
GrayscaleTransformer(Recommendation.BT709),
CalibrationTransformer(CalFile.FromFile("somecal.txt")),
PullupTransformer(16, Range.Full)
).GetTransformer();

Now I just have to create a GUI for it.

Edit: I'll just assume ArgyllCMS, i1Profiler, basICColor and ColorEyes know what they're doing when they mean linear = linear and use the same.

[cyberbeing]

Running 'dispclut.exe --reset' results in an 8-bit ramp similar to hitting the 'Reset' button in Calibration Tester, Windows built-in lut loader, or using RivaTuner to forcibly reload your GPU driver.

If you run 'dispwin.exe -c' from the ArgyllCMS package, it loads the 16-bit ramp (as seen with Calibration Tester). All other Calibration applications do the same, loading a 16-bit linear ramp pre-calibration.

Other other hand, if I run 'dispclut.exe --load' pointing to an exported 16-bit ramp, it does load the 16-bit ramp correctly.

dispclut.exe --reset = 8-bit ramp
dispclut.exe --load = 16-bit ramp

I would conclude that the 8-but ramp is the Windows default, and 16-bit ramps need to be specially loaded. What differs between dispclut.exe --reset and --load?

[nand chan]

Running 'dispclut.exe --reset' results in an 8-bit ramp similar to hitting the 'Reset' button in Calibration Tester, Windows built-in lut loader, or using RivaTuner to forcibly reload your GPU driver.

If you run 'dispwin.exe -c' from the ArgyllCMS package, it loads the 16-bit ramp (as seen with Calibration Tester). All other Calibration applications do the same, loading a 16-bit linear ramp pre-calibration.

Other other hand, if I run 'dispclut.exe --load' pointing to an exported 16-bit ramp, it does load the 16-bit ramp correctly.

dispclut.exe --reset = 8-bit ramp
dispclut.exe --load = 16-bit ramp

I would conclude that the 8-but ramp is the Windows default, and 16-bit ramps need to be specially loaded. What differs between dispclut.exe --reset and --load?

No, there's no such thing as an 8-bit ramp. Windows only provides the 16-bit interface. There's also no “ResetGammaRamp” function. To reset anything, you have to generate a 16-bit ramp and load it.

The cause for the difference in results you are seeing is two different generation methods:

The “8-bit” ramp:
output = input << 8
Results in eg. 1010 1010 0000 0000

The “linear” ramp:
output = (input << 8) + input
Results in eg. 1010 1010 1010 1010

In both cases, a fresh gamma ramp is being created and then loaded. There's no such thing as a “windows default” gamma ramp.

[cyberbeing]

Good to know, but could explain it in a non-technical way?
It's always been a mystery to me why the "8-bit" ramp happens on Windows, yet lut loaders somehow are able to bypass this and load a 16-bit CLUT.

Why was --reset originally doing output = input << 8?
16-bit input=output CLUT loaded, but Windows (?) shifts it to 8-bit?

Why was --load doing output = (input << 8) + input
16-bit input=output CLUT loaded, Windows (?) shifts it to 8-bit, then a 16-bit input!=output linear ramp is loaded yet it sticks? I don't understand the + input part.

[nand chan]

Good to know, but could explain it in a non-technical way?
It's always been a mystery to me why the "8-bit" ramp happens on Windows, yet lut loaders somehow are able to bypass this and load a 16-bit CLUT.

Why was --reset originally doing output = input << 8?
16-bit input=output CLUT loaded, but Windows (?) shifts it to 8-bit?

Why was --load doing output = (input << 8) + input
16-bit input=output CLUT loaded, Windows (?) shifts it to 8-bit, then a 16-bit input!=output linear ramp is loaded yet it sticks? I don't understand the + input part.

Once again, there's no such thing as an 8-bit LUT. Windows doesn't shift anything.

It was previously “input << 8” because that's what I previously wrote in my ResetGammaRamp() code.

Some tools simply use the different logic. They're both loaded the same way (as 16 bit), and they stay that way once loaded.

Note: This is not entirely true, Windows does clamp the LUTs a bit to make sure they're “proper” gamma ramps. You can't do things like inverting the colors or drastically reducing one channel because it will fail windows' sanity check. But other than that, Windows should not modify it (or especially the bit-depth).

-- load was doing nothing, it was simply loading the information from a file.

[cyberbeing]

If I'm understanding you correctly, --reset was loading the exact 16-bit ramp CLUT you had it coded to use:
#GammaRamp
0 0 0 0
1 256 256 256
2 512 512 512
3 768 768 768
4 1024 1024 1024
...

Using --load resulted in a different CLUT, because the 16-bit linear ramp I was trying to load actually was a different CLUT than what -reset was using?

It makes me wonder if
#GammaRamp
0 0 0 0
1 256 256 256
2 512 512 512
3 768 768 768
4 1024 1024 1024
...
is more optimal than
#GammaRamp
0 0 0 0
1 257 257 257
2 514 514 514
3 771 771 771
4 1028 1028 1028
...
if your GPU only does CLUT processing in 8-bit, or if they would both ultimately result in the same thing.

[nand chan]

If I'm understanding you correctly, --reset was loading the exact 16-bit ramp CLUT you had it coded to use:
#GammaRamp
0 0 0 0
1 256 256 256
2 512 512 512
3 768 768 768
4 1024 1024 1024
...

Using --load resulted in a different CLUT, because the 16-bit linear ramp I was trying to load actually was a different CLUT than what -reset was using?

It makes me wonder if
#GammaRamp
0 0 0 0
1 256 256 256
2 512 512 512
3 768 768 768
4 1024 1024 1024
...
is more optimal than
#GammaRamp
0 0 0 0
1 257 257 257
2 514 514 514
3 771 771 771
4 1028 1028 1028
...
if your GPU only does CLUT processing in 8-bit, or if they would both ultimately result in the same thing.

Yes, that is correct. About your question, I don't know - I'm using the linear ramps for now, I figure they'll be correct.

There's probably no difference between the two because I'm guessing that windows just right shifts the values 8 places when on an 8-bit display device.

It will probably only make a difference for 10-bit and higher devices. Can you detect any visible change when switching between the two?

[tschi]

Can you tell me whether this version works or not?

http://www.mediafire.com/?ct1xzd6h492d6e3
it's ok :thanks:

[cyberbeing]

Can you detect any visible change when switching between the two?

Hard to say, but interestingly using the 'reset' one, dispcalGUI reports it as only 99.6% of grayscale values. For 0-128 output=input. For 129-255 output=input-1. Using the 'linear' shows 100% of grayscale input=output 0-255. For all we know, that could just be a bug in dispcalGUI though.

I'm thinking the 'reset' style ramp one must be a legacy thing or misinterpretation, since looking back, that is what older calibration programs like ProfileMaker 5 use.

Maximum value for each color is 65280 instead of 65535. Why? Maximum number of 8-bit number is 0xFF (255). Maximum number of gradations for each channel is 256.

255 * 256 = 65280
A Google search came up with the above by a forum poster, but I'm not sure that is correct logic. 8-bit values range from 0-255 (256) which matches up with the 0-255 (256) gradations. Maybe there was a bug in old GPU hardware or drivers which made 8-bit 0x0 null and mapped 0x1 to 0 or was it just a case of poor documentation by Microsoft confusing people? Not something I think you need to worry about now, since there seems to have been a transition at some point to using 65535 as the maximum value in modern calibration software.

[alph@]

version 0,6 work , no error message,the function 'save as 3dlut ' is not available,only 'save ycms script',it's normal?
http://www.zimagez.com/miniature/sanstitref38ddb19fd2dfab78bb276f842638899.png (http://www.zimagez.com/zimage/sanstitref38ddb19fd2dfab78bb276f842638899.php)

[nand chan]

Hard to say, but interestingly using the 'reset' one, dispcalGUI reports it as only 99.6% of grayscale values. For 0-128 output=input. For 129-255 output=input-1. Using the 'linear' shows 100% of grayscale input=output 0-255. For all we know, that could just be a bug in dispcalGUI though.

I'm thinking the 'reset' style ramp one must be a legacy thing or misinterpretation, since looking back, that is what older calibration programs like ProfileMaker 5 use.


A Google search came up with the above by a forum poster, but I'm not sure that is correct logic. 8-bit values range from 0-255 (256) which matches up with the 0-255 (256) gradations. Maybe there was a bug in old GPU hardware or drivers which made 8-bit 0x0 null and mapped 0x1 to 0 or was it just a case of poor documentation by Microsoft confusing people? Not something I think you need to worry about now, since there seems to have been a transition at some point to using 65535 as the maximum value in modern calibration software.

I'm just going to follow along with it for now, I have higher things on my priority list (such as getting version 0.6 out) than breaking my head over an imperceptible difference.

version 0,6 work , no error message,the function 'save as 3dlut ' is not available,only 'save ycms script',it's normal?
http://www.zimagez.com/miniature/sanstitref38ddb19fd2dfab78bb276f842638899.png (http://www.zimagez.com/zimage/sanstitref38ddb19fd2dfab78bb276f842638899.php)

Tried Update -> Update yCMS? If that doesn't work, it may be a restriction with Windows' security policy. I'll see if I can find a way around that.

[alph@]

ok work,i have updated ycms and launch the 'TI3ParserGUI.exe' with administrator Rights,you are correct:),this week end I start the test with my sony lcd 46 hx 700 and the combo xrite -colorhcfr,thank very match.

[nand chan]

Going to release version 0.6 as-is for now, so you don't get caught up in the delays of a bunch of behind the scenes changes. I will be making greater use of the improved pipeline in 0.7.

Download link for 0.6: http://www.mediafire.com/?112ct1ijm2e2edz

[nand chan]

http://img13.imageshack.us/img13/3766/guir.png

Experimental mock-up design of my proposed 3dLUT Creation Studio.

What do you think? I could also allow stuff like YCbCr -> RGB conversion blocks, or gamut mapping blocks. Then we wouldn't even need yCMS anymore.

Or you could do fun stuff like this:

http://img850.imageshack.us/img850/4587/chanswap.png

[yesgrey]

Which means that I am now being bottlenecked by the limited capacity of the .3dlut file standard, even though my code is perfectly capable of processing something irregular like 10-bit LUTs (as opposed to 8, 16 or 32).
That "limited" capacity you referred was intentional.

The benefit of allowing bit depths other than those would be only a lower 3DLUT file size. We think that a 96 MB (8/16 3DLUT) is a pretty manageable size according to current computer standards, so we favoured a simpler file format structure. In addition to that, I performed some precision tests that showed which input/output bit depth combinations would be useful. For an 8 bit input bit depth using more than 16 bit output is not worth it. The 32 FP was added for future use, when/if we decide to use 9/32 3DLUT files (1.5 GB), but before that we need to have graphic cards with a minimum of 2 GB ram to contain the 3DLUT... ;)

Furthermore, 32 bit FP was preferred over 32 bit integer because the precision is almost the same but 32 FP is much more compressible than 32 bit integer.

[nand chan]

That "limited" capacity you referred was intentional.

The benefit of allowing bit depths other than those would be only a lower 3DLUT file size. We think that a 96 MB (8/16 3DLUT) is a pretty manageable size according to current computer standards, so we favoured a simpler file format structure. In addition to that, I performed some precision tests that showed which input/output bit depth combinations would be useful. For an 8 bit input bit depth using more than 16 bit output is not worth it. The 32 FP was added for future use, when/if we decide to use 9/32 3DLUT files (1.5 GB), but before that we need to have graphic cards with a minimum of 2 GB ram to contain the 3DLUT... ;)

Furthermore, 32 bit FP was preferred over 32 bit integer because the precision is almost the same but 32 FP is much more compressible than 32 bit integer.

Yeah, it's reasonable enough. Something like a 8->10 bit LUT would be horrendously hard to work with if you just write 10 bits into the file at a time, and if you pad the data to 16 bit to make reading easier, might as well just use 16 bit precision.

That being said, the .3dlut file format has some other, ridiculously major flaws imo, because it's lacking a lot of meta-tags.

While these tags are not strictly speaking necessary for the operation of a LUT as such, many tools *do* want to be more content-aware (eg. is the input or output range 0-255 or 16-235?), a prime example being madVR itself - which scans through the parameters file to find out information.

Of course, this breaks as soon as it encounters a .3dlut not created by any known tool, even if the LUT itself would be in the correct format, simply because it isn't searching for the correct strings.

The result is that I have to pad my parameters data with “madVR compatibility hacks”.

Inclusion of this information, as well as stuff like gamut, gamma, and white point would be *critical* to a fully functional 3dLUT format.

Ps. Floating point can have another advantage: Since value ranges are normalized from 0 to 1, that makes gamut mapping independent of the level range. madVR arguably already works this way because it only works with limited range /output/ LUTs (and transforms these further), but it's a bit silly regardless.

Floating point has another advantage of that you're strictly limited to 0-1, you can introduce whiter than white / blacker than black information.

Edit: Also, what about 9->16 bit LUTs for eg. Hi10P content? Though I suppose it doesn't matter because madVR doesn't dither down to 8 bit precision, it interpolates the .3dlut results up to 10 bit instead.

I don't suppose it should make a significant difference.

[nand chan]

Concept of a proposed scripting language I was toying with, as an analog to the above flowchart GUI:

!Input_Range 16 235
!Input_Depth 8 8 8
!Output_Range 0 255
!Output_Depth 16

# Import the CalFile namespace, which gives us access to the CalFile() filter
!Import(Files.CalFile)

# Example .3dlut generation script
!Pixel {

# Since this is going to be a YCbCr script, first convert to RGB using the BT.709 primaries
ConvertToRGB(BT709)

# Now we want to adjust for our screen's somewhat darker gamma curve
Gamma(2.2,2.35)

# And apply the calibration info from an argyll file
CalFile("path_to_file.ti3")

# And just for fun, grayscale it
Grayscale()

}

Things to consider:

- Allow custom scripting logic or not? Eg. declaration of variables, creation of custom transform filters or just allow loading these via managed assemblies
- Maybe include the pixel pulldown / pullup logic into the !Pixel pipeline itself? The reason doing this might be bad though is it would disencourage creation of meta-data, which renderers such as madVR would rely on.

Interesting realization:

If I went through with this scripting language, it obviously wouldn't be limited to just .3dluts. I could, for example, save them as transformation pipeline packages (.tpp?) and apply these to images etc. manually.

I could also introduce filters such as ChangeGamut() or ChangeWhitepoint().

I've also put some thought into how to do multiplexing. In the most basic idea, single-pixel filters would be distinguished by some sort of extra markup in the name, for example a period or $ or _ or something.

# Single filters could be applied like this
Multiplex(BlockR, BlockG, BlockB)

# For example
Multiplex($Gamma(1.0), $Gamma(0.9), $Gamma(0.8))

# A null or noop operation could either be represented as “null”..
Multiplex($Gamma(0.5), null, null)
# ..left out..
Multiplex($Gamma(0.5), ,)
# or perhaps done with $Cat() or $Default() or something
Multiplex($Gamma(0.5), $Cat(), $Cat())

# Larger blocks of multiple filters would be joined by { }
Multiplex({ $Gamma(0.5) $Add(0.1) }, { $Gamma(0.8) $Add(0.2) }, $Cat())

# Which can be stretched over several lines
Multiplex({
$Gamma(0.5)
$Add(0.1)
}, $Cat(), $Cat())

# The basic idea here is that newlines are ignored
Gamma(
2.2,
2.35 )

Further thoughts: Maybe extend my scripting format to custom amounts of channels? For example, a 3 -> 4 channel convertor that would convert from RGB to CMYK.

This obviously wouldn't be saveable as a .3dlut due to the limitation of the file format, but I could still save them as .tpp and use for proofing or something.

Feedback / thoughts?

[yesgrey]

That being said, the .3dlut file format has some other, ridiculously major flaws imo, because it's lacking a lot of meta-tags.
...
Inclusion of this information, as well as stuff like gamut, gamma, and white point would be *critical* to a fully functional 3dLUT format.
3DLUT format spec is not complete yet. In the beginning the current state was more than enough, but now I'm perfectly aware that it's not. Unfortunately I currently don't have time to finish it, so we have to discuss it later.

Ps. Floating point can have another advantage: Since value ranges are normalized from 0 to 1, that makes gamut mapping independent of the level range.
No, it's not. The black and white with FP is not always 0.0 and 1.0. That happens only with levels 0-255.

Also, what about 9->16 bit LUTs for eg. Hi10P content?
Using a 9/16 (768MB) 3DLUT for a small increase in precision it's not worth it. The 9/32 (1536MB) shows a significant theoretical increase, but probably no one would be able see any difference, so it might not be worth it too...

[nand chan]

No, it's not. The black and white with FP is not always 0.0 and 1.0. That happens only with levels 0-255.

I don't see what the motive for this decision is (I've included support for it regardless), any reason?

Since you're pulling down from 16-235 in the first place, why not go all the way and just pull them into 0-1? You actually increase precision instead of losing it, since values don't have to be rounded. The only motive for not normalizing integer 3dLUTs to full range is because you lose WTW/BTB information that way, but with FP it's not an issue.

Using a 9/16 (768MB) 3DLUT for a small increase in precision it's not worth it. The 9/32 (1536MB) shows a significant theoretical increase, but probably no one would be able see any difference, so it might not be worth it too..

Depends on how good the interpolation algorithm is I suppose. I'm currently using linear interpolation (between individual steps) which causes aliasing, but if you were to find out the gamma curve (again, meta-tags) and do a correct non-linear interpolation, the quality of interpolation should be significantly better.

I suppose it would make less of a difference the better the screen is.

[alph@]

i create a 3dlut file for ycms (curve +matrice-120cd-gamma2,35 -testchart 912-profil hight quality) Here are the results
http://www.zimagez.com/miniature/mesures7.png (http://www.zimagez.com/zimage/mesures7.php)
left no ycms-right with,the deltaE after is best
http://www.zimagez.com/miniature/courbes1.png (http://www.zimagez.com/zimage/courbes1.php)

http://www.zimagez.com/miniature/gamma6.png (http://www.zimagez.com/zimage/gamma6.php)
the correction of gamma is very good

http://www.zimagez.com/miniature/color19.png (http://www.zimagez.com/zimage/color19.php)
color correction is different,the delta luma red is very good,but the blue going from 9 to 15??,

http://www.zimagez.com/miniature/color24.png (http://www.zimagez.com/zimage/color24.php)

[nand chan]

color correction is different,the delta luma red is very good,but the blue going from 9 to 15??

Might just be a limitation of the screen, eg. you can't get correct colors unless you set a certain channel to be too bright (especially for black point correction).

At any rate, this would probably be more appropriate in the yCMS topic.

[alph@]

I use a gamma of 2,35 instead of 709 bt,it's a probleme? and why the diagramme cie is out in the green,ycms must compromise?
'you can't get correct colors unless you set a certain channel to be too bright (especially for black point correction)'
Excuse me, but what is the relation with the black point:helpful:

[nand chan]

I use a gamma of 2,35 instead of 709 bt,it's a probleme? and why the diagramme cie is out in the green,ycms must compromise?

I don't know this.

'you can't get correct colors unless you set a certain channel to be too bright (especially for black point correction)'
Excuse me, but what is the relation with the black point:helpful

Well, when you're on an LCD monitor, you have a certain level range (0-255) which you can't go outside of.

So if your black point (of 0/0/0) is too red, the logical thing you'd have to do is decrease the red channel (-5/0/0). However, this is simply physically not possible.

To compensate, to fix the tone balance, you have to increase the other channels instead, eg. (0/5/5).

However, this results in a “brighter” black than 0/0/0. In other words, fixing the black point's tone balance can offset its luminance (brightness).

That's why the luma error increases.

[nand chan]

Version 0.7 is out!

Changelog is in the first post as usual.

Download link: http://www.mediafire.com/?ugaa1y0o5a1452g

Since this release introduces LutScript files (.3ls), I will post an example file here until the formal documentation is complete:

######################################
## Example .3dlut generation script ##
######################################

# Set the properties properly
!Input_Range(Limited)
!Input_Depth(8,8,8)
!Output_Range(Limited)
!Output_Depth(16)

# An example transformation pipeline. Note that this !Pixel pipeline applies pulldown and range adjustment
# so you don't have to do it manually. All numbers are normalized to the range 0-1 within here.
!Pixel( {

# Load the logic from an existing .3dlut
LutFile("some_3dlut.3dlut")

# And apply the calibration info from an argyll file
CalFile("path_to_file.ti3")

# And just for fun, grayscale it
Grayscale()

} )

List of current functions: (All are case-insensitive)

Meta / top-level functions:
!Input_Range(Full | Limited): Sets the range appropriately
!Input_Depth(Number, Number, Number): Sets the depth appropriately
!Output_Range(Full | Limited): Sets the range appropriately
!Output_Depth(Number): Sets the depth appropriately
!Pixel(Function): Creates a pixel transformation pipeline (with appropriate adjustments) from the given function.
Multiple of these can be “glued” together to form a block by using { and }

Pixel functions:
Grayscale(): Converts to grayscale
CalFile(String): Applies calibration information from a given .cal or .ti3 file
LutFile(String): Applies a LUT transformation using a given .3dlut
Constant(Number, Number, Number): Outputs a constant value, for example (1, 1, 1) to create a LUT that blanks to white
Invert(): Inverts the input

The .3dlut is calculated by appending each !Pixel() function together (without further transformations), then mapping each possible value through the list.

Do note that to use the generated .3dlut with madVR you need to tag it with BT.709 primaries first using tag3dlut -7

All LutFile() functions are completely independent of each other and detect ranges/depth for their own .3dlut, that is, you could mix a 16-bit 3dlut with a 10-bit 3dlut and an 8-bit 3dlut freely, or mix together limited and full range .3dluts. It really doesn't matter one bit, as long as they are tagged correctly.

To-do / planned functions:

ConvertFromYCbCr(Primaries): Would convert from YCbCr to RGB
Gamma(): Would adjust values using a given gamma coefficient
Swap(Number, Number): Would swap the two channels
Cat(): Would return the input as output, useful for where you want a .3dlut to do nothing

To show how this LutScript format gradually replaces the dedicated functions for .3dlut generation / modification, I have shown some example !Pixel scripts that implement the functionality currently found in other programs:

applycal.exe:
!Pixel({
LutFile("input.3dlut")
CalFile("calfile.cal")
})

merge3dlut.exe:
!Pixel({
LutFile("file1.3dlut")
LutFile("file2.3dlut")
LutFile("file3.3dlut")
# ...
})

changedepth:
# Set the appropriate range/depth for the new .3dlut

!Pixel(LutFile("old.3dlut"))

[alph@]

'Do note that to use the generated .3dlut with madVR you need to tag it with BT.709 primaries first using tag3dlut -7'

it gets complicated,So you can not use the 3dlut directly in madvr?a small tuto would cool,because I'm confused:rolleyes:
Your software is great,but utilities annexes are mysterious,I wish I could use it if necessary.

[nand chan]

'Do note that to use the generated .3dlut with madVR you need to tag it with BT.709 primaries first using tag3dlut -7'

it gets complicated,So you can not use the 3dlut directly in madvr?a small tuto would cool,because I'm confused:rolleyes:
Your software is great,but utilities annexes are mysterious,I wish I could use it if necessary.

It's an issue with madVR and the .3dlut format in particular rather than the programs, so I'm waiting on madshi to add native compatibility checking so you won't have to do it manually. I've chosen to not add madVR compatibility hacks for now because I don't want to botch up the format further than it has to be.

If he still hasn't added support for it, then I'll add a flag to lutscript.exe to add madVR compatibility (or maybe include it by default).

For now, just do this:

lutscript.exe -i input.3ls -o output.3dlut
tag3dlut.exe -i output.3dlut -7

Or for a shorter version:

lutscript -i input.3ls -o - | tag3dlut -7 -i - -o output.3dlut

The command line utilities are meant for power users that know exactly what they're doing and delivers programs to support the necessary logic so even those people who aren't programmers with free time can benefit from it.

The GUI is intended for simple usage. It's just that I haven't gotten around to actually updating it yet due to lack of interest. I have far more planned for the GUI.

[alph@]

thank,
is what you plan to support icm files in TI3Parser,It would be very convenient.

[nand chan]

thank,
is what you plan to support icm files in TI3Parser,It would be very convenient.

I am planning on that, yes. An estimate would be around version 0.9.

[nand chan]

Version 0.8 release

This build offers no real shiny new features, but a *lot* of behind the scenes improvements, refactoring and overall changes in the rendering path. The most noticeable improvements should be significantly better depth/range conversion logic and some further speed improvements.

I'm also starting to experiment with smaller 3dluts for lower requirement devices, specifically a 4-bit LUT (16x16x16) can get you surprisingly good results if interpolated, with only some very significant banding issues, far less than initially expected. You will still suffer from aliasing until I can match the gamma curve, unfortunately, but I can't really do that unless the .3dlut format includes it sooner or later (or I do ugly detection hacks like madVR does). It's still surprisingly higher quality than what I expected.

The most notable “issue” right now is that you have to tag .3dluts manually using tag3dlut if you change any of the depth or range settings with changedepth, at least until .3dlut version 2 comes out, which I'm hoping to develop within a reasonable timespace.

Download link for version 0.8: http://www.mediafire.com/?e5mqe0u3dllhuj9

Changes in first post.

[alph@]

'Do note that to use the generated .3dlut with madVR you need to tag it with BT.709 primaries first using tag3dlut -7'
still nessecaire for version 0,8?

[nand chan]

'Do note that to use the generated .3dlut with madVR you need to tag it with BT.709 primaries first using tag3dlut -7'
still nessecaire for version 0,8?

Yes. I don't plan on adding gamut mapping or color space aware 3dluts until at least version 0.9 or maybe even 0.10, since I'm waiting for ongoing changes in the .3dlut format.

[nand chan]

Version 0.9 released

Instead of waiting for the ever-so-slow authors of the .3dlut format, I've decided to branch it into a new, .3dl2 format. As such, you shouldn't ever need to use tag3dlut when making changes to .3dluts within my programs.

Do note that other programs, eg. madVR, do not support .3dl2 yet - luckily, I've added a conversion utility to convert a .3dl2 into a .3dlut, this is done by running: tag3dlut -i somefile.3dl2 -o somefile.3dlut --legacy

If you want madVR to support .3dl2 natively, go bug madshi.

Download link for version 0.9: http://www.mediafire.com/?fd05yivc5i50dd1

Changes in the first post as usual.

[Audionut]

Ok, silly question. How do you get .3dl2 files. I can only seem to save .3dlut files.

edit: nevermind, I paid more then 2 seconds attention to the first post.

This is coming along nicely btw too.

[Thunderbolt8]

so this tool is only a front end to ycms (and other lut creating progs), right? does it have any additional functionality ycms does not have, or the other way round?

are there any bugs which for example are not included when I'd use ycms instead, without this frontend? or would creating a 3dlut file for madvr currently result in exactly the same output file?

[nand chan]

Big news

Coming with version 0.10 I will be adding full support for ICC profiles, using LittleCMS as a back-end for these.

A quick comparison: http://screenshotcomparison.com/comparison/78631

The yCMS .3dlut was generated using the .ti3 file and make3dlut.exe (which uses sRGB primaries).

The LittleCMS .3dlut was generated using the ArgyllCMS-generated .icc profile, attached to the sRGB profile in absolute colorimetric mode.

so this tool is only a front end to ycms (and other lut creating progs), right? does it have any additional functionality ycms does not have, or the other way round?

The only tools that act as a front-end to yCMS are “make3dlut” and the “TI3ParserGUI”.

Everything else is significantly outside the scope of yCMS and is all implemented using my own logic.

yCMS and this toolset sort of complement each other. yCMS is a tool for color space conversions, and this is a tool for .3dlut manipulation - yCMS does none of what this tool does, and this tool aims to do none of what yCMS does (functionally). What it does do, however, is parse .ti3 files and input their primaries into yCMS, which is something that yCMS cannot do natively and what originally gave this project its name.

Right now I'd estimate that the yCMS front-end constitutes about 1%-5% of the total toolset.

are there any bugs which for example are not included when I'd use ycms instead, without this frontend? or would creating a 3dlut file for madvr currently result in exactly the same output file?

The make3dlut program uses the exact same script as madVR does and should result in bit-identical LUT output. (Except that my script creates .3dl2 by default instead of .3dlut, but those two can be converted losslessly with tag3dlut)

There are zero additional bugs introduced to the yCMS front-end components.

It is possible that there may be /some/ bugs in the utilities I create myself (changedepth, lutscript, and so on) but I generally do a lot of testing before I publish releases to make sure they work as advertised.

Ok, silly question. How do you get .3dl2 files. I can only seem to save .3dlut files.

edit: nevermind, I paid more then 2 seconds attention to the first post.

This is coming along nicely btw too.

Every tool will automatically generate .3dl2 files (what file name you type doesn't make a difference), except for: tag3dlut --legacy (will generate .3dlut) and TI3ParserGUI.

I have not included .3dl2 support to the TI3ParserGUI whatsoever because it does not use my 3dLut classes, it is simply a front-end for yCMS and nothing more.

[Thunderbolt8]

Big news
The make3dlut program uses the exact same script as madVR does and should result in bit-identical LUT output. (Except that my script creates .3dl2 by default instead of .3dlut, but those two can be converted losslessly with tag3dlut)

There are zero additional bugs introduced to the yCMS front-end components.

It is possible that there may be /some/ bugs in the utilities I create myself (changedepth, lutscript, and so on) but I generally do a lot of testing before I publish releases to make sure they work as advertised.ok, so I take there wont be any bugs at all then when I just use it to create the same lut file I'd create it using ycms, unless that bug is part of ycms as well.

regarding that converter tag3dlut, is it already a part of your tool? if not would it be possible to add such a conversion of .3dl2 to .3dlut so that we wouldnt have to use more additional tools?

edit: whats littlecms btw?

[madshi]

Instead of waiting for the ever-so-slow authors of the .3dlut format, I've decided to branch it into a new, .3dl2 format.
Haha. You're just coming at a bad time. yesgrey is taking a break for personal reasons. And I'm taking a madVR break, working on my money-earning job for once. I'd appreciate if we could do a common extended 3dlut format together once yesgrey is back online. Of course it's possible that we'll like your extended format just as it is. But maybe we have suggestions or wishes for changes. But that's going to take some time...

[nand chan]

Haha. You're just coming at a bad time. yesgrey is in the middle of a move to another country. And I'm taking a madVR break, working on my money-earning job for once. I'd appreciate if we could do a common extended 3dlut format together once yesgrey is back online. Of course it's possible that we'll like your extended format just as it is. But maybe we have suggestions or wishes for changes. But that's going to take some time...

When the time comes, it can be discussed whether or not the changes get merged into the main spec and what further changes to make. Until then, I am keeping the signature strictly separate in order to prevent any compatibility confusions about what supports it and what not.

My current implementation is flexible enough for me to easily change such things, so me implementing the fork already is not a hindrance.

ok, so I take there wont be any bugs at all then when I just use it to create the same lut file I'd create it using ycms, unless that bug is part of ycms as well.

That is correct.

regarding that converter tag3dlut, is it already a part of your tool? if not would it be possible to add such a conversion of .3dl2 to .3dlut so that we wouldnt have to use more additional tools?

tag3dlut is included, yes. If you wish to avoid using tag3dlut, then you can simply use the TI3ParserGUI which does not generate .3dl2 files at all - this is the most intuitive and easiest way to generate a .3dlut from .ti3 data and it is recommended if you simply want a “one-click” solution. That was the original purpose of the GUI in the first place, so I could refer to it within my guides.

edit: whats littlecms btw?

http://www.littlecms.com/

It's basically an alternative to yCMS in that it calculates transformations by using ICC profiles, which are more accurate as a 3d LUT than the primary / grayscale measurements yCMS uses - which defeat the whole point of using a 3dlut in the first place by dumbing everything down to matrix + transfer function.

In general, it is to be expected for ICC profiles containing 3D LUTs to be more accurate than yCMS. Apart from that, I find that yCMS introduces *more* banding than LittleCMS does, which provides a somewhat smoother gradient (and less gray tone deviation), as shown in the screenshot comparison I linked above (make sure you view it on a color-dumb browser like opera or chrome since firefox/safari will apply their own corrections).

[alph@]

thank;
A quick comparison: http://screenshotcomparison.com/comparison/78631 ,Why such a big difference.
you already finished version 0,10 ?

[nand chan]

thank;
A quick comparison: http://screenshotcomparison.com/comparison/78631 ,Why such a big difference.
you already finished version 0,10 ?

Different rendering engine, different input.

Version 0.10 is partially done on my PC but I'm not uploading it yet until I have better integration. I also still need to support YCbCr LUTs for ICC profiles, I'm also probably adding support for RGB -> XYZ + ICC integration, and auto-application of calibration data within an .icc profile before I release v0.10.

[nand chan]

I'm going to upload version 0.10 as-is for now. I haven't done any comprehensive testing yet but linkicc.exe should work. Just make sure you're combining the correct ICC profile types. I'm going to be revising the color pipeline a lot now, especially in regards to LutScript, but I don't want those changes to hold up the release of version 0.10, so I'm scheduling those for 0.11 instead.

Download link for v0.10: http://www.mediafire.com/?w4elng942d4g927

I will start posting the changelog in the release posts instead since I've hit the length cap for the first message.

Version 0.10:
! Added basic support for ICC profiles
+ Color space information is extracted from the ICC profiles and written into the resulting .3dl2
+ New program: linkicc.exe
+ New LutScript function: IccTransform(String, String, Intent), intents: Perceptual, Relative, Saturation or Absolute
+ New LutScript function: !Filetype(Type), possible values: 3DL2 or 3DLUT
+ New LutScript function: !Parameters(String), overrides the parameters data (not recommended for 3DLUT filetype)
+ Added support for LZO compression (CompressionMethod=1). This is used by default.
? Note: Legacy LUTs do not support compression, it will always be disabled when using --legacy mode in tag3dlut
+ Added a --compression (-c) flag to tag3dlut, valid values are: None, LZO
+ New LutScript function: !Compression(Method), possible values: None, LZO
+ LutScript now computes the intermediate gamut after each step (this changes when using eg. IccTransform())
It is recommended to *NOT* set the !Input_Primaries or !Output_Primaries tags when performing gamut mapping!
LutScript will auto-detect these correctly for you, eg. when mapping together multiple ICC profiles
~ LutScript now defaults to Limited range when not specified, since it just makes more sense for video content
? Changelog v0.9 was flawed, !Input_Primaries did not accept a string but an /enum/
(eg. “!Input_Primaries(BT709)” and not “!Input_Primaries("BT709")”)

[Audionut]

How to get a .3ls file? The first post links to the 0.7 release, with no mention of it.

What should the 'input' and 'device' icc profiles be in linkicc?

There are numerous tools to create 3dl2 files. Should one apply each tool and then merge3dlut function?

[nand chan]

How to get a .3ls file? The first post links to the 0.7 release, with no mention of it.

.3ls refers to a LutScript file. It is a scripting language I made up myself. I did provide some examples, but I'm going to write up a proper documentation for it soon.

In the meantime, it's now possible (in v0.11) to do monitor /proofing/ (preview what input would look like on /another/ monitor):

!Filetype(3DLUT)
!Pixel({
IccTransform("sRGB.icc") # RGB -> XYZ
IccTransform("monitor.icm") # XYZ -> RGB
IccTransform("monitor.icm") # RGB -> XYZ
IccTransform("previewm.icm") # XYZ -> RGB
})

Or you could do it manually by generating two RGB->RGB LUTs and merging them, eg:

linkicc -i sRGB.icc -d monitor.icm -o lut1.3dl2
linkicc -i monitor.icm -d previewm.icm -o lut2.3dl2
merge3dlut -o output.3dl2 lut1.3dl2 lut3.3dl2

What should the 'input' and 'device' icc profiles be in linkicc?

“device” should be the ICC profile of your monitor, obtained via ArgyllCMS or whatever.

“input” should be the ICC profile of the input color space, eg. BT.709 for HD video, or sRGB for computer imagery.

There are numerous tools to create 3dl2 files. Should one apply each tool and then merge3dlut function?

What is it exactly that you want to do? There are only three basic programs for LUT creation: make3dlut, linkicc and gen3dlut, and each of these effectively cancel eachother out. (You should only use one)

What you want to do is create one 3dlut and then /modify/ it using the tools such as changedepth, swapchan or applycal.

The LutScript facility is sort of presented as an alternative to all of the individual programs. If you need to do a lot of complex processing, you can combine all of these steps into a single LutScript file (.3ls) instead of generating and modifying a .3dl2 a bunch of times.

One should note that LutScript will be /more precise/ because it doesn't round the output down to 16-bit precision between each step, it keeps the 64-bit pipeline until the very end.

(Or you could just use a 64-bit .3dl2, for example using linkicc -b 64. Make3dlut relies on yCMS and therefore has a limit of 32 bit precision)

[Audionut]

Thanks for your response nand chan.

I would like to be able to use the information from Argyll to generate the necessary 3dlut for madvr.
Alot of the new additions to your program are over my head though, and I am getting lost about what process should be used to accomplish that.
I understand I can use make3dlut and then tag3dlut to convert from 3dl2 to 3dlut for madvr, but I would like to use the tools provided to get the best out of it that I can.

I'm guessing that I would be better served by using linkicc.
But I cannot find a BT.709 icc profile. Do I have to roll my own.

I have my display calibrated with SRGB/BT.709 chromaticity coordinates, and a relative tone curve of gamma 1.9, as my display has a gamma of around 1.9, and I'm lead to believe that it's best to calibrate for the display gamma as to minimise banding.

So if you are displaying images encoded to the sRGB standard, or displaying video through the calibration, just setting the gamma curve to sRGB or REC 709 (respectively) is probably not what you want!
If you're wondering what gamma value you should use, you can run “Report on uncalibrated display device” from the “Tools” menu to measure the approximated overall gamma among other info. Setting the gamma to the reported value can then help to reduce calibration artifacts like banding, because the adjustments needed for the video card's gamma table should not be as strong as if a gamma further away from the display's native response was chosen.

http://dispcalgui.hoech.net/#settings_calibration

[nand chan]

Thanks for your response nand chan.

I would like to be able to use the information from Argyll to generate the necessary 3dlut for madvr.
Alot of the new additions to your program are over my head though, and I am getting lost about what process should be used to accomplish that.
I understand I can use make3dlut and then tag3dlut to convert from 3dl2 to 3dlut for madvr, but I would like to use the tools provided to get the best out of it that I can.

I'm guessing that I would be better served by using linkicc.
But I cannot find a BT.709 icc profile. Do I have to roll my own.

These are the two basic options, yes. Either you read the .ti3 measurements and use yCMS, or you read the ICC profile and use LittleCMS. LittleCMS should have slightly better output.

It seems like you will have to create a BT.709 RGB profile yourself :P

(LittleCMS /does/ provide a function for this, I'll upload a BT.709 profile in a moment)

I have my display calibrated with SRGB/BT.709 chromaticity coordinates, and a relative tone curve of gamma 1.9, as my display has a gamma of around 1.9, and I'm lead to believe that it's best to calibrate for the display gamma as to minimise banding.

http://dispcalgui.hoech.net/#settings_calibration

That's fine, both yCMS and LittleCMS will account for the measured output gamma when translating colors.

[Audionut]

I'll upload a BT.709 profile in a moment

Cheers.

One final question.

Should I use gamma processing in madvr. Or does everything get taken care of in the 3dlut creation?

[nand chan]

Cheers.

One final question.

Should I use gamma processing in madvr. Or does everything get taken care of in the 3dlut creation?

Everything gets taken care of in the 3dlut creation.

Here is a BT.709 .icc profile which /should/ work fine.

<link deleted>

Edit: I should mention that I just noticed a bug in tag3dlut. When using --legacy, you have to specify --compression none, as it will (mistakenly) attempt to apply compression to a .3dlut that does not support it. This is fixed in the next version, just notifying.

Edit 2: The profile I uploaded seems to have incorrect primaries information due to a bug in LittleCMS. (You can still fix it manually by tagging the resulting LUT with BT.709 primaries using tag3dlut -7, or just use the fixed profile below)

[nand chan]

Fixed BT.709 profile: <upload pending, LOL MEDIAFIRE IS DOWN>

In the meantime, have a base64 representation of the file:

AAACYGxjbXMEIAAAbW50clJHQiBYWVogB9sACQAIAAQACQAuYWNzcE1TRlQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
PbWAAEAAAAA0y1sY21zAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAALZGVzYwAAAQ
gAAAA2Y3BydAAAAUAAAABOd3RwdAAAAZAAAAAUY2hhZAAAAaQAAAAsclhZWgAAAdAAAAAUYlhZWgAAAeQAAAAUZ1hZWgA
AAfgAAAAUclRSQwAAAgwAAAAQZ1RSQwAAAhwAAAAQYlRSQwAAAiwAAAAQY2hybQAAAjwAAAAkbWx1YwAAAAAAAAABAAAA
DGVuVVMAAAAaAAAAHABSAEcAQgAgAGIAdQBpAGwAdAAtAGkAbgAAAABtbHVjAAAAAAAAAAEAAAAMZW5VUwAAADIAAAAcA
E4AbwAgAGMAbwBwAHkAcgBpAGcAaAB0ACwAIAB1AHMAZQAgAGYAcgBlAGUAbAB5AAAAAFhZWiAAAAAAAADzUgABAAAAAR
a+c2YzMgAAAAAAAPSf///6GQAAEDH///jBAAECiwAABWMAAAMn///6vwABVJdYWVogAAAAAAAB7pYAAQAAAAAV1FhZWiA
AAAAAAAKQDgABAAAADfqpWFlaIAAAAAAAAIQlAAEAAAAAJ0JwYXJhAAAAAAAAAAAAAlmacGFyYQAAAAAAAAAAAAJZmnBh
cmEAAAAAAAAAAAACWZpjaHJtAAAAAAADAAAAAKPsAABU2QAATyYAAJlWAAAlZgAADpg=

[Audionut]

How about uploading to a host that in turn uploads to various hosts.

http://www.multiupload.com/

[nand chan]

How about uploading to a host that in turn uploads to various hosts.

http://www.multiupload.com/

http://www.multiupload.com/VANWR6WMUF

I won't do that for anything serious though since none of those downloading services are remotely as good as mediafire. (Except maybe the multiupload's own DD)

I can't stand wait times and such, if I'm forced to download from such shitty hosts I'd rather not download the file at all.

Edit: MEDIAFIRE IS BACK UP http://www.mediafire.com/?0sbwzrd8f0nowbw

[Audionut]

I won't do that for anything serious though

No worries. Looks like I successfully converted the base64 anyway.

Never heard of that before.

Thanks again.

[Audionut]

Ok, something is wrong.

I used,

linkicc -i Rec.709_2.35_gamma.icc -d device.icm -o output.3dl2
tag3dlut -i output.3dl2 -o output.3dlut --legacy --compression none

Results in this.

http://farm7.static.flickr.com/6079/6126340766_4aede8e0ab_b.jpg

[nand chan]

Ok, something is wrong.

I used,

linkicc -i Rec.709_2.35_gamma.icc -d device.icm -o output.3dl2
tag3dlut -i output.3dl2 -o output.3dlut --legacy --compression none

Results in this.

<snip>

Yeah, I just noticed that as well. The issue seems to be deeper than simply a bugged tag. It seems like all of the conversions done by LittleCMS are completely different as well when the tags change to the correct (?) values.

I've posted to the lcms-user mailing group in order to find out what the cause of this bug is, but in the meantime, what you can do is use the incorrect profile (re-uploaded here: http://www.mediafire.com/?9g66hzyg92jmlye) and fix the tagging manually with -7.

So, basically,

linkicc -i Rec.709_2.35_gamma.icc -d device.icm -o output.3dl2
tag3dlut -i output.3dl2 -o output.3dlut --legacy --compression none -7

For the next version (0.11) of linkicc.exe, I've made it ignore the ICC profile tags by default and assume everything to be BT.709, since this is simply unacceptably flawed and I don't want to trouble my end users with having to deal with LittleCMS's bullshit.

[Audionut]

That fixed it.

[nand chan]

Another comparison between the two:

http://screenshotcomparison.com/comparison/78975

(BPC) indicates black point correction, a flag which I've enabled starting version 0.11 (without BPC it's basically the same but the blacks are a tiny bit cut off, since it maps the input to a true black level which of course the LCD monitor can't actually reproduce)

I haven't quite tracked down the reason why the yCMS image seems a bit brighter in general, but I'm guessing it's because of different gamma settings.

Edit: yeah, it was a different gamma setting. yCMS uses a gamma of 2.2 by default, BT.709 uses 2.35. That's why the image appears slightly darker. I'll upload an sRGB comparison image in a moment.

[nand chan]

Here is an sRGB comparison image:

http://screenshotcomparison.com/comparison/78977

And here is a 2.2 gamma BT.709 comparison image: (this should, in theory, be closest to the yCMS result)

http://screenshotcomparison.com/comparison/78979

I've uploaded the 2.2 gamma profile if you want to make use of it: http://www.mediafire.com/?8svo69bqavubn4b (the same warning applies as for the last)

What it boils down to: LittleCMS handles the high-sat colors better but yCMS handles the low spectrum better.

I think it's time for me to create a program that can combine the two, by using LittleCMS values above a certain brightness, and yCMS values below them. Then we'll have a perfectly linear tone curve mapping that accounts for LCD inaccuracies, with the full 3D color correction of LittleCMS!

Sounds like a plan? You bet it does!

[Audionut]

Getting an error from mediafire trying to download the 2.2 gamma profile.

[Graeme Gill]

This may be a dumb question, but why all this messing around with .ti3 files ? They are raw device measurement information, and that leaves a lot of work in creating a device model and linking it to a input profile to create a 3D transform.

There are a bunch of tools already out there that do most of this work. ArgyllCMS has the profile creation capability, and both lcms and Argyll have device link capability. (Argyll's is probably a bit more sophisticated though).

ICC device links are a standard "3dlut", so why invent some new format ? All you should have to work on for use with video is 1) Creating suitable input profiles to suite video encoding standards 2) Providing a mechanism to apply a 3Dlut and 1D curves to video in realtime using the GPU, and 3) provide a loader to load ICC device to device links + display calibration curves so the GPU code can use it. Note that the equivalent machinery (implemented as non-real time, general CPU code) exists in Argyll imdi. lcms has similar code.

[nand chan]

This may be a dumb question, but why all this messing around with .ti3 files ? They are raw device measurement information, and that leaves a lot of work in creating a device model and linking it to a input profile to create a 3D transform.

I am aware of the amount of work involved here, this is why I do not do any transformations myself - I use yCMS and LittleCMS (which itself relies on ArgyllCMS for the profile creation), all of which are highly complete and well-done tools.

There are a bunch of tools already out there that do most of this work. ArgyllCMS has the profile creation capability, and both lcms and Argyll have device link capability. (Argyll's is probably a bit more sophisticated though).

linkicc uses lCMS to create the device link. You can also use LutScript (starting version 0.11) to load a single device link, eg. from ArgyllCMS.

ICC device links are a standard "3dlut", so why invent some new format?

ICC profiles are not complete 3dluts. If you look at the filesizes, a typical “3dlut-based” ICC profile is about 20-40 KB large, and a typical .3dlut is about 96 MB large. There are further issues. For example, ICC profile 3dluts are natively 8-bit.

All you should have to work on for use with video is 1) Creating suitable input profiles to suite video encoding standards

This you can/need to do either way, for example the BT.709 profile above.

2) Providing a mechanism to apply a 3Dlut and 1D curves to video in realtime using the GPU

This is not doable in realtime. 64-bit interpolation would unacceptably strain the GPU, especially on GPUs that simply cannot handle it.

and 3) provide a loader to load ICC device to device links + display calibration curves so the GPU code can use it. Note that the equivalent machinery (implemented as non-real time, general CPU code) exists in Argyll imdi. lcms has similar code.

The reason why they are non-realtime / general is because they are so slow.

You're missing the point of the .3dlut format. The .3dlut format is not a general purpose / comprehensive “transformation” format, the point of the .3dlut format is to pre-calculate /every single value/ so you can use the result in real-time during video playback.

Have you ever loaded a 1920x1080 image on a color managed application like Firefox? It takes about ~5 seconds depending on the image to apply the color correction fully, and that's without 64-bit floating point logic.

Using a properly multi-threaded code (linkicc), it still takes about 10 seconds to generate a full .3dlut from ICC profiles with all of the bells and whistles (FLAG_HIGHRESPRECALC, BPC, application of 3x1D TRCs and so forth).

[nand chan]

What it boils down to: LittleCMS handles the high-sat colors better but yCMS handles the low spectrum better.

I think it's time for me to create a program that can combine the two, by using LittleCMS values above a certain brightness, and yCMS values below them. Then we'll have a perfectly linear tone curve mapping that accounts for LCD inaccuracies, with the full 3D color correction of LittleCMS!

Sounds like a plan? You bet it does!

I've implemented this for now,

Here's what I dubbed a composite image vs yCMS:

http://screenshotcomparison.com/comparison/79074

They should basically be the same in terms of appearance and black level handling, but the composite image will be, in general, a /bit/ more accurate (since it accounts for 3D errors as well).

For an explanation of how the system works:

http://img40.imageshack.us/img40/5827/fractalecompositeluma.png

The areas where the screen is red means LittleCMS is being used, the areas where it's blue means yCMS is being used. Purple means they're being interpolated. The determining factor is luma, and you can adjust the cross-over point and amount of spread (interpolation) being performed.

Here is a grayscale comparison: http://img823.imageshack.us/img823/7021/compvsycms.png

On my monitor, I notice that yCMS's grays are slightly too blue, since it doesn't perform the 3D correction. It also shows a tiny bit more banding than the composite.

I've also disabled black point compensation again since composite mode eliminates the problem.

[nand chan]

Version 0.11 release

http://img705.imageshack.us/img705/8360/linkiccgui.png

This release spotlights the addition of a new GUI program called LinkIccGUI. This GUI program serves as a front-end for the entire process from .ti3 parsing to ICC linking up to and including auto-calibration, legacy tagging, composite joining and so forth.

Users who simply wish to get good color management out of the box now have a one-click solution that doesn't require any further in-depth knowledge of the system.

Special note: If “auto-calibrate” is checked, you have to make sure that /no/ calibration curve is interfering with madVR. Either disable Windows color management or set your monitor to the sRGB device profile. Alternatively, you could use one of the many available tools to disable the curves (dispcalGUI, dispclut.exe, ArgyllCMS, CalibrationTester.exe and so forth).

Download link: http://www.mediafire.com/?c6hlon3r1g0ohfa

Changelog:

Version 0.11:
- IccTransform() was removed
+ IccSingle(String, [Intent]) added. Intent is optional and defaults to Relative.
This IccSingle filter decodes between a profiles PCS and ColorSpace and can go both ways (eg. XYZ->RGB, RGB->XYZ)
+ IccLink(String, String, [Intent]) added - this links two profiles together and can only go one way (no PCS support)
* Missing ICC profiles no longer result in a crash
* Comments after LutScript lines with strings in them get filtered out properly now
* Fixed a bug where tag3dlut would still mark a lut as compressed even in legacy mode (the actual output wasn't
compressed, but the tag was incorrectly set)
* linkicc.exe now automatically assumes the input to be BT.709 instead of detecting the primaries from the ICC profile,
due to a strange issue in LittleCMS where the input tags will be completely incorrect.
+ New tag --auto-detect (-a) reverses this behavior [linkicc]
+ New program: join3dlut.exe, this program joins together two .3dl2s, switching between the two depending on
the luminance of the input.
+ New program: LinkICCGUI.exe. This is the new preferred front-end for everything. It accepts both an .icc and a .ti3,
and applies the following steps automatically:
make3dlut
linkicc (auto-generates a BT.709 / 2.2 profile in memory)
join3dlut
applycal
tag3dlut
The result is a fully functional v1 .3dlut ready for madVR compatibility. No further steps are needed.

[Thunderbolt8]

the whole thing still seems rather complicated to me, because I dont know what is required to start, what information or input files and where to take them from

basically, all I want to to is to create a .3dlut file for black level fixing, so changing input levels from default 0-255 to something like input range 16+X - 235+X (x= depending on how much black and/or white is off)
and sometimes changing gamma a little as well which worked in ymcs like this:

Output_Transfer_Function 4.5 0.099 0.45 0.018

by modifying the value 0.45 which then resulted in more or less gamma.

with ycms, I only needed to put some lines into a .txt file, but how does this work with your programm?

[alph@]

can you explain why you mix a icm file with a ti3 ,for the precision of the 3dlut?

[nand chan]

the whole thing still seems rather complicated to me, because I dont know what is required to start, what information or input files and where to take them from

basically, all I want to to is to create a .3dlut file for black level fixing, so changing input levels from default 0-255 to something like input range 16+X - 235+X (x= depending on how much black and/or white is off)

That's not currently possible (with the X offset thing). What you can do is change the input level from 0-255 to 16-235, but you can't apply an offset to either manually. I can add it as a feature for 0.12.

If you still want to create a .3dlut that will map from 0-255 to 16-235, you can do that by using:

gen3dlut.exe -o levels.3dl2 --range Full
changedepth.exe -i levels.3dl2 --output-range Limited

and sometimes changing gamma a little as well which worked in ymcs like this:

Output_Transfer_Function 4.5 0.099 0.45 0.018

by modifying the value 0.45 which then resulted in more or less gamma.

with ycms, I only needed to put some lines into a .txt file, but how does this work with your programm?

If you want to change the gamma afterwards you can create a Limited-range .3dlut that just changes the gamma in yCMS, and then merge it onto the .3dlut from earlier:

merge3dlut -o result.3dl2 levels.3dl2 gamma.3dlut

Now that I have LittleCMS integrated, I can add a program for changing the gamma level of a .3dlut by version 0.12 as well.

Ps. Alternatively, just use LutScript:

!Input_Range(Full)
!Output_Range(Limited)

!Pixel({
LutFile("gammafixing.3dlut")
})

can you explain why you mix a icm file with a ti3 ,for the precision of the 3dlut?

Yes, basically. The .icm file doesn't handle near-black levels the way I want it, so I handle the near-black levels manually using the .ti3 (and yCMS).

[Thunderbolt8]

That's not currently possible (with the X offset thing). What you can do is change the input level from 0-255 to 16-235, but you can't apply an offset to either manually. I can add it as a feature for 0.12.that would be very helpful, because sometimes only black leves are affected, while white leves are fine, or they are affected between 0-16 input range, so simply 16-235 would be too far off.


gen3dlut.exe -o levels.3dl2 --range Full
changedepth.exe -i levels.3dl2 --output-range Limited



If you want to change the gamma afterwards you can create a Limited-range .3dlut that just changes the gamma in yCMS, and then merge it onto the .3dlut from earlier:

merge3dlut -o result.3dl2 levels.3dl2 gamma.3dlut

Now that I have LittleCMS integrated, I can add a program for changing the gamma level of a .3dlut by version 0.12 as well.

Ps. Alternatively, just use LutScript:

!Input_Range(Full)
!Output_Range(Limited)

!Pixel({
LutFile("gammafixing.3dlut")
})I would be able to create these 2 things with whatever aspect or tool is most precise, so whatever gives the best results (and is accepted by madvr as input file of course). so using littlecms for both is recommended atm?

[nand chan]

that would be very helpful, because sometimes only black leves are affected, while white leves are fine, or they are affected between 0-16 input range, so simply 16-235 would be too far off.

It seems like I will have to support dynamic ranges after all :P.

Oh well, this is why I split off the 3DL2 from 3DLUT in the first place, so I can make changes like this and modify the spec as feedback comes in, before pushing it back upstream.

I would be able to create these 2 things with whatever aspect or tool is most precise, so whatever gives the best results (and is accepted by madvr as input file of course). so using littlecms for both is recommended atm?

I personally trust yCMS more than LittleCMS. LittleCMS has a lot of little funny things, and I don't like the way it handles black levels at all.

It's important to note here that LutScript is just a front-end, it's basically just a script-form that wraps all of the functions that the 3dLutLib.dll exposes.

So if I add some feature like YCbCr conversion, gamut mapping or gamma conversion to LutScript, it will just use the respective functions from yCMS and/or LittleCMS.

Ps. If you're doing gamma correction, why not use madVR's built in function for this? Or is the application going to be something other than madVR?

[Thunderbolt8]

I have no clue, no clue what all these progs can go, what gamut correction even means or that madvr has a built in function for this. Ive been given that line once to change gamma and that was the only way I knew how to change this.

[nand chan]

I have no clue, no clue what all these progs can go, what gamut correction even means or that madvr has a built in function for this. Ive been given that line once to change gamma and that was the only way I knew how to change this.

Sorry, I mistyped - I meant /gamma/ correction. madVR has a built-in function for gamma correction, you can scale the input (2.2 gamma) to any pure power curve / BT.709 curve you want, eg. 2.35 or 2.0, without needing a .3dlut for the task.

[Thunderbolt8]

sorry, but thats also over my head :p

do you mean to say that when I use the input values for b/w I desire, I can just apply a curve which is supposed to be "correct" automatically with whatever is used for input? (afaik yesgrey once mentioned that gamma might get screwed up when I change such b/w values myself and therefore gamma correction would be needed) and how would this work, is there a definition of correct gamma in such a situation or is it more a matter of taste? so BT.709 would be the curve I need for HD content?

[nand chan]

sorry, but thats also over my head :p

do you mean to say that when I use the input values for b/w I desire, I can just apply a curve which is supposed to be "correct" automatically with whatever is used for input? (afaik yesgrey once mentioned that gamma might get screwed up when I change such b/w values myself and therefore gamma correction would be needed) and how would this work

Yes, basically madVR /decodes/ the input to a linear level, then /recodes/ it to your desired output gamma. So essentially, if you view 2.3 material with the setting set to 2.3, nothing will be changed.

But if you view 2.2 material with the setting set to 2.3, it will appear darker.

is there a definition of correct gamma in such a situation or is it more a matter of taste? so BT.709 would be the curve I need for HD content?

Not necessarily, no. BT.709 is the correct gamma curve for HD /input/, but your /monitor/ probably has a pure power curve, so you want to output a pure power curve as well (unless you have your monitor calibrated to a BT.709 curve as well).

It's not just a matter of preference, but a matter of the ambient light level. In darker rooms, you generally want higher values (eg. 2.35).

Ps. I implemented the non-uniform level ranges thingy and tested it by producing a .3dlut which would take full range input and output 16-255 output.

Then I realized that didn't help at all because the library takes care of all levels differences when it normalizes the output -.-

Time to add a function for explicitly rescaling the range...

[nand chan]

Time to add a function for explicitly rescaling the range...

Done. Here's what your .3ls would look like: (The ScaleAbs() function will be supported starting version 0.12)

# Set the properties properly
!Input_Range(Full) # set these to whatever you need, make
!Output_Range(Full) # sure they're the same though

!Pixel( {

# Load the gamma correction .3dlut, as generated using yCMS
LutFile("gamma_correction.3dlut")

# Scale the signal to a different value range, relative to absolute space
ScaleAbs(16, 235) # set to whatever you want

# Any further processing here ...

} )

One thing I would like to comment on:

If you /first/ load the LutFile and /then/ scale the value range, the resulting gamma will be scaled as well. So if your input is 0-255 with a given gamma, and you want it to be 16-235 with the same gamma, you would first load the lut and then scale - within the new value range, the gamma will be the same.

If you want to do the reverse, that is, take a 16-235 signal and pull the levels up to 0-255, you can do this with !Input_Range(Limited) and Scale(Full).

(Note the addition of a sister function, ScaleRelative, which will further scale the input relative to what it is already. So if you want to, say, preview what limited range content looks like, inside a limited range .3dlut, you would use !Input_Range(Limited) and ScaleRelative(Limited), which would scale the input even further. ScaleAbs(Limited) would not affect it at all because the input range is already limited and ScaleAbs() works in absolute space.)

[Graeme Gill]

ICC profiles are not complete 3dluts. If you look at the filesizes, a typical “3dlut-based” ICC profile is about 20-40 KB large, and a typical .3dlut is about 96 MB large. There are further issues. For example, ICC profile 3dluts are natively 8-bit.



This is incorrect. ICC device links can be any resolution you like up to 256 ^ input channels. They can be either 8 or 16 bit. There's absolutely no point through, in creating a 3D lookup that vastly exceeds the precision of the input data.



This is not doable in realtime. 64-bit interpolation would unacceptably strain the GPU, especially on GPUs that simply cannot handle it.




Why would you use 64 bit interpolation when the output device is at best 10-12 bits ?




You're missing the point of the .3dlut format. The .3dlut format is not a general purpose / comprehensive “transformation” format, the point of the .3dlut format is to pre-calculate /every single value/ so you can use the result in real-time during video playback.



That's making no sense to me. Why create huge tables of interpolated values - that just leads to slow, memory intensive code. The 3D transformation representation should
be as small as possible to reduce memory size and hence bandwidth needed for the tables, consistent with the precision of the device characterization. Remember, the device
can only be characterized with a few thousand measurements, so there is no point is creating millions of fake interpolated values.



Have you ever loaded a 1920x1080 image on a color managed application like Firefox? It takes about ~5 seconds depending on the image to apply the color correction fully, and that's without 64-bit floating point logic.



They must be doing something wrong then. My general CPU
imdi code, that doesn't even use SSE type parellism takes about 0.12 seconds for that size image RGB->RGB at 16 bit precision. GPU will do it much faster.




Using a properly multi-threaded code (linkicc), it still takes about 10 seconds to generate a full .3dlut from ICC profiles with all of the bells and whistles (FLAG_HIGHRESPRECALC, BPC, application of 3x1D TRCs and so forth).

You shouldn't be generating such huge LUTs then. There is simply no point in it.

[ryrynz]

Thanks nand Chan making some great progress.

[Thunderbolt8]

BT.709 is the correct gamma curve for HD /input/, but your /monitor/ probably has a pure power curve, so you want to output a pure power curve as well (unless you have your monitor calibrated to a BT.709 curve as well).how can I find out what type of curve my LCD TV has? (its a shitty no name product with no kind of drivers or whatsoever I could look into)

[nand chan]

how can I find out what type of curve my LCD TV has? (its a shitty no name product with no kind of drivers or whatsoever I could look into)

The only way to be sure is to either measure it, or even better, calibrate it to a specific curve yourself. Chances are it's a regular sRGB curve though which is approximately equal to a pure power curve of 2.2.

This is incorrect. ICC device links can be any resolution you like up to 256 ^ input channels. They can be either 8 or 16 bit. There's absolutely no point through, in creating a 3D lookup that vastly exceeds the precision of the input data.

Is ArgyllCMS capable of creating such large input profiles? eg. 256x256x256

Why would you use 64 bit interpolation when the output device is at best 10-12 bits ?

Because the lower bit-depth you clamp your result to, the less precise it gets when applying multiple transformations. There's a reason every single decent video filter outputs a higher bit depth then it was fed, usually at least twice the bit-depth of the input. Remember, once you destroy information, it's gone. Why not preserve it if you can?

And taking a higher bit depth + dithering that down to the output depth is significantly more precise than using that bit-depth directly. There's a reason why madVR, for example, has such higher quality upsampling/upscaling than comparable, 8-bit video renderers - even on 6-bit display devices.

That's making no sense to me. Why create huge tables of interpolated values - that just leads to slow, memory intensive code.

Looking up a table index is quite fast, far faster than 64-bit square root calculations etc.

The 3D transformation representation should be as small as possible to reduce memory size and hence bandwidth needed for the tables, consistent with the precision of the device characterization. Remember, the device can only be characterized with a few thousand measurements, so there is no point is creating millions of fake interpolated values.

So you're suggesting re-interpolating the values 49766400 times per second, giving you the same result? Why not interpolate them once and then do nothing, greatly reducing playback stress?

They must be doing something wrong then. My general CPU
imdi code, that doesn't even use SSE type parellism takes about 0.12 seconds for that size image RGB->RGB at 16 bit precision. GPU will do it much faster.

Chances are the calculations you are performing are less in magnitude. Proper gamut mapping should be done with polar coordinates and converting to/from those is quite expensive.

You shouldn't be generating such huge LUTs then. There is simply no point in it.

And you're suggesting doing it in realtime?

[e-t172]

I think the point of Graeme Gill (apparently it's the first post on Doom9 from the author of ArgyllCMS... welcome !) is that you don't need such precision when storing the filter on e.g. your hard disk, and that the actual, 256x256x256 3dlut should be generated on the time it is used (i.e. beginning of playback in the case of madVR).

[nand chan]

I think the point of Graeme Gill (apparently it's the first post on Doom9 from the author of ArgyllCMS... welcome !) is that you don't need such precision when storing the filter on e.g. your hard disk, and that the actual, 256x256x256 3dlut should be generated on the time it is used (i.e. beginning of playback in the case of madVR).

You'll have to talk to/argue with madshi about this - he's unwilling to even /decompress/ the .3dlut at the beginning of playback because it would simply be unacceptably slow, also keep in mind that madVR does in fact reload the .3dlut on several occasions, for example when you switch to fullscreen mode.

Re-calculating the .3dlut when you enter fullscreen mode would mean significantly higher delays - you're also neglecting the fact that there's /still/ no reason to re-calculate the exact same thing 10000 times than calculating it /once/ and keeping it that way.

Can you offer any arguments as to why .3dluts /shouldn't/ be used? 100 MB storage space? We're living in an age where storage space is cheaper than processing power. 100 MB is absolutely nothing.

Even my portable music player could fit hundreds of .3dluts on it, there's no reason to worry about such an insignificant amount.

[Thunderbolt8]

The only way to be sure is to either measure it, or even better, calibrate it to a specific curve yourself. Chances are it's a regular sRGB curve though which is approximately equal to a pure power curve of 2.2.ok, but you have to remember, I have to clue :P so: how to measure or calibrate it?

[nand chan]

v0.12 release

Not all of the things I described above made it into the release, and I spent quite some time reversing some of the changes. Either way, the release should be largely working now, but I haven't tested every program - only the basics.

I also decided to get rid of ScaleAbs() and renamed ScaleRelative to simply Scale. Most importantly, however, this release introduces gamma and gamut functions.

Download link: http://www.mediafire.com/?l3toc3v20l03g6v

Changelog:
Version 0.12:
* Fixed a bug in the merge3dlut description
~ Internal ColorTriple<T> structure re-ordered to have the same channel order for BGR and YCbCr, most tools are now channel
agnostic unless they need to be otherwise. This might cause some issues where the channels end up swapped - please report
them asap.
+ New LutScript functions: GammaDec(Number), GammaEnc(Number), ColorDec(Primaries), ColorEnc(Primaries) - these add primitive
gamma and gamut adjustment. The proper procedure is to first decode gamma, then colors, then encode colors and encode gamma.
“Primaries” can be either of the form BT709 or it can be 8 separate numbers, referring to the x/y values of the primaries'
Red, Green, Blue and White points - in that order. For example:
!Pixel({
GammaDec(2.2) # This converts from weighted R/G/B to linear r/g/b
ColorDec(BT709) # This converts from linear r/g/b to absolute X/Y/Z
ColorEnc(0.682, 0.304, 0.1975, 0.7036, 0.1457, 0.0486, 0.313, 0.3308) # This converts from X/Y/Z to some other linear r/g/b
GammeEnc(2.35) # This converts from linear r/g/b back to weighted R/G/B, here scaled slightly differently
})
+ New LutScript function: Scale(Number, Number) - this scales to a given value range, for example Scale(0.2, 0.8)
It is important to note here that this is relative to the input range - the above function in a Limited range .3dl2 will scale
to something like 59.8 - 198.4.
+ New LutScript function: Clamp(Number, Number) - this clamps output to a certain value range. Useful when you get artifacts
in madVR when dealing with BTB/WTW levels in Limited range .3dluts (my own code does not exhibit this behavior, so it is not
needed for usage with progams like imagecal.exe)
* LinkIccGUI no longer locks up the interface while it is processing

[Graeme Gill]

I think the point of Graeme Gill (apparently it's the first post on Doom9 from the author of ArgyllCMS... welcome !) is that you don't need such precision when storing the filter on e.g. your hard disk, and that the actual, 256x256x256 3dlut should be generated on the time it is used (i.e. beginning of playback in the case of madVR).

Hi,
the point is that there is no reason at all to create a 256x256x256 table. A device->device transform with resolution somewhere between 17 and 50 as a maximum (when combined with higher res. per channel curves) is all that can be justified given the fairly coarse characterisation of the actual output device. It has additional advantages in scaling to greater than 8 bit inputs smoothly. Modern processors have lots of compute speed, but relatively limited memory bandwidth, so trading computation for smaller table size is going to give better performance. A GPU has the advantage of having built in hardware that does the core interpolation - the trilinear 3D texture lookup instruction. See for instance: GPU Gems 2, page 381: "Using Lookup Table to Accelerate Color Transformations", The OpenEXR exrdisplay application, and CompICC:
<http://sourceforge.net/apps/mediawiki/compicc/index.php?title=Main_Page>

[Graeme Gill]

Is ArgyllCMS capable of creating such large input profiles? eg. 256x256x256



Yes, but you would be crazy to do so, and nothing would be gained by it. Why generate 16 million points from a few thousand measurements ?


Because the lower bit-depth you clamp your result to, the less precise it gets when applying multiple transformations. There's a reason every single decent video filter outputs a higher bit depth then it was fed, usually at least twice the bit-depth of the input. Remember, once you destroy information, it's gone. Why not preserve it if you can?


Sounds like a severe case of premature optimisation to me, to go for 2x10^16 times the precision of the final result. A factor of 2 - 10 is typically sufficient, but the final proof is in the viewing. If you can see it in a double blind test amongst all the natural image noise, JPEG quantisation artefacts and motion estimation artefacts, then by all means use a higher precision - say 16 bits.


Looking up a table index is quite fast, far faster than 64-bit square root calculations etc.


Why would you want to do a square root ? That's not likely to be fast....


So you're suggesting re-interpolating the values 49766400 times per second, giving you the same result? Why not interpolate them once and then do nothing, greatly reducing playback stress?


Yes, because that optimises the memory bandwidth, which is one of the main bottlenecks on modern processors. You also get the advantage of scaling to greater than 8 bit input precision.



Chances are the calculations you are performing are less in magnitude. Proper gamut mapping should be done with polar coordinates and converting to/from those is quite expensive.



Yes, but none of that has anything to do with the final transformation. Argyll gamut mapping is actually done in CIECAM02 space, quite a complex transformation in both directions. But the end result of a device -> device link is a cLUT stored in an ICC device link, something that can be executed very fast using just integer CPU code, such as Argyll's IMDI, lcms or a GPU texture lookup.

[cyberbeing]

Yes, but you would be crazy to do so, and nothing would be gained by it. Why generate 16 million points from a few thousand measurements ?
Graeme, you may want to direct your issues with the 3dlut format at yesgrey (yCMS) (http://yesgrey.com/ycms.html) and tritical (ddcc/t3dlut) (http://web.missouri.edu/~kes25c/), since they are the ones who originally created the 3dlut format. madshi in that same time period back in 2009 created the video renderer madVR (http://forum.doom9.org/showthread.php?t=146228) to use a 3dlut for gamut correction.

It has now been over 2 years since people began using 3dluts for color correction on their GPUs with madVR. All nand-chan is doing is making tools to ease the process of 3dlut creation and modification. Previously people were pretty much restricted to manually copying and pasting measurements from TI3 files or tools like ColorHCFR and feeding them to yesgrey's tool yCMS to create a 3dlut for use with madshi's video renderer, madVR.

Recently madshi and yesgrey have added support for 6bit (64x64x64) and 7bit (128x128x128) 3DLUTs where madVR uses trilinear interpolation to generate the missing values, in addition to the full 8bit (256x256x256) lut. I personally have been using a 6bit 3DLUT for awhile, the results are very good, but on average I've measured a yCMS created 6bit 3DLUT having 0.5 worse dE values than a full 8bit 3DLUT with my i1pro rev. D. The difference is indeed nearly invisible (gamma correction aside), but with modern GPUs $80+ having zero performance issues with the full 16.7million color 3DLUT it's more of case of why not? The more significant issue with the current implementation is not gamut correction with 6bit 3DLUT, but rather gamma correction where the difference between 64 points and 256 points (non-smoothed) can be extreme when attempting to represent non-power-curves like REC709 or sRGB in the lower tones.

That said, there is a lot of room for improvement in 3dlut creation. All yCMS created 3dluts take into account are white point, primary colors (100% Red/Green/Blue), and IRE grayscale, making the resulting correction extremely basic and likely worse than an Argyll created Matrix/Shaper ICC profile. This no doubt hurts the potential accuracy of a 6bit 3DLUT.

As I've been saying for the past year or so on Doom9, what would be amazing is if ArgyllCMS colprof could be modified to create a madVR-compliant 3DLUT instead of a ICC profile during a normal ArgyllCMS calibration workflow. Graeme would modifying your existing cLUT code to output a 6bit(?) 3DLUT interest you personally at all (since nobody else has been interested...)? What we are really lacking is a high quality CMS like ArgyllCMS that takes a full set of measurement patches into account so we can actually get high quality results with less data points in the 3DLUT. The work nand chan has been doing is a step in the right direction, but an official ArgyllCMS solution which meets your quality standards and approval would be so much better. madshi as said a few times already that he'd love to be able to make 6bit 3DLUTs the default in madVR if the quality improved.

[Audionut]

Graeme, you may want to direct your issues with the 3dlut format at yesgrey (yCMS) (http://yesgrey.com/ycms.html) and tritical (ddcc/t3dlut) (http://web.missouri.edu/~kes25c/), since they are the ones who originally created the 3dlut format.

It's probably best to keep it here actually, as yesgrey won't be available for a few months and nand chan looks set to improve the 3dlut format.

All nand-chan is doing is making tools to ease the process of 3dlut creation and modification.

As above, it has since been expanded to include a new 3dlut format. (Mainly due to lack of time on yesgreys part atm afaik).

As I've been saying for the past year or so on Doom9, what would be amazing is if ArgyllCMS colprof could be modified to create a madVR-compliant 3DLUT instead of a ICC profile during a normal ArgyllCMS calibration workflow.

That would be nice. I would be keen to donate again if this were to happen.

[cyberbeing]

It's probably best to keep it here actually, as yesgrey won't be available for a few months and nand chan looks set to improve the 3dlut format.
I probably should have stated that differently. If Graeme wants more information on why the 3DLUT format was originally created instead of using existing ICC profile implementation, he would need to talk to tritical or yesgrey, as both I'm sure have a stronger opinion of the merits of a 3DLUT vs an ICC profile compared to nand chan who has only recently taken on the task of expanding the format.

It's very true that yesgrey is unavailable for the time being since he is in the process of moving to another country, but contacting tritical may not be a bad idea if Graeme desires some information on the logic behind the format's creation. Considering tritical's background as a Ph.D in Math as well as a college professor, I'm sure he and Graeme could have a lively discussion on the technical merits of 3DLUT vs ICC if they so desired, which may be much more Graeme's cup of tea than debating things with nand chan or even yesgrey. ;)

Now if Graeme wants to discuss changes and future of the 3dlut format, rather than why it exists as it is now, nand chan (and yesgrey when he returns) are the proper people to direct suggestions, complaints, and constructive criticism.

Either way, I'm happy to see Graeme has finally made his way over to Doom9 to provide his valuable input on all things CMS related. :D

[nand chan]

Yes, but you would be crazy to do so, and nothing would be gained by it. Why generate 16 million points from a few thousand measurements ?

Sounds like a severe case of premature optimisation to me, to go for 2x10^16 times the precision of the final result. A factor of 2 - 10 is typically sufficient, but the final proof is in the viewing. If you can see it in a double blind test amongst all the natural image noise, JPEG quantisation artefacts and motion estimation artefacts, then by all means use a higher precision - say 16 bits.

Why use less when you can use more? That's my line of thinking, anyhow.

Why would you want to do a square root ? That's not likely to be fast....

LittleCMS uses polar coordinates to desaturate colors, the chromaticity of which is calculated as the square root of the sum of squares of the a and b values (from L*ab).

Yes, because that optimises the memory bandwidth, which is one of the main bottlenecks on modern processors. You also get the advantage of scaling to greater than 8 bit input precision.

Modern graphics cards are more than optimized for fast random memory access - this is just as much not a limiting factor as calculation speed, so arguing over the two is sort of pointless.

Yes, but none of that has anything to do with the final transformation. Argyll gamut mapping is actually done in CIECAM02 space, quite a complex transformation in both directions. But the end result of a device -> device link is a cLUT stored in an ICC device link, something that can be executed very fast using just integer CPU code, such as Argyll's IMDI, lcms or a GPU texture lookup.

I see what you mean by this. Since the result is (arguably) the same (as long as LUT precision isn't reduced below measurement precision), it boils down to “computation power” vs “storage size”.

Either way, let me clear up some potential misunderstandings here: I'm not claiming that the benefits of 256x256x256 LUTs are tangible, I'm just providing reasons to prove that a smaller LUT would not be better.

The matter of fact is, madVR uses full .3dluts and it's not within my power to change that - if you can provide a good reason for madshi to change that then by all means do so and I will adapt my toolset accordingly. But, for the time being, all I'm interested in doing is providing the tools to better work with madVR.

Graeme, you may want to direct your issues with the 3dlut format at yesgrey (yCMS) (http://yesgrey.com/ycms.html) and tritical (ddcc/t3dlut) (http://web.missouri.edu/~kes25c/), since they are the ones who originally created the 3dlut format. madshi in that same time period back in 2009 created the video renderer madVR (http://forum.doom9.org/showthread.php?t=146228) to use a 3dlut for gamut correction.

I would like to mention that there is a separate thread to discuss the .3dlut topic. If possible, I would like to keep this thread related only to the tool which I am implementing (eg. bug reports, issues, questions)

Link to .3dlut discussion topic: http://forum.doom9.org/showthread.php?t=139389&page=36

Recently madshi and yesgrey have added support for 6bit (64x64x64) and 7bit (128x128x128) 3DLUTs where madVR uses trilinear interpolation to generate the missing values, in addition to the full 8bit (256x256x256) lut.

I'd just like to mention that madVR also interpolates the 8bit lut when using 10bit video.

That said, there is a lot of room for improvement in 3dlut creation. All yCMS created 3dluts take into account are white point, primary colors (100% Red/Green/Blue), and IRE grayscale, making the resulting correction extremely basic and likely worse than an Argyll created Matrix/Shaper ICC profile. This no doubt hurts the potential accuracy of a 6bit 3DLUT.

As I've been saying for the past year or so on Doom9, what would be amazing is if ArgyllCMS colprof could be modified to create a madVR-compliant 3DLUT instead of a ICC profile during a normal ArgyllCMS calibration workflow.

My program is capable of doing this. The IccSingle() function loads a single .icc profile and transforms its input accordingly (eg. for a source profile, it would input RGB and output XYZ*, or for a link profile, it would transform RGB to RGB).

So you can generate your link profile using colprof + collink and then generate an 8-bit .3dlut using lutscript, which uses LittleCMS internally.

* Currently, only the .3dl2 file format is capable of XYZ output - .3dlut (v1) must have RGB input and RGB output and my toolset will complain to you otherwise.

(I'll go test this myself now so I can back up my claims)

[nand chan]

And, the results are in.

Step 1: Profile generated using collink.exe sRGB.icm standard.icm link.icm -i a

(I fixed a bug here in v0.13 that would cause my tools to reject any ICC profiles that don't have the tags for r/g/bXYZ + wtpt)

Step 2: I ran the following lutscript:

!Filetype(3DLUT)
!Input_Primaries(BT709)
!Input_Depth(6, 6, 6)

!Pixel(IccSingle("link.icc"))
For this .3dlut, I had the standard.icm's CLUT tables loaded to my graphics card

And then the same with !Input_Depth(8, 8, 8)

Here is a grayscale comparison: http://screenshotcomparison.com/comparison/79911
Here is a color image comparison: http://screenshotcomparison.com/comparison/79912 (one of my favorite scenes for testing gamut mapping because of the green tones)

Step 3: In order to compare the quality of the actual gamut mapping to that of LittleCMS, I have modified the script slightly:

!Filetype(3DLUT)
!Input_Primaries(BT709)
!Input_Depth(6, 6, 6)

!Pixel({
IccSingle("link.icc")
CalFile("standard.ti3")
})
For this .3dlut, I have my video card's gamma table reset

Here is a comparison image between the .3dlut generated using the above script, and a previous, 8-bit .3dlut that I had generated using LittleCMS and yCMS:

http://screenshotcomparison.com/comparison/79913

I believe the results can be said to be satisfactory.

[alph@]

nand chan,when I want to create a 3dlut with LinkICC, i have a error message 'input string was not in a correct format'' I forgot something?

[cyberbeing]

And, the results are in.

Step 1: Profile generated using collink.exe sRGB.icm standard.icm link.icm -i a

(I fixed a bug here in v0.13 that would cause my tools to reject any ICC profiles that don't have the tags for r/g/bXYZ + wtpt)

I'd be interested to test that once you release 0.13, since something was causing a large gamma shift using LinkICCGUI.exe in 0.11, even though auto-calibrate was disabled. You probably should add a checkbox to disable yCMS grayscale measurements, if I assume that was the cause and not littlecms2. At first I was thinking some significant modification would need to be done to ArgyllCMS because of all the PCS and LAB code integrated in colprof and ICC profiles, but I completely forgot that profile linking could be used to create a cLUT with src -> dst gamut mapping.

I assume 'standard.icm' is your ArgyllCMS display profile and 'sRGB.icm' could be replaced with a custom 'REC709.icm' source profile?

It's amazing how quickly you put together all these tools in less than a month. I look away for a couple weeks and the number of tools and functions double. :scared:

Keep up the good work, at some point I hope you take a break to write up some documentation w/ example uses and workflows for all these tools.

[nand chan]

nand chan,when I want to create a 3dlut with LinkICC, i have a error message 'input string was not in a correct format'' I forgot something?

Might be the locale issue again, afaik all that could be causing it is Double.Parse() which is used to generate a number from a source string (in the .ti3 files).

I'd be interested to test that once you release 0.13, since something was causing a large gamma shift using LinkICCGUI.exe in 0.11, even though auto-calibrate was disabled. You probably should add a checkbox to disable yCMS grayscale measurements, if I assume that was the cause and not littlecms2. At first I was thinking some significant modification would need to be done to ArgyllCMS because of all the PCS and LAB code integrated in colprof and ICC profiles, but I completely forgot that profile linking could be used to create a cLUT with src -> dst gamut mapping.

yCMS values were only being used for very dark values, the problem with LittleCMS is that it seemingly did not compensate for the screen's minimum brightness, and as such, every value that in theory would decode to a value below the screen's minimum would get cut off - so you lost a lot of shadow detail on screens with poor black levels (like mine).

My (naive) solution was to let LittleCMS handle the bright things (the vast majority of stuff), and yCMS to handle the very dark values.

Can you test if linkicc.exe -i sRGB.icc -d monitor.icc -o - | tag3dlut -i - -o output.3dlut --legacy -7 works?

Also, have you tried modes other than absolute chromatic?

I assume 'standard.icm' is your ArgyllCMS display profile and 'sRGB.icm' could be replaced with a custom 'REC709.icm' source profile?

Standard.icm is the profile for my monitor's “standard” mode (which I use to test as it is a lot wider gamut than my sRGB emulation mode, which is already at delta E 2 average)

sRGB.icm could be replaced by a BT.709 source profile, I don't know for 100% sure whether madVR encodes for the screen using an sRGB gamma curve or a pure power 2.2 curve.

I also don't recommend using the BT.709 source profile I provided earlier since I still don't quite know how to work with LittleCMS in terms of its white point shifting (it automatically normalizes all generated profiles to D50 even though I'm working with D65 everywhere, so if I want to combine them I'll have to use relative chromatic mode to fix the white point back to D65).

[cyberbeing]

Can you test if linkicc.exe -i sRGB.icc -d monitor.icc -o - | tag3dlut -i - -o output.3dlut --legacy -7 works?

Also, have you tried modes other than absolute chromatic?

I'll test that command line later. The other day I tested both absolute colorimetric and relative colorimetric and the results had identically bad gamma shift, as far as I could tell.

For future reference, how do I go about disabling yCMS gamma corrections (aka grayscale measurements) with your tools? You really shouldn't have that enabled by default, unless someone is calibrating to a pure power-curve (i.e. the only type of curve which yCMS agrees with ArgyllCMS) . Both yCMS (grayscale measurements) and littlecms2 (in mpc-hc) are seemingly unable to properly linearize dispcal's CIECAM02 scaled REC709/sRGB gamma curves, or return the curve to its pre-linearized state unmolested.

I'm crossing my fingers that the ArgyllCMS collink method you came up with, will give the results I desire.

sRGB.icm could be replaced by a BT.709 source profile, I don't know for 100% sure whether madVR encodes for the screen using an sRGB gamma curve or a pure power 2.2 curve.
madVR uses a 2.2 pure power-curve by default, not sRGB. I guess that means I need to create a profile with REC709/sRGB primaries, but has 2.2 gamma power-curve as my src link?

I also don't recommend using the BT.709 source profile I provided earlier since I still don't quite know how to work with LittleCMS in terms of its white point shifting (it automatically normalizes all generated profiles to D50 even though I'm working with D65 everywhere, so if I want to combine them I'll have to use relative chromatic mode to fix the white point back to D65).

That's exactly why I always have used Scarse for creating REC709 profiles (up to 16bit 256x256x256). It's too bad that project was abandoned in alpha status 6 years ago, and was left in state where creating non-calibrated custom ICC profiles is all it's good for. Unfortunately, there just aren't many tools that I'm aware for creating uncalibrated ICC profiles with user-defined settings.

[Audionut]

That's exactly why I always have used Scarse

Sorry for taking this off topic, but since Scarse has no documentation, does this CL look like what I want to be doing. Display is calibrated to a Gamma of 1.9

ipb -c i -i RGB -o RGB:1.9 -p HDTV test.icm

[nand chan]

I'll test that command line later. The other day I tested both absolute colorimetric and relative colorimetric and the results had identically bad gamma shift, as far as I could tell.

Okay. Would you mind also giving me a further detailing of the problem? Test case? Screenshots? It might help me either debug a problem with my tools or a potential misconfiguration.

For future reference, how do I go about disabling yCMS gamma corrections (aka grayscale measurements) with your tools? You really shouldn't have that enabled by default, unless someone is calibrating to a pure power-curve (i.e. the only type of curve which yCMS agrees with ArgyllCMS) . Both yCMS (grayscale measurements) and littlecms2 (in mpc-hc) are seemingly unable to properly linearize dispcal's CIECAM02 scaled REC709/sRGB gamma curves, or return the curve to its pre-linearized state unmolested.

You can't currently do that I'm afraid, it includes the grayscale measurements one way or another - however, what's the point in using yCMS if you leave out the grayscale measurements? At that point it's just a simple gamut mapping, which can be done with EncodePrimaries() in LutScript or the tools in ArgyllCMS.

I'm crossing my fingers that the ArgyllCMS collink method you came up with, will give the results I desire.

If it does not then it is either an issue with your profiles, ArgyllCMS or madVR - all my program does with the above described method is interpolate the Argyll values (using LittleCMS).

madVR uses a 2.2 pure power-curve by default, not sRGB. I guess that means I need to create a profile with REC709/sRGB primaries, but has 2.2 gamma power-curve as my src link?

Yes, basically. Actually, if you want to go a step further, what you can do is input a profile that has the same primaries as your display - madVR does gamut mapping to match the .3dlut's input, after all. I'm going to guess it's not as advanced as ArgyllCMS's gamut mapping though. So yeah, basically, just use a BT.709 primaries 2.2 power curve input profile, because that's exactly what madVR spits into the .3dlut.

That's exactly why I always have used Scarse for creating REC709 profiles (up to 16bit 256x256x256). It's too bad that project was abandoned in alpha status 6 years ago, and was left in state where creating non-calibrated custom ICC profiles is all it's good for. Unfortunately, there just aren't many tools that I'm aware for creating uncalibrated ICC profiles with user-defined settings.

Okay, good to know. I'll go check it out.

[alph@]

Might be the locale issue again, afaik all that could be causing it is Double.Parse() which is used to generate a number from a source string (in the .ti3 files)..
ok,it's work now,i have solved the probleme -panels of configuration -region and language-formats-additional parameters-in 'decimal symbol' replace the comma ',' by a point '.', Now I can test linkICC.

[Graeme Gill]

It has now been over 2 years since people began using 3dluts for color correction on their GPUs with madVR. All nand-chan is doing is making tools to ease the process of 3dlut creation and modification. Previously people were pretty much restricted to manually copying and pasting measurements from TI3 files or tools like ColorHCFR and feeding them to yesgrey's tool yCMS to create a 3dlut for use with madshi's video renderer, madVR.


From what I can see of it, it's all a bit weird - a very round about, cumbersome and inflexible approach to dealing with video color management, and an awful lot of re-inventing the wheel.


Recently madshi and yesgrey have added support for 6bit (64x64x64) and 7bit (128x128x128) 3DLUTs where madVR uses trilinear interpolation to generate the missing values, in addition to the full 8bit (256x256x256) lut. I personally have been using a 6bit 3DLUT for awhile, the results are very good, but on average I've measured a yCMS created 6bit 3DLUT having 0.5 worse dE values than a full 8bit 3DLUT with my i1pro rev. D. The difference is indeed nearly invisible (gamma correction aside), but with modern GPUs $80+ having zero performance issues with the full 16.7million color 3DLUT it's more of case of why not? The more significant issue with the current implementation is not gamut correction with 6bit 3DLUT, but rather gamma correction where the difference between 64 points and 256 points (non-smoothed) can be extreme when attempting to represent non-power-curves like REC709 or sRGB in the lower tones.


You simply shouldn't be using the 3D tables to do the per channel curves. That's why any ICC device link worth it's salt has per channel curves as well as the 3D table. For the 3D table to be a reasonable prospect, it needs to be in a colorspace that best leveraged its advantages, while minimises its limitations. So typically the color mapping would be designed to be in a perceptually
linear space, with the input curves (or even more sophisticated transforms if you use ICC V4) serving to transform between the device space and the 3D mapping space.


That said, there is a lot of room for improvement in 3dlut creation. All yCMS created 3dluts take into account are white point, primary colors (100% Red/Green/Blue), and IRE grayscale, making the resulting correction extremely basic and likely worse than an Argyll created Matrix/Shaper ICC profile. This no doubt hurts the potential accuracy of a 6bit 3DLUT.


One of the things that I see as being rather suspect, is the lack of clear distinction between a device profile (which relates the device color values like RGB to device independent color values such as XYZ or L*a*b*), and a device link (which transforms from one device space to another, such as REC709 to display RGB).


As I've been saying for the past year or so on Doom9, what would be amazing is if ArgyllCMS colprof could be modified to create a madVR-compliant 3DLUT instead of a ICC profile during a normal ArgyllCMS calibration workflow. Graeme would modifying your existing cLUT code to output a 6bit(?) 3DLUT interest you personally at all (since nobody else has been interested...)? What we are really lacking is a high quality CMS like ArgyllCMS that takes a full set of measurement patches into account so we can actually get high quality results with less data points in the 3DLUT. The work nand chan has been doing is a step in the right direction, but an official ArgyllCMS solution which meets your quality standards and approval would be so much better. madshi as said a few times already that he'd love to be able to make 6bit 3DLUTs the default in madVR if the quality improved.

This doesn't seem the right approach at all to me. I'd strongly suggest instead implementing direct support for ICC device links in the video display path, taking as a model how ArgyllCMS's cctiff works. You don't even have to do any work as a first cut to see if the quality of transform is of the right order - take a single frame and use the whole of the Argyll toolchain to make a video standard to device RGB transform, and apply it to the frame using cctiff. I'm quite prepared to investigate any quality issues with this (as I've done in the past). This approach leverages established color management tools, and minimises a lot of duplication and rediscovery of how not to do things :-)

[nand chan]

From what I can see of it, it's all a bit weird - a very round about, cumbersome and inflexible approach to dealing with video color management, and an awful lot of re-inventing the wheel.

Re-inventing the wheel is not always a bad thing - if the existing wheels become too large or complex, it can make sense to start a “fresh”, application-specific, limited approach.

You simply shouldn't be using the 3D tables to do the per channel curves. That's why any ICC device link worth it's salt has per channel curves as well as the 3D table. For the 3D table to be a reasonable prospect, it needs to be in a colorspace that best leveraged its advantages, while minimises its limitations. So typically the color mapping would be designed to be in a perceptually
linear space, with the input curves (or even more sophisticated transforms if you use ICC V4) serving to transform between the device space and the 3D mapping space.

Or you increase the bit depth to compensate.

One of the things that I see as being rather suspect, is the lack of clear distinction between a device profile (which relates the device color values like RGB to device independent color values such as XYZ or L*a*b*), and a device link (which transforms from one device space to another, such as REC709 to display RGB).

I personally think one should draw a line between data information and usage information. So, for example, while it makes sense to store, say, the input primaries, or the color encoding, or bit depth - I don't think it makes sense to store things such as which display it's used for, or what the applied transformations are.

Such things, in my opinion, belong in the hands of the usage implementation, and not the data format. If different implementations, for example a monitor calibration system, want to store additional meta-data (which file means what, which display they're associated with), they should introduce their own structures.

Encoding such trivial “information” into the format itself is, in my opinion, more of a waste of space than additional bit depth.

This doesn't seem the right approach at all to me. I'd strongly suggest instead implementing direct support for ICC device links in the video display path, taking as a model how ArgyllCMS's cctiff works. You don't even have to do any work as a first cut to see if the quality of transform is of the right order - take a single frame and use the whole of the Argyll toolchain to make a video standard to device RGB transform, and apply it to the frame using cctiff. I'm quite prepared to investigate any quality issues with this (as I've done in the past). This approach leverages established color management tools, and minimises a lot of duplication and rediscovery of how not to do things :-)

Don't quote me on it but I believe the problem is that yesgrey, madshi, etc. simply don't have enough time to create a full implementation of the ICC profile, and they refuse to use existing libraries such as LittleCMS or ArgyllCMS to do the job, as that would not only take the fun out of it but also deny a learning experience, credit/license and ultimately defeat the point.

So they decided to create their own, stripped down format instead, and it's up to programmers such as me who have both time and no obligations to reject third party code, to create the middleware / bridgeware.

[nand chan]

Version 0.13
This is a bug fix release, mainly to fix a blatant issue in the CalFile parser that was introduced last version. It's a shame I didn't notice it sooner.

Download link (v0.13) (http://www.mediafire.com/?ebf348k5ksjt9w3)

Changelog:

Version 0.13:
* ICC profiles without added primaries information no longer get rejected (the color space gets set as “unknown”)
* Calibration files no longer have swapped primaries, bug introduced in version 0.12
This fix affects: CalFile(), applycal, imagecal and --calibrate in make3dlut

[madshi]

From what I can see of it, it's all a bit weird - a very round about, cumbersome and inflexible approach to dealing with video color management, and an awful lot of re-inventing the wheel.
The basic idea is that having a very big 3dlut allows us to use as complex a calibration math as we want, without affecting rendering performance. Furthermore it allows us to strictly separate calibration programming from video rendering programming. Personally, I've more than enough on my hands just with the video rendering, I don't really want to get into calibration programming too much. So I very much the approach to just use a 3dlut in my renderer and rely on external tools to do the math stuff in offline 3dlut creation tools.

I'd strongly suggest instead implementing direct support for ICC device links in the video display path, taking as a model how ArgyllCMS's cctiff works.
I'd have to write all that as GPU pixel shaders, and I doubt it would run fast enough on any current Intel GPU for 1080p60. Especially considering that I want to use the GPU shaders for other things like chroma upsampling, scaling, sharpening etc etc, too.

And what happens if we aim for a "perfect" calibration? E.g. if we do a couple hundred (or even thousand) measurements and try to achieve perfect correction based on those measurements? Considering all the funny things like too small/big display gamuts, out of gamut source data, perception based stuff etc, the math to achieve "perfect" calibration could quickly become very complicated. Too complicated to run it in realtime on current Intel GPUs at least, I think.

[nand chan]

The basic idea is that having a very big 3dlut allows us to use as complex a calibration math as we want, without affecting rendering performance. Furthermore it allows us to strictly separate calibration programming from video rendering programming. Personally, I've more than enough on my hands just with the video rendering, I don't really want to get into calibration programming too much. So I very much the approach to just use a 3dlut in my renderer and rely on external tools to do the math stuff in offline 3dlut creation tools.

But that doesn't conflict with what he suggests - doing the calculations offline as you mentioned and storing the result in a small RGB -> RGB link profile (ICC), the same as a .3dlut would. His only gripe is with you re-inventing the format when the ICC profiles already exist.

I'm not quite sure I agree with his philosophy though, since implementing such a huge and complex standard as ICC profiles takes a lot more work than re-inventing some simple format.

I'd have to write all that as GPU pixel shaders, and I doubt it would run fast enough on any current Intel GPU for 1080p60. Especially considering that I want to use the GPU shaders for other things like chroma upsampling, scaling, sharpening etc etc, too.

And what happens if we aim for a "perfect" calibration? E.g. if we do a couple hundred (or even thousand) measurements and try to achieve perfect correction based on those measurements? Considering all the funny things like too small/big display gamuts, out of gamut source data, perception based stuff etc, the math to achieve "perfect" calibration could quickly become very complicated. Too complicated to run it in realtime on current Intel GPUs at least, I think.

It would be the same as, say, a 6-bit .3dlut. If those run on Intel GPUs in realtime, then an ICC link profile would run too.

[madshi]

So an ICC link profile can basically contain a 6bit input 16bit output 3dlut? I have no knowledge about ICC at all, so I don't really have any clue.

[nand chan]

So an ICC link profile can basically contain a 6bit input 16bit output 3dlut? I have no knowledge about ICC at all, so I don't really have any clue.

If I've interpreted Graeme Gill's posts correctly, yes.

Ps. Another disadvantage of using .icc profiles is that it might confuse people who are attempting to use the more common /device/ profiles instead of a link profile - since they all have the same extension they might not know the difference, or struggle in figuring out how to create a link profile before settling for some inferior solution.

By forcing .3dlut, we have people actually being forced to search for .3dlut programs in particular, which would be our implementations - and there's no confusion between ICC profiles.

[IceB]

Hi.
You are doing a great job - keep on !

I am trying to generate the 3DLUT file using the LinkICCGUI and getting a following error on Win 32 when i press generate.
"Error: Could not find a part of the path
'C:\Users\#MyUsername#\AppData\Roaming\TI3Parser\dfdd367b-09e8-4086-96b8-373cd55f9f4d'."
There is no TI3Parser folder at AppData\Roaming at all.
My extracted 0.13 version is a C:\ .
Latest .NET Frameworks installed.
I choose both ICM and TI3 files created with dispcalGUI as linked on the guide.
Is there something I miss ?

Thanks for advance.

[nand chan]

Hi.
You are doing a great job - keep on !

I am trying to generate the 3DLUT file using the LinkICCGUI and getting a following error on Win 32 when i press generate.
"Error: Could not find a part of the path
'C:\Users\#MyUsername#\AppData\Roaming\TI3Parser\dfdd367b-09e8-4086-96b8-373cd55f9f4d'."
There is no TI3Parser folder at AppData\Roaming at all.
My extracted 0.13 version is a C:\ .
Latest .NET Frameworks installed.
I choose both ICM and TI3 files created with dispcalGUI as linked on the guide.
Is there something I miss ?

Thanks for advance.

Hmm, it might be a permissions issue? Are you on a restricted account maybe?

(It would surprise me though since even restricted accounts are supposed to have access to their appdata folders)

[IceB]

Hmm, it might be a permissions issue? Are you on a restricted account maybe?

(It would surprise me though since even restricted accounts are supposed to have access to their appdata folders)

It is strange.
I have tried it on 2 computers an the laptop - all with admin account returned the same exact error.
I also tried to copy the TI3Parser folder to the Appdata\Roaming folder - then pressing the "generate" button returns different error:
"Error: The system can no find the file specified "

What file does the app looking for and why at the first place it needs to look at AppData\Roaming\TI3Parser while this folder is not there.

Is there any installation process I should do or should I use the new TI3Parser as is after the extraction.

I tried it with different profiles and different machines - all of them return the same error - i must be missing something.

[nand chan]

It is strange.
I have tried it on 2 computers an the laptop - all with admin account returned the same exact error.
I also tried to copy the TI3Parser folder to the Appdata\Roaming folder - then pressing the "generate" button returns different error:
"Error: The system can no find the file specified "

What file does the app looking for and why at the first place it needs to look at AppData\Roaming\TI3Parser while this folder is not there.

Is there any installation process I should do or should I use the new TI3Parser as is after the extraction.

I tried it with different profiles and different machines - all of them return the same error - i must be missing something.

Come to think of it, try running “updateycms.exe” (you don't have to run this from the console, just double clicking it should be fine).

I just realized that I didn't add the dialog to automatically download yCMS to LinkICCGUI as well, I only added it to TI3ParserGUI.

[IceB]

Well done !

The “updateycms.exe” trick worked for my laptop.
I'll try to manage it on the PC later.

Thank you.

[Audionut]

Come to think of it, try running “updateycms.exe”

Can you add an option to select ycms from a user folder.

Seems a bit redundant needing to have ycms in a specific folder.

Cheers.

[nand chan]

Can you add an option to select ycms from a user folder.

Seems a bit redundant needing to have ycms in a specific folder.

Cheers.

I'll add something like it for ver 0.14. I'm also moving the temporary files into, well, %temp%.

Edit: Where do you think I should save the setting for this? 1. A registry key, 2. A config file in appdata or 3. A local configuration file in the \bin folder?

Advantage of 3. is portability, but 1./2. is invisibility and the ability to retain settings / have settings effect every copy of the program.

[Audionut]

Where do you think I should save the setting for this?

Personally, I would prefer 2 as it makes it easy to use the same config when upgrading.

I prefer 2 over 1 for this, as I think there shouldn't been things added to the registry unless really needed, although I not really that bothered as I don't install 9 million programs.

Also some people get really anal about their registries.

3 would work fine if you switch to an installer which can upgrade over a previous release.

edit: If you switch to an installer, you should provide an option to select the location of YCMS during install, or the option to download.
I'd personally be happy to have the installer copy my download of YCMS to it's own bin folder (in case I delete the YCMS folder for instance without thinking).

[nand chan]

I'm not a fan of installers, I think programs should be extract + run wherever possible - especially if they're command line programs (just add the bin folder to PATH and done).

There won't be such a thing as “upgrading over a previous release” since there's nothing to upgrade - you just download the new version and paste it over the old, replacing all of the files. If any new tool makes further use of this proposed configuration file, it can simply add the keys when it first tries to access them. A formal installer is not needed.

I personally don't see the difference between a file and a registry key since they both end up saved on your hard drive either way. I imagine a registry key has /less/ overhead than a file, though.

If I *do* create an installer (or rather a small setup script), I can provide the option to choose where you want to save the config file. (With the program checking all three on load, local files take the highest precedence).

[Thunderbolt8]

I need a 3dlut file (compatible to madvr) with input_range 16 235 and output_range 0 255. everything else should stay the same. how can I do this in highest quality possible?

[nand chan]

I need a 3dlut file (compatible to madvr) with input_range 16 235 and output_range 0 255. everything else should stay the same. how can I do this in highest quality possible?

Just a question, why don't you just set your monitor's output range to 0-255 in madVR?

[Thunderbolt8]

not sure whether I have done this already (output is set to PC levels 0-255). I need to correct a japanese BD which has wrong levels.

[nand chan]

not sure whether I have done this already (output is set to PC levels 0-255). I need to correct a japanese BD which has wrong levels.

I see. Well, do you mean that the BD compressed the levels twice? Eg. it stores 32-188 instead of 16-235? (Result: Everything in the BD looks more flat and gray than it should)

If so, you can fix that by expanding the levels using the following script:

# Set the properties properly
!Input_Range(Limited)
!Output_Range(Limited)
!Filetype(3DLUT)
!Input_Primaries(BT709)

!Pixel( {

Scale(-0.07306, 1.09)
Clamp(0, 1)

} )

For an explanation:

If we consider 0-1 to be 16-235, then 0-255 will be (-0.07306)-1.09.

As such, scaling with these two coefficients *stretches* the levels out to full range.

(Alternatively, you can just write !Output_Range(Full) since madVR does not check to see if the output is limited range as well, but this is not the ideal way to solve it as it will use slightly different scaling coefficients, those of 16-bit output instead of 8-bit).


If you want to do the opposite of this, that is, correct a BD which has mistakenly encoded 0-255 levels (blacks are crushed and whites are bleached, too high contrast), you would use the same script but change the line to Scale(0.06275, 0.92157).

Edit: Oh and if you're OCD about accuracy, use Scale(-0.07305936073059360730593607305936, 1.0913242009132420091324200913242).

[Thunderbolt8]

alright, but how to create the script, what tool to use in this case? used to do it with ycms with simply input_range & output_range stuff, but you said there seems to be a tiny more accurate way to do it.

[nand chan]

alright, but how to create the script, what tool to use in this case? used to do it with ycms with simply input_range & output_range stuff, but you said there seems to be a tiny more accurate way to do it.

Save it in a file anywhere (the official extension is .3ls), then run:

lutscript -i <path to file> -o Limited2Full.3dlut

[Thunderbolt8]

thanks, but madvr complains that the 3dlut file is missing primaries information (picture stays black)

I could copy & paste

Input_Primaries 0.640000000000000 0.330000000000000 0.300000000000000 0.600000000000000 0.150000000000000 0.060000000000000 0.31272661468101209 0.32902313032606195
Output_Primaries 0.640000000000000 0.330000000000000 0.300000000000000 0.600000000000000 0.150000000000000 0.060000000000000 0.31272661468101209 0.32902313032606195

from the ycms template file into the script you gave me, but would this be valid input for lutscript as well or accurate enough?


and how did you get the values for 0.07306..., 1.09... how to calculate this?

[nand chan]

thanks, but madvr complains that the 3dlut file is missing primaries information (picture stays black)

Did you forget the line !Input_Primaries(BT709) maybe? That line is supposed to add the correct primaries already. Either that, or your region locale uses commas instead of decimals (which wouldn't be the first time the issue has cropped up).

I could copy & paste

Input_Primaries 0.640000000000000 0.330000000000000 0.300000000000000 0.600000000000000 0.150000000000000 0.060000000000000 0.31272661468101209 0.32902313032606195
Output_Primaries 0.640000000000000 0.330000000000000 0.300000000000000 0.600000000000000 0.150000000000000 0.060000000000000 0.31272661468101209 0.32902313032606195

from the ycms template file into the script you gave me, but would this be valid input for lutscript as well or accurate enough?

Not quite, the LutScript format is !Input_Primaries(0.640000000000000, 0.330000000000000, 0.300000000000000, 0.600000000000000, 0.150000000000000, 0.060000000000000, 0.31272661468101209, 0.32902313032606195) (And the same for !Output_Primaries).

and how did you get the values for 0.07306..., 1.09... how to calculate this?

-0.07306... is (0 - 16) ÷ (235 - 16)
1.09... is (255 - 16) ÷ (235 - 16)

Explanation: The way to convert from 0-1 to limited range is: λi ↦ i × (235 - 16) + 16

However, we want this constant to be equal to 255 for the input 1 (while keeping the .3dlut in limited space since that's what madVR works with). In other words, the input 235 (which gets mapped to 1) should output 255 - thus fulfilling our desired goal.

So we want to introduce a scaling function ε so that λ(ε(0)) = 0:

ε(0) × (235 - 16) + 16 = 255 | -16
ε(0) × (235 - 16) = (-16) | ÷ (235 - 16)
ε(0) = (-16) ÷ (235 - 16)
ε(0) = (-16) ÷ (219)
ε(0) ≈ -0.07305936073059360730593607305936

This is now our lower limit.

For the upper limit, we proceed similarly, under the assumption that λ(ε(1)) = 255:

ε(1) × (235 - 16) + 16 = 255 | - 16
ε(1) × (235 - 16) = (255 - 16) | ÷ (235 - 16)
ε(1) = (255 - 16) ÷ (235 - 16)
ε(1) = (255 - 16) ÷ (219)
ε(1) ≈ 1.0913242009132420091324200913242

[Thunderbolt8]

Did you forget the line !Input_Primaries(BT709) maybe? That line is supposed to add the correct primaries already. Either that, or your region locale uses commas instead of decimals (which wouldn't be the first time the issue has cropped up).no, I copy & pasted the whole script box like you wrote it down. also tried it with comma, but that didnt change anything

edit: at the command line, I didnt use .3dl2 as output file extension, but .3dlut (this is what you wrote though). maybe thats the mistake? do I have to use .3dl2 and convert that file afterwards to a madvr compatible .3dlut file (if so, how?)

[nand chan]

no, I copy & pasted the whole script box like you wrote it down. also tried it with comma, but that didnt change anything

No, I mean going to Control Panel -> Region and Language -> Formats -> Additional Settings and setting the “Decimal symbol” to a period.

I'll see if I can definitively fix this issue by the next version.

edit: at the command line, I didnt use .3dl2 as output extension, but .3dlut. maybe thats the mistake? do I have to use .3dl2 and convert that file afterwards to a madvr compatible .3dlut file (if so, how?)

No, .3dlut is correct.

The line !Filetype(3DLUT) means it's a .3dlut file, had you written !Filetype(3DL2) or left it out completely, it would have been a .3dl2 file.

The !Filetype command already ensures compatibility with madVR.

[Thunderbolt8]

No, I mean going to Control Panel -> Region and Language -> Formats -> Additional Settings and setting the “Decimal symbol” to a period.

I'll see if I can definitively fix this issue by the next version.that might have helped a bit. Im also from germany, but use an english version of win7 (region set to ger, lg set to eng)

after that change, I dont get that error any more, but the picture stays black. video plays at least and theres also sound, but screen remains black.

[nand chan]

that might have helped a bit. Im also from germany, but use an english version of win7 (region set to ger, lg set to eng)

after that change, I dont get that error any more, but the picture stays black. video plays at least and theres also sound, but screen remains black.

Copying the script I posted earlier 1:1 and doing nothing else but running lutscript as described, it works fine here (and the levels get expanded appropriately).

Maybe there's still some locale issue going on. (I'm from Germany as well but I have my locale completely set to English formats)

I'll try uploading version 0.14 which has the en-US culture forced, see if it works.

[nand chan]

I'll try uploading version 0.14 which has the en-US culture forced, see if it works.

Link: http://www.mediafire.com/?o8wds588w66ot66

Change log for reference:

Version 0.14:
* LinkICCGUI now automatically updates yCMS on first run as well
* imagecal.exe clamps the levels to 0-255 properly now, before it would overflow
* Another attempt at fixing the locale bug, this time by forcing en-US explicitly instead of the invariant culture

[Thunderbolt8]

with this version, Im back to the no primaries information error message (after reverting the comma to decimal point change again in windows)

[nand chan]

with this version, Im back to the no primaries information error message (after reverting the comma to decimal point change again in windows)

Strange.

Do you think you could upload the output of inspect3dlut -i Limited2Full.3dlut --params?

[Thunderbolt8]

corretion, it now does work, but only if I keep this comma decimal point change applied. when I revert back to how it usually is (what I actually like to do, since no other programs have problems with it) then it doenst work.

[nand chan]

corretion, it now does work, but only if I keep this comma decimal point change applied. when I revert back to how it usually is (what I actually like to do, since no other programs have problems with it) then it doenst work.

Try this version?

http://www.mediafire.com/?ezjufd54ft6o9y6

I set the locale to “German” on my machine and it seems to work using this version. What I did is went through the code and forced every single .Parse() and .ToString() to use the en-US culture... silly culture issues.

If only .NET would just get rid of the whole globalization crap and standardize the sane English number formats we wouldn't have this problem. Oh well.

[Thunderbolt8]

it does work now, thanks :-)

strangely though, it only works properly with LAVsplitter, but not with haali.

with haali, the file doesnt seem to load up properly, playback wont start. in order to get it work with haali, what I needed to do is to start with an invalid 3dlut file and then switch over to that correct 3dlut file. then the file would start playing.

but when trying to get it to work with the correct 3dlut file directly from the start, theres just a black screen with no playback.

[nand chan]

it does work now, thanks :-)

strangely though, it only works properly with LAVsplitter, but not with haali.

with haali, the file doesnt seem to load up properly, playback wont start. in order to get it work with haali, what I needed to do is to start with an invalid 3dlut file and then switch over to that correct 3dlut file. then the file would start playing.

but when trying to get it to work with the correct 3dlut file directly from the start, theres just a black screen with no playback.

What I *THINK* might be causing the issue: Since madVR loads the .3dlut at the start, it causes a short delay before playback starts (same when switching to/from fullscreen). Maybe this short playback delay causes Haali to fail, but not LAV Splitter? I can't test it myself, Haali works fine here.

Does playback with Haali work immediately using any other .3dlut?

Also, what you could try is adding the line !Input_Depth(7) or !Input_Depth(6) which should reduce the file size quite dramatically (at the cost of a bit of quality).

But, if that works right away, it offers further proof for my theory - these are not as large, ergo faster to load.

[Thunderbolt8]

at which position to add this line?

I keep getting error: parsing failure: unexpected '!'; expected end of input (position XX) messages when trying to use the command line

edit: now haali does start up properly with every (valid) 3dlut file I feed to it. I uninstalled LAVsplitter this time, before that I might just have reinstalled haali after having installed LAV instead of proper uninstallaling LAV first. maybe that caused some problems.

[nand chan]

Oh, totally my bad - I forgot that you have to set the bit depth for each input channel, so it's !Input_Depth(7, 7, 7)

And good to hear about the Haali thing.

[nand chan]

In case anybody's interested, I'm working on a system to perceptually expand the gamut of a source file to the maximum representable gamut in your monitor space that won't clip any levels (or clip some of the blue, how far you expand it depends on your preferences).

In other words, you will be able to take a dull and gray source image and expand the gamut to something bright and exciting, depending on how much your monitor can represent.

This is basically like using an uncalibrated wide gamut monitor, but with one very important difference: the hue will stay the same, just the chromaticity increases or decreases. So basically, the red will be the same shade of red - just more vivid.

This solves the “plastic greens” problem of wide gamut monitors while giving you some of their benefits.

The main drawback? On the average wide gamut monitor, you can't get a significant increase since the blue primaries are often very close to the originals. For example on my U2410, I can increase the chromaticity by around 1.5% before I start clipping the blue channel. This is a barely detectable increase, only very slight at best. But if you have something like a projector with a much wider blue primary, you can benefit more from the increase.

Alternatively, you can choose to clip the blue channel slightly, which will begin shifting the hue of colors with a lot of blue saturation.

[fairchild]

Hey nand, I have a question regarding these tools which you are providing. Normally what I had been using was use the following coordinates which had been generated by my calibration software + meter (I used colorHCFR + i1DisplayLT) and entered these values into the MadVR yCMS configuration tab:

Gamut_Measurements 1
8.522508 0.659738 0.332989
27.843536 0.297610 0.595287
3.192925 0.149621 0.066393
46.652584 0.311805 0.327390

Grayscale_Measurements
20 1 3.572751 0.309652 0.327431
30 1 7.791107 0.312203 0.327093
40 1 13.703165 0.312105 0.327861
50 1 21.226063 0.312431 0.329232
60 1 31.068672 0.310517 0.327090
70 1 42.082134 0.312013 0.327941
80 1 53.747086 0.312122 0.327018
90 1 68.256332 0.311853 0.328607
100 1 81.266403 0.312070 0.327252

Is there an easy/quick way to just enter the above values into Agryll/your software and have it+your software generate a 3dlut to be used by MadVR or is there something else that needs to be done? (I'd rather not re-calibrate using Agryll since I like the way my set is already configured for both my HTPC usage + Blu-ray player + Cable TV.

If that's out of the scope of your tools, then that's understandable. I'd like to try an alternative to MadVR + yCMS.

Thanks!

[nand chan]

If that's out of the scope of your tools, then that's understandable. I'd like to try an alternative to MadVR + yCMS.

Thanks!

My tools use yCMS as a back-end for those kind of measurements. The only gamut mapping I've implemented myself is currently just a linear transition, it isn't capable of calibrating the gamma curve like that or anything.

[nand chan]

Previews of the technology I mentioned earlier:

80% source gamut (http://img9.imageshack.us/img9/6438/23008245.jpg)
90% source gamut (http://img52.imageshack.us/img52/2797/15644120.jpg)
100% source gamut (original image) (http://img853.imageshack.us/img853/6200/41836425.jpg)
110% source gamut (http://img215.imageshack.us/img215/5918/93329443.jpg)
120% source gamut (http://img685.imageshack.us/img685/5355/68964914.jpg)
130% source gamut (http://img820.imageshack.us/img820/2463/56361002.jpg)
140% source gamut (http://img560.imageshack.us/img560/5020/20171732.jpg)
150% source gamut (http://img703.imageshack.us/img703/8674/58569394.jpg)

Do note that these images are for the standard mode of my U2410, calibrated to D65 and colors will appear inaccurate on your monitor. It is just to demonstrate how the technology works. At 100%, the colors (as shown on my monitors) are exactly BT.709.

It is important to note, that (on my monitor):


Above 1.5% the blue channel starts clipping
Above 15.2% the red channel starts clipping
Above 41.8% the green channel starts clipping

[nand chan]

Version 0.14 release

Download link: http://www.mediafire.com/?m7nr5qgkdso6bdh

This build mainly highlights some significant back-end changes in terms of color space support. If I keep going at the current rate, I'll be able to replace yCMS and LittleCMS soon. The library is currently already capable of gamma encoding/decoding, color space conversion, chromatic adaption, chromatic desaturation etc.

In addition, I'm working on the proper GUI front-end which will replace the command line utilities / lutscript. (Apart from that, I need to re-work the way color spaces work in LutScript significantly. I may just scrap LutScript completely and switch to an existing scripting language like ruby or haskell and provide bindings instead)

Change log:
Version 0.14:
* LinkICCGUI now automatically updates yCMS on first run as well
* imagecal.exe clamps the levels to 0-255 properly now, before it would overflow
* Another attempt at fixing the locale bug, this time by forcing en-US explicitly instead of the invariant culture
* Gamma encoding for values under 0 clamps the values now (instead of resulting in NaN or Infinity)
* Fixed a very slight error in the predefined BT709 primaries
+ LutScript scripts can now be directly dragged onto lutscript.exe instead of having to run it from the console
You can also assign the program as default program for the .3ls file extension, so you just need to double click it
+ Added new gamuts (usable in LutScript):
BT709 (aliases: HDTV, sRGB, scRGB)
ROMM (aliases: ProPhotoRGB)
AdobeRGB (aliases: AdobeRGB98)
AppleRGB
NTSC (aliases: NTSC53, FCC1953, BT470M)
NTSC87 (aliases: SMPTEC, SMPTE_C, SMPTE_RP_145, SMPTE_170M)
SECAM (aliases: PAL, EBU_Tech_3213, BT470_B)
AdobeWide
CIE1931
ACES
+ Chromaticity expanding in the L*Ch(ab) color space is now supported
+ Chromatic adaptation between two different white points is now supported
+ Separated the white point component from the color space, added more default white points:
A, B, C, D50, D55, D65, D75, E, F1, F2, F3, F4, F5, F6, F7, F8, F9, F10, F11, F12
+ Added a function to calculate the white point from a color temperature
? Note: The above four features are not yet available in LutScript or any program

[nand chan]

Going for a double release since I finished this version up much, much faster than anticipated:

Version 0.15 release

Download link: http://www.mediafire.com/?rowlp5bf44y2205

This build mainly implements the features added in 0.14 into LutScript, doing so by completely rewriting the LutScript engine. Basically, LutScript is now a derivative of C# (some things are modified, for example ! in function names gets changed to _, and a few global constants have been added eg. 3DLUT for !Filetype). What this means is that you can basically extend its capabilities with standard C#. The plaintext of a LutScript file (.3ls2) is stuck inside a code provider's constructor and executed directly, so any sort of variables or other processing logic you declare in it get executed as well. The main syntactic change is that top-level functions (starting with a !) need a semicolon (;) after them, eg. !Filetype(3DLUT); and !Pixel()'s functions need to be separated by a regular comma, eg. !Pixel( GammaDec(2.4), GammaEnc(2.2) );

To complement this, a new filter has been added named Code(Func<ColorTriple, ColorTriple>) which basically provides you with the possibility of writing your own transformation logic in C# source. The !Pixel function has been rewritten and is now !Pixel(params ITransformationProvider<ColorTriple>[] filters), which is a fancy way of saying you can give it a bunch of filters as arguments.

Other than that, the main new filters this release highlights are ChromaticAdapt(ColorTriple source, ColorTriple dest, [ScalingMode mode]) and ChromaticScale(double factor, ColorTriple illuminant).

Using these filters, it's possible to, for example, desaturate or oversaturate colors, which is useful when moving into a large color space (where saturation wouldn't cause clipping).

Here's a new example of what a LutScript file looks like:

!Filetype(3DLUT);

!Pixel(
GammaDec(2.2),
ColorDec(ColorSpace.BT709),
ChromaticScale(1.3),
IccSingle("monitor.icm")
);

This creates a madVR-compatible .3dlut that decodes from BT.709, oversaturates by 130% (increases the chromaticity) and decodes back into the color space defined by my monitor's ICC profile.

Full changelog:

Version 0.15:
! LutScript completely re-worked, instead of creating a DSL from scratch it is now (nearly) syntax-conform C#.NET
This, in practice, doesn't really change much (as far as the scripting goes), but there are a few syntax differences
Most notably, semicolons are required after every command, the { } blocks have been removed from !Pixel, and
individual filters are separated by commas instead, eg. !Pixel( Grayscale(), Inverse(), Scale(0.1, 0.9) );
The new preferred extension is .3ls2
+ The “old” LutScript behavior can still be obtained by appending the --legacy flag to lutscript.exe. If you open
a file ending with .3ls instead of .3ls2, this flag is appended automatically.
+ New LutScript function: ChromaticAdapt(ColorTriple source, ColorTriple dest, [ScalingMode mode])
mode can either be ScalingMode.Bradford, ScalingMode.VonKries or ScalingMode.XYZ - Bradford is the best/default
+ New LutScript function: ChromaticScale(double factor, ColorTriple illuminant)
This will scale the chromaticity in L*Ch(ab) space by a constant factor
+ New LutScript functions: sRGBGammaEncode() and sRGBGammaDecode()
+ New LutScript function: Code(Func<ColorTriple, ColorTriple>), can be used to code your own transform filters, eg:
Code((input) => input * 0.7)
~ The LutScript functions dealing with primaries have changed their signature to (ColorSpace c) instead of a bunch of
double values. If you want to create a color space from the x/y chromaticity values, you can do so using the class's
default constructors, for example: new ColorSpace(0.64, 0.33, 0.30, 0.60, 0.15, 0.06, Illuminant.D65)
+ Added “smart” versions of ChromaticX / ColorX which attempt to guess the required color space based on the output
of the previous filter, the primaries of the 3dlut, or failing that, BT.709
For example, !Input_Primaries(ColorSpace.BT709); !Pixel( GammaDec(2.2), ColorDec(), ColorEnc(), GammaEnc(2.2) );
+ All standard colorspaces and illuminants are now readily accessible from within LutScript, as ColorSpace.<name> or
Illuminant.<name>, eg. ColorSpace.sRGB or Illuminant.D50
+ A white point can be calculated from the temperature using Illuminant.FromTemperature(double temp)
* Exceptions are a bit more descriptive

[Thunderbolt8]

is the input line you gave me for colour correction

# Set the properties properly
!Input_Range(Limited)
!Output_Range(Limited)
!Filetype(3DLUT)
!Input_Primaries(BT709)

!Pixel( {

Scale(-0.07305936073059360730593607305936, 1.0913242009132420091324200913242)
Clamp(0, 1)

} )

still valid with the new version or has anything changed?

[nand chan]

is the input line you gave me for colour correction

<snip>

still valid with the new version or has anything changed?

If you keep the extension as .3ls, it's still valid. However, to take advantage of the new features (eg. chromatic expansion), you can change the extension to .3ls2 and modify the script slightly:

!Filetype(3DLUT);
!Input_Primaries(ColorSpace.BT709);

!Pixel(

Scale(-16.0 / 219.0, 239.0 / 219.0),
Clamp(0, 1)

);

Firstly, instead of just “BT709” it's changed to “ColorSpace.BT709”.

Take note of the semicolons (;) and commas (,) added. I'm trying to figure out how I can add these automatically so the user doesn't have to, but for now, that's the only significant change.

I've also removed the !Input_Range and !Output_Range tags because those are limited range by default either way. And since this version supports mathematical operators, I've replaced the lengthy constants by their mathematical representations instead. (The .0 is important though, otherwise it won't be done with floating point precision math but instead with integer precision)

Once I'm happy with LutScript I'll write up a real documentation / user guide so all of these questions will be unnecessary, but for now I'm keeping it open so I can still make changes before it's too late.

Once I bump the version to 1.0 I'm not going to make any more drastic changes like that.

[alph@]

Once I'm happy with LutScript I'll write up a real documentation / user guide so all of these questions will be unnecessary.
I wait patiently on documentation for benefit from the library of software that you share with us:thanks:

[vega22]

First of all, thank warmly Nand all these tools and all their efforts.
Now, a whimsical (and silly) question: I would like to manually alter some calibration data (namely the primary coordinates) before creating the .3dlut files through your tools. What is the best way to do it? Manipulate data in the .cal and/o ti3 files seems complicated. Maybe I should create a .3dlut for only gamut mapping using yCMS and my custon script, then merge this .3dlut with one creating from LinkICCGUI with only grayscale and gamma corrections? It´s possible? How?
The issue is to get a .3dlut to correct grayscale and gamma, but only correct the gamut interpolating one or two primaries, leaving the other untouched.

[Wile-E-Coyote]

Thank you for keeping your tools updated nand. Any chance you could also update your guides on haruhichan.com ? You said yourself there they were outdated and it's getting hard to sort the right settings in the comments.

[nand chan]

First of all, thank warmly Nand all these tools and all their efforts.
Now, a whimsical (and silly) question: I would like to manually alter some calibration data (namely the primary coordinates) before creating the .3dlut files through your tools. What is the best way to do it? Manipulate data in the .cal and/o ti3 files seems complicated. Maybe I should create a .3dlut for only gamut mapping using yCMS and my custon script, then merge this .3dlut with one creating from LinkICCGUI with only grayscale and gamma corrections? It´s possible? How?
The issue is to get a .3dlut to correct grayscale and gamma, but only correct the gamut interpolating one or two primaries, leaving the other untouched.

I don't quite understand why you want to alter the primaries of your calibration data. If you have an exact measurement available, use those, if you don't, just use a regular transform (without calibration).

If you just want the gamma curves, you could calculate those independently and export them to a 3x1DLUT file like a .cal file or #GammaCurve text file. Then you can do the primary mapping using standard color space logic, and perform the .cal correction on top of that (using LutScript).

Thank you for keeping your tools updated nand. Any chance you could also update your guides on haruhichan.com ? You said yourself there they were outdated and it's getting hard to sort the right settings in the comments.

Sooner or later, maybe

[vega22]

I don't quite understand why you want to alter the primaries of your calibration data


First, my LCD with only hardware calibrations (i use a little boost in "color" setting looking the best ratio for the Y between all colors):
http://img521.imageshack.us/img521/4061/precalibrado.jpg
http://img716.imageshack.us/img716/8002/precalibrado2.jpg
The gamut is unacceptable, highly saturated. Now, automatically corrected by yCMS:
http://img23.imageshack.us/img23/6008/calibradoycms.jpg
http://img846.imageshack.us/img846/7093/calibradoycms2.jpg
The gamut is unacceptable, highly unsaturated. And now, the yCMS gamut corrected manually:
http://img198.imageshack.us/img198/5706/calibradoycmsymanual.jpg
http://img210.imageshack.us/img210/6961/calibradoycmsymanual2.jpg
Although still far from optimal, this gamut is the only decent for watching movies.
The process of finding the right spots manually is slow and cumbersome. It seems that the algorithm implemented in the "Upsilon mixer" tries to find the sweet spots interacting with the probe. This seems a good direction. However, i think it's even better to implement a GUI with "XY" sliders or something, for manually locate the points in the most comfortable way.

[6233638]

So I finally had a chance to try this out. I followed your guide, let dispcalGUI create an "XYZ LUT + matrix" profile using the "Large testchart for LUT profiles" (396 patches) and left it alone for a couple of hours.

When trying to use LinkICCGUI I get this error:

http://i.imgur.com/DXaiI.png

Any ideas?

[nand chan]

So I finally had a chance to try this out. I followed your guide, let dispcalGUI create an "XYZ LUT + matrix" profile using the "Large testchart for LUT profiles" (396 patches) and left it alone for a couple of hours.

When trying to use LinkICCGUI I get this error:

Any ideas?

Post the .ti3 file in question. Judging by the error message, it seems to be in an incorrect format (eg. not an XYZ->RGB LUT).

[6233638]

Post the .ti3 file in question. Judging by the error message, it seems to be in an incorrect format (eg. not an XYZ->RGB LUT).

http://www.mediafire.com/?glwa374o1czy7z7

[nand chan]

http://www.mediafire.com/?glwa374o1czy7z7

Ah, I see what the issue is. Since you are using a spectrophotometer and not a colorimeter, it doesn't store the individual XYZ entries but the actual values at every wavelength. Understandably, yCMS can't work with these values at all.

What you can do is skip LinkICCGUI (which uses yCMS as well as LittleCMS) and use a LutScript:

!Filetype(3DLUT);

!Pixel(
ColorDec(ColorSpace.madVR),
IccSingle("your_monitor_profile.icm")
);

save as output.3ls2 and drag that onto lutscript.exe, the resulting test.3dlut can be used with madVR.

[6233638]

Thanks. I'm now getting:Error: System.Exception: [3] 'TI3ParserLib.ColorSpace' does not contain a definition for 'madVR'

at TI3ParserLib.LutScript.MakeLutFromString(String s)

at TI3ParserLib.LutScript.MakeLutFromStream(Stream s)

at TI3ParserLib.LutScript.MakeLutFromFile(String file)

at lutscript.lutscript.Main(String[] args)

I guess I'm probably supposed to be defining something for "ColorSpace.madVR" but I'm not sure what?

http://www.mediafire.com/?yj5gblrd677j3w1

Here's the ICC profile as well, if that helps. Once I get this working, I'll have to see what the results are like, and then possibly do a longer profile, and make one at 2.40 gamma for film.

Using the ICC profile it generated, I'm getting banding and some discolouration in the greyscale, but I'm hoping that's a result of using the (presumably 8-bit) video card LUT, and that madVR with yCMS will be better.

Another issue I've run into is that when using the ICC profile, it's raising the black level of my display, as if it expected to read something from it. I have an LED backlit LCD that turns the backlight off for black, so that's obviously less than optimal.

I have to change brightness from 50 to 48 to get black to appear black again, and that's obviously going to affect the whole range from black to white. Don't know if the same will also apply when using a 3DLUT file in madVR, but I assume so.

Black point correction was set to 0% in dispcalGUI, which I assumed would take care of this. I guess I need to have a look over the dispcalGUI documentation to see what all the black level options do. (I just followed your guide more-or-less to the letter)

[nand chan]

Oh wow, I totally forgot to upload version 0.16. This was done ages ago.

Download link: http://www.mediafire.com/?d8wmeqlx7dd378y

Changelog:
Version 0.16:
! Color spaces are now aware of their gamma functions by default
~ ColorEnc() and ColorDec() renamed to XYZEnc() and XYZDec(). The new ColorEnc() and ColorDec() functions apply gamma
encoding/decoding by default as well, for color spaces that support it
? Note: This does not apply to all color spaces, eg. ones which define no reference gamma curves, the only gamuts that
automatically apply gamma en/decoding are:
BT.709 (PPC 2.35), Apple RGB (PPC 1.8), sRGB/scRGB (sRGB curve), ROMM/ProPhotoRGB (ROMM curve),
HDTV (709 curve), NTSC87/SMPTE "C" (PPC 2.2), PAL (PCC 2.8), madVR (PPC 2.2) and SECAM/EBU Tech 3213 (PCC 2.8)
+ New gamut: madVR, this gamut uses BT.709 primaries with a 2.2 PPC, to match with what madVR inputs to the .3dlut
~ GammaDec() and GammaEnc() changed to accept an ICurveProvider argument.
+ New functions sRGB() and PPC(double gamma) give you an sRGB or Pure Power Curve. So, for example, GammaEnc(PPC(2.2))
would replace what was previously just GammaEnc(2.2), and GammaEnc(sRGB()) would replace sRGBGammaEncode()
+ New function L_(L_Mode standard) gives you an L* curve, L_Mode can either be Actual or Intent
- Removed sRGBGammaEncode() and sRGBGammaDecode()

Truth be told, I don't exactly know much about the black point compensation options in either LittleCMS or dispcalGUI. If you want to do some more research, maybe in the LittleCMS manual or mailing list, and maybe test some options, it would be a great improvement.

[6233638]

So, I think it just didn't like the ICM filename. Installed 0.16, renamed the LUT from the dispcalGUI generated one, and it seems to have worked. I'm betting it was the "cdm²" that was causing problems.

Still seems to be raising the black level considerably and introducing some discolouration near black. Will give it another try this weekend.

Looks good otherwise though, will need to verify with test patterns and see how it actually measures.

[nand chan]

Still seems to be raising the black level considerably and introducing some discolouration near black. Will give it another try this weekend.

That seems to be an effect done by LittleCMS, even with “black point compensation” disabled (it's disabled in 0.16. I tried enabling it, but it didn't seem to have any effect). I believe it's linked to the fact that the device doesn't have a perfect black level, so something that should be perfectly black in theory will get cut off to the minimum that the display supports.

What I tried doing to work around the issue, at the cost of accuracy near the blacks, is “warping” the curve near the bottom, bending it to fit within the monitor's envelope. This is what “LinkICCGUI” does internally, but it relies on yCMS's calculations, which doesn't currently support spectrophotometer readings.

I'll see if there's another way to get the desired effect, maybe by pre-scaling values to fit into the black level.

If you can work out the ratio of your monitor's black level to your monitor's white level (ie. its contrast), and pre-scale the XYZ values by that factor, it might have a desirable effect. This would be a simple command in LutScript: Scale(contrast, 1) where contrast is the ratio of black to white (a very small figure). This would be put in between IccSingle and ColorDec.

[6233638]

If you can work out the ratio of your monitor's black level to your monitor's white level (ie. its contrast), and pre-scale the XYZ values by that factor, it might have a desirable effect. This would be a simple command in LutScript: Scale(contrast, 1) where contrast is the ratio of black to white (a very small figure). This would be put in between IccSingle and ColorDec.Contrast is effectively infinite, the backlight turns off with black, so I'm not really sure what should be done. (this is a local-dimming LED backlit screen)

[nand chan]

Contrast is effectively infinite, the backlight turns off with black, so I'm not really sure what should be done. (this is a local-dimming LED backlit screen)

You're calibrating a display with dynamic contrast enabled? Doesn't that completely screw up accuracy? Either way, it might just be an issue with your sensor at that point. Spectrophotometers are notoriously bad for very-low-light measurements, it's possible that this is what causes the high inaccuracies around black when using stock LittleCMS settings. Or maybe LittleCMS simply isn't equipped to work with such low light levels. Or maybe it's a curve inaccuracy somewhere along the line. Who knows? (Maybe it's even an issue with the source file, eg. being tagged incorrectly)

[6233638]

You're calibrating a display with dynamic contrast enabled? Doesn't that completely screw up accuracy?It's not dynamic contrast, it's local dimming.

Dynamic contrast simply dims the whole screen with low APL images, and brightens it with high APL images.

LED local-dimming combines both the LC panel and backlight zones to greatly expand the dynamic range of the display. (improving black levels and shadow detail without changing the image)

Either way, it might just be an issue with your sensor at that point. Spectrophotometers are notoriously bad for very-low-light measurements, it's possible that this is what causes the high inaccuracies around black when using stock LittleCMS settings.Spectros can have issues with low light, but I have no issues using CalMAN measuring the kind of levels that are showing big errors in the greyscale.

I didn't get a chance to try calibrating things again. I think what I'm going to do next time is use it as a contact meter (on the screen rather than on a tripod) which I would prefer not to do, and then disable backlight scanning during the calibration which increases the light output from the display significantly, without introducing any colour shifts and should help with readings at lower light levels.

[nand chan]

It's not dynamic contrast, it's local dimming.

Dynamic contrast simply dims the whole screen with low APL images, and brightens it with high APL images.

LED local-dimming combines both the LC panel and backlight zones to greatly expand the dynamic range of the display. (improving black levels and shadow detail without changing the image)

I was under the impression that “dynamic contrast” is an over-arching term for any sort of technology in which the static contrast ratio is improved by adjusting the backlight in any way.

Spectros can have issues with low light, but I have no issues using CalMAN measuring the kind of levels that are showing big errors in the greyscale.

I didn't get a chance to try calibrating things again. I think what I'm going to do next time is use it as a contact meter (on the screen rather than on a tripod) which I would prefer not to do, and then disable backlight scanning during the calibration which increases the light output from the display significantly, without introducing any colour shifts and should help with readings at lower light levels.

Well, if the measured black level is very low, then I don't see why black detail would be removed when using LittleCMS. Have you tried my suggestion regardless? How large is your measured black and white level?

[6233638]

I was under the impression that “dynamic contrast” is an over-arching term for any sort of technology in which the static contrast ratio is improved by adjusting the backlight in any way.It's not the same thing really, though I can see why you might think that. Dynamic contrast is awful.

Well, if the measured black level is very low, then I don't see why black detail would be removed when using LittleCMS. Have you tried my suggestion regardless? How large is your measured black and white level?Black detail is not being removed, but the black level is being raised. (dramatically lowering contrast)

You cannot divide by zero. Black level = 0, White level = 100.

[Wile-E-Coyote]

I have this weird issue when using 3dluts with madVR.

So I calibrated and profiled my monitor with dispcalGUI and created a 3dlut with linkICC. The profile is loaded system-wide after creation.

When I launch a video (using madVR as renderer), it all looks nice. However when I go into madVR settings to set the 3dlut file as calibration, the dark areas of the image are waaaaaay to dark. It's most obvious when using a THX Optimizer-like tool. It's like there's no dark grey anymore. It's either black or light grey. This happens both when the auto-calibrate box is checked or unchecked.

Anybody else had this before? Any idea what could be the issue?

[6233638]

Well, I liked what this seemed to be doing for colour and gamma, though it definitely needed improved somewhat, but I definitely seemed to be noticing a lot more banding, though I never got around to doing a proper test.

From the beginning of Pitch Black:

Original:
http://img818.imageshack.us/img818/8171/originalh.th.jpg (http://img818.imageshack.us/img818/8171/originalh.jpg)

yCMS:
http://img687.imageshack.us/img687/7669/ycms.th.jpg (http://img687.imageshack.us/img687/7669/ycms.jpg)

ArgyllCMS:
http://img403.imageshack.us/img403/9822/argyllcms.th.jpg (http://img403.imageshack.us/img403/9822/argyllcms.jpg)


This is exactly the sort of problem you get when using too much data to create a LUT, as I warned about before.

Any ideas on what can be done to improve things?

[nand chan]

I find it nearly impossible to believe that ArgyllCMS will produce that result, what were the exact steps you took to get it?

Even with 2400 sample points I get nowhere near that amount of banding, I see practically no difference in quality to yCMS except for a few very minor tone differences (especially around the greens).

[Wile-E-Coyote]

I just got this error message after trying to create a 3dlut with LinkICC.

http://i285.photobucket.com/albums/ll68/Petiflo/f8e68a8a.png

First time I ever had this message. Any ideas?

[6233638]

I find it nearly impossible to believe that ArgyllCMS will produce that result, what were the exact steps you took to get it?

Even with 2400 sample points I get nowhere near that amount of banding, I see practically no difference in quality to yCMS except for a few very minor tone differences (especially around the greens).It's the same 396 patch LUT I made from my initial measurements, all I did was play the video.

[rurika]

Hi, nand chan
I can't generate 3dlut with LinkICC. It crashed when it generating 3dlut.

http://i.imgur.com/pHyeU.png

My system is Phenom II x6 1055T + Windows 7 x64.
It work fine when I used other PC to generate ( core2duo E6750 + Windows XP x86 or Phenom II x4 945 + Windows 7 x64 )

I send this program to my friend that has Phenom II x6 & it seem he had a problem like me.
So, I guess the LinkICC can't work with Phenom II x6.

Hope it will work with Phenom II x6.

[markanini]

nand chan, I followed you calibration guide I & II and I'm wondering now what would good for Flash video in Firefox or Chrome.

[nand chan]

Hi, nand chan
I can't generate 3dlut with LinkICC. It crashed when it generating 3dlut.

My system is Phenom II x6 1055T + Windows 7 x64.
It work fine when I used other PC to generate ( core2duo E6750 + Windows XP x86 or Phenom II x4 945 + Windows 7 x64 )

I send this program to my friend that has Phenom II x6 & it seem he had a problem like me.
So, I guess the LinkICC can't work with Phenom II x6.

Hope it will work with Phenom II x6.

Probably an issue due to the number of cores, I'll look into it if I ever get the time. I assume you can just use your other PC for now.

[Wile-E-Coyote]

I just got this error message after trying to create a 3dlut with LinkICC.

http://i285.photobucket.com/albums/ll68/Petiflo/f8e68a8a.png

First time I ever had this message. Any ideas?
Up. Anyone got an answer for me?

[BeNooL]

Up. Anyone got an answer for me?

I'd say you are doing too many measurements (ie too close to each other) and/or your meter is not precise enough.

Try lowering the number of measurements so that each patch is out of the error range of the meter or eventually edit the file and remove the line where the results are not in increasing order.

[IceB]

Nand, thank you for your work and the great tool.
I have used your previous calibration guide with the REC.709 suggestions.
During the weekend I have tried to recalibrate my LCD TV with the updated guide - gamma 2.4 from here : http://haruhichan.com/wpblog/?p=364 .
My rig is HTPC ( ATI 6450 ) -HDMI-> Yamaha RX-V2700 -HDMI-> LCD Philips 42PFL8404.
The calibration processed with DTP-94 (Monaco Optix XR) colorimeter with dispcalGUI + ArgyllCMS (Win7 32bit).
The problem is every time I have tried to calibrate my monitor with 2.4 gamma tone curve the picture is very dark.
As a result I can not see any details on the dark patterns both on pictures in windows and in MPC-HC with madvr.
Creating 3dlut calibration file for madvr improves the colors in the movies, but still very dark with no details on near black parts of the picture.
After several unfortunate tries I had to recalibrate all back to the REC.709 tone curve. As a result - the picture is great, the colors are bit less vivid comparing to the gamma 2.4, but all the near blacks are clear and well detailed with the created 3dlut calibration file in madvr and in windows as well.

Despite my countless calibration related :readfaq: readings, I can not understand why have you changed the guide from REC.709 to gamma 2.4 ? Would you please explain ?
What might be the reason for the explained issues with my calibration with 2.4 gamma tone curve ?
Am I doing something wrong or my TV is not compatible with the gamma 2.4 colors ?

:thanks:

[nand chan]

Nand, thank you for your work and the great tool.
I have used your previous calibration guide with the REC.709 suggestions.
During the weekend I have tried to recalibrate my LCD TV with the updated guide - gamma 2.4 from here : http://haruhichan.com/wpblog/?p=364 .
My rig is HTPC ( ATI 6450 ) -HDMI-> Yamaha RX-V2700 -HDMI-> LCD Philips 42PFL8404.
The calibration processed with DTP-94 (Monaco Optix XR) colorimeter with dispcalGUI + ArgyllCMS (Win7 32bit).
The problem is every time I have tried to calibrate my monitor with 2.4 gamma tone curve the picture is very dark.
As a result I can not see any details on the dark patterns both on pictures in windows and in MPC-HC with madvr.
Creating 3dlut calibration file for madvr improves the colors in the movies, but still very dark with no details on near black parts of the picture.
After several unfortunate tries I had to recalibrate all back to the REC.709 tone curve. As a result - the picture is great, the colors are bit less vivid comparing to the gamma 2.4, but all the near blacks are clear and well detailed with the created 3dlut calibration file in madvr and in windows as well.

Despite my countless calibration related :readfaq: readings, I can not understand why have you changed the guide from REC.709 to gamma 2.4 ? Would you please explain ?
What might be the reason for the explained issues with my calibration with 2.4 gamma tone curve ?
Am I doing something wrong or my TV is not compatible with the gamma 2.4 colors ?

:thanks:

Rec709 is simply incorrect. The black levels you are seeing are unnaturally bright. It's supposed to be as dark as 2.4. If 2.4 is too dark you can try 2.2 or maybe 2.1 instead, but definitely don't use Rec709.

The reason you're experiencing the problem is probably because your TV doesn't have very good detail in the dark regions in the first place. I personally can see the difference between (0/0/0), (1/1/1) and (2/2/2) on my monitor.

[IceB]

Rec709 is simply incorrect. The black levels you are seeing are unnaturally bright. It's supposed to be as dark as 2.4. If 2.4 is too dark you can try 2.2 or maybe 2.1 instead, but definitely don't use Rec709.

The reason you're experiencing the problem is probably because your TV doesn't have very good detail in the dark regions in the first place. I personally can see the difference between (0/0/0), (1/1/1) and (2/2/2) on my monitor.

I have resolved the problem.
The "Ambient light level" measurement was still configured on dispcalGui from my previous calibrations.
Since I have updated the software, the "Ambient light level" configuration was hidden in the new version.
I have disabled it through the advanced settings and re-calibrated the TV with XYZ LUT + Matrix with Gamma 2.4.
Using the ti3 Parser the 3dlut was created for madVR calibration settings.
I have tried several profiles with different brightness/contrast/tint settings.
The best results were received with the "Relative" gamma tone curve.
The movies are much more vivid and the colors are deeper now comparing to the Rec709 profile i used before.
Yet using the gamma curve 2.4 in madVR results a bit darker picture than I would like, but the 2.3 seems to be the sweet spot for me resulting stunning and polished rendering.
:thanks:

[Mama70]

Well, I liked what this seemed to be doing for colour and gamma, though it definitely needed improved somewhat, but I definitely seemed to be noticing a lot more banding, though I never got around to doing a proper test.

From the beginning of Pitch Black:

Original:
http://img818.imageshack.us/img818/8171/originalh.th.jpg (http://img818.imageshack.us/img818/8171/originalh.jpg)

yCMS:
http://img687.imageshack.us/img687/7669/ycms.th.jpg (http://img687.imageshack.us/img687/7669/ycms.jpg)

ArgyllCMS:
http://img403.imageshack.us/img403/9822/argyllcms.th.jpg (http://img403.imageshack.us/img403/9822/argyllcms.jpg)


This is exactly the sort of problem you get when using too much data to create a LUT, as I warned about before.

Any ideas on what can be done to improve things?

The problem is you are using a spectrophotometer (i1 Pro?) and it doesn't give accurate reading in low light measurements.

I have ColorMunki and i1 Display 2 and user ColorMunki only for creating correction matrix for Display2. Then I do the actual calibration with Display2. But still, it doesn't get the low light measurents well.

At first I didn't understand ti3-file is used only for the black end. I tried using MPC-HC's own ICC-CMS and black end seemed ok. I think MPC-HC's CMS uses LittleCMS and it also creates 3dlut internally.

Easy but not perfect solution was to use yCMS and only ti3-file. You don't get perfect calibration though.

Launch TI3ParserGUI.exe and open your ti3-file and you'll see what the problem is.

My grayscale values were:
0, XYZ, 0, 0, 0
1.7857, XYZ, 0, 0, 0
3.5714, XYZ, 0, 0, 0
5.3571, XYZ, 0.027882, 0.02763, 0.042054
7.1429, XYZ, 0.10489, 0.10941, 0.1317
8.9286, XYZ, 0.2645, 0.28805, 0.2925
10.714, XYZ, 0.4104, 0.43535, 0.45801
12.5, XYZ, 0.65471, 0.69213, 0.6966
etc.

The problem in my file is values for 1.7857, 3.5714 and even 5.3571 are too low so my colorimeter doesn't get any values or they are inaccurate. Because program sees to low values, it tries to raise the values in 3dlut, and there you go, gray instead of black.

Simple but not perfect and final solution is just to drop those inaccurate values. So I copy&pasted those values to madVR's yCMS and used these values for gray scale:
0, XYZ, 0, 0, 0
7.1429, XYZ, 0.10489, 0.10941, 0.1317
8.9286, XYZ, 0.2645, 0.28805, 0.2925
10.714, XYZ, 0.4104, 0.43535, 0.45801
12.5, XYZ, 0.65471, 0.69213, 0.6966
etc.

I just wish we could have a version that has an option to only use ICC-files and LittleCMS.

[nand chan]

I just wish we could have a version that has an option to only use ICC-files and LittleCMS.

Use:

!Filetype(3DLUT);

!Pixel(
ColorDec(ColorSpace.madVR),
IccSingle("your_profile.icc")
);

Save it as anything and drag/drop onto lutscript.exe

[Mama70]

Use:

!Filetype(3DLUT);

!Pixel(
ColorDec(ColorSpace.madVR),
IccSingle("your_profile.icc")
);

Save it as anything and drag/drop onto lutscript.exe

Thanks a lot! With SVP and 3DLUT picture is so lifelike.
Why don't we have televisions&projectors that support natively 3dlut :(

[nand chan]

Thanks a lot! With SVP and 3DLUT picture is so lifelike.
Why don't we have televisions&projectors that support natively 3dlut :(

Higher end computer monitors and projectors support reprogrammable 3DLUTs.

[Mama70]

Higher end computer monitors and projectors support reprogrammable 3DLUTs.

But you cannot import 3dlut from pc, can you? Which projectors supports this?

[6233638]

The problem is you are using a spectrophotometer (i1 Pro?) and it doesn't give accurate reading in low light measurements. With the correct software, the i1Pro is accurate down to fairly low light levels—about 0.5cd/m2, which is 11% grey at 2.40 gamma with white at 100cd/m2. The banding shown here is nowhere close to that—it's in the 25–55% luminance range, far outside where meter error comes into play. (P.S. this is another reason why I dislike automated calibration—I can see when a low-light reading is clearly wrong and discard it, automated software doesn't know that)

If I remember correctly, I also disabled the backlight scanning when doing that calibration, which brings the panel brightness up significantly higher, placing the error range down around 6% grey.


I have had far greater success running test patterns in MPC-HC with the madVR yCMS options open and manually editing the xyY values (co-ordinates are far easier to manipulate by hand than XYZ) for each point, taking measurements with CalMAN running on another screen.

It's far more time consuming than it ought to be—if I could simply enter the RGB value I want out rather than tweaking xyY input, I'd arrive at the correct values a lot quicker, but it gives accurate results without the banding that this introduces.

For what it's worth, even then, I won't give it data below about 20% or so. I calibrate that range using the display's own controls, because any changes below 20% or so in yCMS seems to introduce posterisation/discolouration. (not due to meter error)

I also have the option of profiling my i1Pro to a colorimeter, but find that it's unnecessary in most cases, unless I need really low-light readings. I have yet to find a display where you have control over the very dark areas of the picture requiring readings that low, which does not introduce banding, discolouration or some other kind of issue, so while it's nice to know what the display is doing there, you can't fix it without introducing bigger errors anyway. Picture controls are very coarse down near black, and it's obvious when something is wrong.


Perhaps I'll give this another try some time with a significantly reduced dataset. As I've said from the beginning, problems like this tend to arise from using too much data when profiling a display.

[kasper93]

Rec709 is simply incorrect. The black levels you are seeing are unnaturally bright. It's supposed to be as dark as 2.4. If 2.4 is too dark you can try 2.2 or maybe 2.1 instead, but definitely don't use Rec709.

But if we adjust ambient light level (32 lux for me) in dispcal than rec709 is dark enough(little brighter than gamma 2.2), correct me if I'm wrong. I'm using 2.2 for movies since I have shitty monitor and gamma 2.4 is too dark for me.

Thanks for great tool. If I may suggest you should add information about LinkICCGUI "auto-calibration" function in yours madvr calibration guide. It's enabled by default, and in your guide you wrote to enable profile before watching what is wrong, because if auto-calibration is enabled we should disable any other color management, but yo know that :-)

P.S For rec709 default ambient light level is 1000 lux which is good for studio, but definitely not for home :)

[Thunderbolt8]

when having a lut script like this (.3ls2)

!Filetype(3DLUT);
!Input_Primaries(ColorSpace.BT709);

!Pixel(

Scale(-16.0 / 219.0, 239.0 / 219.0),
Clamp(0, 1)

);

here e.g. in order to correct wrong 16/235 BDs, how and where do I have to insert a line or formula to change the gamma value? how does the formula/line for the default gamma look like with this type of script and how do I have to modify it in order to increase or decrease gamma a little?

[tschi]

hi nand chan
I have an error with ti3parser :
System.execption : failed parsing the ti3 file : Sample_ID RGB_R .... not found
I use Argyll 1.4 and dispcalgui 1.0.7.7
here my ti3 file http://www.mediafire.com/?k847ll93lggxp3z
Thank you for your consideration

[killazys]

Hi nand,

Just to confirm the proper way of going about calibration for playback with madVR. (These are the steps I took.)

1. Calibrate using DispcalGUI to Chromaticity 0.312713x, 0.329016y, White level @ 120.0 cd/m^2, Black level as measured, Gamma 2.35 Relative, no ambient adjustment, 100% black output offset, no black point correction (for LCD monitor). Calibration quality set to High. Profile quality High, type = curves + matrix w/large testchart.

2. Use DispcalGUI to apply calibration.

3. Load resulting files into TI3Parser and uncheck "Automatic calibration (disable your video card's calibration for this).

4. In madVR, load the 3DLut file, and specify pure power gamma to 2.35.

Sound about right? The TI3Parser's automatic calibration worries me.

[dansrfe]

I have a couple of questions. But first I need to share my story. I have a 1920 x 1080, 60Hz laptop screen and a 2048 x 1152, 60Hz monitor which is connected to my laptop via HDMI. I have calibrated my laptop monitor with the built-in Windows calibration tool and I ended up simply decreasing the blue tone and green tone by a bit. That was it for that. I calibrated my external monitor with the built-in Windows calibration tool and I basically did nothing but decrease the gamma but 1 point or "registered mouse movement downwards" (since I don't know what units the scale was in). The rest of the calibration for the monitor (Dell SP2309W) was done via it's built-in OSD. I decreased the brightness and contrast from 75 (factory default) to 70. Then I set the colors to "Custom" and it allowed me to change the RGB values individually. I adjusted the red to 92 and the green to 95 and left the blue at 100. The monitor is in "Graphics" mode by default and I left it there.

I use madVR frequently to view Bluray material and I have been meaning to understand and figure out how to best implement a custom calibration solution with yCMS and 3DLUTs but I have come to the conclusion that I don't know nearly enough to even get started in calibrating both my screens in a correct, procedural manner with this technology.

My question is, what tools, devices, and materials do I need to correctly calibrate both monitors. What are the steps to doing so in the current situation and settings I have changed/"calibrated" with. And finally, what is the correct way to go about calibrating my panasonic plasma tv as well? (the tv was corrected only with the built-in OSD. Mainly the contrast and brightness were brought down to 65 in custom mode).

PS: What does it mean to or what are the differences between calibrating with graphics card software, hardware OSDs, Windows calibration tool and making an ICC file, or using some other calibration software?

Thanks! I would truly like to donate to this project if it works for me as well if the author is taking donations at this time.

[hulkss]

I'm a user of JRiver Media Center in an HTPC. Video playback is achieved with LAV Filters and madVR.

I want to apply color management using a .3dLUT in madVR to correct my projector. The short story to accomplish this is:

1. Use ArgyllCMS with dispcalGUI to generate a XYZ LUT + Matrix.
2. Install and load the ICC profile created with Windows 7 color management.
3. Use LinkICCGUI (from the TI3 Parser toolset) to generate the .3dLUT for madVR.
4. Setup madVR to use the 3D LUT.

My question is: If I uncheck the box (no auto-calibrate) in the GUI window shown below when generating the .3dLUT, will the .3dLUT be made to work properly along with the ICC profile loaded by Windows 7 (both the ICC profile and the .3dLUT in madVR will be active):confused:

http://img705.imageshack.us/img705/8360/linkiccgui.png

[kasper93]

IIRC it's exactly as you described, but the best way is to measure colors after correction and you will be sure.

[e-t172]

My question is: If I uncheck the box (no auto-calibrate) in the GUI window shown below when generating the .3dLUT, will the .3dLUT be made to work properly along with the ICC profile loaded by Windows 7 (both the ICC profile and the .3dLUT in madVR will be active):confused:

I'm also interested in the answer. I would suggest, however, that you do the opposite: configure madVR to disable Windows's gamma ramps during playback and then use a full 3DLUT. This is likely to give better results (less banding because the calibration is done before madVR's dithering).

[hulkss]

I'm also interested in the answer. I would suggest, however, that you do the opposite: configure madVR to disable Windows's gamma ramps during playback and then use a full 3DLUT. This is likely to give better results (less banding because the calibration is done before madVR's dithering).

OK, I'll try both methods this weekend. I won't install the ICC profile and I'll check the auto-calibrate box when I make the 3D LUT and the other way as mentioned above.

Update:
Seems to work fine both ways, I checked for banding with a full screen gray scale ramp (I'm using Intel HD4000 graphics with HDMI monitor connection @ 1920 x 1080). I prefer to install the ICC profile with Windows 7 for global gray scale correction. MadVR 3D LUT then corrects color gamut during video playback.

I posted instructions here: How To Get Perfect Video Color (http://yabb.jriver.com/interact/index.php?topic=78302.0)

[zoyd]

What are the specs on the .3dlut created this way, how many gamut points are remapped in madVR? thanks.

[MSL_DK]

What are the specs on the .3dlut created this way, how many gamut points are remapped in madVR? thanks.

Hello ... Are you the same zoyd (http://www.avsforum.com/u/7635732/zoyd) If you are, I thank you for the work with HCFR!

Are you planning gamut & gamma correction (.3 dlut generator) through HCFR? ;)

[zoyd]

Hi! yes that's me. :) No, sorry, I couldn't get close to what dispcalGUI/ArgyllCMS could do. I was just curios about 3DLUTs in general, especially for displays like mine which have good onboard CMS systems. I did some testing I might post about on avsforums with the Argyll profiles->3dlut->madVR corrections.

[MSL_DK]

Ahh .. Okay :-) I will look for your result :-)

[MvB]

hi nand chan
I have an error with ti3parser :
System.execption : failed parsing the ti3 file : Sample_ID RGB_R .... not found
I use Argyll 1.4 and dispcalgui 1.0.7.7
here my ti3 file http://www.mediafire.com/?k847ll93lggxp3z
Thank you for your consideration

Hi all,

i get the same error. I use the newset argyl 1.5 and the newest dispcalgui. Profile created was XYZ + matrix.

When i try to convert the created profile to .3dlut i get the same error. Does anybody know what i did wrong?
Thanks
MvB

[darkbasic]

hi nand chan
I have an error with ti3parser :
System.execption : failed parsing the ti3 file : Sample_ID RGB_R .... not found
I use Argyll 1.4 and dispcalgui 1.0.7.7
here my ti3 file http://www.mediafire.com/?k847ll93lggxp3z
Thank you for your consideration

Hi all,

i get the same error. I use the newset argyl 1.5 and the newest dispcalgui. Profile created was XYZ + matrix.

When i try to convert the created profile to .3dlut i get the same error. Does anybody know what i did wrong?
Thanks
MvB

http://files.linuxsystems.it/files/film_notte/ti3parser_error.jpg

Same problem here, Argyll 1.5.1.

Note that the .ti3 file does not contain the calibration curves because the monitor is hardware calibrated, as a consequence there is also no vcgt tag in the profile.

This is how I created the monitor profile (after the hardware calibration):

targen -v -d3 -f2000 film_notte
dispread -v -H -Ibw film_notte
colprof -v -qh -aX -S C:\Windows\System32\spool\drivers\color\AdobeRGB1998.icc -cmt -dmt film_notte

ti1: film_notte.ti1 (http://files.linuxsystems.it/files/film_notte/film_notte.ti1)
ti3: film_notte.ti3 (http://files.linuxsystems.it/files/film_notte/film_notte.ti3)
icm: film_notte.icm (http://files.linuxsystems.it/files/film_notte/film_notte.icm)

I hope it helps and the problem will be fixed soon :)

[IceB]

Hello nand chan and all.

I have some troubles with the TI3 parsing results with my config.
The stream is HTPC with Nvidia GT430 --> AVR Anthem MRX 300 --> JVC X30 PJ all properly connected with the HDMI cables.
The color space is YCbCr through the Nvidia CP with limited range 16-235.
The HTPC usage is for watching HD material only - Blu Ray mostly.
The software is MPC-HC with Core AVC as a preferred H264 decoder ( 16-235 IN/16-235 OUT ) / LavVideo decoder for VC1 with CUDA ( 16-235 ) / madVR as a rendered (16-235 ).
The projectors HDMI is configured for the limited range.
This way I have reached the best PQ with BTB/WTW .
I have calibrated the projector greyscale/gamma first with AVS HD 709 MP4 patterns through the MPC-HC with DTP-94 meter.
Then I have decided to run through the dicpcalGUI with Argyll CMS since my previous results with this calibration were good with my TV after the 3DLUT implementation on the madVR.
I have checked the black/white level as measured on dispcalGUI with no black point compensation and gamma 2.4.
The result seems to be fine for the desktop, but I do not care of that.
Once I have parsed the TI3 file and placed it on the madVR i have lost my "blacks".
The overall color seems to be fine but I have not made any data meters since the blacks were red.
Removing the calibration 3dLUT file from the madVR settings brings back the blacks immediately.
Googling the issue I have found the following "Chroma Subsampling" (http://www.glennchan.info/articles/technical/chroma/chroma1.htm) article which seems like a description for the reason of the issue.
It is the first time I am using the dispcalGUI with TI3 parser on the YCbCr color space. Is it the reason ?
I would like to use the 3dLUT file for the color correction with the MPC-HC and madVR on top of the gamma/greyscale manual correction .

Is there any workaround ?

Thank you in advance.

[alamagar]

I just got this error message after trying to create a 3dlut with LinkICC.

http://i285.photobucket.com/albums/ll68/Petiflo/f8e68a8a.png

First time I ever had this message. Any ideas?

Hi:

I'm getting the same error reported here. I've tried to profile with less measures: 238, but the error remains.

I've not seen the solution to this issue.
Any idea?

Thanks in advance.

[IceB]

Hi:

I'm getting the same error reported here. I've tried to profile with less measures: 238, but the error remains.

I've not seen the solution to this issue.
Any idea?

Thanks in advance.

I've got the same error previously with the new version of dispcalGUI.

It seems to be related to the white/black point drift compensation option that was implemented in the newer dispcalGUI.

In case this option is enabled, try to disable it and to recalibrate.

It solved this error in my case.

[alamagar]

I've got the same error previously with the new version of dispcalGUI.

It seems to be related to the white/black point drift compensation option that was implemented in the newer dispcalGUI.

In case this option is enabled, try to disable it and to recalibrate.

It solved this error in my case.

White and black point drift compensation are disabled. Tried to profile disabling "Black Point Compensation" in profile settings with same error.
any other idea?

[alamagar]

White and black point drift compensation are disabled. Tried to profile disabling "Black Point Compensation" in profile settings with same error.
any other idea?

I solved the error by chnaging the Reference Control in Calibration Settings from Daylight to Blackbody.

Perhaps I misinterpreted the meaning of this setting. Does It refer to the ambient light when you are calibrating/profiling or the intended ambien light when you usually watch movies?

[James Freeman]

What in the world is this???

http://i557.photobucket.com/albums/ss18/ilya-v/Other/3dlutFail_zps373571bc.jpg~original

[kasper93]

You shouldn't care and you shouldn't use this tool. Here is a proper way to make 3dlut for madVR http://www.avsforum.com/t/1471169/madvr-argyllcms/

[markanini]

I know this is an ignorant question but here goes: Why can't MadVR have the option of quick-dirty remapping to 601/709? My monitor is already calibrated to sRGB through DispcalGUI + Spyder colorimeter.