Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
22nd May 2010, 22:33 | #1 | Link |
Registered User
Join Date: Sep 2004
Posts: 1,295
|
yCMS - Color Management System
Introduction:
yCMS is a Color Management System (CMS) that uses 3-dimensional lookup tables ("3D LUTs") to perform the conversion between the input and output color representations. You can read a little more about CMS here. The idea of using 3D LUTs was due to the intensive computing that needs to be performed when converting video data between different color spaces. When using 3D LUTs all computing is performed offline during its creation. The computed 3D LUTs (three, one for each component) are then written to a file which can later be used, when playing videos or viewing photos, simply by mapping the input data through the 3D LUTs. This way, the conversion can be performed at runtime using lower computational power and leaving more CPU available for other tasks. Since all the computing is performed offline its precision can be increased by using 64bit floating point (per component) through out all the conversion chain. Features: - Color space conversion - Gamut correction - Chromatic adaptation - Gamma correction - HD, SD, xvYCC, sYCC & Computer video standards - Handles R'G'B', RGB, Y'CbCr and YCbCr video data - 6 bit (64x64x64), 7 bit (128x128x128) or 8 bit (256x256x256) input bit depth 3D LUTs - 8 or 16 bit output bit depth 3D LUTs - Full 64 bit FP processing chain - Both x86 and x64 versions Requisites: - Windows XP or newer - x86 or x64 CPU with SSE or above Changelog: Code:
v1.12 - 2012/11/07 - New command Denoise_Black_Level to denoise black level range - Changed Input/Output_Primaries range to allow negative numbers in R,G,B http://yesgrey.com/ycms.html Last edited by yesgrey; 13th November 2012 at 22:09. |
22nd May 2010, 22:33 | #2 | Link |
Registered User
Join Date: Sep 2004
Posts: 1,295
|
How to use yCMS:
(1)Read the detailed instructions inside the Manual and create 3DLUT file(s) that fit your needs. (2)Use your 3DLUT file(s) with madVR, t3dlut or rgb3dlut, the software options which support 3DLUT files. Below are instructions on how to use them. How to use madVR v0.66+: See instructions on this post. How to use madVR up to v0.61: madVR is a DirectShow video renderer created by madshi. If you don't know what madVR is or how to set it up and use it go here. madVR supports four different 3DLUT files. Two for watching HD media (PC and video levels) and two for watching SD media (PC and video levels). The selection between HD and SD files is automatic, according to media resolution. For "height > 576" or "width > 1024" it uses the HD files, otherwise it uses the SD files. You can manually choose between PC and video levels in madVR's settings dialog. You should create a 3DLUT file for each of the available options. To make madVR use them, just copy them into madVR's installation directory (where madVR.ax is contained) and name them properly. The file names expected by madVR are: Code:
HD media with PC levels: "hd - pc.3dlut" HD media with video levels: "hd - video.3dlut" SD media with PC levels: "sd - pc.3dlut" SD media with video levels: "sd - video.3dlut" note: madVR only supports Y'CbCr 8 bit input / R'G'B' 16 bit output 3DLUT files. The output is dithered to screen bit depth by madVR. How to use t3dlut/rgb3dlut: t3dlut/rgb3dlut are Avisynth plugins created by tritical. If you don't know what they are or how to set them up and use them go here. rgb3dlut is included in ddcc. You can use t3dlut/rgb3dlut for watching your media or for processing your video files using Avisynth. t3dlut supersedes rgb3dlut by allowing any combination of RGB and YCbCr input/output, while rgb3dlut only supports RGB output. rgb3dlut is faster because it has some assembly optimizations, so you will have to decide which would fit better your needs. Using t3dlut with a 3DLUT file should give you slightly more accurate colorspace conversions than using ColorMatrix. Due to the high number of possible configurations no example is offered at this time. I will add examples when requested. note: t3dlut supports 8 bit input / 8 or 16 bit output 3DLUT files, and all of the video formats combinations available in yCMS. When using 16 bit output 3DLUT files, the output is automatically dithered to 8 bit by t3dlut. Example using rgb3dlut with ffdshow: Instructions here. Last edited by yesgrey; 27th June 2011 at 17:19. |
22nd May 2010, 22:34 | #3 | Link |
Registered User
Join Date: Sep 2004
Posts: 1,295
|
Calibration example:
Here are the results after calibrating my projector (JVC M15) using yCMS and madVR. I started by measuring my projector primaries and white point at 75 IRE. For that I ran a video with 75 IRE R,G,B color fields, unchecking the 3DLUT option in madVR and selecting PC levels (my projector only accepts this): Code:
Red: x = 0.6662; y = 0.3280 Green: x = 0.2654; y = 0.6818 Blue: x = 0.1425; y = 0.0551 White: x = 0.3109; y = 0.3324 Code:
# Set input format Input_Format HD YCbCr 8 # Set output format Output_Format HD RGB_PC 16 # Gamut correction Gamut_Measurements 0.6662 0.3280 0.2654 0.6818 0.1425 0.0551 0.3109 0.3324 In the picture, the HD color gamut (BT.709) is the black triangle, and my projector's color gamut is the white triangle. After using gamut correction the results are slightly better. The green primary is nearer to the HD green primary, but the red and blue primaries were not corrected at all. I then measured my display's grayscale response (gamma curve): As you can see, it's not very nice... I tried again, but this time including gamma correction by using my display's grayscale response: Code:
# Set input format Input_Format HD YCbCr 8 # Set output format Output_Format HD RGB_PC 16 # Gamut correction Gamut_Measurements 0.6662 0.3280 0.2654 0.6818 0.1425 0.0551 0.3109 0.3324 # Gamma correction Grayscale_Measurements 30.0 6.29 40.0 11.82 50.0 20.12 60.0 30.89 70.0 44.31 80.0 62.52 90.0 83.82 100.0 100.00 As you can see in the picture, all primaries are in the correct positions now, over the HD primaries. Last edited by yesgrey; 23rd June 2010 at 11:17. |
23rd May 2010, 02:10 | #4 | Link |
Registered User
Join Date: Sep 2006
Posts: 2,197
|
will your version of the 3d lut be automatically included in madvr? if not, is there any reason why I should or shouldnt get it when I just want to watch blu-rays via my PC & hdmi connection to a tv? are there any advantages or disadvantages compared to when I don't use your version?
|
23rd May 2010, 09:03 | #5 | Link | ||
Registered Developer
Join Date: Sep 2006
Posts: 9,140
|
Congrats to yesgrey for (finally ) releasing yCMS!
Quote:
Quote:
Now if you're asking whether you should or should not use 3dlut files, that's a more complicated question. Let me give you a list of advantages and disadvantages of using 3dlut files vs. not using them in madVR: advantages: (1) yCMS can make your display show a perfect gamma response (2) yCMS can correct the gamut/primaries of your display (color management system) (3) when watching SD content, yCMS can convert the SD gamut/primaries to HD (4) madVR needs to do less math (faster performance?) disadvantages: (1) ever so slightly higher loading times (2) higher memory usage on your GPU (3) madVR needs to do more GPU memory accesses (slower performance?) In order to use advantages (1) and (2), you will need to have a meter (e.g. Spyder3 or Chroma5 etc) and a calibration software (e.g. HCFR or Calman). |
||
23rd May 2010, 10:16 | #6 | Link |
Broadband Junkie
Join Date: Oct 2005
Posts: 1,859
|
What exactly is yCMS doing when you input values for Grayscale_Measurements?
When I use it, it greatly brightens up the picture and reduces overall contrast considerably. Blacks turn gray. Whites get slightly crushed. It's like a light gray film gets placed over the video. From left to right. | madVR TV->PC | yCMS LUT TV->PC | yCMS LUT with IRE TV->PC | Funnily enough, yCMS LUT /w IRE 16-235 -> 0-255 makes what should be a 0 black, brighter than yCMS LUT 0-255 -> 0-255 (seen below)... From left to right. | madVR PC->PC | yCMS LUT PC->PC | yCMS LUT with IRE PC->PC | Code:
# Set input format Input_Format HD YCbCr 8 # Set output format Output_Format HD RGB_PC 16 # Grayscale IRE Measurements Grayscale_Measurements 10 0.486085 0.472363 0.497613 20 2.855507 2.751755 2.748106 30 7.040504 6.968139 7.044012 40 13.197106 13.020103 13.015621 50 21.422743 21.428576 21.564960 60 31.921806 31.852612 31.998531 70 45.422878 45.317829 45.406380 80 60.435123 60.174709 60.314659 90 78.802399 78.381996 78.620575 100 97.811592 98.124657 98.652870 # Display Gamut Measurements Gamut_Measurements 0.626 0.343 0.294 0.607 0.149 0.078 0.3127 0.3290 Last edited by cyberbeing; 23rd May 2010 at 10:28. |
23rd May 2010, 10:36 | #7 | Link |
And so it begins...
Join Date: Nov 2005
Location: Hannover, Germany
Posts: 64
|
@yesgrey
Good work! Would it be possible to input multiple gamut measurements? Right now we measure primaries/secondaries at 75% or 100%. Like calibrating greyscale there should be the possibility to input primary/secondary measurements for different IRE levels (e.g. in 10 IRE steps). I think this would higher the accuracy of calibration. Another question: in which color space do you interpolate the colors? |
23rd May 2010, 11:05 | #9 | Link |
Registered User
Join Date: Mar 2002
Location: Sofia, Bulgaria
Posts: 661
|
yesgrey,
I've just looked over the madVR thread and I can confirm that the selection of SD or HD luts is done not only based on the height but also on the width of the video - HD = width > 1024 or height > 576. It will be nice if you correct the definition in your second post |
23rd May 2010, 14:28 | #10 | Link |
Registered User
Join Date: Jan 2010
Posts: 479
|
I have the same problem what cyberbeing described.
Here is my input file built with copy-pasteing HCFR 2.1 measurements: Code:
# Source video format Input_Format HD YCbCr 8 #3DLUT output format Output_Format HD RGB_PC 16 # Gamut correction - measured native x;y coordinates Gamut_Measurements 0.657073 0.329413 0.231911 0.690324 0.140940 0.090552 0.311880 0.329555 #Gamma correction - measured grayscale luminance Y;Z values Grayscale_Measurements 10 1.469370 1.549970 1.633230 20 4.510128 4.779058 5.217576 30 9.946090 10.541579 11.273838 40 18.012747 18.977659 20.425848 50 28.926823 30.672146 32.918686 60 42.545696 44.938530 48.570366 70 60.392231 63.847328 69.020111 80 80.578819 85.209976 92.178696 90 105.731606 111.789406 121.205627 100 134.328751 141.941650 154.436478 The result is a brightened image with low contrast feeling. I also tried to use the videoStandard preset (I calibrated my monitor with sRGB curve): Output_Transfer_Function 0 0 Output_Transfer_Function 1 0 It is much better than it was with cr3dlut but it is still noisy. I could not see it from normal viewing distance but I had the "something isn't right" feeling, so I lean closer to the display and it is clearly visible. (There is a lot of noise on dark areas.) I made some screenshots with Output_Transfer_Function 0 0 enabled/disabled: Sometimes I can see more geometry details with slightly less color accuracy: 01 - 02 ; 11 - 12 But sometimes it causes some strange "sour cream" look: 21 - 22 ; 31 - 32 I think it works well with SD sources but still useless with HD materials. The image is too bright and there is still too much quality loss. (I didn't noticed this with PAL DVDs but that was only a quick test, I do not watch them anymore...) After all, we may need that 10 bit output (and I need a new display then...). Last edited by janos666; 23rd May 2010 at 17:00. Reason: affixing |
23rd May 2010, 19:53 | #11 | Link |
Registered User
Join Date: Feb 2010
Posts: 127
|
I only have one question: why do you offer two different options for setting the chromaticity coordinates of the primaries and the white point? I mean, we can set it with the option:
Gamut_Measurements redX redY greenX greenY blueX blueY whiteX whiteY and also, we can set it with: Output_Primaries redX redY greenX greenY blueX blueY whiteX whiteY What's the difference(s) between them? Thanks for your great work. Now I can get rid of ffdshow's levels filter. Finally!!! |
23rd May 2010, 21:01 | #12 | Link | ||
Registered Developer
Join Date: Sep 2006
Posts: 9,140
|
I'll let yesgrey answer most of the questions, because he can do that much better than me, but I know the answer to some questions, at least.
Quote:
Quote:
The remaining commands under the section "standards definition" are mainly meant to be used by people who want to convert a video from one standard/format to another. E.g. if you want to reencode video by upscaling SD to HD, you should also modify the primaries. The "Input_Primaries" should then be PAL or NTSC and the "Output_Primaries" should be HD. When reencoding video, you do NOT want to measure your display and use calibration commands. You want to define standard conversion parameters instead. "Gamut_Measurements" is for display calibration during media playback. This command only supports measured coordinates. "Output_Primaries" is for special purposes like e.g. reencoding video. This command mainly supports shortcuts to the well knowns standards (e.g. "0" for "HD"). But it also supports custom coordinates, in case a reencoding shall be done to a custom format, not directly supported by yCMS through a shortcut. |
||
23rd May 2010, 21:45 | #13 | Link | |
Registered User
Join Date: Feb 2010
Posts: 127
|
Quote:
|
|
23rd May 2010, 22:05 | #14 | Link |
Broadband Junkie
Join Date: Oct 2005
Posts: 1,859
|
According to the manual it should be | Red Luminance | Blue Luminance | Green Luminance | in the three IRE value columns. You shouldn't be putting X or Z values there unless I'm misunderstanding something.
From your example, 1.469370 should be Red Luminance, 1.549970 should be Green Luminance, and 1.633230 should be Blue Luminance. If you have a good display, all three should have near identical values with very little deviance. Optionally you can also just list a single value (only using the first column) for overall Gray Luminance at a given IRE. @yesgrey Could the overly bright problem be because you aren't applying any sort of Black Point Compensation with yCMS? I've also noticed yCMS only allows for 10 IRE values, but it would be nice if you could make it 11 (increments of 10 + zero) or even 21 (increments of 5 + zero), so you could have a value for 0 IRE to represent your Black Point. Last edited by cyberbeing; 23rd May 2010 at 22:17. |
23rd May 2010, 23:14 | #15 | Link | ||
Registered User
Join Date: Sep 2004
Posts: 1,295
|
Thanks.
Quote:
Quote:
yCMS can overcome the incorrect gamut/primaries of your display and let you watch the colors as accurately as possible. If your display's gamut is wider than the source video gamut (if it contains entirely the source video gamut) all colors are shown accuratelly, otherwise the colors that lay outside the display's gamut would be clipped. note: madshi answer was not wrong in a user's point of view, because it appears that the display's gamut/primaries are corrected by yCMS, but in reality they aren't. That could be done only within the display, and yCMS is external to it. To (3) I would like to add: (3) when watching SD content, yCMS can convert the SD gamut/primaries to HD, which should be done if you have a HD display. |
||
23rd May 2010, 23:15 | #16 | Link | ||
Registered User
Join Date: Jan 2010
Posts: 479
|
Quote:
I already figured it out that I should use the RGB values from HCFR. And yes, they are nearly identical values and the 100% IRE is close to my calibration target luminance. But the problem is still exist with the correct matrix. The image is too bright and it has low contrast... And I would ask the same thing: I want to add 11 IREs... Quote:
But you can take a look at my actual screenshots above. They meant to represent the loss in color quality but you may see some strange on dark areas. The main problem there that the overall image looks too bright and this brightening results in washed-out colors. (Ok, it is not a very serious mess -as it was with cr3dlut- but I can easily notice it on a S-PVA display...) PS: I almost forgot to say thanks for your work. Aside from my continuous "problem reports" I appraise it. So, may be it isn't perfect but better than anything else which I can use now... Last edited by janos666; 23rd May 2010 at 23:27. |
||
24th May 2010, 00:26 | #17 | Link | |||||||
Registered User
Join Date: Sep 2004
Posts: 1,295
|
Quote:
Quote:
Quote:
Both are on the ToDo list. Though, don't forget that in the second a perfect Y might run against a perfect xy, so it might ends up as an optional setting. Sorry, but i didn't understand your question. What did you mean with "interpolate colors"? Quote:
Quote:
Quote:
With only one saturation allowed there is an overlapping between the two commands, but we decided to still introduce this command now because it's more user friendly than the Output_Primaries. A big part of the users would not need (or dare ) to go past the Warning, so I must say that you are one of the braves... Quote:
The "Gamut_Measurements" command should always be used when the display's coordinates are measured, no matter what the source is. The source should be reflected only in the Input commands, and in none of the output commands. The commands relative to measures are all output commands, because you are measuring the output (display). Last edited by yesgrey; 25th May 2010 at 00:21. |
|||||||
24th May 2010, 01:49 | #18 | Link | |
Registered User
Join Date: Jan 2010
Posts: 479
|
Quote:
I made a 3dlut with these settings: Code:
Input_Format HD YCbCr 8 Output_Format HD RGB_PC 16 Output_Transfer_Function 0 0 Code:
[scaling settings] luma upsampler=Spline64 luma downsampler=Spline64 chroma resampler=Spline64 [output settings] video levels=0 use 3dlut=1 [trade quality for performance] don't use dithering=0 use 10bit luma buffer=0 use 10bit chroma buffer=0 disable anti-tearing fix=0 Code:
use 3dlut=0 I take these from a real Blu-Ray source (AVC_L4.1 @ 39.2 Mbps) to represent how this option kills the color detail: 21 - 22 I take these from a re-encoded 1080p source (x264_L4.1 @ 8706 kbps) to show that image is sometimes too bright: 31 - 32 And these comes from the Blu-Ray again to show them all (too bright scene with noisy dark spots and washed-out colors...): 41 - 42 And here is the result of these new settings: 51 - 52 Code:
Grayscale_Measurements 10 1.564805 1.551411 1.491997 20 4.668003 4.810841 4.791349 30 10.406371 10.604137 10.320132 40 19.016638 18.991844 18.722298 50 30.179890 30.871260 30.149662 60 44.580005 45.085168 44.541965 70 63.151944 64.109925 63.294323 80 84.184549 85.582548 84.539725 90 110.365791 112.272175 111.200465 100 140.120883 142.500616 141.767822 Last edited by janos666; 24th May 2010 at 02:07. |
|
24th May 2010, 07:44 | #19 | Link | |||
Registered Developer
Join Date: Sep 2006
Posts: 9,140
|
Quote:
All commands you'll ever need for madVR and video playback are in the first 3 sections of the manual (mandatory, optional, calibration). Please simple *forget* the commands in the "defining standards" section. Quote:
Quote:
(1) Full 3dlut processing (without Gamut_Measurements, though, since it doesn't seem to work properly yet). (2) 3dlut turned off. I expect to see the "noise in dark areas" in image (1), but not in image (2). Thanks! |
|||
24th May 2010, 09:27 | #20 | Link | |||
And so it begins...
Join Date: Nov 2005
Location: Hannover, Germany
Posts: 64
|
Quote:
Quote:
Quote:
|
|||
Tags |
cms, gamma correction, gamut calibration, ycms |
Thread Tools | Search this Thread |
Display Modes | |
|
|