gamut conversions through Avisynth ?

[pitch.fr]

hello world,

there's a new SAMSUNG projector, that's ISF(Imaging Science Foundation) certified and made under the supervision of Joe Kane(he's quite a video guru)

that's the SP-A800B(DLP 1080p), and as they say "Paramount, DreamWorks, ABC, and Universal Studios are among the professional users of this model."

http://www.projectorcentral.com/Samsung-SP-A800B.htm

it's one of the very few projectors(if not the only one) where you can change the gamut on the fly.

mostly, the SD movies are encoded through the REC601 matrix, and the HD movies through REC709.

but this is the theory...

in practice(and I know that sounds crazy), the mastering engineers still use this kind of monitoring displays :

http://www.soundandvisionmag.com/assets/image/2007/W49/1242007175956.jpg

the engineer is using a BVM CRT monitor(with SMPTE-C phosphores gamut)

this thing :
http://assets.sonybiz.net/products/BVM-A14F5M(img1).jpg
http://www.sony.co.uk/biz/view/ShowProduct.action?BIZ_SESSIONID=yndHL19ZmGTb7HJwGSbJyQRmYPJYqtv1GjHhgvYr6TzXZNjLbf26!-1724456250&product=BVM-A14F5M&pageType=Overview&category=BVM&site=biz_en_GB

so basically, SD and HD movies are not mastered on sRGB/HDTV gamuts as we've all been led to believe.

video monitors used in Europe :
SONY BVM-A14F5M (EBU phosphores)
SONY BVM-A20F1M (EBU phosphores)
SONY BVM-A32E1WM (EBU phosphores)
SONY BVM-L230 (LCD)
SONY BVM-L420 (LCD)

and in USA:
SONY BVM-A20F1U(SMPTE-C phosphores)


so basically, to see movies the way they were mastered, you'd need :

1)YCbPr decoding :

- Rec. ITU-R BT.601-5 for SD primaries (in theory only)
Y’ = 0,299R’ + 0,587G’ + 0,114B’ (601)

- Rec. ITU-R BT.709-4 for HD primaries (in theory only)
Y’ = 0,2126R’ + 0,7152G’ + 0,0722B’ (709)

2)display gamut :

- REC-601 SMPTE-C for SMPTE-C primaries (NTSC SD & HD, 90% of bluray discs as well, and some PAL DVD's)

- REC-601 EBU Tech. 3213 for EBU primaries (PAL/SECAM SD and european bluray discs)

- REC-709 HDTV (for US and EUR HDTV)


that's what my DLP projector looks like, when calibrated in sRGB(the gamut is identical to the HDTV gamut) :

http://pix.nofrag.com/a/3/c/deff50ab2a4b5c063c43e15a16155.png

and here's a gamut comparison :

http://www.homecinema-fr.com/forum/download/file.php?id=54227

so is there any AVS guru that'd be interested to find a way to get 1:1 colors with the way movies are being mastered in the first place ?

of course it'd have to 10bit if possible in order to avoid banding and stuff.

I know that sounds crazy, but that's the way it is.
this Samsung projector is the ultimate thing to watch movies "the way they were meant to be watched"(it supports sRGB/HDTV/EBU/SMPTE/etc.. gamuts).

surely enough, you could revert this colorspace hiccup in AVS ?

....and run it in realtime in ffdshow ? that'd be the shiznit :D

here's a list made by the french ISF CEO "Julien Berry", that lists which gamut has to be used for each Bluray movie :

http://www.wysios.com/jkp/gamut2.asp

from what he told me, here's what happens when using a display with sRGB/HDTV gamut :

for telecines that were made on an EBU monitor, only the green primary will be off.

for telecines that were made on an SMPTE-C monitor, the colors will be over-saturated.

when you read that most of the bluray's are made on SMPTE-C monitors, and that usually you end up with over-saturated colors on sRGB/HDTV displays.....that suddenly would make a lot of sense to fix the colors.....wouldn't it ?

here's a technical PDF about gamuts :
http://www.teksite.co.kr/osilo/down/Understandin%20color%20and%20gamut.pdf


and basically you can do this gamut conversion with a scaler :
http://www.google.com/search?hl=en&q=gamut+conversion&btnG=Search&lr=

its job is to convert gamuts, but that costs a hell lot of money(and usually only SDI input and DVI output).....so doing it in AVS in 10bit would be beyond words :)

some ppl have worked on it with PS scripts :
http://www.avsforum.com/avs-vb/showthread.php?t=912720

the idea would be to :

-do REC-601/REC-709 matrix conversions for SD/HD to get sRGB content
-do REC-601 SMPTE-C / REC-601 EBU Tech. 3213 > HDTV gamut conversions to get proper primaries on sRGB/HDTV displays :)

[pitch.fr]

an even easier solution would be to input the primaries locations(after calibration) and input them in AVISYNTH :
http://www.avsforum.com/avs-vb/showthread.php?p=11726737#post11726737

is that doable ?

here are the primaries locations on my sRGB CRT :
R:[0.658,0.328]; G:[0.334,0.619]; B:[0.150,0.070]

EDIT : another option might be to use ColorMatrix.
but it doesn't support either "REC-601 EBU" or "REC-601 SMPTE-C" :(

anyhow here's a BT.709>RGB32 conversion(through ffdshow)
+ the same with ColorMatrix(mode="Rec.601->Rec.709") added upfront :

http://thumbnails8.imagebam.com/915/73e4979147374.gif (http://www.imagebam.com/image/73e4979147374)http://thumbnails8.imagebam.com/915/0b83a59147373.gif (http://www.imagebam.com/image/0b83a59147373)
http://thumbnails8.imagebam.com/915/7778be9147377.gif (http://www.imagebam.com/image/7778be9147377)http://thumbnails8.imagebam.com/915/36e09b9147376.gif (http://www.imagebam.com/image/36e09b9147376)
http://thumbnails8.imagebam.com/915/a0f7ef9147379.gif (http://www.imagebam.com/image/a0f7ef9147379)http://thumbnails8.imagebam.com/915/30d22c9147378.gif (http://www.imagebam.com/image/30d22c9147378)

I would say that it looks A LOT better on my sRGB/HDTV calibrated displays :)

this is a US bluray source btw, so most likely telecined in a "REC-601 SMPTE-C" gamut ;)

hopefully I've convinced a few people now :D

[yesgrey]

some ppl have worked on it with PS scripts....but they are confusing "decoding matrix" and "display gamut" :
http://www.avsforum.com/avs-vb/showthread.php?t=912720

It's me who have started the AVSForum thread you refer, and no, I am not confusing "decoding matrix" and "display gamut".

It seems to be you who are confusing "encoding matrix" and "monitoring color gamut". The authoring process being monitored with a different color gamut, does not means that the colors are wrong. They could be correcting the colors on their monitoring displays the same way I suggested in that thread.

All the thread is considering the mastering process is monitored with displays the same color gamut of the encoding matrix; what you are saying is that it's not done that way. In fact, if the above correction in the monitoring displays is not performed, the correction should be a little different, but the math is all there, we just need to adapt it to this situation. I will take a look and update the thread accordingly.

When I started the thread I started using avisynth, but when I realized the correction must be done in the color linear space JohnAd suggested to use pixel shaders and mplayerc.
It seems to be the better option for now, you have it already done and working, why requesting it in avisynth?

[pitch.fr]

well many people speak about different ways to do it in this thread, sorry I didn't understand it this way :(

anyhow, great to have you here :)

tritical is improving ColorMatrix to allow this kind of CIExyz conversions on the fly in AVS/ffdshow :)

well, for one I only use Haali's Renderer, which is the de facto video renderer on PC to get smooth video....so these PS scripts are not usable.

and ColorMatrix will allow to input the primaries coordinates we get from colorimetry applications, so it will offer tailor-made gamut conversions between SMPTE-C > your own calibrated display for 1:1 colors(no need for an expensive scaler anymore).

well the ISF CEO and many other professionals are showing that the only way to watch bluray with 1:1 colors is to do REC.709 conversion, and watch it on an SMPTE-C gamut if it was mastered in the US, and an EBU gamut if it was done in europe.

these guys are calibrating broadcasting studios with Minolta CS colorimeters all over the world, and they are pushing this new Samsung as the ultimate projector because it supports any kind of gamut you could possibly want :eek:

even PLANAR does that now with their newest projectors as well.

you can find above a list with all the bluray that are supposed to be watched on an SMPTE-C gamut.

OTOH, the only thing you need to use this kind of PS script is 2D convolution, that some AVS plugins can do.....only problem is that converting to RGB in AVS is single threaded and very slow compared to ffdshow, and you also get those pesky "chroma upsampling bugs" :
http://forum.doom9.org/showpost.php?p=1137196&postcount=1868

[pitch.fr]

alright, so this PS script is really nice :eek:

http://www.avsforum.com/avs-vb/showthread.php?t=912720

...too bad that doesn't work with HR :(

here's my 19" iiyama CRT calibrated CIE :

http://thumbnails9.imagebam.com/931/c3ad2b9303768.gif (http://www.imagebam.com/image/c3ad2b9303768)

here's my XLS script with the primaries coordinates measured with an Eye One Display 2 colorimeter in Color.HCFR :

http://thumbnails9.imagebam.com/931/fdb6989303770.gif (http://www.imagebam.com/image/fdb6989303770)

and here's the correction from SMPTE-C to my CRT gamut, with REC.709 RGB32HQ HD content from ffdshow in VMR9.

left is untouched, right is corrected to match my CRT gamut.

http://thumbnails9.imagebam.com/931/4ad07f9303793.gif (http://www.imagebam.com/image/4ad07f9303793)http://thumbnails9.imagebam.com/931/8bfb8d9303796.gif (http://www.imagebam.com/image/8bfb8d9303796)
http://thumbnails9.imagebam.com/931/76e0769303773.gif (http://www.imagebam.com/image/76e0769303773)http://thumbnails9.imagebam.com/931/01b8319303776.gif (http://www.imagebam.com/image/01b8319303776)
http://thumbnails9.imagebam.com/931/b562359303778.gif (http://www.imagebam.com/image/b562359303778)http://thumbnails9.imagebam.com/931/7e556c9303781.gif (http://www.imagebam.com/image/7e556c9303781)
http://thumbnails9.imagebam.com/931/25028c9303784.gif (http://www.imagebam.com/image/25028c9303784)http://thumbnails9.imagebam.com/931/10f5e19303785.gif (http://www.imagebam.com/image/10f5e19303785)
http://thumbnails9.imagebam.com/931/2aedfd9303788.gif (http://www.imagebam.com/image/2aedfd9303788)http://thumbnails9.imagebam.com/931/3d12c09303791.gif (http://www.imagebam.com/image/3d12c09303791)

I'd say it removes the green cast on faces(my biggest problem at this point!), and it gives much more natural colors :cool:

too bad my red level is now lower, and because it's 8bit/pixel this could possibly introduce banding.

that's also too bad that I've never managed to get good results with Reclock in VMR9......HR is way smoother, so this is not a viable solution for me at this point.

there is some AVS plugins that can do 2D convolution :
http://www.google.com/search?hl=en&q=avisynth+convolution&btnG=Search&lr=

but as I said earlier RGB32 conversion in AVS is slow as hell(not multithreaded and/or optimized), and full of "chroma upsampling bugs".......except if tritical can make miracles happen :)

otherwise maybe you could create an .icm file ?
or a .cal file to be opened with dispwin.exe(part of ARGYLLCMS www.argyllcms.com) ?

ATi's CLUT is 30bit AFAIK :)

ARGYLL's coder is very knowledgeable and always willing to help :)

it's still 8bit/pixel, but better get proper 8bit corrected colors than greenish movies :D

[Wilbert]

but as I said earlier RGB32 conversion in AVS is slow as hell, and full of "chroma upsampling bugs"
If you feed interlaced material, you need to use

ConvertToRGB32(interlaced=true)

[pitch.fr]

ok, thanks for the tip but I've never managed to get RGB32 content out of AVS w/o chroma bugs.

here's a good test sample :
http://forum.doom9.org/showpost.php?p=1137196&postcount=1868

only HR and ffdshow(with "high quality YV12 to RGB conversion" checked) manage to do bug-free conversions AFAIK...

and HR/ffdshow BT.709 matrixes are also different.....ffdshow's is 1 notch greener, and 1 notch less blue and less red...

[Wilbert]

ok, thanks for the tip but I've never managed to get RGB32 content out of AVS w/o chroma bugs.

here's a good test sample :
http://forum.doom9.org/showpost.php?...postcount=1868
Ok. Post a script and a screenshot which shows these chroma upsampling bugs.

[pitch.fr]

Ok. Post a script and a screenshot which shows these chroma upsampling bugs.

http://img207.imageshack.us/img207/2374/blaud9.png

there is also some AVS commands to increase the chroma upsampling, but that didn't work either.

I'm not an AVS guru, so maybe I'm missing something :D

what is required is "progressive chroma upsampling", at this point this is not done :eek:

[Didée]

That's fine and all, but where's the bug you're talking about? There's nothing wrong in your pic. :)

When converting YUV to RGB, it's a matter of choice if you replicate the subsampled UV planes 1:1, or if you interpolate during resampling. The former is what Avisynth and ffdshow(default) are doing, the latter is what ffdshow's HQ=true does.

Both methods are valid, there is no "right" or "wrong".

Definetly, it is not a bug. A missing option, yes. But not a bug.

[pitch.fr]

oh ok thanks for the clarification Didée...well everyone calls it "chroma upsampling bug" :
http://www.google.com/search?hl=en&q=%22chroma+upsampling+bug%22&btnG=Search&lr=

it basically makes the red blocky to hell...and I'm not watching HD to get blocky reds :D

so is there a way to overcome this issue in AVS ?

I think I read that when you checked "HQ" in ffdshow, it was using some YV12>YUY2 conversion script(from Avisynth actually).....I read it in the ffdshow thread :
http://forum.doom9.org/showpost.php?p=1106325&postcount=3250

anyhow ConvertToRGB32 can't be multithreaded, so this can't be used in realtime in ffdshow :(

[tritical]

pitch.fr how did you generate that image? Avisynth uses interpolation in its yv12->yuy2->rgb conversions, but the image for converttorgb32() looks more like replication was used. Btw, 'chroma upsampling bug' usually refers to using progressive upsampling for yv12->yuy2 when interlaced is needed or vice versa.

Maybe someone who knows for sure can comment on this, but I was under the assumption that the high quality rgb conversion in ffdshow actually used the yv12->yuy2->rgb conversion code from avisynth, and that the other colorspace conversions in ffdshow used xvid's conversion routines? I took a quick glance at ffdshow-tryout's subversion repository, and saw avisynth's/xvid's conversion code in there, but couldn't quickly see when each was called.

[pitch.fr]

hi tritical, did you get my PM ? any chance getting a reply please ? :D

well I was using Haali's Renderer in MPC, then I pressed the "print screen" key, zoomed to 300% in photoshop and did some cut/paste.

yes it seems that ffdshow is using some YV12>YUY2>RGB32 conversion when you tick the "HQ conversion" option, I dunno why it ends up like that with ConvertToRGB32 :(

anyhow, I'll try to watch a movie with EVR in MPC HC, Reclock and that PS script......but I don't think this will be as smooth as HR in 1920*1080 24Hz :(

[tritical]

Yeah, I'll get back to you, gonna go on a bike ride right now though. I quickly downloaded that file, and I get pretty much exactly the same image using ffdshow's hq rgb conversion versus using converttorgb24() in avisynth. image (http://bengal.missouri.edu/~kes25c/tt.bmp) (top is ffdshow default, second is ffdshow hq, third is converttorgb24()) (you might have to zoom in to see the differences). Are you sure that when you tested with converttorgb in avisynth that you didn't accidently still have ffdshow outputting rgb when it decoded the video or something along those lines?

[pitch.fr]

I didn't use ConvertToRGB24, only ConvertToRGB32.....as even if I do RGB24 in AVS, then I will need to do convert to 32 again....which will mean more CPU time required :(

I've just tried again with ConvertToRGB24(matrix="rec709") + RGB24>RGB32 conversion in ffdshow(as neither VMR9/EVR or HR will accept RGB24 input) and I still get pixelated red.....are you sure you got this output from ConvertToRGB24 ?!

maybe it's ffdshow messing the whole stuff up, then ? even tho I kinda doubt it :(

[yesgrey]

The problem is the color correction must be done in the linear space. So, after you convert from YUV to R'G'B', you must remove gamma and convert it to RGB, apply the color correction, reapply gamma and then output to your screen. This process has a big advantage, you can also perform gamma correction all in the same operation, to apply a gamma to the signal more indicated for your display.

This is not a Convolution 2D, is simply multiplying vectors and matrices. The problem is it being slow, and the PS have the great advantage of being highly multithreaded and all the clip math is already implemented, no cpu work.

I use reclock and it is great. Tell me more about your problems, maybe I can help you.

When I created that thread, I asked Haali to add that PS funcionallity to HR, since he is already using PS for the YUV->RGB conversion. He told me he would look into it, but never replyed me again about the subject, so I think he did not approve the idea... Let's keep using mplayerc and vmr9...

[tritical]

@pitch.fr
Something is wrong in your chain. Here is how I tested:

1.) ffdshow default. Use directshowsource("Bronz_s.mkv") in avisynth to open the mkv file using ffdshow, and set ffdshow to output RGB w/o hq conversion. Open in vdub.
2.) ffdshow hq. Same as method 1 setup, but check the hq rgb conversion box in ffdshow.
3.) avisynth conversion. Again use directshowsource, but make sure ffdshow is set to output YV12 (check that it is in vdub), and use converttorgb24 or converttorgb32 after directshowsource() in the avisynth script.

Using this, I get that 2 and 3 give almost identical output (image (http://bengal.missouri.edu/~kes25c/tt2.bmp)).

[pitch.fr]

Let's keep using mplayerc and vmr9...
yeah, except if tritical can make some highly multithreaded and optimized code, I don't see it working in realtime in ffdshow :(

anyhow, I've just run through all my test movies on my DLP projector.

I'd say the colors look way more realistic :eek:

the best example being an HD bollywood movie, where red clothes look too dark, dull and artificial and skin tones slightly too green.....and when I enable the tailor-made PS script, bam! it looks natural (again) :devil:

well my main issue with not using HR w/ Reclock is that it's simply not as smooth.....HR is just so smooth, that's quite amazing...

I can watch a 2H movie in 24fps@24Hz with very low jitter and no dropped frame.

EVR or VMR9 can't offer me this kind of super low jitter.

but I'll run more tests, considering I'm starting to enjoy getting 1:1 colors on a 2 meters wide projection screen :p

@pitch.fr
Something is wrong in your chain. Here is how I tested:

1.) ffdshow default. Use directshowsource("Bronz_s.mkv") in avisynth to open the mkv file using ffdshow, and set ffdshow to output RGB w/o hq conversion. Open in vdub.
2.) ffdshow hq. Same as method 1 setup, but check the hq rgb conversion box in ffdshow.
3.) avisynth conversion. Again use directshowsource, but make sure ffdshow is set to output YV12 (check that it is in vdub), and use converttorgb24 or converttorgb32 after directshowsource() in the avisynth script.

Using this, I get that 2 and 3 give almost identical output (image (http://bengal.missouri.edu/~kes25c/tt2.bmp)).
that's really weird ?! I'll run more tests tomorrow ?!

and could you do that gamut conversion in 10 bit in AVS(even if it's not officially supported yet) ?

[yesgrey]

yeah, except if tritical can make some highly multithreaded and optimized code, I don't see it working in realtime in ffdshow :(

It would be great if the gpu drivers could allow this. It would be so simple to them, but I don't believe they would do this kind of stuff to just a few guys...

I'm starting to enjoy getting 1:1 colors on a 2 meters wide projection screen :p
I understand you. My projector is a bit old (JVC M15), and the green is slightly off. It never bothered me. In fact, I never noticed it. After starting using mplayerc with the PS for correcting the colors, I started noticing and prefering the standard colors...:)

[yesgrey]

and could you do that gamut conversion in 10 bit in AVS(even if it's not officially supported yet) ?

For this the PS is better. I think it operates with more than 14 bit precision. But the biggest problem is we not having bit depths higher than 8 bit per color in the pc world. Due to this, in theory, you could get some banding with the color correction, in practice, I never noticed it.

[pitch.fr]

well even BD's are 8bit, there's nothing further on the market at this point AFAIK..

oh really 14bit ? I had no idea :)

well I've run some tests this evening, and EVR/VMR9 still can't compete with HR's smoothness...

maybe I'm willing to give up some smoothness to get perfect colors :rolleyes:

lucky you, your projector has a Xenon lamp so you get massive reds, due to my UHP lamp mine are orangey now :(

[tritical]

Whether avisynth is the right place or not, I created a filter to do the conversions. ddcc v1.2 (http://bengal.missouri.edu/~kes25c/ddcc.zip). It takes in rgb24 or rgb32, and does the following conversion:

gamma corrected RGB -> linear RGB -> CIE XYZ -> CIE XYZ -> linear RGB -> gamma corrected RGB

Chromaticity coordinates and transfer functions of the source and output are fully adjustable... some presets are built in. If white points of chromaticity coordinates differ, then a chromatic adaptation is done using Bradford matrix. More details are in the readme. I tested some, but not enough to really be sure it is correct (I simply implemented the conversions as I understand them). For this to run real time on HD video you're gonna need a processor with SSE3 support (I didn't write SSE or SSE2 routines, so its either SSE3 or the C routine), and probably more than 1 core (using only 1 core on my 2.8Ghz Q6600 doesn't quite make it). Hopefully someone can comment on whether the results are anywhere near correct :).

[yesgrey]

tritical,
I will take a look and update my AVSForum thread if it's working. It's nice to have other options.:)

Have you think about doing it using the GPU instead of the CPU? Since this is a highly parallelizable task the GPU could speed it up a lot!... The mplayerc Pixel Shader version works great without any cpu overhead...

Thanks for your work!

[pitch.fr]

cool thanks tritical, gonna give it a shot!

wow 32float, hope this will work in realtime :D

I've spoken to a friend of mine who's a PS script coder, and he said PS scripts also work in 32bit float :
http://msdn.microsoft.com/en-us/library/bb509646(VS.85).aspx

EDIT : ooh, your plugin is highly technical stuff :eek:

all I wanna do is map SMPTE-C to my DLP projector gamut, which is

Your Display xy

Red Green Blue White
x 0.656 0.339 0.151 0.311
y 0.329 0.611 0.068 0.328
z 0.015 0.05 0.781 0.361

maybe you could give some real life examples in the readme for the Avisynth challenged amongst us :D

well I can check very easily if the conversion is done properly, by doing a Color.HCFR calibration using the official test patterns DVD in MPC

I could even compare the MPC PS script and your plugin.

I'm gonna try to see if it works in realtime, but some little help on the syntax would be much appreciated :)

[pitch.fr]

well it's actually very usable in realtime with HR :eek:

man, 32bit gamut correction with HR and Reclock, this is too awesome :)

at this point I'm decoding h264 w/ ffdshow, then it enters the AVS filter in YV12, for 1) LSF 2) multithreaded ConvertToRGB32 conversion and 3) your colorimetry plugin.

[pitch.fr]

oh btw yesgrey3, tritical is using 6 digits after the coma for the xyz coeffs, may wanna update your script ?

for BT470-2 you have more digits after the coma than he does.......but you simply call it "NTSC" so I dunno if it's the same exact matrix.

and for "Offset" which is set to 24 in the XLS script, what's a good value ? what does it stand for ?

[tritical]

@yesgrey3
I'll look into it. Unfortunately, I don't have any experience using pixel shaders... so it may be unlikely.

@pitch.fr
Not sure if you figured this out already, but to map to a custom set of chromaticity coordinates you'll have to use the ofile parameter:

ddcc(ofile="file.txt")

file.txt would contain your values as follows:

0.151
0.068
0.781
0.339
0.611
0.050
0.656
0.329
0.015
0.311
0.328
0.361

You also need to specify the 5 values for the transfer function. You can just take one of the sets of values out of the readme, or to match the transfer function of the pixel shader code use the following five values:

1.0
0.0
0.45
0.0
0.0

I set the defaults of the filter to assume SMPTE-C input primaries and BT.709 input transfer function... so you'll probably also need to change the input transfer function depending on what values you use for the output transfer function.

Also, I put up v1.1... it fixes one small bug, and adds one new built in transfer function (gam_i/gam_o = 5) which matches the transfer function of the pixel shader code.

[pitch.fr]

great, thanks!

actually I've spent quite some time trying to find a way to run this thing in realtime in ffdshow :)

I've had to cut quite a lot of my audio enhancements in ffdshow audio(96KHz resample and stuff), because it sucks a lot more CPU to use your script + RGB32 conversion in the AVS filter, than to let ffdshow do the RGB32HQ conversion + using the PS script to fix the gamut :rolleyes:

what do you think is a good/best transfer function to choose ?
any theoritical improvement over the way the PS script works ?

and what do you guys think of that SAMSUNG pj that can auto-detect whether the YCbPr input is receiving BT.601 or BT.709 ?!

they've got to use some sort of auto-detection algorithm I guess ? or maybe they actually found a way to do that ?

AFAIK this is not possible :confused:

there is already a PS script parser for AVS :
http://www.google.com/search?hl=en&q=avisynth+pixel+shader&btnG=Search&lr=

maybe there is room for improvement ?

only the GPU works on these scripts, that's a good thing because I've always heard ppl saying that CPU's were pretty lousy DSP's

[yesgrey]

I'll look into it. Unfortunately, I don't have any experience using pixel shaders... so it may be unlikely.


I was not thinking exclusivelly in PS. CUDA would be too complicated? Haven't you tried something with the gpu for your NNEDI?;)

[tritical]

@yesgrey3
A CUDA implementation would be easy to do, but I was thinking that it would be pretty limiting (in terms of supported video cards) compared to a pixel shader.

@pitch.fr
The best choice of transfer functions would be the one corresponding to what was originally used to produce the gamma corrected RGB, and the one corresponding to the transfer function of your display. Of course, finding those out may be impossible. The one in the pixel shader code is just a generic one with ~2.222 display gamma and no linear segment... it's a reasonable choice if the true one is not known. I have no idea how autodetection of BT.601 vs BT.709 would work.

[pitch.fr]

yeah, I prefer the PQ of ATi cards........please no nvidia proprietary stuff :D

so what do u think of the PS AVS plugin, maybe a good starting point ?

I'm not sure I understand what you call the "Transfer Function", but it's 4AM here :D

you mean that it's a wild guess at what the gamma curve was before they encoded their RGB master to YV12 ?

I know what the gamma curve looks like on my DLP pj, it start at 2.3 and ends at 2.1

BUT the professional broadcasting equipment runs 2.5 gamma, that's the rule.......only consumer equipment is 2.22

do you know what I should change in the PS script generator to reflect this please ?

there's also many ways to compute gamma, and there's 2 formulas being used the most.

I've spoken about that with the HCFR color engineers, the most widely used formula seems to be this one :

y' = (y - black) / (white - black)
gamma = log (y') / log (x)
or y' = x ^gamma

which can also be written as follow
we agree that
Y = Ymin when V = 0
Y = Ymax when V = 1
gamma = log ((Y - Ymin) / (Ymax - Ymin)) / log (V)
for 0<V<1

if we wanna take the offset in account, we need to use the following formula :

Y = ((V+offset)/(1+offset))^gamma

if V = 1 then Y = 1
if V = 0 then Ymin = ( offset / (1 + offset)) ^ gamma

it's the one used by Colorfacts, X-Rite, Color.HCFR, Calman etc...

[tritical]

When I say 'transfer function' I just mean the function which maps linear RGB to gamma corrected RGB. The inverse of the transfer function maps gamma corrected RGB to linear RGB. In ddcc they are implemented as:

x = lv*C if C < thresh1
(1.0+av)*pow(C,pv)-av otherwise

x = C/lv if C < thresh2
pow((C+av)/(1.0+av),1.0/pv) otherwise


Here, x is the output value, C is a linear or gamma corrected RGB value in the range [0,1], and lv, av, pv, thresh1, and thresh2 are the variables. This is the same as the function you describe, if thresh1 and thresh2 are set to 0 (i.e. no linear segment around 0), offset = av, and pv = 1.0/gamma. All of the presets that I put into ddcc (with the exception of the one to match the pixel shader code, and the srgb one which I got from http://en.wikipedia.org/wiki/SRGB) I took from the mpeg2 specs ('transfer_characteristics', Table 6-8).

[pitch.fr]

humm OK, I will read that slowly again I'm not quite a coder :D

BTW, what happens to off-gamut colors ?

white is my DLP pj, black is SMPTE-C :

http://pix.nofrag.com/a/3/c/deff50ab2a4b5c063c43e15a16155.png

what happens to the green colors that my pj can't output ? they're clipped ? :eek:

so they don't appear(banding?), or it still shows yellow instead like it does w/o correction ?

and are you sure this can't be done through the CLUT ?!

ARGYLLCMS has pretty advanced CLUT capabilities, and it's GPL software.

considering you know the formula part, why would that be impossible to generate CLUT entries ?

then we can "capture" it with powerstrip, and we're free from any ressource hogging process.

of course I will test it in every possible way with my colorimeter :D

DESCRIPTOR "Argyll Device Calibration State"
ORIGINATOR "Argyll dispcal"
CREATED "Tue Jun 17 13:01:45 2008"
KEYWORD "DEVICE_CLASS"
DEVICE_CLASS "DISPLAY"
KEYWORD "DEVICE_TYPE"
KEYWORD "TARGET_WHITE_XYZ"
TARGET_WHITE_XYZ "47.889141 50.384253 54.837810"
KEYWORD "TARGET_GAMMA"
TARGET_GAMMA "2.200000"
KEYWORD "BLACK_POINT_CORRECTION"
BLACK_POINT_CORRECTION "1.000000"
KEYWORD "QUALITY"
QUALITY "high"

KEYWORD "RGB_I"
NUMBER_OF_FIELDS 4
BEGIN_DATA_FORMAT
RGB_I RGB_R RGB_G RGB_B
END_DATA_FORMAT

NUMBER_OF_SETS 256
BEGIN_DATA
0.0000 0.084613 0.0000 2.0434e-003
3.9216e-003 0.087834 0.0000 6.7763e-003
7.8431e-003 0.091075 1.0884e-003 0.011527
0.011765 0.094334 8.1334e-003 0.016295
0.015686 0.097613 0.015385 0.021081
0.019608 0.10091 0.022857 0.025885
0.023529 0.10423 0.030658 0.030707
0.027451 0.10757 0.038978 0.035546
0.031373 0.11092 0.047935 0.040408
0.035294 0.11430 0.057523 0.045302
0.039216 0.11771 0.067448 0.050240
0.043137 0.12117 0.077293 0.055230
0.047059 0.12466 0.086703 0.060272
0.050980 0.12820 0.095542 0.065369
0.054902 0.13178 0.10384 0.070522
0.058824 0.13540 0.11162 0.075722
0.062745 0.13906 0.11897 0.080956
0.066667 0.14275 0.12592 0.086205
0.070588 0.14647 0.13252 0.091446
0.074510 0.15019 0.13878 0.096661
0.078431 0.15392 0.14473 0.10184
0.082353 0.15765 0.15039 0.10697
0.086275 0.16136 0.15583 0.11205
0.090196 0.16507 0.16107 0.11710
0.094118 0.16877 0.16614 0.12210
0.098039 0.17246 0.17107 0.12706
0.10196 0.17616 0.17589 0.13197
0.10588 0.17986 0.18059 0.13685
0.10980 0.18357 0.18519 0.14168
0.11373 0.18728 0.18972 0.14646
0.11765 0.19100 0.19416 0.15121
0.12157 0.19471 0.19853 0.15589
0.12549 0.19843 0.20279 0.16054
0.12941 0.20214 0.20699 0.16514

and many more lines.......

[yesgrey]

BTW, what happens to off-gamut colors ?
what happens to the green colors that my pj can't output ? they're clipped ? :eek:
so they don't appear(banding?), or it still shows yellow instead like it does w/o correction ?

The final color gamut will be the intersection of the two triangles; a display cannot output colors out of it's own color gamut. What this correction does is remapping the "original" colors into the display gammut so they are the same. It does this changing the RGB values sent to your display in a way that the resulting colors would be the same as the "original".

As I told you before, in theory could exist banding, in practice, I've never noticed it. As you see, the colors clipped are the more saturated ones, which don't appear too often in a movie...

There are two kinds of possible banding, one avoidable and the other not:
1-colors out of display gammut: there is nothing to do, due to the clipping, some colors will be the same. Not many and not very frequent in a movie.
2-colors inside of display gammut: If you perform the correction and have a display with >= 10 bit per color, you could avoid this kind of banding, but in the pc we are currently "limited" by 8 bit per color. Maybe there is any hardware solution...

[yesgrey]

A CUDA implementation would be easy to do, but I was thinking that it would be pretty limiting (in terms of supported video cards) compared to a pixel shader.

Well, it would be a start... and much of the people who will use this probably have a Nvidia card.
The ATI "CUDA version" is too complicated?
I understand that currently it's not very appealing working with these new GPU/CPU APIs, because probably only one will survive or even none of them will survive (a Microsoft one maybe?...)

Also, we cannot forget that the currently existing alternative to this, using mplayerc with PS, works in all of them and gives pretty good results. I believe you're solution in theory is better, because you are using the correct gamma functions, but at the end, the result should be very similar...

I left the decision in your ends (as it should be).:)

[pitch.fr]

As I told you before, in theory could exist banding, in the practice, I've never noticed it. As you see, the colors clipped are the more saturated ones, which don't appear too often in a movie...

There are two kinds of possible banding, one avoidable and the other not
ok, gracias for the explanation ;)

well CLUT is 30 bit, so no banding :)

I've spoken to Graeme Gill(the author of www.argyllcms.com) and he said this could be achieved through the CLUT.

then I could use it on top of Haali's Renderer.............this would be awesome for me coz this is the smoothest renderer ever :)

here's what he told me :

do you think that would be doable through the CLUT ?

Yes. It's been done many times. See the example code in the openEXR package and their exrdisplay utility. Rather than doing simple matrix arithmetic, you set up a texture to hold the CLUT, and have the fragment shader do a texturelookup for each pixel.

the issue is that the smoothest video renderer on PC(Haali's Renderer) doesn't support Pixel Shaders......and I was hoping to get SMPTE-C corrected colors with it, but I guess it's just a wild dream at this point...

The code here http://www.avsforum.com/avs-vb/showthread.php?t=912720 is already using a shader, it's just a matter of converting it to use a text3D and loading the ICC device link into the texture, instead of the pow/mul etc. functions.

so are you saying that once the CLUT is built, it could work w/o further realtime pixel manipulations ?

Yes, it only needs initializing if you change the device link it represents.
The OpenEXR utility was written to preview motion picture effects in real time,that's why they use a shader program to do it.

if any of you two code gurus could do that, that'd save a helluvalot of CPU time :)

as I understand it, this should be possible to output an .ICC/.ICM file instead of a realtime correction through PS or AVS ? :p

maybe this PDF would help ?
http://www.inventoland.net/imaging/JEI/159.PDF

OTOH, I'm trying your AVS plugin in ffdshow....as nothing's as smooth as HR on my box with Reclock in 24/48Hz :(

I've copied this to C:\PJ_SMPTE.txt :

0.151
0.068
0.781
0.339
0.611
0.050
0.656
0.329
0.015
0.311
0.328
0.361
1.0
0.0
0.45
0.0
0.0

and I'm using :

MT("""ConvertToRGB32(matrix="rec709")""",4)
ddcc(chr_i=3,ofile="C:\PJ_SMPTE.txt",threads=4,opt=1)

all I get is a black screen, ddcc(chr_i=3,chr_o=0,threads=4,opt=1) works fine.

[IanB]

@Tritical,

One very minor issue, you min(max(X, 0.0f), 1.0f) in 3 places. The input gamma table, the result of the matrix multiply and again for the output gamma table. I would have expected only the output gamma table which generates the 8bit output pixels needed to be clamped, all the other cases could be free to have temporary head room excursions outside the nominal range.

i.e.

1.001*0.55 + 0.7*0.20 + 0.9*0.25 = 0.91555
1.000*0.55 + 0.7*0.20 + 0.9*0.25 = 0.91500

I guess it's a style thing, allow headroom on intermediate results or clamp rigorously at every stage.

[tritical]

@IanB
Well, only one of those three clamps is actually doing anything. The first clamp (on building the igamma table) shouldn't ever trigger since the inverse transfer function maps [0,1]->[0,1], and at that point it will only get [0,1] input. The second clamp, after the linear RGB -> XYZ -> linear RGB conversion, will trigger for out-of-gamut colors (i.e. a color in the input gamut is outside of the output gamut). I believe this is the right place to clamp because the final RGB values should be in the range [0,1], and the transfer function is defined as taking [0,1] input. The final clamp (on building the fgamma table) will again never trigger since it maps [0,1]->[0,1] and because of the previous clamp it will only be getting [0,1] input. I put those clamps in the gamma table creation mainly to remind myself that those values should be in the range [0,1], and since it is run only in the constructor the cost is negligible.

@pitch.fr
Having the filter output a 3D LUT is possible, just need to know the format to use. Also, the dark screen when using the ofile parameter was due to a bug I introduced in v1.1 (a missing multiply if the white points differed causing chromatic adaptation to be used). I put up a fixed version.

[pitch.fr]

well, Graeme Gill's told me that an .ICC file would do the trick! possibly v4 ?

the LUT is 30 bit on ATi cards, dunno with nvidia ?

If you could generate an .ICC file, this would be really awesome........as the Avisynth support in ffdshow is too slow(or maybe it's ConvertToRGB32's fault), and this PS script isn't working with HR..

at this point, nothing could be better than a way to generate .ICC profiles

I'm actually very happy to hear that it's possible, because getting 1:1 colors with Haali's Renderer and Reclock would be just TOO AWESOME :eek:

OK I'm gonna try again with the new version

there is another slight detail, though.
a friend of mine believes that gamut color conversion can only be done on the original 16-235 signal........applying the gamut conversion on 0-255 expanded content would require different coeffs ?!

when we do REC.xxx conversions to RGB32 in ffdshow, luma goes from 16-235 to 0-255 and chroma from 16-240 to 1-255.....so are you still able to map the 0-15 and 241-255 colors :confused:

or should we output in full range and let your ICC do the PC>TV conversion ? or a custom range ?

also, here's a very interesting thread about these gamut stories :
http://www.avsforum.com/avs-vb/showpost.php?p=14071122&postcount=1

and another one on doom9 :
http://forum.doom9.org/showthread.php?t=132745

and a PDF about ICC corrections in the movie industry :
http://www.color.org/ICC_Chiba_07-06-19_PM_DMP_Float.pdf

thanks for your help tritical !

[tritical]

I don't know anything about ICC profiles... How exactly would that solve the problem (assuming you had one, how would you use it)? Also, is there a document which describes the format of icc profiles, i.e. how to create one?

a friend of mine believes that gamut color conversion can only be done on the original 16-235 signal........applying the gamut conversion on 0-255 expanded content would require different coeffs ?!

when we do REC.xxx conversions to RGB32 in ffdshow, luma goes from 16-235 to 0-255 and chroma from 16-240 to 1-255.....so are you still able to map the 0-15 and 241-255 colors

All of the equations in this filter work on values in the [0,1] range. The conversion to and from [0,1] currently assumes the min/max values of the RGB channels are 0 and 255. I could allow the user to specify different min/max values if desired (for instance if the RGB channels had a range of [16,235]), but usually RGB is [0,255]. The range question is usually related to whether YUV is [16,235],[16-240] or [0-255]. Depending on which it is, you need different scalings of the YUV->RGB conversion coefficients.

[pitch.fr]

well an ICC profile is the most universal solution by far.

people could compute the numbers on windows, then use the ICC profile on any OS and even hardware device :)

and it would enable gamut correction in any software/player(except PowerDVD because it only works in OVERLAY and this doesn't support ICC's).......but all the other players could beneficiate from this......even games, and there's no CPU time required, just like Pixel Shaders :)

I believe you can find all kind of white papers through google considering it's an open file format I think :
http://www.google.com/search?hl=en&q=ICC+specs&btnG=Search&lr=

this one maybe ?
http://www.color.org/ICC1v42_2006-05.pdf

I can ask Graeme Gill as well.

you can apply it within windows(for automatic correction) or with various apps, but I would personally use dispwin.exe(from the ARGYLLCMS package) with a batch for each of my displays(CRT/pj)

[pitch.fr]

I'm still a bit confused between LUT and profiles.
LUT is the final result of the profile, so you can't quite output LUT raw data.

here's what he told me

maybe he could simply output "raw lut data", that can be read with this software : http://www.exactscan.com/lutmanager/

You are confusing calibration and profiles.
The "lut data" is the calibration curves.
A device link is the result of linking two device ICC profiles,and is a clut, a 3D dimensional interpolation table.
Gerhard's code http://www.mail-archive.com/lcms-user@lists.sourceforge.net/msg02326.html , which uses lcms to link two device profiles and load it into a textureis basically a complete example.
I can't help you much more than pointing youat that code. Graeme Gill.

I've asked him for simple specs to an .ICC file

ok, so using the settings from the previous page with DDCC 1.2, compared to these settings with the .XLS script(updated the script with your coeffs that have 5 digits after the coma) :
http://rapidshare.com/files/130373214/pj.xls.html

the correction is quite different actually.

this is the original test video :

http://rapidshare.com/files/130375581/rec709.mkv.html

http://thumbnails8.imagebam.com/950/b5e6859499794.gif (http://www.imagebam.com/image/b5e6859499794)

this is with the PS script, and SMPTE-C conversion :
http://thumbnails8.imagebam.com/950/3498ff9499795.gif (http://www.imagebam.com/image/3498ff9499795)

this is with ddcc 1.2, and SMPTE-C conversion :
http://thumbnails8.imagebam.com/950/3c2ca29499796.gif (http://www.imagebam.com/image/3c2ca29499796)

I'd say the gamma has been lowered way too much....clearly not 2.2 ?

or maybe we need to finetune the settings ? :D

EDIT : there's an ICC profile SDK btw :
http://www.littlecms.com/downloads.htm

interestingly enough, they have used that SDK to make a simple example, that is said to :

This command line program does compute colorspace conversion based on icc profiles. Additionally, it can show XYZ and Lab values of PCS, and up to 16 bits of precision (48, 64 bits per pixel). If you ever have been searching for a universal colorspace conversion utility, check this one!

http://www.littlecms.com/newutils.htm

but these things are going way over my head :scared:

[tritical]

The differences to the pixel shader are due to two things:

1.) ddcc and the ps code are using different input gamma functions. You need to set gam_i=5 in ddcc to have it use the same one as the ps code. So the script line would become:

ddcc(chr_i=3,gam_i=5,ofile="ofile.txt")

2.) When the white point coordinates differ between the source and destination, ddcc performs chromatic adaptation using the Bradford method. The pixel shader excel file does not. This will slightly change the resulting coefficients. (Other than this, the ps excel file and ddcc produce the same coefficients).

I don't think I'll mess with ICC profiles. However, the 3D lut method via a 3d texture on the gpu looks easy enough. I will try to implement it when I have the time.

[pitch.fr]

OK that sounds great, indeed they're identical now...tritical yo'ure my hero :D

original :

http://thumbnails8.imagebam.com/956/b5e6859555408.gif (http://www.imagebam.com/image/b5e6859555408)

PS :

http://thumbnails8.imagebam.com/956/3498ff9555409.gif (http://www.imagebam.com/image/3498ff9555409)

AVS :

http://thumbnails8.imagebam.com/956/9a4b919555410.gif (http://www.imagebam.com/image/9a4b919555410)

I will make colorimeter measures anytime soon :)

too bad the ConvertToRGB32 sucks so much CPU......it's hardly usable in real time :(

[pitch.fr]

OK I've played around with Profile Maker 5.0

http://pix.nofrag.com/a/e/5/7812bec244bd2cba7acec57c666bdtt.jpg (http://pix.nofrag.com/a/e/5/7812bec244bd2cba7acec57c666bd.html)

here's what their help file says :

Mechanisms of the ICC Profile
There are a variety of mathematical mechanisms for color conversion that are used by the Color Management Module. The data required for this operation is contained in the profile.

There are two methods of defining a conversion:

1)Mathematical Functions (TRC: Tone Reproduction Curves and Matrix Models)

2)Conversion Tables (LUT: Look Up Table)

The ICC profiles of the different device categories work with a variety of mathematical models.
Using Matrix Functions and TRCs
For the quite simple conversions between purely additive color systems, ICC profiles use Matrix Functions. These models are sufficiently accurate and produce relatively small color profiles.

The advantage of functions is the compactness of the parameter sets, but unfortunately they are poorly suited to accurately describe complex mapping characteristics such as exist between color models that have different dimensions (RGB > LAB versus CMYK > LAB).

Monitor profiles work mostly with TRCs and 3x3 Matrix Operations.

Using LUTs
For more complex color systems, for example in the case of color transformations involving a change in dimension, and for the description of non-linear color spaces, Tables (LUTs) must be used.

Tables offer unlimited accuracy and an unlimited number of dimensions, but are also unlimited in terms of size.

As a rule, the direction of conversion from Scanner RGB to LAB is described in a scanner profile using three-dimensional LUTs. Normally, a separate table would have to be created for each Rendering Intent, for a total of four tables. However, since the Rendering Intents for Relative and Absolute colorimetric interpretation can be calculated from the same table, only three LUTs are required. In output profiles, both conversion directions are described using LUTs. In this case, three four-dimensional LUTs are created for the direction CMYK to LAB and three thee-dimensional LUTs for the direction LAB to CMYK.

so clearly a LUT is more accurate than an ICC :)

and Powerstrip's coder had given me infos on how to input custom LUT's within his app :

LUTs are stored by PowerStrip in the registry, and read/written to SRAM in the GPU. Here's the text format for a Broadcast D65 LUT - it uses 8 bit values because it predates deeper depths, but PowerStrip has been able to read LUTs stored with 16 bit data since the Matrox Parhelia.

So you have here 768 bytes (or words) in the form R[0], R[1]...R[254],R[255],G[0]...G[255],B[0]...B[255].


"Broadcast 2.2 Gamma D65"=hex:00,00,00,01,01,02,02,02,03,03,04,04,04,05,05,06,\
07,08,09,0a,0b,0c,0d,0f,10,11,12,13,14,15,16,17,19,1a,1b,1c,1d,1f,20,21,22,\
23,24,26,27,28,29,2a,2b,2c,2d,2f,30,31,32,33,34,35,36,38,39,3a,3b,3c,3d,3e,\
3f,40,42,43,44,45,46,47,48,49,4a,4b,4c,4d,4f,50,51,52,53,54,55,56,57,58,59,\
5a,5b,5c,5e,5f,60,61,62,63,64,65,66,67,68,69,6a,6b,6c,6d,6e,6f,70,71,72,73,\
74,76,77,78,79,7a,7b,7c,7d,7e,7f,80,81,82,83,84,85,86,87,88,89,8a,8b,8c,8d,\
8e,8f,90,91,92,93,94,95,96,97,98,99,9a,9b,9c,9d,9e,9f,a0,a1,a2,a3,a4,a5,a6,\
a7,a8,a9,aa,ab,ad,ae,af,b0,b1,b2,b3,b4,b5,b6,b7,b8,b9,ba,bb,bc,bd,bd,be,bf,\
c0,c1,c2,c3,c4,c5,c6,c7,c8,c9,ca,cb,cc,cd,ce,cf,d0,d1,d2,d3,d4,d5,d6,d7,d8,\
d9,da,db,dc,dd,de,df,e0,e1,e2,e3,e4,e5,e6,e7,e8,e9,ea,eb,ec,ed,ee,ef,f0,f1,\
f2,f3,f4,f5,f6,f7,f8,f9,fa,fb,fb,fc,fd,fe,ff,00,00,00,01,01,02,02,02,03,03,\
04,04,04,05,05,06,07,08,09,0a,0b,0c,0d,0f,10,11,12,13,14,15,16,18,19,1a,1b,\
1c,1d,1f,20,21,22,23,24,26,27,28,29,2a,2b,2c,2e,2f,30,31,32,33,34,35,37,38,\
39,3a,3b,3c,3d,3e,3f,40,42,43,44,45,46,47,48,49,4a,4b,4c,4e,4f,50,51,52,53,\
54,55,56,57,58,59,5a,5b,5c,5e,5f,60,61,62,63,64,65,66,67,68,69,6a,6b,6c,6d,\
6e,6f,70,71,72,74,75,76,77,78,79,7a,7b,7c,7d,7e,7f,80,81,82,83,84,85,86,87,\
88,89,8a,8b,8c,8d,8e,8f,90,91,92,93,94,95,96,97,98,99,9a,9b,9c,9d,9e,9f,a0,\
a1,a3,a4,a5,a6,a7,a8,a9,aa,ab,ac,ad,ae,af,b0,b1,b2,b3,b4,b5,b6,b7,b8,b9,ba,\
bb,bc,bd,be,bf,c0,c1,c2,c3,c4,c5,c6,c7,c8,c9,ca,ca,cb,cc,cd,ce,cf,d0,d1,d2,\
d3,d4,d5,d6,d7,d8,d9,da,db,dc,dd,de,df,e0,e1,e2,e3,e4,e5,e6,e7,e8,e9,ea,eb,\
ec,ed,ee,ef,f0,f1,f2,f3,f4,f5,f6,f7,f8,f9,fa,fb,fc,fd,fe,ff,ff,00,00,00,01,\
01,02,02,02,03,03,04,04,04,05,05,06,07,08,09,0a,0b,0c,0d,0f,10,11,12,13,14,\
15,16,17,19,1a,1b,1c,1d,1e,20,21,22,23,24,25,27,28,29,2a,2b,2c,2d,2f,30,31,\
32,33,34,35,36,38,39,3a,3b,3c,3d,3e,3f,40,41,43,44,45,46,47,48,49,4a,4b,4c,\
4d,4e,50,51,52,53,54,55,56,57,58,59,5a,5b,5c,5d,5e,60,61,62,63,64,65,66,67,\
68,69,6a,6b,6c,6d,6e,6f,70,71,72,73,74,75,77,78,79,7a,7b,7c,7d,7e,7f,80,81,\
82,83,84,85,86,87,88,89,8a,8b,8c,8d,8e,8f,90,91,92,93,94,95,96,97,98,99,9a,\
9b,9c,9d,9e,9f,a0,a1,a2,a3,a4,a5,a6,a7,a8,a9,aa,ab,ac,ad,ae,af,b0,b1,b2,b3,\
b4,b5,b6,b7,b8,b9,ba,bb,bc,bd,be,bf,c0,c1,c2,c3,c4,c5,c6,c7,c8,c9,ca,cb,cc,\
cd,ce,cf,d0,d1,d2,d3,d4,d5,d6,d7,d8,d9,da,db,dc,dd,de,df,e0,e1,e2,e3,e4,e5,\
e6,e7,e8,e9,ea,eb,ec,ed,ee,ef,f0,f1,f2,f3,f4,f5,f6,f6,f7,f8,f9,fa,fb,fc,fd,\
fe,ff

maybe using your voodoo magic, you could offer a way to output powerstrip compatible LUT's :eek:

he has also told me that you can output 16bit LUT, which will improve the accuracy to 10 bit on ATi cards, dunno about nvidia

If you use 16-bit values, PowerStrip automatically adjusts down to the maximum bit depth of the DAC.

If you use 8-bit values, PowerStrip will make 8 bit correction.

the advantage of doing the conversion through the LUT is that it doesn't use any CPU time and it's compatible with any picture viewer/video renderer(except OVERLAY) w/o any requirement(like PS script compatibility, that only MPC and KMPlayer support at this point)

and the LUT can be ported to other OS such as Linux :)

thanks!

[pitch.fr]

OK I've read the AVS thread again.

it seems that people were confusing ICC and LUT(just like me, but the other way around)

an ICC only works in compatible applications, Adobe Gamma can force it though.....but only in window mode.

so the best solution is to output 16 bit LUT to powerstrip, which will render it in 10 bit on the Radeon.

the LUT is compatible with any application, except the OVERLAY video renderer.

here's the powerstrip coder's notes about importing 16bit pstrip-compatible LUT's to the registry(for instant use in pstrip in 10 bit) :

Here is the same Broadcast D65 LUT, but with 16 bit values instead of 8 bit.


"Broadcast 2.2 Gamma D65 (16 bit)"=hex:00,00,00,00,00,00,00,01,00,01,00,02,00,\
02,00,02,00,03,00,03,00,04,00,04,00,04,00,05,00,05,00,06,00,07,00,08,00,09,\
00,0a,00,0b,00,0c,00,0d,00,0f,00,10,00,11,00,12,00,13,00,14,00,15,00,16,00,\
17,00,19,00,1a,00,1b,00,1c,00,1d,00,1f,00,20,00,21,00,22,00,23,00,24,00,26,\
00,27,00,28,00,29,00,2a,00,2b,00,2c,00,2d,00,2f,00,30,00,31,00,32,00,33,00,\
34,00,35,00,36,00,38,00,39,00,3a,00,3b,00,3c,00,3d,00,3e,00,3f,00,40,00,42,\
00,43,00,44,00,45,00,46,00,47,00,48,00,49,00,4a,00,4b,00,4c,00,4d,00,4f,00,\
50,00,51,00,52,00,53,00,54,00,55,00,56,00,57,00,58,00,59,00,5a,00,5b,00,5c,\
00,5e,00,5f,00,60,00,61,00,62,00,63,00,64,00,65,00,66,00,67,00,68,00,69,00,\
6a,00,6b,00,6c,00,6d,00,6e,00,6f,00,70,00,71,00,72,00,73,00,74,00,76,00,77,\
00,78,00,79,00,7a,00,7b,00,7c,00,7d,00,7e,00,7f,00,80,00,81,00,82,00,83,00,\
84,00,85,00,86,00,87,00,88,00,89,00,8a,00,8b,00,8c,00,8d,00,8e,00,8f,00,90,\
00,91,00,92,00,93,00,94,00,95,00,96,00,97,00,98,00,99,00,9a,00,9b,00,9c,00,\
9d,00,9e,00,9f,00,a0,00,a1,00,a2,00,a3,00,a4,00,a5,00,a6,00,a7,00,a8,00,a9,\
00,aa,00,ab,00,ad,00,ae,00,af,00,b0,00,b1,00,b2,00,b3,00,b4,00,b5,00,b6,00,\
b7,00,b8,00,b9,00,ba,00,bb,00,bc,00,bd,00,bd,00,be,00,bf,00,c0,00,c1,00,c2,\
00,c3,00,c4,00,c5,00,c6,00,c7,00,c8,00,c9,00,ca,00,cb,00,cc,00,cd,00,ce,00,\
cf,00,d0,00,d1,00,d2,00,d3,00,d4,00,d5,00,d6,00,d7,00,d8,00,d9,00,da,00,db,\
00,dc,00,dd,00,de,00,df,00,e0,00,e1,00,e2,00,e3,00,e4,00,e5,00,e6,00,e7,00,\
e8,00,e9,00,ea,00,eb,00,ec,00,ed,00,ee,00,ef,00,f0,00,f1,00,f2,00,f3,00,f4,\
00,f5,00,f6,00,f7,00,f8,00,f9,00,fa,00,fb,00,fb,00,fc,00,fd,00,fe,00,ff,00,\
00,00,00,00,00,00,01,00,01,00,02,00,02,00,02,00,03,00,03,00,04,00,04,00,04,\
00,05,00,05,00,06,00,07,00,08,00,09,00,0a,00,0b,00,0c,00,0d,00,0f,00,10,00,\
11,00,12,00,13,00,14,00,15,00,16,00,18,00,19,00,1a,00,1b,00,1c,00,1d,00,1f,\
00,20,00,21,00,22,00,23,00,24,00,26,00,27,00,28,00,29,00,2a,00,2b,00,2c,00,\
2e,00,2f,00,30,00,31,00,32,00,33,00,34,00,35,00,37,00,38,00,39,00,3a,00,3b,\
00,3c,00,3d,00,3e,00,3f,00,40,00,42,00,43,00,44,00,45,00,46,00,47,00,48,00,\
49,00,4a,00,4b,00,4c,00,4e,00,4f,00,50,00,51,00,52,00,53,00,54,00,55,00,56,\
00,57,00,58,00,59,00,5a,00,5b,00,5c,00,5e,00,5f,00,60,00,61,00,62,00,63,00,\
64,00,65,00,66,00,67,00,68,00,69,00,6a,00,6b,00,6c,00,6d,00,6e,00,6f,00,70,\
00,71,00,72,00,74,00,75,00,76,00,77,00,78,00,79,00,7a,00,7b,00,7c,00,7d,00,\
7e,00,7f,00,80,00,81,00,82,00,83,00,84,00,85,00,86,00,87,00,88,00,89,00,8a,\
00,8b,00,8c,00,8d,00,8e,00,8f,00,90,00,91,00,92,00,93,00,94,00,95,00,96,00,\
97,00,98,00,99,00,9a,00,9b,00,9c,00,9d,00,9e,00,9f,00,a0,00,a1,00,a3,00,a4,\
00,a5,00,a6,00,a7,00,a8,00,a9,00,aa,00,ab,00,ac,00,ad,00,ae,00,af,00,b0,00,\
b1,00,b2,00,b3,00,b4,00,b5,00,b6,00,b7,00,b8,00,b9,00,ba,00,bb,00,bc,00,bd,\
00,be,00,bf,00,c0,00,c1,00,c2,00,c3,00,c4,00,c5,00,c6,00,c7,00,c8,00,c9,00,\
ca,00,ca,00,cb,00,cc,00,cd,00,ce,00,cf,00,d0,00,d1,00,d2,00,d3,00,d4,00,d5,\
00,d6,00,d7,00,d8,00,d9,00,da,00,db,00,dc,00,dd,00,de,00,df,00,e0,00,e1,00,\
e2,00,e3,00,e4,00,e5,00,e6,00,e7,00,e8,00,e9,00,ea,00,eb,00,ec,00,ed,00,ee,\
00,ef,00,f0,00,f1,00,f2,00,f3,00,f4,00,f5,00,f6,00,f7,00,f8,00,f9,00,fa,00,\
fb,00,fc,00,fd,00,fe,00,ff,00,ff,00,00,00,00,00,00,00,01,00,01,00,02,00,02,\
00,02,00,03,00,03,00,04,00,04,00,04,00,05,00,05,00,06,00,07,00,08,00,09,00,\
0a,00,0b,00,0c,00,0d,00,0f,00,10,00,11,00,12,00,13,00,14,00,15,00,16,00,17,\
00,19,00,1a,00,1b,00,1c,00,1d,00,1e,00,20,00,21,00,22,00,23,00,24,00,25,00,\
27,00,28,00,29,00,2a,00,2b,00,2c,00,2d,00,2f,00,30,00,31,00,32,00,33,00,34,\
00,35,00,36,00,38,00,39,00,3a,00,3b,00,3c,00,3d,00,3e,00,3f,00,40,00,41,00,\
43,00,44,00,45,00,46,00,47,00,48,00,49,00,4a,00,4b,00,4c,00,4d,00,4e,00,50,\
00,51,00,52,00,53,00,54,00,55,00,56,00,57,00,58,00,59,00,5a,00,5b,00,5c,00,\
5d,00,5e,00,60,00,61,00,62,00,63,00,64,00,65,00,66,00,67,00,68,00,69,00,6a,\
00,6b,00,6c,00,6d,00,6e,00,6f,00,70,00,71,00,72,00,73,00,74,00,75,00,77,00,\
78,00,79,00,7a,00,7b,00,7c,00,7d,00,7e,00,7f,00,80,00,81,00,82,00,83,00,84,\
00,85,00,86,00,87,00,88,00,89,00,8a,00,8b,00,8c,00,8d,00,8e,00,8f,00,90,00,\
91,00,92,00,93,00,94,00,95,00,96,00,97,00,98,00,99,00,9a,00,9b,00,9c,00,9d,\
00,9e,00,9f,00,a0,00,a1,00,a2,00,a3,00,a4,00,a5,00,a6,00,a7,00,a8,00,a9,00,\
aa,00,ab,00,ac,00,ad,00,ae,00,af,00,b0,00,b1,00,b2,00,b3,00,b4,00,b5,00,b6,\
00,b7,00,b8,00,b9,00,ba,00,bb,00,bc,00,bd,00,be,00,bf,00,c0,00,c1,00,c2,00,\
c3,00,c4,00,c5,00,c6,00,c7,00,c8,00,c9,00,ca,00,cb,00,cc,00,cd,00,ce,00,cf,\
00,d0,00,d1,00,d2,00,d3,00,d4,00,d5,00,d6,00,d7,00,d8,00,d9,00,da,00,db,00,\
dc,00,dd,00,de,00,df,00,e0,00,e1,00,e2,00,e3,00,e4,00,e5,00,e6,00,e7,00,e8,\
00,e9,00,ea,00,eb,00,ec,00,ed,00,ee,00,ef,00,f0,00,f1,00,f2,00,f3,00,f4,00,\
f5,00,f6,00,f6,00,f7,00,f8,00,f9,00,fa,00,fb,00,fc,00,fd,00,fe,00,ff


As you can see from the very last LUT entry above (0xFF00 for B[256]) when represented as text the msb are in the 2nd byte of the couplet.

[pitch.fr]

OK I've made some colorimetry measurements with my Eye One Display 2 on my HC3100 as promised.

the black triangle is SMPTE RP 145 in all these graphs, and the ghost gamut is the original one.

I also give the Delta-E and the Distance from the reference in the CIExy space

the projector has been D65 calibrated with a Delta-E <3 before doing the tests(with the official test patterns DVD and Color.HCFR in MPC) :

http://pix.nofrag.com/b/2/6/c48e09b6e79d3305ce11b26fa27edtt.jpg (http://pix.nofrag.com/b/2/6/c48e09b6e79d3305ce11b26fa27ed.html)

that's the original gamut :

http://pix.nofrag.com/2/0/8/2b4fde6c873d93324cb13e538db76tt.jpg (http://pix.nofrag.com/2/0/8/2b4fde6c873d93324cb13e538db76.html)

http://pix.nofrag.com/d/a/7/993d1813bee28cdd09acd5f647378.png


now with the Pixel Shaders script :

http://pix.nofrag.com/4/1/0/6146664a9e7e23f8f006fc5e3875dtt.jpg (http://pix.nofrag.com/4/1/0/6146664a9e7e23f8f006fc5e3875d.html)

http://pix.nofrag.com/6/8/f/de5e32a36e2732028bbe1ab14e1f3.png


now with ddcc(chr_i=3,gam_i=5,ofile="C:\PJ.txt",threads=4,opt=1) :

http://pix.nofrag.com/2/c/c/aea54933ce35849d7bf9774522bf3tt.jpg (http://pix.nofrag.com/2/c/c/aea54933ce35849d7bf9774522bf3.html)

http://pix.nofrag.com/1/2/f/1b9c92223b2fef175a7d7fe61cb10.png


now with ddcc(chr_i=3,gam_i=2,ofile="C:\PJ.txt",threads=4,opt=1) :

http://pix.nofrag.com/a/f/a/7b96e6cba9e016e1c02893d743e26tt.jpg (http://pix.nofrag.com/a/f/a/7b96e6cba9e016e1c02893d743e26.html)

http://pix.nofrag.com/f/3/4/941c1cae7095aa14a0fed85b9d41d.png

PJ.txt contained :

0.151
0.068
0.782
0.338
0.611
0.052
0.653
0.330
0.002
0.312
0.328
0.360
1.0
0.0
0.45
0.0
0.0


it's really impressive how close the PS script managed to get to the refs in the CIExy space....I'm speechless :eek:

maybe some brush-up of the DDCC maths and/or the text file could improve the results ?

together with a way to output 16 bit Powerstrip-compatible LUT's please ? :D

TIA,

[pitch.fr]

OK forget it, this can't be done through the LUT :

> I'm afraid there is a misunderstanding. The VCGT LUTs which can be
> loaded into the graphics card with ARGYLL are just one-dimensional
> LUTs. Such LUTs do not allow arbitrary color transformation, but only
> VERY LIMITED ones (like e.g. adjusting the gamma of the RGB channnels).
> For the kind of color transformation you desire, you would need a
> three-dimensional LUT. I'm not aware of graphics cards which support
> that (except in conjunction with a pixel shader program), but I also
> don't want to rule out that such graphics cards might exist.

tritical, any idea what I could do to get the same accurate results that the PS script is giving ?

thanks,

[yesgrey]

Now I understand what is a LUT. You could change it through the color calibration you could perform in the graphic card drivers, this was debated in the AVS Forum. The problem of it is that each component only depends on itself, and for this kind of correction, each component should depends on all three components.
Example:
LUT
Rd = f(Rs)
Gd = f(Gs)
Bd = f(Bs)

What we need:
Rd = f(Rs,Gs,Bs)
Gd = f(Rs,Gs,Bs)
Bd = f(Rs,Gs,Bs)

It's the 3D LUT table you referred...

There is also a very simple and fast solution for your problem... Haali uses Pixel Shaders in his renderer for the YUV->RGB conversion, you could ask him to add this PS code to his renderer, in which we just need to specify our conversion matrix.
In fact, I have already tried that some time ago without any luck. Give it a try, maybe if more people start to request this to him he feel the need to do it.;)

[tritical]

Your measurements aren't telling the whole story here for a number of reasons. First, you are ignoring white point differences between source/destination, and chromatic adaptation. As I said before, the only difference between the PS script and ddcc (assuming ddcc is setup to use the same gamma function assumed by the ps script, etc...) is that ddcc performs chromatic adaptation when the white points between the source and destination differ. Chromatic adaptation, with regards to the HVS, refers to the ability of the HVS to make an object appear the same (color wise) under a wide range of illuminants. What this means is that to a person different points in the CIE XYZ colorspace will appear to be the same color when viewed under different illuminants. When it is used in regards to colorspace conversions (specifically those based on CIE XYZ where the white point differs) it refers to trying to alter the XYZ tristimulus values such that to a person the new color under the new illuminant would look the same as the original color under the original illuminant. The are three main methods for doing chromatic adaptation: XYZ scaling, Von Kries, and Bradford. ddcc uses Bradford. To see how this effects the coefficients in the BGR->CIE->BGR transform, I had ddcc print them out. The first set shows the coefficients calculated when using the ddcc() line you specified. Each line after that modifies your white point to be progressively closer to the D65 illuminant, such that in the last case no chromatic adaptation is needed (in this case the PS script's coefficients match ddcc's to 7 or more decimal places).
0.312713,0.329016,0.358271 -> 0.312,0.328,0.360

1.109563,-0.129243,0.016116
0.028285,0.972018,-0.001302
0.008913,0.054374,0.938853


0.312713,0.329016,0.358271 -> 0.313,0.329,0.358

1.105802,-0.123381,0.019919
0.028030,0.971643,-0.000281
0.008032,0.052882,0.938751


0.312713,0.329016,0.358271 -> 0.3127,0.3290,0.3583

1.108651,-0.127532,0.018811
0.028307,0.971852,-0.000172
0.008176,0.053085,0.938775


0.312713,0.329016,0.358271 -> 0.312713,0.329016,0.358271

1.108620,-0.127480,0.018860
0.028306,0.971848,-0.000154
0.008164,0.053063,0.938774

While the coefficients aren't altered all that much (since the white point you specified was already pretty close to D65), it is definitely enough to account for the minor differences you measured between the PS script and ddcc.

Another thing to remember is that the assumed gamma function (in this case x^2.2222) is not an exact match to any real world device (let alone an exact match to both the source/destination). This will introduce error into the calculations.

Basically, while measurements are nice, the only way to truely know if the colors are accurate would be to see the original source on the original display device, and see if to you it looks the same as what you are seeing on your display device. Unfortunately this is unlikely to ever happen.

Anyways, to get back to the point, there isn't anything wrong with the math in ddcc.