gamut conversions through Avisynth ?

[leeperry]

Coding a yuy2 to rgb 3d lut filter wouldn't take much work (small modifications to rgb3dlut). I might include it in the next release of ddcc.
well mark0077 has been recently pointing out that the nvidia drivers in YV12(using software renderers) were offering better chroma upsample than even ConvertToRGB32() :
http://forum.doom9.org/showpost.php?p=1230065&postcount=27

I'm not sure how they do it, but could you please set up a very HQ chroma upsample scheme ? like spline36 ? apparently Convert() is using bicubic, and so does ffdshow....but the ugly ATi drivers would be using pointresize(from what Leak said) :rolleyes:

the test sample VOB is available here :
http://www.mediafire.com/?9g9ddlfzxhv
@leeperry

It will be... it just reads fp values (sscanf(buf,"%lf",&val)).

On the topic of saturation correction. Maybe you could ask JohnAd what he did, because I still don't know how you could make any corrections based on the data you have.
well the real issue will be to verify how it went, the HCFR coders are too busy with the next release to set up some sort of automatization with MPC and the DVD manual patterns....and going through 24 manual patterns is too cumbersome. nevertheless, I will ask JohnAd :)

if at some point, you could sorta give some hints for the gam_i/o formulas to use, that'd be really great.
atm I'm using 5/5 and it looks good to me on my ±2.2 calibrated display....maybe I should put 0.454545454545 instead of 0.45 ?

[tritical]

What playback chain do I need to use to ensure that nvidia's drivers are doing the upsampling (I have a 9800gtx with 178.28 drivers)? Avisynth's color conversions operate as described here: http://avisynth.org/Sampling. The only problems with the descriptions on that page are that the rgb->yuy2 conversion doesn't use a [1 2 1] kernel, and the C code version of yuy2->rgb doesn't average to get every other chroma sample (see http://forum.doom9.org/archive/index.php/t-129316.html). ffdshow's hq rgb conversion uses avisynth's code (as in taken out of avisynth cvs) so it uses the same sampling. Any differences between avisynth's conversion and ffdshow's hq rgb conversion are probably due to coefficient scaling.

My plan for yuy2 to rgb lut is to use linear interpolation to create the chroma for the odd pixels (same method described on the avisynth sampling page).

[leeperry]

the test was conducted in YV12 with EVR on Vista SP1 :
http://forum.doom9.org/showthread.php?t=143818

apparently what also helped was Leak's progressively upsampling PS script(embedded in MPC HC) :
http://forum.doom9.org/showpost.php?p=1184975&postcount=32

that'd be more convenient if you could allow YV12 input instead of YUY2 if any possible, so ddcc gets the untouched video stream(no ConvertToYUY2() implied).

[yesgrey]

The LUT size would be: 3x220x254x254 ~ 40.6MB < 48MB.
This is wrong. Since we also want to convert values of Y<16 and Y>235, the correct size would be:
3x254x254x254 ~ 48MB.
In YUV values 0 and 255 should be clipped. So, I think the better is just keeping the same size of the RGB, 3x256x256x256...

If you could write a program to actually calculate the LUT files for use with it that would great :).
Yes, I am thinking in this. I will use parts of your code, and am also thinking in finding a way of performing the correction for the saturations problem.
Probably I will use Fortran - is my prefered language -, are you ok with it?

[yesgrey]

that'd be more convenient if you could allow YV12 input instead of YUY2 if any possible

This is not possible.
A LUT maps an input value to an output value.
In YV12 you don't have all the input values, only half, so you cannot map it through the 3D LUT. You have to compute the values that are missing, so, you always have to convert to YUY2. The idea of including the YUV->RGB conversion is only to avoid the extra cpu load of YUV->RGB conversion, you cannot avoid the YV12->YUY2 conversion...

Of course rgb3dlut function could support YV12 input, but then it will have to perform the YV12->YUY2 internally beforing mapping it out via the 3D LUT, so there is no gain in avoiding ConvertToYUY2...;)

[leeperry]

you cannot avoid the YV12->YUY2 conversion
right :D
well still, seeing how inaccurate ConvertToRGB32() is, maybe the YV12>YUY2 conversion could also benefit from some higher quality upsample :o

[tritical]

In YUV values 0 and 255 should be clipped. So, I think the better is just keeping the same size of the RGB, 3x256x256x256...
I agree... it also makes the code simpler. My code makes the lookup in to the table as: ((V<<16)+(U<<8)+Y)*3, then the three bytes at that location are stored in b,g,r order.

Yes, I am thinking in this. I will use parts of your code, and am also thinking in finding a way of performing the correction for the saturations problem.
Probably I will use Fortran - is my prefered language -, are you ok with it?
Yep.

[IanB]

YV12 to RGB is possible in exactly the same way as the proposed YUY2 to RGB, you just have to pre-interpolate vertically as well.

As for out of gamut values, maintaining Hue is the all important factor. The eye is very sensitive to hue, especially skin tones (pink), but is quite ambivalent about saturation particularly at high saturations.

In YUV space, Hue is analogous to the ratio of U/V, saturation is analogous to the length of the UV vector, i.e. U**2+V**2, so just clamp the offending U or V value at the gamut limit and scale the other value to maintain the original ratio of the preclamped values.

e.g. Using normalised values, conversion results in U=1.06, V=0.42. Clamp U at 1.0, set V=1.0*(0.42/1.06)=0.396

In RGB space hue is the ratio of the 2 dominant primaries after the value of the minority primary has been subtracted from each. These values are best corrected in linear (non-gamma) space.

e.g. Using normalised values, conversion results in R=0.72, G=1.07, B=0.42.
B is the current minority primary, R'=0.72-0.42=0.3 and G'=1.07-0.42=0.65
Maintain Hue by keeping the ratio G'/R'=0.65/0.3=2.167 constant.
Clamp G to 1.0, scale R''=R'*G''/G'=0.3*(1.0-0.42)/0.65=0.267
Thus adjusted values become R=R''+B=0.267+0.42=0.688, G=1.0, B=0.42
Apply Gamma correction and scale to output range.

[tritical]

I agree that maintaining hue while decreasing brightness and saturation is the best way to handle the case of values greater than 1.0, but I don't follow your math. You say that the ratio of G'/R' must be kept constant, and that ratio is equal to (G-B)/(R-B). Yet you only modify G and R. It's not possible for that ratio to remain the same without scaling all three values by the same factor (and it will stay constant if you do). I think the way to do it is simply to divide all three values by the maximum value... which in your example would give:

R=0.72/1.07, G=1.07/1.07, B=0.42/1.07
R=0.673, G=1.000, B=0.393

[yesgrey]

YV12 to RGB is possible in exactly the same way as the proposed YUY2 to RGB, you just have to pre-interpolate vertically as well.
Yes, I referred that, but it's not possible to include the interpolation in the 3D LUT values. The interpolation should be done before mapping with the 3D LUT, it's the only way to know what to map from...;)

[yesgrey]

These values are best corrected in linear (non-gamma) space.
Well, these values only appear in the linear space, when we are performing the gammut conversion, so we are safe.:)

[yesgrey]

that ratio is equal to (G-B)/(R-B). Yet you only modify G and R. It's not possible for that ratio to remain the same without scaling all three values by the same factor...
It's possible. Look:
a) C = (G-B)/(R-B); you know R,G,B and calculate C
b) C = (G''-B)/(R''-B)
c) Set G''=1.0
d) R'' = (G''-B)/C + B; you know G'',B,C and calculate R''

I think this method is better because affects less the values not to be clipped.

[tritical]

You're are right. I must have been up to late last night.

However, I still don't think it's clear cut that that method is better. In terms of HSL coordinates IanB's method is better... both methods produce the same H/S values, but IanB's method reduces L less. However, in terms of HSV coordinates my method is better... both methods produce the same H/V values, but IanB's method reduces S while mine does not.

[yesgrey]

tritical,
Your method has the advantage of being much easier to code.
Since the hue is kept with both methods, and the HSL vs HSV is not conclusive, maybe it's better just doing the easiest...;)
IanB, what do you think?

[IanB]

Yes, both methods are compromises, the value is out of gamut, you must compensate it.

Tritical's all scaling method maintains hue and saturation at the expense of luminance.

My dominant pair scaling method maintains hue at the expense of both saturation and luminance, but to a lesser degree.

Another alternative is to prune saturation even harder and maintain luminance. i.e.

R=0.72, G=0.42, B=1.07.
Y=0.2126*0.72+0.7152*0.42+0.0722*1.07=0.531

All scaling
R=0.673, G=0.393, B=1.0
Y=0.496

Dominant pair scaling
R=0.688, G=0.42, B=1.0
Y=0.519

Plus minority primary scaling
R=0.695, G=0.437, B=1.0
Y=0.531

Each method has weaknesses. Probably a selection of methods might be needed, depending on how the colour is out of gamut. I chose to make the Blue channel over valued in this example because it was not possible to achieve the Y correction needed with the original Green over value example. Minority primary scaling can work best for excess Blue excursions and worst for excess Green excursions.

Likewise dominant pair scaling should not be used for low saturation examples, the eye is quite sensitive to changes in low saturation values. Think of white point difference between 9300K, 6500K and 5000K to the eye these are all very low saturation blues and reds

And of course small changes in mid scale luminance for high saturation colours are more noticable, making all scaling a less desirable choice.

We are probably being excessively picky here, but gamut correction is for the perfectionist's anyway. Most of the population watch coloured TV and just don't care/don't know any better.

[yesgrey]

We are probably being excessively picky here, but gamut correction is for the perfectionist's anyway.
Well, with the 3D LUT method, implementing all of this will not increase the cpu load, and it's not too hard to code, so let's go be picky.:)

[tritical]

I'll admit that I'm one of those people who don't really care. When I watch stuff on my computer I never worry about colorimetry or gamut correction. I have a cheap 19 inch flat panel monitor :).

Anyways, initial speed tests show the yuy2->rgb lut method is much faster than calling converttorgb32().rgb3dlut(), but it isn't going to beat out just converttorgb32(). Some tests on my quad core Q6600 using 720x480 mpeg2 video decoded with dgdecode to yv12:

220fps converttorgb32()

76fps converttorgb32().ddcc(threads=1)
111fps converttorgb32().ddcc(threads=2)
143fps converttorgb32().ddcc(threads=4)

118fps converttorgb32().rgb3dlut(threads=1)
138fps converttorgb32().rgb3dlut(threads=2)
145fps converttorgb32().rgb3dlut(threads=4)

175fps converttoyuy2().rgb3dlut(threads=1)
200fps converttoyuy2().rgb3dlut(threads=2)
210fps converttoyuy2().rgb3dlut(threads=4)

These results seem a little strange... have to investigate.

[leeperry]

I'll admit that I'm one of those people who don't really care. When I watch stuff on my computer I never worry about colorimetry or gamut correction. I have a cheap 19 inch flat panel monitor :).
I believe you start to care when you use a projector, as these things have wide gamuts.....and a flashy picture on a big projection screen looks really ugly :o

nice speed improvement! with proper chroma upsampling at that...can't wait to try a new beta :thanks:

[yesgrey]

I'll admit that I'm one of those people who don't really care. When I watch stuff on my computer I never worry about colorimetry or gamut correction. I have a cheap 19 inch flat panel monitor :).
But have you tryed it? probably not, because I think that maybe you don't know the coordinates of your monitor primaries.
I also don't care much about it, because since I'm not english native, sometimes I have to use subtitles, and with it I spend more time looking at the subtitles than at the movie colors...:D but when I disable the subtitles and listen to the audio directly, the difference is very noticeable and pleasant.:)

Anyways, initial speed tests show the yuy2->rgb lut method is much faster than calling converttorgb32().rgb3dlut(), but it isn't going to beat out just converttorgb32().
Well, if we could load the 3D LUT in the graphics card memory and performing the mapping through it, maybe it would beat the convertorgb32() alone. Even as it is now, is very fast, but it's strange that the speed changes with the number of threads... it should be limited by the memory access speed, right?

[tritical]

I put up the new version. I thought the fps numbers were weird compared to tests I ran two weeks ago, but those numbers were on a dual core using xvid encoded input. For rgb3dlut speed increases going from 1 to 2 threads, but hardly at all from 2 to 4. I think it is probably because this quad core is actually two dual cores each with its own 4MB L2 cache.

[leeperry]

nice! trying it as we speak...it looks fast as hell :eek:

but because ddcc takes care of the RGB conversion, shouldn't it let us choose the YCbCr>RGB decoding matrix ?

Rec. ITU-R BT.601-5 => PAL / SECAM / NTSC (SD)
Rec. ITU-R BT.709-4 => HD

you can have a SD video w/ EBU gamut, or a HD video w/ SMPTE-C gamut as you know.

also letting us choose the input levels would be nice, as I'm sending 0-255 content to ddcc(or just assume 0-255 input and let ppl use colorYUV(levels="tv->pc"))

and I've played around again w/ gam_i/o=3 or 5 but I always go back to 5/5.

[yesgrey]

well mark0077 has been recently pointing out that the nvidia drivers in YV12(using software renderers) were offering better chroma upsample than even ConvertToRGB32() :
http://forum.doom9.org/showpost.php?p=1230065&postcount=27

I'm not sure how they do it, but could you please set up a very HQ chroma upsample scheme ? like spline36 ?
Try using avisynth 2.6. With it you can select the resizer for the chroma upsampling, even spline64.

[yesgrey]

but because ddcc takes care of the RGB conversion, shouldn't it let us choose the YCbCr>RGB decoding matrix ?
Yes. I will write a little program, based on some of ddcc code, for creating the 3D LUT files. I will consider this, and even the possibility of defining custom levels for the conversion, like it's possible now with ffdshow. This way, I could use just rgb3dlut and control myself the yuv->rgb conversion.
I also want to add custom gamma correction, including the possibility of using the display gamma curve.

[leeperry]

Try using avisynth 2.6. With it you can select the resizer for the chroma upsampling, even spline64.
ORLY ? well MT 0.7 doesn't support Avisynth 2.58.
I know chromaresample="spline36" exists, but it's not used for chroma upsampling in RGB32 conversion from what IanB told me.
Yes. I will write a little program, based on some of ddcc code, for creating the 3D LUT files. I will consider this, and even the possibility of defining custom levels for the conversion, like it's possible now with ffdshow. This way, I could use just rgb3dlut and control myself the yuv->rgb conversion.
I also want to add custom gamma correction, including the possibility of using the display gamma curve.
oh OK, I thought tritical would simply add an option to choose REC601/709(and assume 0-255 input), and that everything would be fine :o

anyhow, I failed to understand how I could build the LUT in RGB32 in ddcc() and then use it in YUY2 in rgb3dlut()....so that would explain :D

so rgb3dlut() will assume 0-255 input for YUY2>RGB32 conversion then ?

I'm back to ddcc 1.5 for the time being..or maybe I'll rebuild the LUT's as off-gamut colors seem to be even better handled now :cool:

[leeperry]

OK so I've rebuilt the LUT's, my 2 displays(CRT/DLP pj) both carry off-gamut colors, and the new ddcc looks absolutely stunning :eek:

playing HD SMPTE-C movies yields very true to life colors....actually they've never looked so good :cool:

as soon as I started playing around w/ the PS script, I quickly realized that when you watch movies in their native gamut the contrast ratio increases accordingly....mainly coz colors are perfectly spot-on and not "polluted" anymore.

a friend of mine, who got me into that gamut craziness also told me the same thing....he's got the holy Samsung SP-A800B pj and runs a website about gamuts & stuff : www.hdsoir.com (sorry, french only :o )

@yesgrey : if you can get ddcc working equally accurately in YUY2, that sure will be a blast! :D

[cyberbeing]

I've read the documentation, but I'm a bit confused about how to properly use this filter. I currently have two GDM-F520 CRT monitors calibrated and profiled with an Eye-One Pro spectrophotometer to D65 2.2 for photoediting/graphic design use.

Could someone give me a step-by-step of how to create the required txt file(s)?

Do I only need to create an ofile?
Why would I need an ifile?
Is lutfile automatically created by ddcc or do I have to create that as well?

It sounds like rgb3dlut is faster and therefore preferred to ddcc?
Once I have the required files, what would be an example script for bt.709 video? For bt.601 video?
How should I setup ffdshow's output panel?

[yesgrey]

First you have to measure the chromaticity coordinates of your displays, in this case of both your crt's, you will need it for filling the data in the ofile.
Generally you don't need the ifile, only if you will use source material from a standard different than the already included in ddcc.

Currently the 3D lut file is created by ddcc when you specify a name for it when calling the function. Then, rgb3dlut uses that file.

I'm writting a little program, using some parts of ddcc, to create the 3D LUT files in an alternative way. These files would then be used by rgb3dlut.
Wait a few more days (I'm waiting for the weekend to finish it), I will post my program with instructions and scripts for the more usual setups.;)

[cyberbeing]

First you have to measure the chromaticity coordinates of your displays
That is my main confusion. I'm unsure what chromaticity coordinates you are talking about.

What program do I use? If not easy to figure out, how would I measure what is needed?
What do I need to measure?
After measuring, where in the results would I find the chromaticity coordinates needed for the ofile?

[yesgrey]

What program do I use? If not easy to figure out, how would I measure what is needed?

Well, you can look here (http://www.avsforum.com/avs-vb/showthread.php?t=912720) to understand what are the coordinates we need, and you can look here (http://www.homecinema-fr.com/colorimetre/index_en.php) for a free software to help you measuring the coordinates.

[cyberbeing]

OK, I think I figured it out, but ddcc is throwing an error:
ddcc: error reading from file (C:\Program Files\Avisynth 2.5\bt709.txt)

I was using the following script to try to generate the lut file:

dss2("F:\test.mkv",fps=23.976)
ConvertToRGB24()
ddcc(chr_i=0, gam_i=1, ofile="C:\Program Files\AviSynth 2.5\bt709.txt", lutfile="C:\Program Files\AviSynth 2.5\3D_LUT_BT709.txt", threads=0, opt=-1)


the bt709.txt contains:
http://img144.imageshack.us/img144/296/ofilegi8.png

What am I doing wrong?

[yesgrey]

What am I doing wrong?
If you are using ddcc v1.6, you should supply only the x and y coordinates. In your file you have specified x,y and z.

[cyberbeing]

Oh, I guess that explains the error. I was just basing my file off one of the old examples earlier in this thread, but I didn't catch it had changed. I removed the z value and it worked.

Now I've run into another problem. Using YUY2 input with rgb3dlut gives me messed up chroma (http://img144.imageshack.us/img144/5751/0000ur6.png):
ConvertToYUY2()
rgb3dlut(lutfile="C:\Program Files\AviSynth 2.5\3D_LUT_BT709.txt", threads=2)

If I use RGB input the colors seem correct:
ConvertToRGB32(matrix="Rec709")
rgb3dlut(lutfile="C:\Program Files\AviSynth 2.5\3D_LUT_BT709.txt", threads=2)

Am I missing something else, or in ddcc 1.6 are you unable to use YUY2 input with rgb3dlut?

[yesgrey]

Now I've run into another problem. Using YUY2 input with rgb3dlut gives me messed up chroma...
Am I missing something else, or in ddcc 1.6 are you unable to use YUY2 input with rgb3dlut?
Read this (http://forum.doom9.org/showthread.php?p=1236588#post1236588).

[leeperry]

Read this (http://forum.doom9.org/showthread.php?p=1236588#post1236588).
I was reading again that you would let us input the display gamma curve.

does that mean that we could input all the data from ColorHCFR ? 9 points for each primary color ? :cool:

http://pix.nofrag.com/2/f/a/731617f4652b055fc4d4498489ac4.png

and to get back on the orangey red problem in SMPTE-C, I'm often seeing this kind of UGLY red's :
http://pix.nofrag.com/d/e/c/18ccaea39e7817bbb9a670494f9a8tt.jpg (http://pix.nofrag.com/d/e/c/18ccaea39e7817bbb9a670494f9a8.html)

this is the Dumb & Dumber BD, mostly the damn cameras didn't sample the Ferrari red properly...the major flaw in SMPTE-C.

in oversaturated REC709 demos, red cars are actually red :
http://img141.imageshack.us/img141/7003/31bv.jpg

it's discussed here :
http://www.avsforum.com/avs-vb/showthread.php?t=1038602

and here :
http://www.google.com/search?hl=en&q=SMPTE-C+orangey+red&btnG=Search&lr=

EDIT: using this Ferrari colors panel, maybe it was a "Rosso Corsa" red after all....but still undersaturated :rolleyes:

http://www.jb330gt.com/color/1995FerrariBig.jpg

[cyberbeing]

Read this (http://forum.doom9.org/showthread.php?p=1236588#post1236588).

I don't see the answer to my question in what you linked.

It would be much easier if you could just give me simple answers to my questions instead of linking me all over the place... I've already previously read through everything you have linked me to so far before I even thought about asking for help... I need a clear answer beyond what has already been posted... I don't want to play 20 questions...

Can rgb3dlut be used with YUY2 input? The documentation suggests the answer is yes.

What is causing the problem I'm seeing with YUY2 input? The lutfile? Something else?

If the lutfile isn't the problem, how did tritical get it to work with YUY2?

If the lutfile is the problem, then I assume that you are working on an app for creating the needed lutfile for YUY2 input?

[leeperry]

If the lutfile is the problem, then I assume that you are working on an app for creating the needed lutfile for YUY2 input?
it'll be possible when the app will be ready, now just wait like all of us :o

[yesgrey]

It would be much easier if you could just give me simple answers to my questions instead of linking me all over the place...
Some answers are not simple, and is boring having to repeat long answers several times. Each long answer that I repeat, is less time I can put in coding...;)

If the lutfile is the problem, then I assume that you are working on an app for creating the needed lutfile for YUY2 input?
Yes.

[yesgrey]

I was reading again that you would let us input the display gamma curve.

does that mean that we could input all the data from ColorHCFR ? 9 points for each primary color ?
Yes, I am thinking in it, but not in the first release. First it will be just the same as ddcc v1.6, but including YCbCr->RGB conversion.

[leeperry]

Yes, I am thinking in it
http://forum-images.hardware.fr/images/perso/nico54.gif

[cyberbeing]

Yes.
Thank you, that is all I needed to know.

The lutfile is the problem and without a properly created lutfile, it won't work with YUY2 input.

You are working on a solution to make that lutfile, and until it is ready I'll have to wait.
:thanks:

[yesgrey]

Here is a link (http://www.megaupload.com/?d=JAVPPV3Q) with 4 3D LUT files just for testing purposes, to keep you busy while I finish the first version of my little program...;)

The 3D LUTs are only for performing the YCbCr->RGB conversion. This way, we can test the speed and the quality of using rgb3dlut versus other options available for the conversion (like ConverToRGB32, ffdshow, graphic cards drivers).
There are 4 files:
-BT.601 with 16-235 levels
-BT.601 with 0-255 levels
-BT.709 with 16-235 levels
-BT.709 with 0-255 levels

Don't forget that the speed will be exactly the same with the full options available for creating the 3D LUT files.

[leeperry]

awesome, thanks!

so from the readme :

itype -

For yuy2 input this sets how to compute the u/v values for the second y value in each yuyv set.
Possible settings:

0 - duplicate (use u/v of first y value)
1 - linear interpolation (average u/v of first y with u/v of first y in next yuyv set)

ConvertToRGB32(matrix="rec601")
http://www.image-load.eu/out.php/t141613_convert32601.png (http://www.image-load.eu/out.php/i141613_convert32601.png)

ConvertToYUY2()
rgb3dlut(lutfile="C:\3dluts\3dlut_ycbcr_16-235.txt",itype=0)

http://www.image-load.eu/out.php/t141614_ddccitype0.png (http://www.image-load.eu/out.php/i141614_ddccitype0.png)

ConvertToYUY2()
rgb3dlut(lutfile="C:\3dluts\3dlut_ycbcr_16-235.txt",itype=1)

http://www.image-load.eu/out.php/t141615_ddccitype1.png (http://www.image-load.eu/out.php/i141615_ddccitype1.png)

major changes :eek:

the chroma looks a lot more accurate, not some smearing pixelating hollow anymore :eek:

PS: I didn't have the BT.709 LUT when I ran the tests, but this doesn't really matter.

[yesgrey]

YV12 to RGB is possible in exactly the same way as the proposed YUY2 to RGB, you just have to pre-interpolate vertically as well.

Yes, I referred that, but it's not possible to include the interpolation in the 3D LUT values. The interpolation should be done before mapping with the 3D LUT, it's the only way to know what to map from...;)

IanB,
Now that I am messing with YCbCr->RGB conversion, I realized that I haven't understood exactly what you have said in that quote. I thought that YUY2 had all the CbCr data for the correponding Y values, but now I know it doesn't, it only has 1 CbCr data for 2 Y data.
Now I see that the YV12 input could also be used in rgb3dlut. I hope it's not too much work, and that tritical could find the time for doing it...;)

[yesgrey]

the chroma looks a lot more accurate, not some smearing pixelating hollow anymore :eek:
Yes, I agree that rgb3dlut YCbCr->RGB conversion looks better.
I have zoomed to compare, and when the linear interpolation is used (itype=1), the color grading is exactly the same from ConvertToRGB32, but it doesn't have the smearing around it...
This makes me think that if rgb3dlut would also accept YV12 input the result could be even better... maybe ConvertToYUY2 is also adding some smearing of it's own...

[leeperry]

This makes me think that if rgb3dlut would also accept YV12 input the result could be even better... maybe ConvertToYUY2 is also adding some smearing of it's own...
exactly my words when you guys started that YCbCr thingie....tritical's algorithms are a lot more accurate than Convert(), so we should bypass it altogether if any possible :cool:

until your code is ready, I will now use rgb3dlut() twice in a row, once for RGB32 conversion and once for gamut conversion :D

EDIT: I might also need to lower my sharpening in ffdshow, it's too damn sharp now :eek:

[mark0077]

so there is no direct yv12 to rgb32 lut? Is one in production :)

[yesgrey]

so there is no direct yv12 to rgb32 lut? Is one in production :)
The yv12 to rgb32, as the yuy2 to rgb32 conversions could not be performed by a 3D lut. The 3D LUT converts only betwwen YCbCr and RGB, and for that you must have the Y and CbCr values of each pixel.
In YUY2 you have 2 Y and 1 CbCr for 2 pixels, and in YV12 you have 4 Y and 1 CbCr for 4 pixels. The CbCr values for the other 1 or 3 pixels, have to be calculated by rgb3dlut. Currently, it's already doing it for YUY2, let's hope the same could happen for YV12...

So, what we need for YV12->RGB32 is not a new lut, but a new rgb3dlut version...;)

[yesgrey]

leeperry,
Could you also post a screenshot using ffdshow's conversion?
Thanks!

[leeperry]

here's ffdshow in RGB32/601 :

http://thumbnails3.imagebam.com/2490/fe9a1124894461.gif (http://www.imagebam.com/image/fe9a1124894461)

and in RGB32HQ/601 :

http://thumbnails9.imagebam.com/2490/6a0f6e24894469.gif (http://www.imagebam.com/image/6a0f6e24894469)

http://img132.imageshack.us/img132/1697/31370080kq2.png

the file sizes are pretty self-explanatory....the smearing red costs, and RGB32HQ=ConvertToRGB32()

also the black background is actually R0-G0-B0 in ddcc, but in Convert()(full range 601)/ffdshow(full range 601) it's R2-G0-B1 :confused:

the sample is still available here :
http://forum.doom9.org/showpost.php?p=1137196&postcount=1868

and here's some real world comparisons, top is Convert(709), bottom is ddcc(709) :

http://thumbnails16.imagebam.com/2490/f405bd24894487.gif (http://www.imagebam.com/image/f405bd24894487)http://thumbnails15.imagebam.com/2490/f137f324894509.gif (http://www.imagebam.com/image/f137f324894509)http://thumbnails10.imagebam.com/2490/054af024894527.gif (http://www.imagebam.com/image/054af024894527)http://thumbnails11.imagebam.com/2490/208cb824894559.gif (http://www.imagebam.com/image/208cb824894559)
http://thumbnails11.imagebam.com/2490/257a6b24894500.gif (http://www.imagebam.com/image/257a6b24894500)http://thumbnails2.imagebam.com/2490/922c4d24894517.gif (http://www.imagebam.com/image/922c4d24894517)http://thumbnails10.imagebam.com/2490/aa353524894543.gif (http://www.imagebam.com/image/aa353524894543)http://thumbnails14.imagebam.com/2490/635f1324894578.gif (http://www.imagebam.com/image/635f1324894578)

http://img262.imageshack.us/img262/4982/42859825xo0.png

the red shade is slightly different in ddcc, more saturated :confused:

[yesgrey]

as I told you in PM the 2 BT.709 LUT's are mixed up, the 0-255 is actually 16-235 ;)
The Luts designation refers to the black and white in RGB.
The BT.709_0-255 is
Y:16-235 -> RGB: 0-255
and the BT.709_16-235 is
Y:16-235 -> RGB: 16-235

and RGB32HQ=ConvertToRGB32()
As it's supposed to be, the HQ mode uses Avisynth's code.

also the black background is actually R0-G0-B0 in ddcc, but in Convert()(full range 601)/ffdshow(full range 601) it's R2-G0-B1 :confused:

Probably some small error in the matrix coefficients...