gamut conversions through Avisynth ?

[Yellow_]

@Dogway, I understand now. :-)

yesgrey, if you're still about, is it possible to use yCMS to create a LUT that does this, the way Cineform handles levels, quoted by Cineform:

"Downstream tools generally don't handle full range YUV well, so we take the 0-255 input, bump that to 10-bit 0-1023, then range correct to 64-940, compressing the results. So we made a standard range YUV without clipping, and without loss of codewords as we bumped to 10-bit first. This greatly simplifies YUV playback or HDSDI/HDMI devices."

I'm aware yCMS can do the work in 16bit, then dither down to 8bit and input and output levels can be specified. So is it possible to do the above at 16bit?

Also I find out that Canon DSLR video is BT601 not 709, so should I use YCbCr or HD or PAL or specify 601 for primaries and colour coefficients when going to RGB?

Cheers

[yesgrey]

"Downstream tools generally don't handle full range YUV well, so we take the 0-255 input, bump that to 10-bit 0-1023, then range correct to 64-940, compressing the results. So we made a standard range YUV without clipping, and without loss of codewords as we bumped to 10-bit first. This greatly simplifies YUV playback or HDSDI/HDMI devices."
yCMS already does better than that. It takes the 0-255 input, converts that to 64 bit fp 0.0-1.0, then range correct according to specified output bit depth, which could be 8bit or 16bit. If you use 16bit it would be better than the above.

I'm aware yCMS can do the work in 16bit, then dither down to 8bit and input and output levels can be specified. So is it possible to do the above at 16bit?
yCMS never uses dithering, because it only creates the 3DLUT file. Dithering should be used only when reducing the bit depth for the processed streams/images, not for the 3DLUT. What you're referring is the t3dlut working mode, and that I'm afraid could not be changed, because Avisynth is limited to 8 bit processing. So, it's not a yCMS limitation, but Avisynth's. However, by dithering from 16 bit to 8 bit the compression would not be a problem, only a slightly higher noise level on the images,

Also I find out that Canon DSLR video is BT601 not 709, so should I use YCbCr or HD or PAL or specify 601 for primaries and colour coefficients when going to RGB?
You should select PAL, if you're from PAL land, or NTSC, if you're from NTSC land.

[Yellow_]

yesgrey, thanks for the clarifications, my bad with regard to 8bit 16bit in yCMS, not dither but range correct, dithering in t3dlut.

Thanks for the tool, it's excellent.

[yesgrey]

not dither but range correct
Yes, you can do range correct in yCMS. I decided to use only values between 0-255 to simplify the interface, because it's easier to think that we will convert 0-255 -> 16-235. However, the accurate range values are calculated internally. So, if you select 16-235 as output, but you're using a 16 bit bit-depth, the real output range values would be 4096-60160.

[Yellow_]

Just to clarify.

For doing 601 full range PAL to 8bit RGB:

Input_Format PAL YCbCr 8
Output_Format PAL RGB_PC 16

Gives me a LUT suitable for taking 0 - 255 and 'squashing' into 16 - 235 out

Substituting 'PAL' for 'HD' would give me a LUT suitable for the same but with for example HDV video camera sources?

Adding the line 'Output_Range 0 255' to either would give me full range output in either PAL or HD?

Is it as simple as that, no other parameters?

[yesgrey]

For doing 601 full range PAL to 8bit RGB:

Input_Format PAL YCbCr 8
Output_Format PAL RGB_PC 16

Gives me a LUT suitable for taking 0 - 255 and 'squashing' into 16 - 235 out
No.

When using the mandatory commands (Input/Output_Format) YCbCr is always considered to be within the standard range:
Y: 16-235
CbCr: 16-240

For full range you should add the line:
Input_Range 0 255

And for 'squashing' to 16-235 you should add this other line:
Output_Range 16 235

Substituting 'PAL' for 'HD' would give me a LUT suitable for the same but with for example HDV video camera sources?
Only if the 3DLUT's are meant to 'squash' the sources and keep the video format. If you will use them for watching the files then you should set as output format your display's video standard, and not the same as the source.

[Yellow_]

Thanks again yesgrey.

Considering that the Canon DSLR is BT601 but HD. Then really I should use the transfer function to 709 as well when going to an intermediate file for editing and reencoding?

Just thinking about final playback, aiming for widest compatibility with playback devices, should I really be encoding HD size material with 709, I assume that's what players will expect, or is 601/709 flagged in the encoding added by the encoder, say with x264.

I've read players are renowned for doing there own thing, but as best practice, 709?

[yesgrey]

Considering that the Canon DSLR is BT601 but HD. Then really I should use the transfer function to 709 as well when going to an intermediate file for editing and reencoding?
What do you mean with "Canon DSLR is BT601 but HD". Are you referring to the primaries, the transfer function, or the encoding matrix?
The transfer function is not a problem, because it's the same in both BT.601 and BT.709. The differences between them are the encoding matrices and the primaries.

Just thinking about final playback, aiming for widest compatibility with playback devices, should I really be encoding HD size material with 709, I assume that's what players will expect
Yes. According to what I've read everywhere, the players and TV sets use BT.709 when they detect a HD signal. It might happen that some equipments don't, but that's the exception (manufacturer mistake) and not the rule.

[Yellow_]

What do you mean with "Canon DSLR is BT601 but HD". Are you referring to the primaries, the transfer function, or the encoding matrix?
The transfer function is not a problem, because it's the same in both BT.601 and BT.709. The differences between them are the encoding matrices and the primaries.

Whether it's colour primaries or matrix, I'm unsure, all i know is that 601 is said to be "flagged in the bitstream"? FFMPEG for example reads Canon 7D and 550D DSLR video as 601 where as HDV video from my other vidcam is read as 709.

[yesgrey]

That's strange... I've googled a bit about those two models and I saw no reference whether it's PAL or NTSC, so it would be very strange if the cameras use BT.709 primaries and BT.601 encoding matrix... Did you see any reference to PAL or NTSC in the user manuals?

[poisondeathray]

Color primaries : BT.709-5, BT.1361, IEC 61966-2-4, SMPTE RP177
Transfer characteristics : BT.709-5, BT.1361
Matrix coefficients : BT.601-6 525, BT.1358 525, BT.1700 NTSC, SMPTE 170M


This is the mediainfo metadata for the 7D, it's the same for 5D MK2

[yesgrey]

Color primaries : BT.709-5, BT.1361, IEC 61966-2-4, SMPTE RP177
Transfer characteristics : BT.709-5, BT.1361
Matrix coefficients : BT.601-6 525, BT.1358 525, BT.1700 NTSC, SMPTE 170M
For characterizing this source in yCMS it should be used:

Input_Format HD YCbCr 8
Input_Matrix_Coefficients 3

if you prefer a simpler form, or

Input_Primaries 0
Input_Transfer_Function 1
Input_Matrix_Coefficients 3
Input_Bit_Depth 8
Input_Range 16 235

if you prefer the advanced/detailed form.

[Yellow_]

Thanks guys. I've now installed Mediainfo, a fine tool. Canon 550D/T2i is same as 7D & 5D mkII. All sources are PAL.

Prefer the advanced / detailed form for yCMS. :-)

So, for example, using the detailed form above gives me suitably 'decoded', colourmetrics 'neutral' source which I can then use to go to full range RGB using RGB_PC 16 and / or Output_Range 0 255 and then from that RGB intermediate to whatever, 601, 709 just using matrix co efs without the need for transfer function?

Looking at some HD encoded files done previously from edits and also source mpeg2 from a HDV cam, mediainfo just lists colormetry as 4:2:0, assume then these are not flagged sufficiently, so does that become more of a lottery at playback, as to what colourmetrics the player uses, perhaps based on frame size/pixel count where colourmetrics are not flagged?

Thanks for your patience with this, it must quite frustrating. :-)

[yesgrey]

Thanks guys. I've now installed Mediainfo, a fine tool. Canon 550D/T2i is same as 7D & 5D mkII. All sources are PAL.
If it's the same as above it can't be PAL, it has different primaries.

Prefer the advanced / detailed form for yCMS.
Me too, but I'm suspect.;)

So, for example, using the detailed form above gives me suitably 'decoded', colourmetrics 'neutral' source which I can then use to go to full range RGB using RGB_PC 16 and / or Output_Range 0 255 and then from that RGB intermediate to whatever, 601, 709 just using matrix co efs without the need for transfer function?
RGB_PC assumes always full range (0 255), so you won't need to use the Output_Range command. Of course you could also use the advanced/detailed form for the output, you just need to know what you want to get.

The RGB intermediate only makes sense if you plan to process the image in RGB, but then, if you want to output as YCbCr, you would need to use another 3DLUT for performing the RGB->YCbCr conversion. Is that your intention?

Looking at some HD encoded files done previously from edits and also source mpeg2 from a HDV cam, mediainfo just lists colormetry as 4:2:0, assume then these are not flagged sufficiently, so does that become more of a lottery at playback, as to what colourmetrics the player uses, perhaps based on frame size/pixel count where colourmetrics are not flagged?
If you know which camera was used for the capture you could try to know, otherwise it's safer to stick to the standards. Another option would be using the standard and the other and decide which look best.

[Yellow_]

If it's the same as above it can't be PAL, it has different primaries.

I'll check again. The 550D/T2i does both PAL & NTSC, but I use it mainly on PAL, just NTSC to get the 60fps for slow mo. (720P)

The RGB intermediate only makes sense if you plan to process the image in RGB,

Unfortuneatley the NLE / Compositor / 3D app I use stuffs up video sources with regard to levels and colourmetrics, (FFMPEG based) as it works solely in RGB. So I prefer to do a controlled conversion with AVISynth, then finally encode back to video for final delivery. Also want to maintain full range levels in editing / grading then squash 16 - 235 for final delivery.

but then, if you want to output as YCbCr, you would need to use another 3DLUT for performing the RGB->YCbCr conversion. Is that your intention?

Yes, I've used ConvertToYUY2 or YV12. However since finding yCMS + t3dlut I've created some 'Output' LUTS via yCMS instead which include 16 - 235 levels. I assume encoders don't provide method to set levels, generally just pass what they get through?

If you know which camera was used for the capture you could try to know, otherwise it's safer to stick to the standards. Another option would be using the standard and the other and decide which look best.

The 4:2:0 files are from a Canon HV30 HDV cam and the other 4:2:0 files are generally XVid and h264 final encodes that don't flag colourmetrics I guess.

Thanks again.

[Yellow_]

Check mediainfo and get this for 550D videos:

Color primaries : BT.709-5, BT.1361, IEC 61966-2-4, SMPTE RP177
Transfer characteristics : BT.709-5, BT.1361
Matrix coefficients : BT.601-6 525, BT.1358 525, BT.1700 NTSC, SMPTE 170M

The camera is set to it's PAL mode as opposed to NTSC. Whether that's strictly to spec I don't know.

[yesgrey]

The BT.709 primaries are not much different from the PAL primaries, only the green changes, and it's a very small change, so the effect would not be significant. What's more important is using the right matrix, and that it does. However, due to the slight difference of the green primary it would not follow exactly the standards.
Just FYI the PAL's green coordinates are: 0.2900, 0.6000, while BT.709's are 0.3000, 0.6000 .

Just for curiosity, what do you get when the camera is set to NTSC mode?

[Yellow_]

What's more important is using the right matrix, and that it does.

You mentioned in an earlier post that it would be strange for the camera to use 709 primaries and 601 matrix. But above that it uses the correct matrix, ie: 601?

Just for curiosity, what do you get when the camera is set to NTSC mode?

Mediainfo gives exact same for NTSC mode from camera.

Assume th mix of primaries and matrix, increases chances of apps misjudging how to handle it? For example FFMPEG suggests 601?

I see variations in different apps with regard to brightness (assume levels stretching there) and colour hue mainly a difference in red. One app will show something of reddish colour more orange, generally when using 709 and the same source with a definite pink hue instead of orange, generally when using 601.

There's much 'folk law' about Canon being on the 'red' side, wonder how much that is to do with camera settings (I use Neutral profile and knocked magenta down a bit in custom white balance) or assuming incorrect colour primaries 601 instead of 709, would that make any real difference I wonder?

Lightworks NLE goes Free as Open Source on the 29th and offers handling of DSLR sources + GPU effects and playback/editing without transcoding to DNxHD or similar first. Time will tell how successful that is. Which may or may not result in a second workflow, other than the RGB one, staying YCbCr. I think there are still benefits to using your LUT + td3lut for precision and eeking most out. Many advocate Cineform products for similar reasons, high precision, upsampling and more lightweight codec.

Considering whether it's beneficial to decode the Canon sources as we've discussed 709 primaries & 601 matrix and put them into 709 and use a lossless codec like UT / lossless h264 / Matrox I Frame ? Sort of a 'poor mans' Neoscene. :-) Or maybe better?

[yesgrey]

You mentioned in an earlier post that it would be strange for the camera to use 709 primaries and 601 matrix. But above that it uses the correct matrix, ie: 601?
Yes, but I did not remember that BT.709 and PAL primaries are almost the same. Yes, it uses the correct matrix.

Mediainfo gives exact same for NTSC mode from camera.
With NTSC it would not be that good... Look at all the primaries (R;G;B):
BT.709: 0.6400, 0.3300; 0.3000, 0.6000; 0.1500, 0.0600
PAL: 0.6400, 0.3300; 0.2900, 0.6000; 0.1500, 0.0600
NTSC: 0.6300, 0.3400; 0.3100, 0.5950; 0.1550, 0.0700

There's much 'folk law' about Canon being on the 'red' side... or assuming incorrect colour primaries 601 instead of 709, would that make any real difference I wonder?
It might be from the primaries... If you compare the Red coordinates from above you could see that the NTSC Red is more saturated than the BT.709 one. If you capture using BT.709 and then process assuming NTSC the end result might be just that... To be sure you would need to process the same file considering both options and compare.

Assume th mix of primaries and matrix, increases chances of apps misjudging how to handle it? For example FFMPEG suggests 601?
Ideally you should use all correct parameters, but considering not all applications allow the selection of the primaries at least choose the right coefficients matrix.

Considering whether it's beneficial to decode the Canon sources as we've discussed 709 primaries & 601 matrix and put them into 709 and use a lossless codec like UT / lossless h264 / Matrox I Frame ? Sort of a 'poor mans' Neoscene. :-) Or maybe better?
I can't help you on that. I'm not in the editing area, so I really don't know the results...

[Yellow_]

yesgrey, I really appreciate your continued replies.

In trying to read around the subject and try to get a better understanding I found info on xvYCC, http://en.wikipedia.org/wiki/XvYCC and stated there:

xvYCC-encoded video retains the same color primaries and white point as BT.709, and uses either a BT.601 or BT.709 RGB-to-YCC conversion matrix and encoding. This allows it to travel through existing digital YCC data paths, and any colors within the normal gamut will be compatible.

Which sounds like what the Canon DSLR is doing? It's h264 AVC.

Although it suggests for final output:

xvYCC is not supported by DVD-Video or Blu-ray, but is supported by the high-definition recording format AVCHD and PlayStation 3.

However when reading about AVCHD it is suggested it is compatible and designed for Blu-ray, I assume there are various profiles and the higher end of AVCHD accommodate the xvYCC gamut?

Is it possible to use a LUT and retain the xvYCC data in an intermediate for grading / image manipulation and then encoding out to a lesser gamut like 709 or 601?.

Would it need to be as RGB? maybe described in a wider colourspace than sRGB, AdobeRGB perhaps? I see yCMS includes an AdobeRGB output option.

[Yellow_]

yesgrey, could you clarify whether I have this right:

Source is 709 primaries, 601 matrix. Want to convert to 709 primaries and matrix and export from Vdub as uncompressed YCbCr HD. The bit I'm unsure about is the need for the Output transfer matrix or not required?

# Set input format


Input_Primaries 0

Input_Transfer_Function 1

Input_Matrix_Coefficients 3

Input_Bit_Depth 8

Input_Range 0 255



# Set output format



Output_Primaries 0

Output_Transfer_Function 0

Output_Matrix_Coefficients 1

Output_Bit_Depth 16

Output_Range 0 255

re Adobe RGB & xvYCC query from previous post, when converting using 709 pri and 601 matrix to AdobeRGB, I get less colour info per frame using AdobeRGB than sRGB, I'm ensuring that Gimp uses the correct ICC under colour management for importing the image frames, I'd assumed I see better colour info per frame, ie: wider gamut.

For AdobeRGB I was using the same input details as above with these outputs:

Output_Primaries 4

Output_Transfer_Function 5

Output_Matrix_Coefficients 0

Output_Bit_Depth 16

Output_Range 0 255

Does all look correct for generating the LUT's?

[yesgrey]

Source is 709 primaries, 601 matrix. Want to convert to 709 primaries and matrix and export from Vdub as uncompressed YCbCr HD.
# Set input format
Input_Primaries 0
Input_Transfer_Function 1
Input_Matrix_Coefficients 3
Input_Bit_Depth 8
Input_Range 0 255

# Set output format
Output_Primaries 0
Output_Transfer_Function 0
Output_Matrix_Coefficients 1
Output_Bit_Depth 16
Output_Range 0 255

Almost correct. You shouldn't change the TF. You should use:
Output_Transfer_Function 1

If you're not using any of the mandatory commands you must use all STANDARDS DEFINITION commands for input and output.

re Adobe RGB & xvYCC query from previous post, when converting using 709 pri and 601 matrix to AdobeRGB, I get less colour info per frame using AdobeRGB than sRGB, I'm ensuring that Gimp uses the correct ICC under colour management for importing the image frames, I'd assumed I see better colour info per frame, ie: wider gamut.

For AdobeRGB I was using the same input details as above with these outputs:
Is your source full range YCbCr? If not, try using:
Input_Range 16 235

[diffid]

Post removed, wrong thread.

[Yellow_]

Almost correct. You shouldn't change the TF. You should use:
Output_Transfer_Function 1

If you're not using any of the mandatory commands you must use all STANDARDS DEFINITION commands for input and output.

Just to clarify, I have missed the mandatory? So should add:

Input_Format HD YCbCr 8 & Output_Format HD YCbCr 8

When you say almost correct, which is almost correct the 709pri /601 matrix to 709 or the AdobeRGB route? Obviously I want to get the LUTs right before using them on a full project. :-)

If I'm for example going from 709 pri / 601 matrix to 709 for both, then will either YCbCr or HD work the same in Output_Format? Which would be correct?

Is your source full range YCbCr? If not, try using:
Input_Range 16 235

Yes always full range from the cameras I'm using. But if I haven't been doing the LUT generation correctly for these latest tests then that might explain the AdobeRGB problem. :-)

[yesgrey]

Just to clarify, I have missed the mandatory? So should add:
Input_Format HD YCbCr 8 & Output_Format HD YCbCr 8
If you're using all STANDARDS DEFINITION commands the mandatory lose their mandatory state, so, no, you don't need to add those two lines. If you miss any command yCMS would let you know, because it never assumes any default values.

When you say almost correct, which is almost correct the 709pri /601 matrix to 709 or the AdobeRGB route?
The first. You were using the sRGB transfer function (0) instead of the BT.601/BT.709 one (1). The AdobeRGB route is correct.

If I'm for example going from 709 pri / 601 matrix to 709 for both, then will either YCbCr or HD work the same in Output_Format? Which would be correct?
If you are comfortable in using the standards definition commands stick to them. The mandatory always consider YCbCr to be in the standard range, so it's safer for you to have full control on the Input and Output settings.

Yes always full range from the cameras I'm using. But if I haven't been doing the LUT generation correctly for these latest tests then that might explain the AdobeRGB problem. :-)
Right. Let me know the results with the correct LUTs then.

[Yellow_]

Right. Let me know the results with the correct LUTs then.

Ok, I've been trying to establish whether Canon DSLR h264 with 709 priimaries / 609 matrix is xvYCC and whether it is more beneficial to try putting that into a wider RGB gamut than sRGB.

My assumptions to establish either way are:

That sRGB is not sufficient for conversion from xvYCC to RGB and hold all data. ie enough for 709 primaries and matrix, but not xvYCC.

That a wider gamut like AdobeRGB would be needed.

However using the following spec for the 3DLUT:

# Set input format

Input_Primaries 0

Input_Transfer_Function 1

Input_Matrix_Coefficients 3

Input_Bit_Depth 8

Input_Range 0 255


# Set output format

Output_Primaries 4

Output_Transfer_Function 5

Output_Matrix_Coefficients 0

Output_Bit_Depth 16

Output_Range 0 255


Gives me far less unique colours in the exported image frames, (using .tif and reading in Gimp with assumed ICC profile AdobeRGB) than exactly the same process and t3dlut settings.


The sRGB LUT used the same LUT config but substituting sRGB instead and reading the images with an sRGB ICC profile.

# Set input format

Input_Primaries 0

Input_Transfer_Function 1

Input_Matrix_Coefficients 3

Input_Bit_Depth 8

Input_Range 0 255


# Set output format

Output_Primaries 0

Output_Transfer_Function 0

Output_Matrix_Coefficients 0

Output_Bit_Depth 16

Output_Range 0 255

I've tried different methods of wrting the frames out, copy to clip in Vdub, image seq out in Vdub, imagewriter to tif in AVISynth and even Wilberts Imagemagick write to tif, all AdobeRGB ICC images are identical output and far less colours than the sRGB ones. :-(

Are my assumptions about xvYCC totally wrong or is the LUT config still not right?

[yesgrey]

Are my assumptions about xvYCC totally wrong or is the LUT config still not right?
yCMS does not support xvYCC colorspace yet, so that might be the reason for your current results. I plan to support it, but I don't know when.

[Yellow_]

lol yep that could be it. :-)

Oh well, thanks for your time and I'll look out occasionally in case you feel the urge to add support. :-)

[yesgrey]

I'll look out occasionally in case you feel the urge to add support. :-)
I guess I haven't expressed myself clearly enough...

It's not a question of feeling the urge for to add support, is because I don't have the xvYCC standard's documentation. So, until I could be able to access reliable information about it I could not do it. For the current supported formats the specs are widely spread, but for xvYCC I don't.

If you have it let me know and I will consider it soon.;)

[Yellow_]

Sorry for misunderstanding, urge was really wrong word anyway, time and resources.

I'll see what I can find, maybe others on Doom9 may shout out.

Don't want to provide links to what might not be 'reliable', it is Sony after all, but here's a link.

http://www.sony.net/SonyInfo/technology/technology/theme/xvycc_02.html

KMO offers two links in the bottom of first post here and a test pattern in second post. I think you are a part time resident at avsforum anyway. :-)

http://www.avsforum.com/avs-vb/showthread.php?t=1170632

And Neuron2s site for h264AVC additional info:

http://www.avsforum.com/avs-vb/showthread.php?p=17051065#post17051065

I'll try to find written, ratified, for purchase doc if necessary.

Thanks

[yesgrey]

Thanks for the links.

I've also searched in the documents I have and I think I have all the info needed. If it's not too much work I will try to add xvYCC support to yCMS v1.9.

[Yellow_]

Hey, that's good news.

I notice the one link I gave above to avsforum, Neuron2's site needs a password, Neuron2's suggested you PM him at his site if you'd like access.

[yesgrey]

OK, thanks.

[Yellow_]

Hi, I'm unable to eek out anymore from the Canon DSLR video source using xvYCC, maybe the camera doesn't record it. :-( Although from what I've read it appears to be an extension of ITU601 and ITU709 where the range 1 to 254 is used so assumed the camera was xvYCC as I can capture above and below 16 - 235, especially with a flatter picture style camera curve.

Did you see anything / were successful with the gold source?

Sorry to ask again but with the following header info from the camera .mov files:

Color primaries : BT.709-5, BT.1361, IEC 61966-2-4, SMPTE RP177
Transfer characteristics : BT.709-5, BT.1361
Matrix coefficients : BT.601-6 525, BT.1358 525, BT.1700 NTSC, SMPTE 170M

I've tried various combinations of settings to generate the LUT. If I want full range RGB from the above source which is 0 - 255 what settings would I need again if I want to assume the source is xvYCC.

Also will using 16bit output dithered down skew the results, been using 8bit out only anyway.

And last query :-), Do you think when going to RGB via the LUT, AdobeRGB or sRGB?

[yesgrey]

Hi, I'm unable to eek out anymore from the Canon DSLR video source using xvYCC, maybe the camera doesn't record it. :-(
That's what I thought too. The Gold sample seems to have the same number of available colors when using BT.709 or xvYCC.

I've tried various combinations of settings to generate the LUT. If I want full range RGB from the above source which is 0 - 255 what settings would I need again if I want to assume the source is xvYCC.
With that you can go as simple as:
Input_Format xvYCC_SD YCbCr 8
Output_Format videoStandard RGB_PC 16

on the Output format use as videoStandard the RGB space you want.

Also will using 16bit output dithered down skew the results, been using 8bit out only anyway.
Using the 16 bit output dithered to 8 bit is always preferable, unless you plan to compress the source, but since you're talking about RGB space I don't think you are considering any compression...

[yesgrey]

Do you think when going to RGB via the LUT, AdobeRGB or sRGB?
If your display supports AdobeRGB it would be preferable. It's always a good idea to use the widest color gamut supported by the display.

[Yellow_]

In your Config above you've used SD rather than HD is that deliberate ie because the transfer is BT601?

I wonder whether the cameras 'Picture Style' camera curves play a part? Would the curve be applied in the conversion from linear to gamma encoded I wonder, could that reduce the chances of seeing xvYCC? More testing I think. :-)

I don't think the curve styles affect actual captured range ie 0 - 255 but only colour content.

However I think I used 'Faithful' for the Gold source as I had the thought in mind then. I also use the Marvel Styles that emulate a LOG space.

Whats peoples general opinion of camera curves here I wonder?

Also about colour gamut, the app I'm using is linear RGB is that a wide gamut compared to AdobeRGB or does gamut not apply to liear space?

Would it be worth transforming to linear RGB or detrimental many apps appear to do all processing in linear space such as Nuke and AE.

[yesgrey]

In your Config above you've used SD rather than HD is that deliberate ie because the transfer is BT601?
Yes. According to the spec the BT.601 coefficients matrix is to be used with SD and the BT.709 with HD, hence why I named them like I did. I think it's more user friendly than xvYCC601 and xvYCC709.;)

I wonder whether the cameras 'Picture Style' camera curves play a part?
Sorry, but I don't know what you're referring to...

Also about colour gamut, the app I'm using is linear RGB is that a wide gamut compared to AdobeRGB or does gamut not apply to liear space?
The colour gamut depends on the RGB primaries only. Being linear or non linear doesn't matter, the colour gamut would be exactly the same. You need to know which are the primaries coordinates of the linear RGB space the application works with.

Would it be worth transforming to linear RGB or detrimental many apps appear to do all processing in linear space such as Nuke and AE.
If the application could accept the 16 bit output of the 3DLUT, then it might be beneficial, but if it only accepts 8 bit I think you should go with a non linear output, and let the application perform the linearisation itself.

[zcream]

A Q about xvYCC. For this extended color gamut, do we need YUV 10-bit 4:2:2 ?
The HDC-SD9 advertises xvYCC output via HDMI.
However, every other camcorder I have seen gives 8-bit YUV 4:2:2 via HDMI. For greater color depth would we need to go from 8-bit to 10-bit HDMI ?

[yesgrey]

A Q about xvYCC. For this extended color gamut, do we need YUV 10-bit 4:2:2 ?
No. xvYCC is an extension of BT.709, so it's still 8-bit. The gamut extension is achieved by using the CbCr values of 1-15 and 241-254.

[zcream]

So, if I use a HDMI capture card to get lossless 8-bit 4:2:2 data, do I need to do something to map the extra data ?
I would assume that it gets recorded normally.

[yesgrey]

So, if I use a HDMI capture card to get lossless 8-bit 4:2:2 data, do I need to do something to map the extra data ?
I would assume that it gets recorded normally.
Sorry, but I don't quite understand what you are intending...

[zcream]

Hi! Chroma is usually 16-240 for YUV video. For xvYCC it is 0-255. My question related to capturing this signal via HDMI. If I use a video capture program like VirtualDub, does it capture the extra chroma values for chroma ?

Problem is that a HDMI capture card outputs HDYC, which is then converted to RGB24 (as the video preview is seen on the monitor), then it is converted to YUV 4:2:2 for capture.
The extra chroma values would give a negative RGB value, so I would think they get thrown away.

[Yellow_]

Hi! Chroma is usually 16-240 for YUV video. For xvYCC it is 0-255. My question related to capturing this signal via HDMI. If I use a video capture program like VirtualDub, does it capture the extra chroma values for chroma ?

If your camera captures outside the 16-240 range as most do, certainly my Canon DSLR, HV30 & old JVC DV do then you have access to both. But just how much is xvYCC data is the thing. :-)

f your NLE or whatever is strictly 16 - 240 then you'll loose the rest and I think it's the case that if you're NLE or whatever works above 8bit processing(ie 10, 16 or 32bit) then they may well allow a full range workflow holding onto whatever extra that may have been captured.

Problem is that a HDMI capture card outputs HDYC, which is then converted to RGB24 (as the video preview is seen on the monitor), then it is converted to YUV 4:2:2 for capture.
The extra chroma values would give a negative RGB value, so I would think they get thrown away.

From what I've found so far doing xvYCC / full range to R'G'B is that out of the potential far wider gamut that xvYCC is said to have by the time the 4:2:0 subsampling has been done there is I think only the potential for 2.75million colors to be utilised out of the 16million and therefore where I thought I needed to use AdobeRGB or some other wide gamut profile to work with xvYCC converted to R'G'B I don't think it's necessary. :-) However if it were possible to capture 4:2:2 or 4:4:4 then something bigger than sRGB gamut might be needed I guess?

To eek out most there maybe some more to find using certain 'picture profiles' to pack more values in, possibly. Shoot Neutral or with Marvels profile but wonder whether a vivid maybe better for xvYCC?


Which leads me onto a couple of questions. I think I know the reason why I'm seeing no more colour values than before the xvYCC addition to yCMS. Here my thinking to be put right. :-)

Normal NLE handling would be 16 - 235/240 in 8bit processing, The conversion to RGB by taking reference black as 16 and reference white as 235 and putting them on that 0 1 abstract scale. To my understanding that is Rec709 to R'G'B. sRGB.

What I've been doing is using the full range because the camera captures it and in AVISynth using PC.709 as the matrix and yesgrey's yCMS with Full Range Input/Output, getting all the range and putting it in R'G'B so already doing a xvYCC conversion to R'G'B.

Which raises the question about the abstract scale, the conversion to R'G'B and handling there on in an NLE.

If an NLE only extracts 16 - 235/240 and puts it on that 1 0 scale and applies 8bit processing, I make the distinction because I think that 10,16 or 32bit and that it's float, is required for holding onto the negative values and values over 1 ie xvYCC. Then applies colour processing on a 0 1 scale set to 0 to 255 that would be incorrect handling unless they had used the full range in a bit depth greater than 8bit in the initial YCbCr to R'G'B conversion or I guess this is why they scale 16 - 235 to 0 - 255 for R'G'B processing in 8bit thus staying inside the supposed 'legal' boundaries for video?

[yesgrey]

Hi! Chroma is usually 16-240 for YUV video. For xvYCC it is 0-255.
Not quite. For xvYCC is 1-254. The 0 and 255 values should not be used with video data.

Problem is that a HDMI capture card outputs HDYC, which is then converted to RGB24 (as the video preview is seen on the monitor), then it is converted to YUV 4:2:2 for capture.
The extra chroma values would give a negative RGB value, so I would think they get thrown away.
I think it would be easier if you tell me which HDMI capture card are you thinking of (post a link, please)...

[yesgrey]

From what I've found so far doing xvYCC / full range to R'G'B is that out of the potential far wider gamut that xvYCC is said to have by the time the 4:2:0 subsampling has been done there is I think only the potential for 2.75million colors to be utilised out of the 16million and therefore where I thought I needed to use AdobeRGB or some other wide gamut profile to work with xvYCC converted to R'G'B I don't think it's necessary. :-) However if it were possible to capture 4:2:2 or 4:4:4 then something bigger than sRGB gamut might be needed I guess?
4:2:0, 4:2:2 and 4:4:4 have nothing to do with the number of available colors. In all the modes you will have the same number of available colors. The difference between them is the color resolution of an image. With 4:4:4 the chroma and luminance resolutions are the same. With 4:2:2 the chroma is half the resolution of the Luminance, and with 4:2:0 it's 1/4.

I will not comment any of the other parts of your post because it's very hard to understand what you are trying to say. Sorry.

[zcream]

Hi
I am using the Blackmagic intensity capture card.
http://www.blackmagic-design.com/products/intensity/

The colorspace output by the card was discussed here, and a special build of Huffyuv created.
http://forums.virtualdub.org/index.php?act=ST&f=6&t=16116&#entry66552

We did a test posted at vimeo
http://vimeo.com/10174263 - Comparison of Blackmagic MJPEG and Huffyv colorspace on Vimeo
We played out a SMPTE color bar from a 5d Mark II. It was ingested with a BM Intensity card using Virtualdub. There were 2 files, a special Huffyuv build for HDYC and the default BM MJPEG. As you scroll through the video it can be seen that the colorspace changes when Huffyuv records. This is a confirmation of the problem first noticed at
etfinder.net/capturepics/

To my knowledge, only AMV2-MT and Cineform handle the color conversion properly to capture a direct feed.



Not quite. For xvYCC is 1-254. The 0 and 255 values should not be used with video data.


I think it would be easier if you tell me which HDMI capture card are you thinking of (post a link, please)...

[Yellow_]

yesgrey, here again. :-)

I've started to read up on DCP (Digital Cinema Package).

http://reduser.net/forum/showthread.php?t=33118

Open Source route:

https://github.com/wolfgangw/digital_cinema_tools/wiki/Open-source-tools-for-a-digital-cinema-pipeline

Part of the process is getting from sRGB to X’Y’Z’ (Gamma 2.6) however my interest is from Canon DSLR YCbCr 1 - 254 xvYCC to X’Y’Z’ (Gamma 2.6) and then output Jpeg2000 image sequences via Wilberts Imagemagick AVISynth plugin.

What would be involved in getting a 3D LUT created to X’Y’Z’ (Gamma 2.6) is it possible within what you've created so far in yCMS?

[yesgrey]

What would be involved in getting a 3D LUT created to X’Y’Z’ (Gamma 2.6) is it possible within what you've created so far in yCMS?
Yes, it should be possible, but I haven't tested it yet. I'm planning to include that option in next yCMS version, but you can already use it with the current one by defining it on your own by using:

Output_Primaries 1.000 0.000 0.000 1.000 0.000 0.000 0.3127 0.3290
Output_Transfer_Function 1.0 0.0 0.384615384615385 0.0

Note: I've used D65 white point, but I think you can use any other you would prefer. I've read the DCI spec and did not find any reference for a specific white point.

[Yellow_]

Excellent, once again thanks, going to give it a go. :-)

Looks like last bit of the jigsaw is a avisynth plugin using Openjpeg to export the jpeg2000 images unless imagemagick does it now, via Wilberts plugin.