Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 13th April 2005, 09:23   #1  |  Link
Dimwitted
Registered User
 
Join Date: Apr 2005
Posts: 5
YCbCr *into* RGB channels.

Is there any way that I can take the Y, Cb, and Cr data straight from a DV AVI and put it into the R, G and B channels of an uncompressed AVI, for processing in After Effects?
Dimwitted is offline   Reply With Quote
Old 13th April 2005, 09:42   #2  |  Link
communist
Registered User
 
Join Date: Jul 2003
Posts: 1,152
After Effects filters work in RGB, hence if you load your DV files it well get converted by After Effects automatically to RGB - no need to do so manually.
Also you cant have RGB DV files. DV is stored as YV12 regardless of what the codecs output or input (requirement).

But if you (unnessecarily) want to create LARGE uncompressed RGB files from your DV files you're free to do so
communist is offline   Reply With Quote
Old 13th April 2005, 20:19   #3  |  Link
Dimwitted
Registered User
 
Join Date: Apr 2005
Posts: 5
Sorry, I don't think you understood my question.

I've written some filters to process the Cb and Cr information in order to do better greenscreen compositing using (PAL) DV footage in After Effects.

However, the camera that we're using - the Canon XM2 - like most DV cameras, stores the Y, Cb and Cr data in the range 0-255. Consequently, when the codec presents the RGB data to After Effects (which, as you correctly observe, requires RGB input) you can get "luma leakage": because one or more of the R, G or B values that correspond to the YCbCr values would need to be less than zero or greater than 255, when you re-convert the RGB data back to YCbCr for processing, artifacts that ought to be confined to the chroma channels can appear in Y.

Additionally, converting back and forth introduces quantization error.

Hence, I would like to find a method (presumably using Avisynth) that will output an uncompressed (RGB) AVI file, in which R contains the Y, G contains the Cb, and B contains the Cr data from the original DV AVI.
Dimwitted is offline   Reply With Quote
Old 13th April 2005, 22:15   #4  |  Link
Wilbert
Moderator
 
Join Date: Nov 2001
Location: Netherlands
Posts: 6,364
Quote:
Hence, I would like to find a method (presumably using Avisynth) that will output an uncompressed (RGB) AVI file, in which R contains the Y, G contains the Cb, and B contains the Cr data from the original DV AVI.
Although it's trivial to make such a plugin. I don't think it can be done with existing filters (at least i'm not sure how).

Perhaps bb can move this thread to AviSynth forum?
Wilbert is offline   Reply With Quote
Old 14th April 2005, 12:53   #5  |  Link
trevlac
budala
 
Join Date: Oct 2003
Location: U.S.
Posts: 545
Quote:
Originally posted by Dimwitted

However, the camera that we're using - the Canon XM2 - like most DV cameras, stores the Y, Cb and Cr data in the range 0-255. Consequently, when the codec presents the RGB data to After Effects (which, as you correctly observe, requires RGB input) you can get "luma leakage": because one or more of the R, G or B values that correspond to the YCbCr values would need to be less than zero or greater than 255, when you re-convert the RGB data back to YCbCr for processing, artifacts that ought to be confined to the chroma channels can appear in Y.

Additionally, converting back and forth introduces quantization error.
I think you would be better served to simply convert from YCbCr to Studio RGB which runs in the range of 16-235 leaving the top and bottom to cover the overshoot that might be in your YCbCr. Quite a few DV codecs will do this. Sony comes to mind. I wrote an avs filter when I was playing with this stuff that converts from YUY2 to studioRGB.
http://trevlac.us/colorCorrection/YUY2toRGB219.zip

With this kind of conversion, luma will not change. AKA no quantization error on luma. Plus, because YCbCr 4:2:0 is such crap as far as color ... both in terms of color depth and detail ... the minor color shifts that occur are a nit in the grand scheme.

--------------------------
On the other hand ... if you really want YCbCr inside of RGB ... you are going to have to deal with the subsample issue 1st. You only have 1 Cb/Cr for every 4 G/B. Are you just going to duplicate them? This would be like a nearest neighbor 2x2 upsize of the chroma. Quite blocky.

Just some things to consider ....
trevlac is offline   Reply With Quote
Old 14th April 2005, 19:10   #6  |  Link
bb
Moderator
 
bb's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 2,665
Quote:
Originally posted by Wilbert
Perhaps bb can move this thread to AviSynth forum?
Sounds reasonable. Thread moved.

bb
bb is offline   Reply With Quote
Old 14th April 2005, 20:32   #7  |  Link
Dimwitted
Registered User
 
Join Date: Apr 2005
Posts: 5
Thanks for all the help so far!

I'd prefer not to scale any of the values until I've done my processing. Because my filter is specifically tailored to extracting mattes from bluescreen footage, I'm already dealing with a relatively small range - I don't want to make that range any smaller than I have to.

The way that I'm currently upsizing the chroma uses cubic interpolation horizontally, and a weighted version vertically, because you get two rows of "good" chroma information, and two rows of nothing in each of the channels. I also distribute the high frequency Y data according to the magnitude of the chroma, which isn't in any way "accurate", but doesn't seem to make things look any worse, and sometimes makes them look a little bit better. Here's a "before and after" example (the top image is the unmodified input, using the Panasonic codec) magnified 5 times:

http://www.cgideas.btinternet.co.uk/beforeafter.jpg

This is just processed with my "baseline" filter. The filter that I'm working on uses analytical methods to find the transitions between background and foreground.
Dimwitted is offline   Reply With Quote
Old 14th April 2005, 21:08   #8  |  Link
trevlac
budala
 
Join Date: Oct 2003
Location: U.S.
Posts: 545
That panasonic code sure looks terrible. I don't recall, but some are quite a bit better. That one looks like it just dulicates pixels. You did a hell of a lot better than it.

Of course if you do it yourself, you get to do what you want. You sound quite capable of writing an avisynth filter. They are not too bad. I wrote one and I hardly know squat. There is a sticky post at the top of the avisynth development forum with a sample. No special tools beyond a c++ compiler required.
trevlac is offline   Reply With Quote
Old 15th April 2005, 03:17   #9  |  Link
khmann
Registered User
 
Join Date: Aug 2004
Location: NTSC
Posts: 9
YES YES YES!

I very much want to "fakeconverttorgb()" so that I can use virtualdub RGB plugin "deshaker" to stabilize my NTSC DV footage without any colorspace conversions or luma/chroma range clipping (my Canon Optura Xi outputs a wider range than 16-235)

should be fairly easy to script with something like 'pointresize' to upconvert chroma to fit 4:4:4, but I have not yet done it. please post your success!
khmann is offline   Reply With Quote
Old 15th April 2005, 10:24   #10  |  Link
E-Male
mad computer-scientist
 
Join Date: Mar 2002
Posts: 1,375
the problem is that the chroma values are the same for a block of 4 pixels (right?)
so after converting youd have 2*2 blocks with identical U & V
after processing you'd have to convert back and during that average the 4 resulting Us & Vs [so it's NOT totally free of conversion]
if that is all fine i can have a look at doing such a filter next week
E-Male is offline   Reply With Quote
Old 15th April 2005, 11:23   #11  |  Link
Dimwitted
Registered User
 
Join Date: Apr 2005
Posts: 5
That's the way MPEG 4:2:0 works. PAL DV is a bit more peculiar. Assuming we put the samples into a 720 x 576 frame:

The first row of pixels gets 360 2-pixels-wide V (Cr) blocks, and no U (Cb) samples.

The second row of pixels gets another 360 2-pixels-wide V blocks (because it's the first row of the next field) and no U samples.

The third row of pixels (the second row of the upper field) gets 360 2-pixels-wide U blocks and no V samples.

The fourth row of pixels gets another 360 2-pixels-wide U blocks and no V samples.

Hope that makes sense; it's hard to describe. The way that DV codecs generally deal with the absense of U and V samples is to either duplicate the samples from two rows above in the case of V and two rows below in the case of U (the Panasonic codec does this, as does Quicktime) or interpolate the missing samples (as far as I'm aware, though, they all do this on a field by field basis, which is why my method gives better results for progressive footage).

Anyway, if you could have a stab at it, that'd be marvellous! I've been looking over the example code and it's pretty baffling to me...
Dimwitted is offline   Reply With Quote
Old 15th April 2005, 13:24   #12  |  Link
E-Male
mad computer-scientist
 
Join Date: Mar 2002
Posts: 1,375
i see, so it's 2*2 blocks, but interlaced

but is it also stored in avisynth that way or already 'sorted' by the "source"-filter?
because i'm not sure how you'd store this in YUV2 or YV12 in avisynth

i haven't worked with dv material yet, but i daubt it'll be that different
avisynth will use the same colorspaces as for other sources

EDIT:
just check http://www.avisynth.org/ColorSpaces and YV12 interlaced seems to work this way, too

so my questions is:
do you need the material to be interlaced?
or could you just run a deinterlacer before doing the pseudo-conversion

Last edited by E-Male; 15th April 2005 at 13:30.
E-Male is offline   Reply With Quote
Old 15th April 2005, 13:35   #13  |  Link
Wilbert
Moderator
 
Join Date: Nov 2001
Location: Netherlands
Posts: 6,364
No, he's not talking about interlaced YV12 sampling. The pb is that PAL-DV sampling is different as MPEG-2 YV12 sampling (although both are 4:2:0 YV12). Have a look at the thread

http://forum.doom9.org/showthread.ph...interpolate420 (my last post on the first page)

also read http://www.mir.com/DMG/chroma.html

Last edited by Wilbert; 15th April 2005 at 13:38.
Wilbert is offline   Reply With Quote
Old 15th April 2005, 13:58   #14  |  Link
trevlac
budala
 
Join Date: Oct 2003
Location: U.S.
Posts: 545
Re: YES YES YES!

Quote:
Originally posted by khmann
I very much want to "fakeconverttorgb()" so that I can use virtualdub RGB plugin "deshaker" to stabilize my NTSC DV footage without any colorspace conversions or luma/chroma range clipping (my Canon Optura Xi outputs a wider range than 16-235)

should be fairly easy to script with something like 'pointresize' to upconvert chroma to fit 4:4:4, but I have not yet done it. please post your success!
Hello

I'm not sure what deshaker does on the inside .... but I'm sure it tries to figure things out based upon the pixel values. If you give it screwy values ... i can't see how it would work.

On color space conversions ....

Blah blah blah ... color space conversions are bad ... don't do em .... blah blah blah .....

This is a real crock. There is zero change to luma on a conversion between YCbCr and RGB if you don't change the range. This is because the standard coefficients add to 1. And any decent process gets it right. There are many DV codecs that output RGB in 16-235 range. As I mentioned.

In addition .... if you start with crappy color YCbCr 4:2:0 or 4:1:1 and go to 4:4:4 RGB, the MAXIMUM color change is miniscule. This is because the RGB color space has 4 times the depth of (4:4:4) YCbCr . There may well be changes introduced by the up sample down sample (4:2:0 -> 4:4:4 -> 4:2:0) .... but what you are talking about does not get around that problem. The problem Dimwitted is addressing is that a crappy codec is not doing a decent job at exactly this. Point sample is exactly what it is doing .....


--------------------

@Dimwitted

Having said all of that ....

I put together a quick filter that stuffs YUY2 into RGB24. You gotta get it to YUY2. YUY2inRGB(clip, true/flase). The true/false option does a simple average to come up with the missing UVs.

In case this is not clear ...

YUYVYUYV -> YUVYUVYUV
11113333 -> 1112xx333

The xx UV are U1+U3/2 and V1+V3/2

http://trevlac.us/YUY2inRGB.zip

Sorry ... I couldn't test it because I can't install avs on my work machine. I included the source. You gotta get it back to a usable colorspace/packaging. Which sounds like what you are doing.

PS: If you want code comments ... let me know. I just didn't want to do that typing if you didn't really care.

[edit]
Also, In case it was not clear. I want you to do the YV12 to YUY2 conversion ... because this is an area you are trying to address and I did not want to 'get in the way'. I'm sure there are lots of ways to do this with avs as is being pointed out. I guess you might have wanted some sort of YV12 in a packed format. But YV12 is planar so I really don't know what that would look like.

Last edited by trevlac; 15th April 2005 at 14:10.
trevlac is offline   Reply With Quote
Old 15th April 2005, 14:02   #15  |  Link
Wilbert
Moderator
 
Join Date: Nov 2001
Location: Netherlands
Posts: 6,364
Quote:
This is because the RGB color space has 4 times the depth of (4:4:4) YCbCr
cough
Wilbert is offline   Reply With Quote
Old 15th April 2005, 14:13   #16  |  Link
trevlac
budala
 
Join Date: Oct 2003
Location: U.S.
Posts: 545
Quote:
Originally posted by Wilbert
cough
Thank god that ba$tard i4004 can't post here. He'd probably get me so worked up I'd get banned ....
trevlac is offline   Reply With Quote
Old 15th April 2005, 14:44   #17  |  Link
khmann
Registered User
 
Join Date: Aug 2004
Location: NTSC
Posts: 9
Re: Re: YES YES YES!

Quote:
Originally posted by trevlac
Hello :)

I'm not sure what deshaker does on the inside .... but I'm sure it tries to figure things out based upon the pixel values. If you give it screwy values ... i can't see how it would work.
any RGB plugin which does not make decisions based upon color or try and "manipulate" color information should work fine with this method, as long as color resolution is appropriately up/down sampled.

deshaker stabilizes video by looking at a luma representation of image. it's perception will probably be skewed by this bogus RGB data, but that shouldn't effect it's efficacy.

colorspace conversions are perfectly accurate with respect to levels, but there _will_ be quantization error as a result of rounding and 8 bit precision of RGB and YCrCb. but as much as anything for me they take unnecessary time.

thankyou very much for your filter, I believe it meets my need exactly. I will do some tests with embedding virtualdub-mod RGB filters in my scripts and see how it performs...

Now I just need RGBINYUY2, but I think it a worthwhile exercise that I learn how to do that myself anyway :)

THANKYOU!
khmann is offline   Reply With Quote
Old 15th April 2005, 15:50   #18  |  Link
trevlac
budala
 
Join Date: Oct 2003
Location: U.S.
Posts: 545
Re: Re: Re: YES YES YES!

Quote:
Originally posted by khmann
Now I just need RGBINYUY2, but I think it a worthwhile exercise that I learn how to do that myself anyway
Good for you!

On the following I don't really mean to pick at you but I think this exemplifies the colorspace conversion bitching that bugs me.

Quote:

any RGB plugin which does not make decisions based upon color or try and "manipulate" color information should work fine with this method, as long as color resolution is appropriately up/down sampled.

deshaker stabilizes video by looking at a luma representation of image. it's perception will probably be skewed by this bogus RGB data, but that shouldn't effect it's efficacy.
How do you think it comes up with luma ? ~ R*.3 + G*.6 + B*.1

Now you are going to give it YUV in place of RGB. The U value will have the biggest effect on the luma. U runs in a range of 0-127 and 128-255 where the low numbers represent yellow and the high numbers represent blue. Worth a shot ... but I'm thinking you are going to get a babble fish translation.

Quote:
colorspace conversions are perfectly accurate with respect to levels, but there _will_ be quantization error as a result of rounding and 8 bit precision of RGB and YCrCb.
ehm .... Sounds nice but in practice when done well, there can be ZERO error in the luma and the quantization error is already huge in YCbCr. Going to RGB will not make it worse. As I said before ... RGB has a much finer measure than YCbCr. Said another way, the CbCr are already 7 bit. Even at 4:4:4.

Quote:
but as much as anything for me they take unnecessary time.
Yes it does save time to not do things. No doubt. But in your case, you are converting and creating more bits.


Sorry for the rant ... but this colorspace conversion thing is out of hand around here. If you seperately address the issue of upsample downsample ... conversion from YCbCr-> RGB ->YCbCr is benign compared to any filtering you might do.

As a side note .... Have you ever thought about the fact that any YCbCr filter chain saves the intermediate results in what is effectively a 8bit, 7bit, 7bit form? AKA YCbCr.

So ... In conclusion .... You can't have your cake and eat it to. YCbCr is faster because you can subsample it. But it is also a lower quality colorspace.

... pant ... pant ... pant
trevlac is offline   Reply With Quote
Old 15th April 2005, 16:25   #19  |  Link
Wilbert
Moderator
 
Join Date: Nov 2001
Location: Netherlands
Posts: 6,364
Please explain why you think that CbCr is 7 bit. Thx.
Wilbert is offline   Reply With Quote
Old 15th April 2005, 18:36   #20  |  Link
trevlac
budala
 
Join Date: Oct 2003
Location: U.S.
Posts: 545
Quote:
Originally posted by Wilbert
Please explain why you think that CbCr is 7 bit. Thx.
Of course technically Cb and Cr are stored as 8 bits. 7 bits is an effect of the colorspace cube orientation.

Short Answer

The YCbCr color cube covers an area of mixed light that is larger than what can be seen. Positive RGB mixed light can be seen. The Y axis of the YCbCr color cube covers light completely within the RGB cube. Cb and Cr however, cover areas of negative light. For example, there is no such thing as highly saturated bright (meaning a high luma) blue. You'd have to add negatively saturated red and green to get the luma up.

The net effect of this is to reduce both Cb and Cr to only producing visible colors 1/2 the time each , depending on the luma. 1/2 8 bit is 7 bit.


Visual Answer
http://www.doom9.org/index.html?/cap...deo_color.html

There are the cubes. To Draw your own .... make a 6 sided shape with a point at the top. Put a dot in the middle. Draw a line from the dot to the bottom center point. Draw a line from the dot to the upper left point. Draw a line from the dot to the upper right point.

Now we have a cube. Label the outer points moving clockwise RMBCGY. Those are the primary and 2ndary colors.

Luma or Y is the dot in the center. White is close to you, black farthest away.

Draw an XY axis (a plus) centered on the dot. These are Cb(x) and Cr(y). You are now looking down at the top of the YCbCr cube. The RGB cube is inside, tilted on a corner. You can't precieve the luma dimension because you are looking right at it. This is the view of a vectorscope which show only color info.

The RGB cube does not fill the YCbCr one. Some ZXY cordinates into this colorspace don't hit the RGB cube.

I leave it to you to work out the volume differenes in the cubes. I think it's around 1 vs 3.5 if RGB fit snug. I don't think this is an exact number anyway ... because the colorspaces we use are scaled and I don't think the max cb wall touches max blue.


Did i go off topic ?

[edit]
For a more detailed coverage of this look to poynton here
http://www.poynton.com/PDFs/Merging_RGB_and_422.pdf

Quote:
The color difference components are bipolar. Unscaled, they range from
roughly --1 to +1. For analog engineers, the doubled excursion represents
a 6 dB SNR penalty for the chroma components. Digital engineers
should consider the sign to consume an extra bit in each of CB
and CR.
This codeword utilization issue represents a serious limitation of
8-bit Y’CBCR performance.
AKA ... Cb and Cr are effectively 7 bit

The central point of his paper is one should stick with RGB, but that's not an option for us. His numbers are more dire than mine. He says you only get 17% of the possible colors out of 601 YCbCr. I think he is comparing this to 0-255 RGB.

Last edited by trevlac; 15th April 2005 at 19:27.
trevlac is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 04:25.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.