csp
13th June 2005, 00:56
Hi, I'm a newbie and I'm lost in all the technical terms.
I am a coder and currently I am working on a Greenscreening app. Greenscreening has been very unsatisfying with the MiniDV PAL format and some people told me this is due to the 4:2:0 color interpolation.
So I had an idea to write a filter that will re-interpolate the colors in a more intelligent way. Depending on changes in luminance the filter would recognize object contours and decide if it should use the color of the right neighbour or the one of the left neighbour, or if it should interpolate as usual if no contour was detected, instead of blindly interpolating between them.
I didn't find a lot of information on how the DV Codecs are interpolating the missing color info at all, so I investigated some (onscreen decoded) frames myself. It seems to me, the 2*2 Pixel field of the 4:2:0 format is relative to an INTERLACED frame-field (bottom field or top field), so in practice, the 2*2 Field is 2*4 pixels big (!),although interlaced. (Someone correct me if I'm wrong)
My conclusion so far is a frame looks like this:
F2a---F2a---F2a---...
---F1b---F1b---F1b
---F2b---F2b---F2b
F1a---F1a---F1a---
F2a---F2a---F2a---
---F1b---F1b---F1b
---F2b---F2b---F2b
F1a---F1a---F1a---
F2a---F2a---F2a---
...
F1a and F1b are on the odd scanlines of the interlaced Image.
F2a and F2b are on the even scanlines. So F2-lines represent the Top-Field and F1-lines represent the Bottom Fields of the interlacing.
A-B-Pairs are part of the same 2*2 Bock, as well as two "---" Pixels.
Again the matrix, this time ignoring the second interlace field and assuming the F-a pixel is the true color information pixel:
---+++---+++---+++
F1a---F1a---F1a---
---+++---+++---+++
F1a---F1a---F1a---
...
So a 2*2 block is represented by:
---+++
F1a---
where ---, +++ or F1a are individual Pixels. I simply don't know which one of them is the pixel that has the oringial color information that is used to interpolate the other pixels. I also don't know if Hue and Saturation are interpolated together.
I'd be very happy if somebody could enlighten me about how this interpolation is done, thank you!
EDIT-------
BTW: this one gave me a lot of information, tho I am still looking for more:
http://www.hometheaterhifi.com/volume_8_2/dvd-benchmark-special-report-chroma-bug-4-2001.html
If the color information we've got in a 4:2:0 block is already the average color of an even pixel column - does this really mean we won't ever be capable of determining the true color of one of the 2 Pixels of this even column? (aka one of the 4 Pixels of a block)
However, my idea may still be useful since the second scanline of a 2*2 block is on line 3 (interlaced interpolation), so the first line is using the color with 75% accuracy while the second line only has a 25% accuracy (because most decoders interpolate with a 50:50 mix, if they mix it at all and do not only dublicate it)
While seeking for contours by luminance and intelligently choose the color from left or right works perfectly on the horizontal level, there isn't a lot to win since we got every 2nd pixel anyway.
Much more important would it be to do this intelligent interpolation on the vertical level because, if you split the interlaced fields into 2 individual frames, all of a sudden you are using this single pixels color information for a block of 2*4 pixels - that's exactly where the inaccurate and blocky Matte Line is born when you are using chroma keying / compositing as an indy movie maker (like me).
I am a coder and currently I am working on a Greenscreening app. Greenscreening has been very unsatisfying with the MiniDV PAL format and some people told me this is due to the 4:2:0 color interpolation.
So I had an idea to write a filter that will re-interpolate the colors in a more intelligent way. Depending on changes in luminance the filter would recognize object contours and decide if it should use the color of the right neighbour or the one of the left neighbour, or if it should interpolate as usual if no contour was detected, instead of blindly interpolating between them.
I didn't find a lot of information on how the DV Codecs are interpolating the missing color info at all, so I investigated some (onscreen decoded) frames myself. It seems to me, the 2*2 Pixel field of the 4:2:0 format is relative to an INTERLACED frame-field (bottom field or top field), so in practice, the 2*2 Field is 2*4 pixels big (!),although interlaced. (Someone correct me if I'm wrong)
My conclusion so far is a frame looks like this:
F2a---F2a---F2a---...
---F1b---F1b---F1b
---F2b---F2b---F2b
F1a---F1a---F1a---
F2a---F2a---F2a---
---F1b---F1b---F1b
---F2b---F2b---F2b
F1a---F1a---F1a---
F2a---F2a---F2a---
...
F1a and F1b are on the odd scanlines of the interlaced Image.
F2a and F2b are on the even scanlines. So F2-lines represent the Top-Field and F1-lines represent the Bottom Fields of the interlacing.
A-B-Pairs are part of the same 2*2 Bock, as well as two "---" Pixels.
Again the matrix, this time ignoring the second interlace field and assuming the F-a pixel is the true color information pixel:
---+++---+++---+++
F1a---F1a---F1a---
---+++---+++---+++
F1a---F1a---F1a---
...
So a 2*2 block is represented by:
---+++
F1a---
where ---, +++ or F1a are individual Pixels. I simply don't know which one of them is the pixel that has the oringial color information that is used to interpolate the other pixels. I also don't know if Hue and Saturation are interpolated together.
I'd be very happy if somebody could enlighten me about how this interpolation is done, thank you!
EDIT-------
BTW: this one gave me a lot of information, tho I am still looking for more:
http://www.hometheaterhifi.com/volume_8_2/dvd-benchmark-special-report-chroma-bug-4-2001.html
If the color information we've got in a 4:2:0 block is already the average color of an even pixel column - does this really mean we won't ever be capable of determining the true color of one of the 2 Pixels of this even column? (aka one of the 4 Pixels of a block)
However, my idea may still be useful since the second scanline of a 2*2 block is on line 3 (interlaced interpolation), so the first line is using the color with 75% accuracy while the second line only has a 25% accuracy (because most decoders interpolate with a 50:50 mix, if they mix it at all and do not only dublicate it)
While seeking for contours by luminance and intelligently choose the color from left or right works perfectly on the horizontal level, there isn't a lot to win since we got every 2nd pixel anyway.
Much more important would it be to do this intelligent interpolation on the vertical level because, if you split the interlaced fields into 2 individual frames, all of a sudden you are using this single pixels color information for a block of 2*4 pixels - that's exactly where the inaccurate and blocky Matte Line is born when you are using chroma keying / compositing as an indy movie maker (like me).