Chroma upsampling using luma detail ?

[trevlac]

I'm a bit of a novice with video, programming, and signal processing; so forgive me if I don't make any sense. (And please tell me and point me in the right direction) :)

I'm talking about using the Luma detail to determine how to upsample the chroma. This may allow for better chroma edges ...


Q1 Has anyone / does any filter use the Y detail when up-scaling 4:2:2 (or maybe 4:2:0) chroma up to 4:4:4 YCbCr ?

http://www.ics.ele.tue.nl/~dehaan/pdf/99_VCIP04Zhao.pdf

Above article talks about doing this by categorizing the 'types' of pixels by actually doing the upscale using different FIR filters (I assume) and measuring the error. Then it uses the best filter for the given type.

Now in my own simple mind I thought you could do a test for 3 cases where you use the existing Y to determine the missing UV.

Given 4:2:2->4:4:4 Y1 UV1 .. Y2 uv2 .. Y3 UV3, uv2 needs to be created.

So depending on what is closest to Y2; Y1, (Y1+Y3)/2, or Y3
Use U1, (U1 + U3)/2, or U3 ....


I put this in a filter that does YUY2->RGB conversion and the results look a little better than a plain 2 value average.

Q2 Any other ideas on 'classifying' pixels?
Q3 Any suggestions on testing and other methods I should compare?

Thanks

fixed the link

[rfmmars]

I couldn't get into that link

In analog work there was a IC chip TDA 4565 called the CTI "Colour Transisit Improver" which detected the change in R-Y & B-Y information and produced an artifical edge on those choma edges after demoduation.

I inturn modified the circuit to also produce a sharp edge on the Y channel. The resulting video was near RGB quality. There are still data sheets on that chip for downloading.

I see no reason why you couldn't do it with your idea. I am very interested in your work and results.

richard
www.photorecall.net

[Guest]

That's interesting. Thank you for bringing it to our attention.

The paper assumes progressive downsampling. It needs to be extended to allow for both progressive and interlaced downsampling, but that should not be so difficult.

[MfA]

AFAIR most modern philips chips basically use warpsharp on the upsampled chroma planes, using the luminance gradient ... although I like this one better, warpsharp is such an ugly hack (faster than classification based methods though).

This (http://www.semiconductors.philips.com/acrobat/applicationnotes/AN10233_1.pdf) has a decent description, it is the digital version of CTI (Colour Transient Improvement).

[kassandro]

Very interesting idea, indeed. However, there are some problems to overcome. For avisynth the n-th yv12 chroma (starting with n=0) is taken at vertical position 2n + 0.5. This is also the specification for mpeg2. Thus with this assumption, a chroma line doesn't belong to a single luma line, rather it lies just in the middle of two luma lines. This approach has also the severe disadvantage that

ConvertToYUY2()
ConvertToYV12()

doesn't leave the clip unchanged. Rather the clip is slightly blurred vertically (of course ony the chroma). To avoid these problems, Microsoft suggests to assume that the n-th chroma is taken at vertical position 2n rather than 2n + 0.5. With this assumption the even luma lines have a chroma and the odd ones don't. With this assumption the ConvertToYV12() becomes simpler and faster, while ConvertToYUY2 becomes a simple separation routine without any averaging. Also trevlac's idea can be easily realised, though it will be very slow. it involves two extremely slow divisions per pixels and I see no chance to use integer SSE, though floating point SSE may work here and may be the fastest.

[trevlac]

Thanks for the replys. I'm glad there is some interest.

I made a YUY2->RGB32 converter that has 5 'interpolation' modes to test with. Dup values, Avg, Use Luma, Make Unknown Grey, drop unmatched Y (gives 1/2 width)

http://trevlac.us/arachno/YUY2toRGB219.dll


LoadPlugin("C:\av\avisynth\myplugins\YUY2toRGB219.dll")
c=AVISource("..\368Source.avi", pixel_type="YUY2")
c = SeparateFields(c)

a1= c.YUY2toRGB219(TVtoPC=true,Interpolate=1) # Dup
a2= c.YUY2toRGB219(TVtoPC=true,Interpolate=2) # Avg
a3= c.YUY2toRGB219(TVtoPC=true,Interpolate=3) # Luma
a4= c.YUY2toRGB219(TVtoPC=true,Interpolate=4) # Kill
#a5= c.YUY2toRGB219(TVtoPC=true,Interpolate=5).LanczosResize(368,240)

#s1 = subtract(a3, a2).Levels(127, 1, 129, 0, 255)
o1 = stackhorizontal(a1, a2)
o2 = stackhorizontal(a3, a4)

stackvertical(o1,o2)


With the above script I made the following pic:
http://trevlac.us/arachno/Bruce.jpg

This was DVB source captured as YUY2. Can't say I found much that makes a difference. I can use an artificial red letters on blue background, but the RGB->YUY2 does a bunch of damage, so this does not seem to be a good test because the 'repair' probably depends upon how the damage was created.

Anyway, perhaps 4:2:0 (or 4:1:1) would benefit, but I'm not sure it matters for 4:2:2.


@MfA

I really liked that link. From the pictures (graphs) it does appear to be sharpening of the chroma. I will have to read it many times to truely understand. "The idea is to vary the data path delay on the basis of a function of the second derivative of the U and V signal." is not really a good clarification to me. :)


@kassandro

In my implementation, I cheated and only did YUY2-> 4:4:4. The
(Y1+Y3)>>1 just truncates 1/2 values. I would think you could keep more precision by shifting up, or just give it up and assume the differences in the classifications are great enough so that float values don't matter.

[MfA]

As I said trevlac, it is basically just warpsharp :)