View Single Post
Old 10th July 2002, 19:42   #13  |  Link
poptones
Registered User
 
Join Date: Jun 2002
Posts: 135
There are a couple of sources of error. One is simply the math is based on 255 (8 bits) but then it is divided by 256 (shift 8). So right there you have a tiny bit of error. And because of the weightings applied to colorspace, it's going to effect blue the least, and green the most. No getting around it.

One could use "c math" to do it all, but then it's significantly slower. The tradeoff is a tiny bit of error in exchange for much faster execution.

This is the place where I started thinking about going to 16 bits before. If all calculations were done in 16 bits there'd be far less error (which is a good reason the "high end" applications have gone there, considering the much smaller speed tradeoffs) but it would be a LOT of work to port all these filters to supporting 16 bit planes.

For merging there's going to be far mroe error. Especially when you're just "merging." Color is determined by luminance and chroma, so just blindly overwriting one with the other is going to have indeterminate results.

const int scaled_y = (y1+y2 - 32) * int(255.0/219.0*32768+0.5);
const int b_y = ((rgb[0]+rgb_next[0]) << 15) - scaled_y;
yuv[1] = ScaledPixelClip((b_y >> 10) * int(1/2.018*1024+0.5) + 0x800000); // u
const int r_y = ((rgb[2]+rgb_next[2]) << 15) - scaled_y;
yuv[3] = ScaledPixelClip((r_y >> 10) * int(1/1.596*1024+0.5) + 0x800000); // v

The most "perfect" solution would be to go to 16 bits. this would also put avisynth on par (in this respect) with stuff like premeire and vegas. I think I could even handle most of the filters, but I have no idea what's going on deeper where frames get created and passed around.
poptones is offline   Reply With Quote