View Single Post
Old 22nd April 2009, 00:30   #3  |  Link
a uio
Registered User
 
Join Date: Jul 2005
Posts: 30
Quote:
Originally Posted by Blue_MiSfit View Post
You sure you're decoding the outputs correctly? The TV / PC levels issue is pretty well known and causes lots of issues.

~MiSfit
Yes. Those typically cause errors in the range of 5-6% and are easy to see. These 1% shifts are quite subtle unless you look at the histograms and toggle back and forth.

My first guess would be that there is truncation instead of rounding going on in calculations that consistently shifts all values in some direction.
a uio is offline   Reply With Quote