redfordxx
3rd December 2007, 12:57
I made these function for calculations with values larger than 255 which do not fit in one lut. Using scaling can be also used for decimal numbers.
Input of these functions are
value...lut-formated expresion to be converted
precision...number of bits used for storing the base number (the rest within 8 bits is used for exponent)
function Convert2Float(string value, int precision) {
return string(value)+" 2 "+string(precision)+" 1 + ^ < "+string(value)+" "+string(value)+" 2 1 "+string(precision)+" - ^ * 0.000010 + log 2 log / 0.500000 - round 2 "+string(precision)+" ^ * "+string(value)+" 2 1 "+string(value)+" 2 1 "+string(precision)+" - ^ * 0.000010 + log 2 log / 0.500000 - round - ^ * 2 "+string(precision)+" ^ - 0.499990 - round + ?"
# "(x<2^(y+1)) ? x : round(log(x*2^(1-y)+0.00001)/log(2)-0.5)*2^(y)+round(x*2^(1-round(log(x*2^(1-y)+0.00001)/log(2)-0.5))-2^(y)-0.49999)"
}
function Convert2Int(string value, int precision) {
return string(value)+" 2 1 "+string(precision)+" + ^ < "+string(value)+" "+string(value)+" 2 "+string(precision)+" ^ % 2 "+string(precision)+" ^ + 0.500000 + 2 "+string(value)+" "+string(value)+" 2 "+string(precision)+" ^ % - 2 0 "+string(precision)+" - ^ * 1 - ^ * ?"
# "(x<2^(1+y)) ? x : (x % (2^y)+2^y+0.5)*2^((x-(x % (2^y)))*2^(-y)-1)"
}
####### example
e=mt_lutxy(a, b, Convert2Float("x y *",4) ,y=3,u=3,v=3)
f=mt_lutxy(c, d, Convert2Float("x y *",4) ,y=3,u=3,v=3)
mt_lutxy(e, f, Convert2Int("x",4)+" "+Convert2Int("y",4)+" 256 + /",y=3,u=3,v=3)
I hope the formulas copied correctly.
precision, max value stored and max error table (I think so):
8,7 255 0%
6 512 0.7%
5 4K 1.5%
4 512K 3.1%
3 2G 6.2%
Input of these functions are
value...lut-formated expresion to be converted
precision...number of bits used for storing the base number (the rest within 8 bits is used for exponent)
function Convert2Float(string value, int precision) {
return string(value)+" 2 "+string(precision)+" 1 + ^ < "+string(value)+" "+string(value)+" 2 1 "+string(precision)+" - ^ * 0.000010 + log 2 log / 0.500000 - round 2 "+string(precision)+" ^ * "+string(value)+" 2 1 "+string(value)+" 2 1 "+string(precision)+" - ^ * 0.000010 + log 2 log / 0.500000 - round - ^ * 2 "+string(precision)+" ^ - 0.499990 - round + ?"
# "(x<2^(y+1)) ? x : round(log(x*2^(1-y)+0.00001)/log(2)-0.5)*2^(y)+round(x*2^(1-round(log(x*2^(1-y)+0.00001)/log(2)-0.5))-2^(y)-0.49999)"
}
function Convert2Int(string value, int precision) {
return string(value)+" 2 1 "+string(precision)+" + ^ < "+string(value)+" "+string(value)+" 2 "+string(precision)+" ^ % 2 "+string(precision)+" ^ + 0.500000 + 2 "+string(value)+" "+string(value)+" 2 "+string(precision)+" ^ % - 2 0 "+string(precision)+" - ^ * 1 - ^ * ?"
# "(x<2^(1+y)) ? x : (x % (2^y)+2^y+0.5)*2^((x-(x % (2^y)))*2^(-y)-1)"
}
####### example
e=mt_lutxy(a, b, Convert2Float("x y *",4) ,y=3,u=3,v=3)
f=mt_lutxy(c, d, Convert2Float("x y *",4) ,y=3,u=3,v=3)
mt_lutxy(e, f, Convert2Int("x",4)+" "+Convert2Int("y",4)+" 256 + /",y=3,u=3,v=3)
I hope the formulas copied correctly.
precision, max value stored and max error table (I think so):
8,7 255 0%
6 512 0.7%
5 4K 1.5%
4 512K 3.1%
3 2G 6.2%