Spent ages yesterday struggling with Math.floor() giving me unexpected results – Math.floor(1) was coming at as 0, Math.floor(2) was coming out as 1.
The problem arose when testing out a routine for getting the minimum number of coins to make up a value. Numbers that were tracing to the output window as integers were not being processed by Math.floor as expected. My guess is that these numbers aren’t quite the integers they claim to be and have a tiny discrepancy (e.g .99999999999999999 instead of 1) which is why they are flooring in an unexpected way. I’m sure I’ve read something about this, and this problem has a name, but I’ll be jiggered if I can find it.
The code below hopefully demonstrates the problem. As indicated, I got rid of the small difference by multiplying the number up, rounding it and then dividing it back down.
var a:Number = 0.6 - 0.5; //should equal 0.1
//uncomment the line below to get rid of the tiny difference
//a = Math.round(a * 10) / 10;
var c:Number = a / .1; // 0.1 / 0.1 should equal 1
trace(Math.floor(c)); //should be 1, but is 0
Jan Grant from the Underscore mailing list answered my question as follows:
“The problem is that numbers are represented as double-precision floats under the hood; ie, a mantissa and a binary exponent (a power of two).
Because all doubles (excepting things that aren’t numbers) must be of the form n * 2 ^ m for some integers n and m, you can’t represent fractions like 0.1 exactly, since the denumerator isn’t a power of two.”