(Back to index)


Confusion about floating point numbers

Everybody who has ever programmed with almost any programming language, and even many people who never have (but use computers) have heard the story: Fractional and real numbers are approximated in computers by using the so-called floating point numbers, and these numbers suffer from accuracy problems which means that direct comparisons (ie. if you have two floating point numbers calculated by two different means, comparing them with a direct "is equal to" operator will often give the wrong answer) should be avoided and instead a ranged comparison using an epsilon value should be used instead (ie. "abs(a-b)<epsilon").

This is basically true. The problem with floating point numbers is that they only have a limited number of bits to represent such a vast set of numbers as the reals, and thus they are prone to rounding errors when calculations are performed with them. The typical example is: Make a loop which sums 0.1 ten times to a variable (which initial value is zero), and then compare the result to 1.0. In theory the comparison should give true, because ten times 0.1 is 1.0. However, any programming language using regular floating point numbers will give false as the result of the comparison.

This is because the value 0.1 cannot be represented accurately with floating point numbers used by computers (this is related to the fact that these floating point numbers use base-2 numbers instead of base-10, as 0.1 is). The reason is the same why the value of the division 1/3 cannot be represented accurately with decimal representation (only a finite approximation can be represented, for example "0.333333333", which is close, but not equal). Thus when you start adding 0.1 to itself this slight rounding error will show itself in the end result, making it not equal to 1.0.

What amuses me, however, are the misconceptions that this phenomenon causes in some people.

Apparently some people think that floating point numbers suffer from some kind of Heisenberg uncertainty principle: They never have an exact value, but only a value which is close to it. It's like the orbit of an electron around an atom: The electron is never in an exact place, but somehow distributed around that place with a certain probability. Thus you can never trust a floating point number to have some exact value, you can only know with a high degree of probability that it's inside a range.

Of course this is BS, and shows a great lack of understanding about how computers work. Computers use bits, and bits are either 0 or 1, and they stay like that until they are explicitly changed. They do not hover randomly somewhere in-between. If a bit has the value 1, then it has the value 1, exactly, and nothing else, until it's changed explicitly. In other words, bits are completely deterministic and predictable.

A floating point value consists of bits. Nothing more, nothing less. Thus a floating point value, by extension, is also determinstic and predictable, and it doesn't change value unless explicitly made to, and it doesn't hover in any in-between state like an electron around in an atomic orbit. You can read every single individual bit in a floating point value, and they will form an exact, accurate and deterministic representation of that floating point value. There's no Heisenberg uncertainty principle.

What these people confuse is the fact that rounding errors caused by a limited number of bits used to represent the value will make the result of many operations to become inexact. It's nothing more than that. There's no quantum uncertainty here. It's just a rounding error, nothing more, nothing less.

A floating point number can be used to represent an exact numerical value. For example, the value 1.0 can be represented exactly with a floating point variable. It's not "something very close to 1.0", it's "exactly 1.0". There's no uncertainty there.

What can happen, as mentioned earlier, is that when you are performing some calculations which in theory should give 1.0 you end up having something very close but nevertheless not equal. This is not because of any uncertainty principle nor because 1.0 couldn't be represented accurately. It's simply because rounding errors during the calculations caused the end result to be different.

To understand this with an example, let's use decimal representation with a limited amount of decimals, let's say 8. We represent the value of "1/3" in our decimal representation, which would thus be "0.33333333". Now we sum this value with itself two times. The first time we get "0.66666666", and the second time we get "0.99999999". We know from theory that 1/3+1/3+1/3 sould give us 1.0. However, when we compare "0.99999999" to "1.0" we see that they are not equal.

This is exactly the same phenomenon which happens with floating point numbers and the value 0.1 (instead of 1/3).

For the same reason things like "square root of 100" might not give exactly 10. Not because 10 cannot be represented accurately, but because the calculations required to calculate the square root of 100 may carry some rounding errors along the way.


(Back to index)