Substraction should be zero but it isn't!

  • Thread starter Thread starter DirkS
  • Start date Start date
D

DirkS

Hello all,

Performing the following substraction on my Psion Workabout pro, which
should be zero, I get a very small negative value!

110.10 - 60.10 - 49.99 - 0.01 = 0 (This is what it's supposed to be)
At first the result seemed 0, but when I multiplied it with
10.000.000.000 I got -9.095155...
All these above values are double values which were read as a string
from a file and convert to a double.

The individual values do not show any strange behaviour (even after
multiplying them with 10.000.000.000). The strange behaviour shows
after the substraction.

Is this behaviour due to my processor type (Intel XScale-PXA2) , or any
bug in the CF or in something else I haven't thought of?
 
I forgot to add: the behaviour in reproduceable

Code: 
Dim a, b, c, d As Double
a = 110.1
b = 60.1
c = 49.99
d = 0.01

If (a - b - c - d) = 0 Then
MessageBox.Show("zero!")
Else
MessageBox.Show("not zero!")
End If

This code alse does not evaluate to 'not zero'.
 
I forgot to add: the behaviour in reproduceable

Code: 
Dim a, b, c, d As Double
a = 110.1
b = 60.1
c = 49.99
d = 0.01

If (a - b - c - d) = 0 Then
MessageBox.Show("zero!")
Else
MessageBox.Show("not zero!")
End If

This code does evaluate to 'not zero'.
 
And if floating point numbers had infinite resolution, you'd get the right
answer. However, since we don't want to eat the performance penalty of
having that, we limit the resolution of floating point numbers so that they
fit in 4 or 8 bytes of storage (8, in your case). This causes potential
problems, however, as we represent all floating point numbers as mantissa
and exponent. The resolution of the values isn't perfect. You're running
into the resolution problem. The result of your subtraction is -9.xyz E-15,
so it's a *very* small error, but, yes, it's not a perfect result. If you
really need to get perfect results for things like this, you'll need to use
binary-coded decimal or something to do your math, rather than just using
built-in types.

Paul T.
 
I think you need to learn about floating point mathematics in binary systems
(Google should give you loads of info). Basically, not all real numbers can
be expressed exactly in a binary system, so you get some degree of error.
If you need precision like what you're example shows, then you need to move
to integer math and multiply everything by some factor of 10 to get rid of
the decimals.

-Chris
 
Floating point numbers have limited precision. Decimal has greater
precision and will work for the numbers you have indicated. You should test
your application using the expected number ranges.

Dim a, b, c, d as Decimal
 
Thanks all for your replies!
I was suspecting answers like you all gave, but since I lack the
in-depth knowledge of the low level handling of maths one can expect
questions like mine ;-)
I will certainly look into your suggestions !

For now I know it is behaviour due to limits of the type I'm working
with, so I can adjust my code and handle it properly.
 
Thanks all for your replies!
I was suspecting answers like you all gave, but since I lack the
in-depth knowledge of the low level handling of maths I was not sure
about it.
Good things there are newsgroups :)

As for your suggestions, I will certainly look into them!

For now I know it is behaviour due to limits of the type I'm working
with, so I can adjust my code and handle it properly.
 
Back
Top