VB.NET Cant add up!!!

  • Thread starter Thread starter Merlin
  • Start date Start date
M

Merlin

Can someone please explain this as it has serious implications for me.

The following code if added together on a calculator adds to 100, but if you
run my example in a vb form with a standard textbox and check the value of D
at the stop statement the value does not equal 100, but 99.999999999999986.
If you resume the program the TextBox1.text will show as 100.

(Why all these number you say - I have a program that stores a large list of
values and performs complex calculations on them).

Dim d As Double

d = d + 63.528588
d = d + 30.013506
d = d + 0.100045
d = d + 1.850833
d = d + 3.001351
d = d + 0.005002
d = d + 0.300135
d = d + 1.20054
If d <> 100 Then MsgBox("wtf!")
Stop
TextBox1.Text = d.ToString
 
Merlin said:
Can someone please explain this as it has serious implications for
me.

The following code if added together on a calculator adds to 100, but
if you run my example in a vb form with a standard textbox and check
the value of D at the stop statement the value does not equal 100,
but 99.999999999999986. If you resume the program the TextBox1.text
will show as 100.

(Why all these number you say - I have a program that stores a large
list of values and performs complex calculations on them).

Dim d As Double

d = d + 63.528588
d = d + 30.013506
d = d + 0.100045
d = d + 1.850833
d = d + 3.001351
d = d + 0.005002
d = d + 0.300135
d = d + 1.20054
If d <> 100 Then MsgBox("wtf!")
Stop
TextBox1.Text = d.ToString
Click to expand...

Welcome to the nature of floating point numbers: Doubles (and Singles) are
not accurate. Use Decimal instead.

Links collected from this group:
http://research.microsoft.com/~hollasch/cgindex/coding/ieeefloat.html
http://www.math.grin.edu/~stone/courses/fundamentals/IEEE-reals.html
http://support.microsoft.com/default.aspx?scid=kb;[LN];42980
 
Merlin,
As Armin suggested use Decimal instead. Just be aware that you need to use
Decimal for both the variable and the literals.

Something like:

Dim d As Decimal

d = d + 63.528588D
d = d + 30.013506D
d = d + 0.100045D
d = d + 1.850833D
d = d + 3.001351D
d = d + 0.005002D
d = d + 0.300135D
d = d + 1.20054D
If d <> 100D Then MsgBox("wtf!")

The D on 63.528588D says it is a Decimal literal, where as the default
(63.528588) is a Double literal.

See Armin's links for more details.

Hope this helps
Jay
 
Can someone please explain this as it has serious implications for me.
Click to expand...

There have been many discussions about this on the group. Basically,
(someone correct me if I'm wrong), a decimal number cannot be accurately
represented in binary which is why you get the inconsistency you get. This
problem is not just a .Net or VB problem.

To get the results you anticipate, use the Decimal data type instead of
Double.
 
Inexact representation of floating-point numbers in computers has been an
issue as long as computers have been using binary as their internal storage
mechanism. This always comes as a surprise to anyone who has never had the
opportunity to study computer architecture at the hardware level (do they
even teach this any more in the computer science curriculum?).

The fact is that most floating-point numbers have no exact representation in
binary. Over the years, a lot of brain cells have been used up in devising
binary encodings of floating-point numbers that maintain the highest
possible precision, but for general-purpose computers you can only go so far
and retain good performance. Most computer architectures these days use the
IEEE encoding scheme. Here's a link to a Microsoft Knowledge Base article
that might be of some interest. It's a tutorial on IEEE floating-point and
will probably answer a lot of your questions:
http://support.microsoft.com/default.aspx?kbid=42980

For more information, go to msdn.microsoft.com, and in the search box in the
upper-right corner select the Knowledge Base radio button and enter search
terms such as the following:

precision accuracy floating point


Hope this helps,
Tom Dacon
Dacon Software Consulting
 
In addition to what everyone else has said, use the decimal type only if
precision is actually important for the application. If "close enough"
results are sufficient and you need to maximize performance, stick with
floating point math. Although decimals are more accurate, they consume more
memory and perform calculations many times slower than with floating point
math.
 
Thanks to everyone for your replies.

I read the MS KB article and understand why, but I always thought this issue
was for larger numbers and divisions, not simply adding a few fixed 6
decimal place values - it's a bit crazy that a child could get the right
answer but my programming language can't.

Oh well! One to archive in the grey matter; looks like my D key is going to
get some exercise.

Merlin.
 
Back
Top