Double is doing incorrect subtraction

  • Thread starter Thread starter Nathan Sokalski
  • Start date Start date
N

Nathan Sokalski

I have the following code:

Dim x As Double = 45.333
Dim y As Double = 45
Dim z As Double = x - y
Me.Label1.Text = z.ToString()

The result should obviously be 0.333, but the value displayed in
Me.Label1.Text is:

0.332999999999998

I have seen stuff like this in other situations, but I never managed to
figure out how to fix it. Can somebody help me here?
 
Nathan Sokalski said:
I have the following code:

Dim x As Double = 45.333
Dim y As Double = 45
Dim z As Double = x - y
Me.Label1.Text = z.ToString()

The result should obviously be 0.333, but the value displayed in
Me.Label1.Text is:

0.332999999999998

I have seen stuff like this in other situations, but I never managed to
figure out how to fix it. Can somebody help me here?
Click to expand...

In order to show the correct result you have to format it. For example you
could do this:
Me.Label1.Text =z.ToString("0.000")

Doubles are floating point numbers and that causes this behavior.

If you want to find more information try keyword "IEEE 754" with Google.

-Teemu
 
Consider using decimal instead of double if you want to get the answer you
expect below.
 
I have the following code:

Dim x As Double = 45.333
Dim y As Double = 45
Dim z As Double = x - y
Me.Label1.Text = z.ToString()

The result should obviously be 0.333, but the value displayed in
Me.Label1.Text is:

0.332999999999998

I have seen stuff like this in other situations, but I never managed to
figure out how to fix it. Can somebody help me here?
Click to expand...

Double is doing entirely correct subtraction.

The problem is that Double is an implementation of IEEE floating point, and
IEEE floating point is incapable of representing all numbers exactly.

Read this:

http://support.microsoft.com/kb/42980
 
Nathan,

This is basic

Float, Double or whatever floating poing calculation for mathimatical
programming,
Decimal for business programming,

Cor
 
Nathan Sokalski said:
Dim x As Double = 45.333
Dim y As Double = 45
Dim z As Double = x - y
Me.Label1.Text = z.ToString()

The result should obviously be 0.333, but the value displayed in
Me.Label1.Text is:

0.332999999999998

I have seen stuff like this in other situations, but I never managed to
figure out how to fix it. Can somebody help me here?
Click to expand...

In addition to the other replies, take a look at these articles:

IEEE Standard 754 Floating Point Numbers
<URL:http://steve.hollasch.net/cgindex/coding/ieeefloat.html>
 
Back
Top