Error in Simple Math

  • Thread starter Thread starter Chris Baxter
  • Start date Start date
C

Chris Baxter

In my program I am trying to do a simple calculation of 5.1 * 100 which
should =510 if my gradeschool level math is correct. My problem is vb.net
is returning 509.99999999999994 as the answer. Am I doing something wrong?
Try it for yourself in your Command Window in the .Net IDE.
Type ?5.1 * 100 in the command window during debugging.


Thanks,

Chris Baxter
(e-mail address removed)
 
* "Chris Baxter said:
In my program I am trying to do a simple calculation of 5.1 * 100 which
should =510 if my gradeschool level math is correct. My problem is vb.net
is returning 509.99999999999994 as the answer. Am I doing something wrong?
Try it for yourself in your Command Window in the .Net IDE.
Type ?5.1 * 100 in the command window during debugging.

<http://research.microsoft.com/~hollasch/cgindex/coding/ieeefloat.html>
 
Hi Chris,

5.1 * 100 is so obviously 510 to us because we think in decimal.

Convert 5.1 to a binary floating point number, however, and the digits
after the 'binary point' go on somewhat longer than the few bytes available.
Thus the very fact of <storing> 5.1 is going to introduce an error - let alone
multiplying it.

You'd think that it wouldn't need many bits to store 5.1 given that 51
only needs 6!

51 = 32 + 16 + 2 + 1 = 110011

But 5.1 is more complicated than this as it is built up from fractional
powers of two:

5.1 = 4 + 1 + 1/16 + 1/32 + 1/256 + 1/512 + 1/2048 + ...

= 11.00011001101000 ...

We've already got to 14 digits after the 'binary point' and it's still not
accurate, being only 5.09999765625.

Some numbers just don't like being converted to sums of powers of two.

Regards,
Fergus
 
Hi Fergus,

Excellently explained, very good indeed.

Nick.

--
/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
"No matter. Whatever the outcome, you are changed."

Fergus - September 5th 2003
/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
 
Hi Cor,

10 fingers?!! With peas and chips as well, I'm full up after only 4. I bet
you can eat more than 3 Shredded Wheat as well.

Regards,
Fergus
 
* "Fergus Cooney said:
10 fingers?!! With peas and chips as well, I'm full up after only 4. I bet
you can eat more than 3 Shredded Wheat as well.

ROFL again.
 
Fergus,
10 fingers?!! With peas and chips as well, I'm full up after only 4. I bet
you can eat more than 3 Shredded Wheat as well.

Maybe this looks all clear for you, but to eat more than 3 Shredded Wheat,
you first have to dimension it. There are 3 dimensions, which will not fit
in a single bit structure.

In a single bit structure you can only have 1 piece or nothing sometimes
called "null".

So if you want to eat more than 1 piece you have to add an overload bit.

When the overload bit is on, you can bring that to an extra one bit
register.

In that case you can eat 3 pieces, although you have only 2 single bit
registers.
The first one has the human value 2 and the second one the value 1. When you
add that together you get the human value 3.

It is a total different way humans are calculating. Humans have ten fingers
and they don't say on or off, but just one finger, two fingers, three
fingers and with 10 they are at the end.
So with this example of you, they would use 3 fingers.

Humans do use a virtual calculating system that is in almost every culture
the same although there are too who use also there toes. But most say in
virtual thinking 11 means 2 full hands of fingers plus one finger. For a
computer this would be 3. Therefore when I can eat 3 peas, you eat 11 peas
and that is a lot I think.

But Humans can calculate with half fingers but do not cut off them further.

So in there mind, when they have to use half or partial fingers, they bring
it first back to a value that does not need half fingers, so 5.1 * 100
becomes the same as 1 * 510.

If the computer software would do that (and as far as I know some do or
did), the results would be the same as humans calculate even if the computer
uses binary values for calculating. That is what I remind me from the past
when the system of binary calculating was developed the system was build on.

But maybe this has to do because that the importance of the rule is
shifting, first were humans important, but now he or she is only the one who
enters values using the keyboard, the computer rules.

This means, when you think you can eat 1000 chips, you maybe only get 999 or
something. But who cares

I thought you did not understand what I did mean, so I made a little
explanation.

:-)) Have fun be happy

Cor
 
Hi Cor,

LOL, and LOL again and again. ;-))

I understand much better now and so I realise that you don't have 10
fingers - you have 10.

If I had 1000 chips, I'd go to a casino!! Do you want to come?

Best Regards,
Fergus
 
Back
Top