What Algorithm Do Single and Double Use when Storing Imprecise Values?

  • Thread starter Thread starter Nathan Sokalski
  • Start date Start date
N

Nathan Sokalski

I know that Single and Double can only store values that can be stored as
x/2^y, such as 0.5 and 0.125. But when they attempt to store values such as
0.1 or 0.2 whose exact value cannot be stored, what algorithm do they use to
determine what value to store? Thanks.
 
Nathan Sokalski said:
I know that Single and Double can only store values that can be stored as
x/2^y, such as 0.5 and 0.125. But when they attempt to store values such as
0.1 or 0.2 whose exact value cannot be stored, what algorithm do they use
to determine what value to store? Thanks.
Click to expand...

As the documentation
http://msdn.microsoft.com/en-us/library/system.single.aspx
says, the types comply with IEEE 754 which also defines rounding rules.
Search for it; you'll find wikipedia etc.



Armin
 
Nathan Sokalski said:
I know that Single and Double can only store values that can be stored as
x/2^y, such as 0.5 and 0.125. But when they attempt to store values such as
0.1 or 0.2 whose exact value cannot be stored, what algorithm do they use
to determine what value to store? Thanks.
Click to expand...

They use a mantissa and an exponent, so that a number is expressed as
(mantissa)*10^(exponent). We do this frequently in everyday engineering work
when we say for instance, that something took "1.23*10^-6 seconds". However,
when speaking about Single and Double, the "10" has to be interpreted in
base 2 (meaning that it is a "2" in base 10). The exponent and mantissa are
also in base 2. The precission of the mantissa is limited, because the
mantissa and exponent and their signs have to fit into 4 bytes (Single) or 8
bytes (Double). The numbers that can be represented exactly are those that
can be converted into base 2 within that number of bits. For instance, 0.5
(base10) is 0.1 (base 2), but some numbers that have a small number of
decimals in base 10 have an infinite number of decimal places once converted
to base 2, so they will be truncated when assigned to the mantissa, and
therefore they will not be "exact".
 
Nathan Sokalski said:
I know that Single and Double can only store values that can be stored as
x/2^y, such as 0.5 and 0.125. But when they attempt to store values such as
0.1 or 0.2 whose exact value cannot be stored, what algorithm do they use
to determine what value to store? Thanks.
Click to expand...

What Every Computer Scientist Should Know About Floating-Point Arithmetic
http://docs.sun.com/source/806-3568/ncg_goldberg.html

Here's a PDF version
http://www.physics.ohio-state.edu/~dws/grouplinks/floating_point_math.pdf
 
Ken said:
What Every Computer Scientist Should Know About Floating-Point
Arithmetic http://docs.sun.com/source/806-3568/ncg_goldberg.html

Here's a PDF version
Click to expand...
http://www.physics.ohio-state.edu/~dws/grouplinks/floating_point_math.pdf

Interesting link, thanks. I can use such a .pdf from time to time. It
is amazing that CS people know so little about it.

--
Rudy Velthuis http://rvelthuis.de

"I once heard two ladies going on and on about the pains of
childbirth and how men don't seem to know what real pain is. I
asked if either of them ever got themselves caught in a zipper."
-- Emo Philips.
 
I understand how they are stored, that is very simple. My question was what
algorithm they use to determine what value to store. Let me rephrase what I
meant by this. When a value is assigned to a variable that cannot be stored
as an exact value, such as the following:

Dim x As Byte = 1.2

Since the exact value cannot be stored, what algorithm is used to determine
what value will be stored? Since the value is obviously not exact, it must
be rounded up or down, so how do I know which it is?
 
Since the exact value cannot be stored, what algorithm is used to
determine what value will be stored? Since the value is obviously not
exact, it must be rounded up or down, so how do I know which it is?
Click to expand...

But you've already been told: IEEE 754.

SteveT
 
Back
Top