precision loss??

  • Thread starter Thread starter hunt n peck
  • Start date Start date
H

hunt n peck

does anyone know why this expression:
str(csng(18333.332))

equals:
"18333.33"

?

why am i losing precision from this operation?
 
hunt n peck said:
does anyone know why this expression:
str(csng(18333.332))

equals:
"18333.33"

?

why am i losing precision from this operation?
Click to expand...

Welcome to the precision of a single value. ;-) To store more precise
values, you can use the Double data type.
 
But of course. Thanks.
It is confusing when in the command window you type:
? csng(18333.332)

and you get:
18333.332


Another question:

When should I use Decimal? Is it better than Double?
 
hunt n peck said:
But of course. Thanks.
It is confusing when in the command window you type:
? csng(18333.332)

and you get:
18333.332
Click to expand...

Good question. Hmmm... currently I don't know. Maybe it does not really
convert it to a single value like it is done by the compiler.

If I write

dim s as single
s = 18333.332
msgbox s

the msgbox returns "18333.33" whereas

? s

in the immediate window displays "18333.332". Seems to use a different
conversion mechanism.
Another question:

When should I use Decimal? Is it better than Double?
Click to expand...

It depends on the purpose. The data types are described in the docs.
 
Back
Top