Problem with data precision

  • Thread starter Thread starter RP
  • Start date Start date
R

RP

Hi,

I've got following problem:

in new table there are two fields "a" and "b" - data type for both are
"double"

in field "a" there is "488.25" value
in field "b" = "494.36"

both input manually !!

and main problem - query:

SELECT [a]- AS DIFF
FROM Table1;

return not "-6.11" but "-6.11000000000001" - WHY ????

I've installed SP3 for Office (2000) - and it doesn't help ;-(((

what's wrong with it ??

regards,
RP
 
Computers cannot accurately do math on floating point numbers. In fact, they
have trouble with anything that can't be divided by 2. This might help
explain it:

http://oopweb.com/Assembly/Documents/ArtOfAssembly/Volume/Chapter_14/CH14-3.
html
--
Arvin Meyer, MCP, MVP
Microsoft Access
Free Access downloads:
http://www.datastrat.com
http://www.mvps.org/access

RP said:
Hi,

I've got following problem:

in new table there are two fields "a" and "b" - data type for both are
"double"

in field "a" there is "488.25" value
in field "b" = "494.36"

both input manually !!

and main problem - query:

SELECT [a]- AS DIFF
FROM Table1;

return not "-6.11" but "-6.11000000000001" - WHY ????

I've installed SP3 for Office (2000) - and it doesn't help ;-(((

what's wrong with it ??

regards,
RP
Click to expand...
 
U¿ytkownik "help said:
because you user the Data Type "Double"

you can try the "long integer"
Click to expand...

how ?? - did you see - both values are not integer !! - so I can't use "long
integer" at all ;-(((

RP
 
RP said:
how ?? - did you see - both values are not integer !! - so I can't use "long
integer" at all ;-(((
Click to expand...

See Arvin's response and/or try the Currency DataType.
 
U¿ytkownik "Rick Brandt said:
See Arvin's response and/or try the Currency DataType.
Click to expand...

I've already try Currency - it also doesn't help :-(

I'll try on other comp - maybe I've got problem with my machine ?

regards,
RP
 
RP said:
I've already try Currency - it also doesn't help :-(

I'll try on other comp - maybe I've got problem with my machine ?
Click to expand...

I doubt it, your machine is functioning the way they all do. The currency
datatype allows 19 places. 15 to the left and 4 to the right of the decimal
point. It should work for you and you needn't format as currency, format as
General number and [Auto] decimal places and you'll get all 4 places
showing:

23.1101 = 23.1101
23.110149 = 23.1101
23.11015 = 23.1102

and the numbers will add according to the second group:

23.110149
+23.110149
___________
46.2202

That's because the number will automatically (and correctly) round to 4
places after you enter it.
--
Arvin Meyer, MCP, MVP
Microsoft Access
Free Access downloads:
http://www.datastrat.com
http://www.mvps.org/access
 
Back
Top