How to control decimal > 4 digit

  • Thread starter Thread starter S.L.
  • Start date Start date
S

S.L.

My application need to keep 5 decimal digits. So, I can not use Curreny
datatype. But when I use single datatype, I not sure what exact value it
keeps.

My field value is derived from function round(<exp>,4) which result is
1.8745 but when it mutiply by 1.0 (VB automatically convert to 1# ) result
become 1.87450003623962. But if I multiply by 1, it reply 1.8745.

What exactly value does it keep ? And if it keep 1.87450003623962, How to
solve ?
 
Hi,

It is a question of precision. When different precisions are involved,
the higher one is the one generally used. In general, an integer is
considered less precise than a float, which is itself less precise than a
decimal. So, you can multiply by CDec(1.0), a decimal number, rather than
by 1.0, which is a floating point number.


The fact is that if an arithmetic expression involves a floating point
number and an integer-like datatype, the result is in a double (floating
point), unless you use a Decimal data type, ( Currency is a decimal where
the number of decimal is fixed to 4), in which case, the Decimal precision
will be kept (it is more precise than a double floating point number):


? ( (1E10) + (1E-10) ) - 1E10
0

? ( CDec(1E10) + CDec(1E-10)) - CDec(1E10)
0.0000000001



showing that the double float has just lost the 1E-10 when having to
handle 1E10, but the Decimal number didn't loose that precision.


Note that you just have to use the Decimal number somewhere in the
expression to get the precision (just be sure it is the first number
involved in the operation:

? ( CDec(1E10) + (1E-10)) - 1E10
0.0000000001


By contrast, if we use

? ( (1E10) + (1E-10)) - CDec(1E10)
0


The decimal number is just used "too late", the precision is already
lost when it get involved. So, use your data field, in the table, as
Decimal, and in case the arithmetic is a problem, specify you want CDec(
1 ), not just 1, neither 1.0, as example, and the precision should be
better.



Hoping it may help,
Vanderghast, Access MVP
 
Back
Top