vbscript/vba function help

  • Thread starter Thread starter joe blow
  • Start date Start date
J

joe blow

I took over someone's project and there are some glitches
in the program. I narrowed it down to the cdbl function.
There is a number in a variable with the value 148.10

after converting that to a double, the number is 148.1

These are really 2 different records, so things are
getting mixed up. I'm trying to figure out if the cdbl
function is necessary. anyways, converting a number with a
zero on the end to a double drops the zero. Is there some
way to keep it on?


tia!
 
Keep it as a text string. Showing the extra zero when it's a number is a
function of how you display it, not how you store it.
 
I don't consider that 148.10 is actually 148.1 a problem.

What not have 148.10000000000000000000000

The above are all the same.

if you are talking about a report, them simply use formatting to display 2,
3 or 4 decimals, or whatever you want. However, to the computer..it is
always 148.1. It is up to YOU to display the data the way you want.
 
I don't consider that 148.10 is actually 148.1 a problem.

What not have 148.10000000000000000000000
Click to expand...

That's because you're not a mathematician.

The number 148.1 is actually shorthand for the set of all numbers
{x: 148.05 <= x < 148.10}

while 148.10 is really
{x: 148.095 <= x < 149.105}

and the last number is very much more precise. The wonder (or the drawback,
depending on your needs) of IEEE floating point numbers is that all these
things have the same binary representation, and the only way of indicating
5-give-or-take-20% is to use a text format.

B Wishes


Tim F
 
Actually, significant figures play into this as well.

To a scientist, 14.1 is not the same as 14.10. The former indicates that the
accuracy of the number is known only to the first decimal place, while the
latter number indicates that the accuracy is known to 2 decimal places.

Thus, they can be "different" numbers in this context!
 
Back
Top