Losing Accuracy in reading numbers

  • Thread starter Thread starter AGP
  • Start date Start date
A

AGP

I thought i had this down but i think i am missing something. i read in a
series of strings in CSV format that are actually numbers. i do something
like

Dim gVal as Double
Dim sData as String = "77.05788457660967"

gVal = Double.Parse(gArrCoords(0),
Globalization.NumberFormatInfo.InvariantInfo)
Debug.printline(gVal)

and this is what i get printed
77.057884576609666

This one seems to be small but i thought the double was supposed to hold the
exact number. I think the conversion has to do with the accuarcy built into
the format type but I cant figure out how to convert the exact number.
Suppose this was a scientific app that required extreme accuarcy in reading
the source data. How would i read it or assign it differently?

tia

AGP
 
AGP,

Simple search on Internet about accuracy and doubles. This is a dead horse.

Cor
 
Gotcha. I've been doing that search for the last hour or so. Will do some
more reading and learning but so far I've learned that the double can hold
15 significant digits of accuarcy. I think I was mistakingly assuming the
range of the double would take care of my numbers but that is not the case.

In my case the documentation for the source data only states that the value
is consisting of floating point value.Looks like one solution is changing my
type to Decimal but i have to weight the performnce hit.

AGP
 
Dim sData As String = "77.05788457660967"



Dim GVal As Decimal = Decimal.Parse(sData,
Globalization.NumberFormatInfo.InvariantInfo)

Debug.WriteLine(GVal)







HTH



Michel
 
Back
Top