Why does 28.08 show up as 28.080000000000002 in DataGridView

  • Thread starter Thread starter Daniel Manes
  • Start date Start date
D

Daniel Manes

I'm baffled. I have a column in a SQL Server Express database called
"Longitude," which is a float. When I view the table in a DataGridView,
some of the numbers, which only have two decimal places in the database
show up with *15* decimal places and are ever so slightly off (in the
example in the subject line, by about 2E-15).

I'm not doing any operations on this column. It's just running a stored
procedure which performs a pretty basic SELECT on the table. If I run
the stored procedure in Management Studio Express, all numbers show up
fine (just two decimal places).

What's going on here?

Thanks,

-Dan
 
Daniel said:
I'm baffled. I have a column in a SQL Server Express database called
"Longitude," which is a float. When I view the table in a DataGridView,
some of the numbers, which only have two decimal places in the database
show up with *15* decimal places and are ever so slightly off (in the
example in the subject line, by about 2E-15).

I'm not doing any operations on this column. It's just running a stored
procedure which performs a pretty basic SELECT on the table. If I run
the stored procedure in Management Studio Express, all numbers show up
fine (just two decimal places).

What's going on here?
Click to expand...

float is an approamite datatype. It consists of a 53-bit mantissa and a
nine-bit exponent. That permits us to store a very broad range of
numbers. From 1E310 to 1E-310. The price is that far from all decimal
numbers can be represented exactly.

When a floating-point number is presented, it appears that different
applications are more or less zealous to show the decimals. If you were
to look at that column from Query Analyzer, the query tools that came with
SQL 2000, you would probably see that random noise at the end. QA shows
more decimals than Mgmt Studio does.

What is the best suggestion for your case I don't know, but you should
probably ask in a newsgroup where DataGridView is popular.


--
Erland Sommarskog, SQL Server MVP, (e-mail address removed)

Books Online for SQL Server 2005 at
http://www.microsoft.com/technet/prodtechnol/sql/2005/downloads/books.mspx
Books Online for SQL Server 2000 at
http://www.microsoft.com/sql/prodinfo/previousversions/books.mspx
 
Back
Top