Really weird rounding

  • Thread starter Thread starter alsmeirelles
  • Start date Start date
A

alsmeirelles

Hi all,

I Have run this test:

Private Sub test()
Dim d As Double
Dim f As Single
Dim output As String

d = 8888888888888.8887
f = 9999999988888.8887

output = "Original double is: " & d.ToString() & " Formated
Double is: " & FormatNumber(d, 2, TriState.True) & vbCrLf _
& "Original single is: " & f.ToString() & "Formated Single is: "
& FormatNumber(f, 2, TriState.True)

MsgBox(output)
End Sub


The result is this:

Original Double is: 8888888888888.8887 Formated Double is:
8.888.888.888.888,89
Original Single is: 1E+13 Formated Single is 10.000.000.000.000,00

Now cames the big question:
How come that 9999999988888.8887 was rounded to 10.000.000.000.000,00
(or 1E+13 in scientific notation)?

This test was run in VB.Net 2003, 1.1 Framework.

Thanks for any help.
 
Alsmeirelles,

Not weird and thousands of time written in the dotNet newsgroup.

In dotNet is ISO bankers rounding used. Not the way as humans do it.

If you search for this in this and the C# newsgroup on Google you get maybe
more pages than the Phone book of by instance New York.

Cor
 
I undestand that VB.Net uses the bankers rounding, what was done with
the double variable is perfect.
My question is about the behavior of the single. See, the original
number was 9999999988888.8887 .

In no way it could have been converted to 10.000.000.000.000,00.

And, as R.MacDonald said, it's true floating-point numbers have
"strange" behaviors sometimes, basically due to the lack of
representability they have in binary bases. For example,
there's no way to express exactly 0.1 in binary. Although I haven't
read the suggested article yet (but I willl), I think that its main
subject is about this lack of representability and what happens to
the numbers when they are in the edge of an overflow (which is not the
case here). So, I consider that my question is a little more tricky
than "it's the bankers rounding that does this".

Of course, I know very little of the .Net framework, and I may be an
ignorant concerning floating-points, so, any help is welcome.

Regards,
Alsm
 
If you do this:

Dim g as Double = 9999999988888.8887

and then output it, you get 9,999,999,988,888.89

Doubles are more accurate.

Francesco Balena says in his VB2005 book "you should use Double variables
instead of Single variables if you care about performance because they
don't require any conversion when stored in and read from the CPU
floating-point registers." Maybe that conversion accounts for or
contributes to the round-off error you are seeing.

Robin S.
 
I undestand that VB.Net uses the bankers rounding, what was done with
the double variable is perfect.
My question is about the behavior of the single. See, the original
number was 9999999988888.8887 .

In no way it could have been converted to 10.000.000.000.000,00.
Click to expand...

this is not a banking rounding.
This is due to the accurancy of a System.Single: over a certain amount it
looses the digit infos and itroduces the 10based-exponent.
 
this is not a banking rounding.
This is due to the accurancy of a System.Single: over a certain amount it
Click to expand...

Yes, definitely it´s not the rounding. The max value of a single is
3.40282347E+38,
so, if Single looses accuracy beyond a certain point, does anyone know
where is this point?

Finally, if the accuracy loss is the case here, that´s a bug or what? I
´m within the possible values range.
looses the digit infos and itroduces the 10based-exponent.
Click to expand...

Once again, thanks for everyone that has contributed.
 
Yes, definitely it´s not the rounding. The max value of a single is
3.40282347E+38,
so, if Single looses accuracy beyond a certain point, does anyone know
where is this point?

Finally, if the accuracy loss is the case here, that´s a bug or what?
I ´m within the possible values range.
Click to expand...

It isn't accuracy, it's precision:
http://en.wikipedia.org/wiki/Accuracy_and_precision

You should also be aware of the concept of significant figures:
http://en.wikipedia.org/wiki/Significant_figures

And before you wonder about numbers like 0.1 (base 10) not being represented
exactly in the binary system (it's been done to death in many newsgroups),
remember that it's a recurring "decimal" in binary, the same as 1/3 is a
recurring decimal in base 10, and therefore cannot be represented exactly
with a limited number of digits for the representation. Books on numerical
analysis should deal with the matter rigorously.

The Decimal data type is another kettle of fish.

HTH

Andrew
 
Yes, definitely it´s not the rounding. The max value of a single is
3.40282347E+38,
so, if Single looses accuracy beyond a certain point, does anyone know
where is this point?
Click to expand...

As a floating point number is never accurate, it can't lose the accuracy.

If you mean at what point the limitation of precision is starting to
spill into whole numbers, it's when you are expressing numbers with more
than seven digits on the left of the decimal separator.
Finally, if the accuracy loss is the case here, that´s a bug or what? I
´m within the possible values range.
Click to expand...

Yes, but you are outside the range of the precision.
 
Back
Top