Strange float behaviour

  • Thread starter Thread starter tedd_n_alex
  • Start date Start date
T

tedd_n_alex

I have a problem with a program i have that only shows when the code is
compiled in "Release" mode and ONLY when the .NET debugger is not
attached to the process.

I have some drawing code that checks to see if a data point is inside
the bounding rectangle of a graph. if it is not then the point is not
shown.

I find that in the deployed code the final data point does not show. I
therefore went through the code adding message boxes to see if i could
spot any problems.

I eventually did something like this

Dim graphRectangle as RectangleF
graphRectangle = graph.GridRect

MessageBox.Show(graph.GridRect.Right = graphRectangle.Right).ToString)
MessageBox.Show(graph.GridRect.Right - graphRectangle.Right).ToString)

MessageBox.Show((graph.GridRect.Right = CType(graphRectangle.Right,
Single)).ToString())

MessageBox.Show(BitConverter.ToInt32(BitConverter.GetBytes(graph.GridRect.Right),
0).ToString)

MessageBox.Show(BitConverter.ToInt32(BitConverter.GetBytes(graphRectangle.Right),
0).ToString)

now you would expect the graphRectangle variable to be the same as the
graph.GridRect property but it is not.

The first message box returns false
The second message box shows 2.098..E-05
The third and fourth message boxes show 11443800 indicating that the
binary values are the same.

So what is happening!! I have found using ctype to convert the value
to a single on each side of the equals operator yields the correct
value. I also found it completly impossible to reporduce this problem
outside my project.

Any ideas anyone??

VB.NET on .NET 1.1 sp1
 
now you would expect the graphRectangle variable to be the same as the
graph.GridRect property but it is not.

The first message box returns false
The second message box shows 2.098..E-05
The third and fourth message boxes show 11443800 indicating that the
binary values are the same.

So what is happening!! I have found using ctype to convert the value
to a single on each side of the equals operator yields the correct
value. I also found it completly impossible to reporduce this problem
outside my project.

Any ideas anyone??

VB.NET on .NET 1.1 sp1
Click to expand...

Just a visitor around here.... If floats are causing you greif, here's some
good info.

What Every Computer Scientist Should Know About Floating-Point Arithmetic
http://docs.sun.com/source/806-3568/ncg_goldberg.html

(Complete) Tutorial to Understand IEEE Floating-Point Errors
http://support.microsoft.com/kb/42980/en-us

FP Representation
http://www.cprogramming.com/tutorial/floating_point/understanding_floating_point_representation.html

fwiw, do a web search on comparing numbers to EPSILON instead of each
other... for example,

If X = Y Then
Do Something
End If

can be replaced by

If Abs(X - Y) < EPSILON Then
Do Something
End If

fwiw, you can find the value for EPSILON in the C header files that ship
with VS... or, just figure out the minimum resolution you need and create
your own EPSILON_LIKE constant.
 
Back
Top