Problem with compaison of Single to Double

  • Thread starter Thread starter DK
  • Start date Start date
D

DK

Observe the following snippet of code and see if you can explain why
it happens.

Dim sAnything as Single

sAnything = 31.308

If sAnything > 31.308 then
msgbox "value greater"
else
msgbox "value less or equal"
endif


Why does this code always go thru the "value greater" portion?

changing the IF condition to If sAnything > CSng(31.308) then it
behaves as expected but why?
 
Observe the following snippet of code and see if you can explain why
it happens.

Dim sAnything as Single

sAnything = 31.308

If sAnything > 31.308 then
msgbox "value greater"
else
msgbox "value less or equal"
endif


Why does this code always go thru the "value greater" portion?

changing the IF condition to If sAnything > CSng(31.308) then it
behaves as expected but why?
Click to expand...

Because your now comparing two numbers that using the same
level of precision.
 
For those new to programming who may not have run into this, fractions which
can be precisely represented in a few digits in one base, e.g. .308 in base
10, may require many more digits in another base, e.g. base 16. So your
number, 31.308, may look like it should require only a few digits to
represent it, but may, in base 16, require more digits. Some fractions in
some bases cannot be represented in any finite number of digits. 1/3 in
base 10 for example. These are called irrational numbers, the most famous
of which is pi.

31.125 can be precisely represented in base 10 using only 5 digits and can
also be precisely represented in a Single's worth of base 16 digits. So if
in your sample code you substitute 31.125 for 31.308 you will get the result
you expect.

Bob
 
Back
Top