Prcesion bug VBA Log function

  • Thread starter Thread starter Mathew Uthup
  • Start date Start date
M

Mathew Uthup

Does any body know why the following code returns wrong value
Int(log(128)/log(2)) returns 6 intsead of 7 ?

Executing the code without the Int function returns the currect value
for example log(128)/ log (2) returns correctly 7 ..

Is this a bug in the int function or the log function..?
Mathew
 
No, it's not a bug, it's a design limitation.

Whenever you're using floating point numbers (such as the result of the log
function), there's a high chance that the number won't be able to be
represented accurately, so it ends up being rounded off.

Because of that rounding, Log(128) / Log(2) comes out to ever so slightly
less than 7, as you can see if you calculate Log(128) / Log(2) - 7.

The most common way to address this is to add a miniscule amount to the
response (effectively rounding it off) before applying the Int() function.
For example: Int(Log(128) / Log(2) + 5e-16)



Rob
 
Mathew Uthup said:
Does any body know why the following code returns wrong value
Int(log(128)/log(2)) returns 6 intsead of 7 ?

Executing the code without the Int function returns the currect value
for example log(128)/ log (2) returns correctly 7 ..

Is this a bug in the int function or the log function..?
Click to expand...

I suspect neither, but a matter of understanding the way in which numbers
are represented internally, in binary, and how they are coerced to different
number types behind the scenes. And, at the point where log(128)/log(2) is
handed over to the Int function, the binary representation is a tiny
fraction less than 7 'way out in the fraction.

Larry Linson
Microsoft Office Access MVP
 
Thanks, I am aware of binary representation of the number. the question I
have is did this behvioir recently change as part of any security patches? We
have production code written more than 5 years ago which all of a sudden is
breaking after February Security patch. All I am trying to do is find out
if this is the behavior of the log function all along or did it change
recentyl?
 
Back
Top