Why does this decimal division not give an exact result?

  • Thread starter Thread starter Elroy Flynn
  • Start date Start date
E

Elroy Flynn

Decimal a = 42.485M;
Decimal b = 48M;
Decimal c = a / b;
Console.WriteLine(c.ToString());

gives the result 0.8851041666666666666666666667

The correct, exact answer is .885.

I know that floating point types cannot represent some values
precisely, but the decimal type (which is a base 10 FP type) CAN
represent these three values (the dividend, divisor, and quotient)
exactly.

Furthermore, doc states that "The result of an operation on values of
type decimal is that which would result from calculating an exact
result (preserving scale, as defined for each operator) and then
rounding to fit the representation."

Could someone explain this unexpected result?
 
Hmmm... I just ran this on Excel and got a result of
0.885104166666667. So, I calculated it by hand and got the same
result as Excel. Am I not understanding the setup of you problem? I
don't think the .885 is the propper result.

John
 
Decimal a = 42.485M;
Decimal b = 48M;
Decimal c = a / b;
Console.WriteLine(c.ToString());

gives the result 0.8851041666666666666666666667

The correct, exact answer is .885.

I know that floating point types cannot represent some values
precisely, but the decimal type (which is a base 10 FP type) CAN
represent these three values (the dividend, divisor, and quotient)
exactly.

Furthermore, doc states that "The result of an operation on values of
type decimal is that which would result from calculating an exact
result (preserving scale, as defined for each operator) and then
rounding to fit the representation."

Could someone explain this unexpected result?
Click to expand...

It is the correct answer. Using a TI-30X IIS, I get:

0.885104167

As the result. Using GNU bc (an arbitrary precision calculator,
accurate to as many digits as are specified), I get:

Friday, 2008-Oct-17 at 14:38:25 - mbt@zest - Linux v2.6.27
Ubuntu Intrepid:[0-7/6187-0]:~> bc
bc 1.06.94
Copyright 1991-1994, 1997, 1998, 2000, 2004, 2006 Free Software
Foundation, Inc. This is free software with ABSOLUTELY NO WARRANTY.
For details type `warranty'.
scale=20
42.485/48
.88510416666666666666

Doing it on paper, I get (use monospaced font to see it correctly):

___
0.8851041666
)-=-----------
48 )42.485
38 4
----
408
384
---
245
240
---
50
48
--
20
0
--
200
192
---
80
48
--
320
288
---
320
288
---
320

So...

Check your math before you post a mathematical error. :-)

--- Mike

--
My sigfile ran away and is on hiatus.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)

iEYEAREDAAYFAkj43bIACgkQ0kE/IBnFmjBNpACeJ7fl47r58URWiYI+lNNP3bQK
wrwAn27lSGAb0QlG7p6ZihVKvvGAA0nL
=IeNb
-----END PGP SIGNATURE-----
 
Decimal a = 42.485M;
Decimal b = 48M;
Decimal c = a / b;
Console.WriteLine(c.ToString());

gives the result 0.8851041666666666666666666667

The correct, exact answer is .885.
Click to expand...

Methinks you were using the numeric keypad and accidentally hit 5 when you
hit 8, because 42.48 / 48 = .885, not 42.485.
 
Elroy Flynn said:
Decimal a = 42.485M;
Decimal b = 48M;
Decimal c = a / b;
Console.WriteLine(c.ToString());

gives the result 0.8851041666666666666666666667

The correct, exact answer is .885.
Click to expand...

It's really not. 0.885 * 48 = 42.48.
I know that floating point types cannot represent some values
precisely, but the decimal type (which is a base 10 FP type) CAN
represent these three values (the dividend, divisor, and quotient)
exactly.

Furthermore, doc states that "The result of an operation on values of
type decimal is that which would result from calculating an exact
result (preserving scale, as defined for each operator) and then
rounding to fit the representation."

Could someone explain this unexpected result?
Click to expand...

Your expectations are flawed :) Make a = 42.48M and you'll get your
expected result.
 
where's the "delete from web" command?
Click to expand...

This is a newsgroup. You cannot delete your post once you have sent
it, because most servers on Usenet do not honor the commands that used
to be used for that since they've been abused in the past.

As far as Google Groups, you'll have to go through the same process
that anyone that wants to have their messages pulled from their
archives has to do: Find the form if they still have it and request
that the message(s) be removed by their Message ID. You will also have
to prove that you own the email address associated with the Message ID.

I don't know where that is, because I don't use the G interface... it's
really lacking in features when it comes to dealing with the Usenet.

--- Mike

--
My sigfile ran away and is on hiatus.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)

iEYEAREDAAYFAkj5CUsACgkQ0kE/IBnFmjB3OACfeu4ZnUJ3CaGI+W2BBDnooCan
xaUAn1Ad79iOJ6Supn4QruYVII9ivlF/
=TKO1
-----END PGP SIGNATURE-----
 
Michael said:
This is a newsgroup. You cannot delete your post once you have sent
it, because most servers on Usenet do not honor the commands that used
to be used for that since they've been abused in the past.

As far as Google Groups, you'll have to go through the same process
that anyone that wants to have their messages pulled from their
archives has to do: Find the form if they still have it and request
that the message(s) be removed by their Message ID. You will also have
to prove that you own the email address associated with the Message ID.
Click to expand...

I don't think it was meant that literally.

Arne
 
I don't think it was meant that literally.
Click to expand...

Possibly, though there was no indication of that...

--- Mike

--
My sigfile ran away and is on hiatus.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)

iEYEAREDAAYFAkj5VvAACgkQ0kE/IBnFmjCqFgCfZ/tZ5yNgRFw6ZaARRIze9V1Z
LWcAni8ikvcyG6Ay26zyJ64lzORNKinR
=QsGg
-----END PGP SIGNATURE-----
 
Are you posting in HTML format or something? To me your posts are empty but
have two attachments...

01: ATT0096.txt
02: signature.asc
 
As Peter Morris indicated with your other post, something strange is
happening. You actual text is showing as an attachment to a blank post.
The post on this part of the thread that Arne V. responded to did not come
through at all. I'm using Windows Mail to view the group.

Both appear fine at the microsoft discussion groups page though, but it's
strange...

news:20081017232431.338ecb01@zest...
 
Family said:
As Peter Morris indicated with your other post, something strange is
happening. You actual text is showing as an attachment to a blank post.
The post on this part of the thread that Arne V. responded to did not
come through at all. I'm using Windows Mail to view the group.

Both appear fine at the microsoft discussion groups page though, but
it's strange...
Click to expand...

I use Thunderbird to view the newsgroups, and there the posts show up
just as usual.
 
Are you posting in HTML format or something? To me your posts are
empty but have two attachments...

01: ATT0096.txt
02: signature.asc
Click to expand...

(This post is not digitally signed.)

It would appear that Windows Mail and Outlook Express (I believe
they're the same program, aren't they?) do not seem to support
cryptographically signed Usenet messages. You may want to seek out a
plugin that will teach it how to read such messages (though this should
be trivially easy; they are sent using PGP/MIME, which is a well-known
message standard).

The message is properly parsed by all other newsreader software known
to me, including the Google Groups Web interface. You may want to
submit a bug report to Microsoft.

--- Mike
 
As Peter Morris indicated with your other post, something strange is
happening. You actual text is showing as an attachment to a blank
post. The post on this part of the thread that Arne V. responded to
did not come through at all. I'm using Windows Mail to view the
group.

Both appear fine at the microsoft discussion groups page though, but
it's strange...
Click to expand...

(This message is not digitally signed.)

You may, like I mentioned to Peter Morris in his response, send a bug
report to Microsoft; you appear to be using the same software
(Microsoft Windows Mail 6.0.6001.18000) that Peter is. It would appear
that there is a bug in that particular software that is prohibiting it
from reading cryptographically signed messages.

Google Groups also shows the posts as intended, as does all other
newsreading software that I am aware of.

Is there an add-in or plugin for Windows Mail that perhaps works around
this functionality?

It would appear that the bug is well-known, as well.[1]

Sorry for the trouble. Maybe Microsoft will listen...?

It'd appear that Arne is using Thunderbird 2 on Windows, as well as
Göran. Not sure why, but in another thread on SFTP, K. Viltersten
(using Microsoft Outlook Express 6.00.2900.5512) isn't appearing to
have that same issue.

--- Mike

[1] http://is.gd/4l5k#Handling_of_PGP.2FMIME_signed_messages
 
It'd appear that Arne is using Thunderbird 2 on Windows, as well as
Göran. Not sure why, but in another thread on SFTP, K. Viltersten
(using Microsoft Outlook Express 6.00.2900.5512) isn't appearing to
have that same issue.
Click to expand...

I have the same problem in OE 6.00.2900.2180. And MS is NOT going to fix
OE/Windows Mail. They don't seem to give a rat's ass about it these days.
 
I have the same problem in OE 6.00.2900.2180. And MS is NOT going to
fix OE/Windows Mail. They don't seem to give a rat's ass about it
these days.
Click to expand...

Eh, well. I don't want my posts to be illegible, so...

At some point in the near future I'll have to modify my newsreader
software so that I can set per-group preferences on applying
cryptographic signatures. For now, I'll just keep them disabled; that
ought to help.

--- Mike
 
Back
Top