A
and192_2003
Hello there!
We have found some rather odd behaviour in the SqlDecimal type, which
is not displayed in the
standard Decimal type.
It appears that mathematical methods choose a precision and scale that
is inappropriate for the result. Not only
is the result truncated to 6 dp, but the rounding is also incorrect.
This does not occur in Decimal.
Can anyone either tell me what I am doing wrong, or point me in the
direction of a workaround or fix. We can't use
Decimal since our requiments are for a SQL standard of accuracy in our
mathematical operations.
I have enclosed some VB to replicate it and the output from the
program.
Code:
Imports System.Data.SqlTypes
Module Module1
Sub Main()
Dim sLeftSide, sRightSide, sResult As SqlDecimal
Dim dLeftSide, dRightSide, dResult As Decimal
sLeftSide = SqlDecimal.Parse("6252")
sRightSide = SqlDecimal.Parse("342")
sResult = SqlDecimal.Divide(sLeftSide, sRightSide)
dLeftSide = 6252
dRightSide = 342
sResult = SqlDecimal.Divide(sLeftSide, sRightSide)
Console.WriteLine("SqlDecimal Result: " + sResult.ToString)
dResult = Decimal.Divide(dLeftSide, dRightSide)
Console.WriteLine("Decimal Result: " + dResult.ToString)
End Sub
End Module
Output:
SqlDecimal Result: 18.280701
Decimal Result: 18.280701754385964912280701754
Andy
We have found some rather odd behaviour in the SqlDecimal type, which
is not displayed in the
standard Decimal type.
It appears that mathematical methods choose a precision and scale that
is inappropriate for the result. Not only
is the result truncated to 6 dp, but the rounding is also incorrect.
This does not occur in Decimal.
Can anyone either tell me what I am doing wrong, or point me in the
direction of a workaround or fix. We can't use
Decimal since our requiments are for a SQL standard of accuracy in our
mathematical operations.
I have enclosed some VB to replicate it and the output from the
program.
Code:
Imports System.Data.SqlTypes
Module Module1
Sub Main()
Dim sLeftSide, sRightSide, sResult As SqlDecimal
Dim dLeftSide, dRightSide, dResult As Decimal
sLeftSide = SqlDecimal.Parse("6252")
sRightSide = SqlDecimal.Parse("342")
sResult = SqlDecimal.Divide(sLeftSide, sRightSide)
dLeftSide = 6252
dRightSide = 342
sResult = SqlDecimal.Divide(sLeftSide, sRightSide)
Console.WriteLine("SqlDecimal Result: " + sResult.ToString)
dResult = Decimal.Divide(dLeftSide, dRightSide)
Console.WriteLine("Decimal Result: " + dResult.ToString)
End Sub
End Module
Output:
SqlDecimal Result: 18.280701
Decimal Result: 18.280701754385964912280701754
Andy