Error when multiplying 111,111,111 by 111,111,111

  • Thread starter Thread starter Supervisdr
  • Start date Start date
S

Supervisdr

Did any one notice that in excel, when 111,111,111 or 1,111,111,111 is
multiplied by the same number, the answer shown ends with few 0s. The same
multiplication on Windows calculator (Use the scientific calc- it
accommodates large numbers) the answer is correctly shown.

Funnily, Open office - CALC too makes the same mistake. !!!!!
 
MS Excel, like most numerical software (and AFAIK all spreadsheet software)
follows the IEEE standard for double precision. Numbers in this format only
assure 15 digit accuracy. MS chose to pad with trailing zeros instead of
showing the actual impact of 53-bit binary approximation beyond the 15th
digit. The Windows calculator uses a different number storage format.

Jerry
 
Back
Top