Hi,
Here's what I get on exp(0.0267329893588985) - 1 with Windows 7 calculator
0,02709352124749775420561
Here's the display from a high precision calculator online http://ift.tt/1n5BJnP
0,0270935212474977542056098047663
As you may see, Casio calculator online and Windows 7 calculator give consistent results up to the 1E-22
Here's what get with EXP() function on Excel 2010 SP3
0,0270935212474979
Here's what I get using Taylor series on Excel 2010 SP3
0,0270935212474976
So, both SQL 2005 and SQL 2012 provide similar precision : 1E-16.
SQL 2005 gives a result 1E-16 below the "correct" (?) result, while SQL 2012 result is above.
As you may know, it's impossible to give an exact answer to EXP(x), and I assumed (naively?) that both
Windows 7 calculator and Casio Online gave accurate answers to 22 digits.
Sebastian Sajaroff Senior DBA Pharmacies Jean Coutu
No comments:
Post a Comment