Tag Archives: IEE 754

Floating Point Precision Problems

A question on Quora : prompted me to look at how these numbers are handled in Excel, in VBA called from Excel, and in Python called from Excel via pyxll. The results are shown in the screenshot below: Column A … Continue reading

Posted in Excel, Link to Python, Maths, Newton, NumPy and SciPy, PyXLL, UDFs, VBA | Tagged , , , , , , , | 1 Comment

When does 35 not equal 35?

… and when does 35 – 35 not equal zero? In the first case the answer is when one “35” is on an Excel spreadsheet and the other 35 is in VBA.  The question arose from the ongoing continuous beam … Continue reading

Posted in Excel, Maths, Newton, VBA | Tagged , , , | 12 Comments