Tag Archives: IEE 754

Floating Point Precision Problems

Posted on 05/04/2021 by dougaj4

A question on Quora : prompted me to look at how these numbers are handled in Excel, in VBA called from Excel, and in Python called from Excel via pyxll. The results are shown in the screenshot below: Column A … Continue reading →

Posted in Excel, Link to Python, Maths, Newton, NumPy and SciPy, PyXLL, UDFs, VBA | Tagged , , , , , , , | 1 Comment

When does 35 not equal 35?

Posted on 20/12/2011 by dougaj4

… and when does 35 – 35 not equal zero? In the first case the answer is when one “35” is on an Excel spreadsheet and the other 35 is in VBA.  The question arose from the ongoing continuous beam … Continue reading →

Posted in Excel, Maths, Newton, VBA | Tagged , , , | 12 Comments