LinEstGap with non-linear functions

The LinEstGap user defined function (UDF) allows the built-in Linest function to be used on data with empty cells or hidden rows, returning results that agree with the chart trend-line function, which automatically ignores blank cells and hidden data.  It also optionally allows any lines with Error values to be ignored, rather than returning an error.

Following a comment here I have modified the code and added additional functions to work with all the non-linear curve  types provided by the chart trend line (Exponential, Logarithmic, Polynomial, and Power).

The revised spreadsheet includes examples and full open-source code, and can be downloaded from:

Linest-PolyDA.zip

The download zip file contains two versions of the spreadsheet.  Linest-PolyDA uses the new dynamic array feature.  Linest-PolyA has fixed size array functions that will work with older versions of Excel (see Using Array Formulas for more details).

I have extracted the code for extracting blanks and errors into a separate function (RemoveGaps), so the LinEstGap function is now reduced to:

Function LinEstGap(Ycells As Range, XA As Variant, Optional Const0 As Boolean = True, Optional Stats As Boolean = False, _
    Optional IgnoreErrors = False)
    Dim iErr As Variant
    Dim YA2() As Double, XA2() As Double

    iErr = RemoveGaps(Ycells, XA, XA2, YA2, Const0, Stats, IgnoreErrors)

    If iErr = 0 Then
        LinEstGap = WorksheetFunction.LinEst(YA2, XA2, Const0, Stats)
    Else
        LinEstGap = iErr
    End If
End Function

I have also added separate functions that call the RemoveGaps function, then Linest with modified X and Y data to return parameters for non-linear curves (see Using LINEST for non-linear curve fitting for more information on using Linest, and the download file for the VBA code).  Examples of the new functions are shown below (click on any image for a full size view):

The PolyEstGap function will return parameters for any order of polynomial function (but see Fitting high order polynomials for warnings on using orders greater than 3 or 4).

The functions ExpEstGap, PowerEstGap, and LogEstGap all return a two element array with values a and b for the functions:

  • ExpEstGap:   y = b. e^a
  • PowerEstGap; y = b.x^a
  • LogEstGap:     y = a.ln(x) + b

The screen shots below show the PolyEstGap function used on data with gaps and errors and filtered data, but the other Gap functions all work the same way.

Using the built in LinEst on data with gaps returns an error, but the Gap functions return the same results as the chart trend line; i.e. any line with a gap in the data is ignored:

Filtered data is also ignored:

The chart trend line treats error values as zero, so returns invalid results.  The Gap functions return an error by default (line 19 below), but if the final optional argument ( IgnoreErrors) is set to True they will return the correct results for the data without the error lines.  Note that the results with errors in rows 12 and 13 below are exactly the same as those with rows 12 and 13 hidden above.

The Gap functions may also be used on data arranged in rows as shown below:

For the polynomial examples above the LinEstGap function may also be used with modified input, or with multiple columns of x data, to generate the same results. For the other non-linear curve types however modifying the LinEstGap input data will generate an error, but the new Gap functions return the correct results.

Posted in Arrays, Curve fitting, Excel, Maths, Newton, UDFs, VBA | Tagged , , , , , , , , , | 4 Comments

DigitGraph Spreadsheet and WebPlot Digitiser

I have posted here previously a spreadsheet that allows XY data to be extracted from images of graphs, maps or other images of objects in a single plane: How to digitise a scanned image.

I have updated the instructions for the procedure to cover Excel’s new dynamic array feature, which can return an array of data from a function entered in a single cell.  The new version can be downloaded from:

DigitGraph2.zip

I also recently discovered the Webplot Digitizer program, that can either be used on-line, or as a download.  The download is free (with no advertising), with a button for voluntary donations.  The screen-shot below shows the downloaded version.  The program is considerably more sophisticated than my spreadsheet, providing not only a magnified image of the graph at the cursor location, but also allowing the option of either manual selection of data points, or a fully automated process to detect the graph line(s) and generate the data points.

Output from the WebPlot program (both manual and automated) is shown below plotted together with output from my Digitgraph function, showing close agreement for all three lines.

Posted in Charts, Charts, Computing - general, Curve fitting, Drawing, Excel, UDFs, VBA | Tagged , , , , , | 5 Comments

Katherine Johnson, NASA Legend

Cartoon by Steve Breen from The San-Diego Union Tribune.

More on Katherine Johnson.

Posted in Maths, Newton | Tagged , , | Leave a comment

Microsoft Python Resources

In addition to the documentation of their software products, Microsoft now have a huge amount of on-line resources of more general content, including:

YouTube: Python for Beginners

Probably the largest hurdle when learning any new programming language is simply knowing where to get started. This is why we, Chris and Susan, decided to create this series about Python for Beginners! (44 Videos)

For those who prefer written tutorials, with interactive examples:

Introduction to Python

Learn how to write basic Python code, declare variables, and work with console input and output

In this module, you will:

Explore the choices available to run Python apps
Use the Python interpreter to execute statements and scripts
Learn how to declare variables
Build a simple Python app that takes input and produces output

Also see Microsoft Learn for other Python based tutorials, or search for a huge range of other topics.

For documentation of the use of Python in Visual Studio see Python at Visual Studio

Welcome Python Developers

See how Microsoft is making our platform better for Python developers
Python Blog
Posted in Computing - general, Excel, Link to Python | Tagged , , , | Leave a comment

Paul Kelly and Joni Mitchell

Leading Australian singer-songwriter Paul Kelly is about to release a powerful new single that tackles Australia’s lack of action on climate change and environmental destruction.

Entitled Sleep, Australia, Sleep, Kelly says the song is both “a lament in the form of a lullaby” and “a critique of the widespread attitude among humans that we are the most important life form on the planet”. (From The Sydney Morning Herald).

50 years ago, Joni Mitchell had a similar message:

… which seems to support this comment:

How sad – all these years have passed and it’s still relevant. For an intelligent animal, we’re awfully slow learners.

Lyrics to Sleep Australia Sleep:

Sleep, Australia, sleep
The night is on the creep
Shut out the noise all around
Sleep, Australia, sleep
And dream of counting sheep
Jumping in fields coloured brown
Who’ll rock the cradle and cry?
Who’ll rock the cradle and cry?
Sleep, Australia, sleep
As off the cliff the kingdoms leap
Count them as they say goodbye
Count down the little things
The insects and birds
Count down the bigger things
The flocks and the herds
Count down our rivers
Our pastures and trees
But there’s no need to hurry
Oh, sleep now, don’t worry
‘Coz it’s only a matter of degrees
Fog, Australia, fog
Just like the boiling frog
As we go we won’t feel a thing

Who’ll rock the cradle and cry?
Who’ll rock the cradle and cry?
Sleep, my country, sleep
As off the cliff the kingdoms leap
Count them as they pass on by
Our children might know them
But their children will not
We won’t know ’til it’s gone
All the glory we’ve got
But there are more wonders coming
All new kinds of shows
With acid seas rising
To kiss coastal mountains
And big cyclones pounding
And firestorms devouring
And we’ll lose track of counting
As the corpses keep mounting
But hey, that’s just the way this old world goes
Sleep, my country, sleep
As we sow so shall we reap
Who’ll rock the cradle and cry?

Posted in Bach, Climate | Tagged , , , | Leave a comment