Non linear regression – 1

Recent posts have described how to fit linear equations (or equations that can be made linear) to a set of data, using the Excel Linest functions, or User Defined Functions (UDF’s) linking to the ALGLIB library (see Using LINEST for non-linear curve fitting and ALGLIB linear and polynomial fitting functions).

This post will describe how to perform non-linear regression using the Excel Solver, and the following one using ALGLIB non-linear regression functions, and the relative advantages and disadvantages of the two methods.

Non-linear regression uses an iterative process to minimise the square of the difference between the values in the data being fitted, and the values generated by the regression equation.  This is exactly the sort of problem that the Excel Solver is designed to solve, and setting up the solution is fairly straightforward.  The steps are:

  • Paste the base data into a spreadsheet, with independent variables and dependent variable in adjacent columns.
  • Enter a list of coefficients that will be used in the regression function, using guessed values.
  • Enter the regression function in a cell adjacent to the top row of data, using absolute addresses for the coefficients, and relative addresses for the variables.
  • In the adjacent column enter a formula returning the square of the difference between data values and the value of the regression function
  • Copy these two cells down over the full list of data.
  • Sum the “square of the difference” column
  • Use the Solver to minimise the sum of the squares by adjusting the regression function coefficients.

This process is illustrated in the spreadsheet NonLinFit-Solver.zip (click to download), which uses the Solver to fit two alternative functions to data for concrete shrinkage.

The screenshot below shows the top of the data, and the columns calculating the square of the differences for each function:

Non-linear regression using Solver (click for full size view)

The values to be minimised are in cells E23 and G23, by adjusting the values in ranges B15:B18 and D15:D20 respectively.  This is done (in two separate operations) by simply entering these ranges into the Solver dialog box, and clicking the Solve button.  The set-up for the first equation is shown in the screenshot below:

Solver dialog box

The results for the first function are seen in the screen shots below:

k1calc.1 = A*LN(T*B)-C*T-D*Th; natural scale

k1calc.1 = A*LN(T*B)-C*T-D*Th; Log scale

It can be seen that although the general trend has been captured there is still a considerable difference between the regression lines and the data points in some place.

The second function provides a much better fit:

k1calc.2 = (A+B*exp(C*Th))*T^D/(T^E + F * Th); natural scale

k1calc.2 = (A+B*exp(C*Th))*T^D/(T^E + F * Th); log scale

The good fit is to be expected, because the base data was generated with an equation of the same form.  The solver analysis has generated coefficients (A to F) that are close to those used to generate the data, but not an exact fit.  The performance of the ALGLIB functions with the same data will be examined in the next post.

Posted in AlgLib, Concrete, Excel, Maths, Newton | Tagged , , , | 6 Comments

Charts for discussion

See the comment from Georg Ströhlein under 2D Spline Interpolation with ALGLIB

Here are the charts referred to:

drgst__aliasing-1deriv-512.png

drgst__aliasing-2deriv-512

drgst__aliasing-spec-512all

drgst__aliasing-spec-512detail

drgst__aliasing-time-512

Posted in Charts, Excel, Maths, Newton | Tagged , | 1 Comment

3 year report

As in previous years, I have downloaded the statistics for this blog for the previous year, and pasted them into a worksheet.  The link to each post is preserved in the spreadsheet, so it makes a convenient index to what has been posted over the year, which can be downloaded here

Most viewed posts - click for full size view

 

Of the 2010 posts, the most popular overall was Secondary axes in Excel 2007.  The most popular in the Newton category was Lateral pile analyis with PY curves …, and the most popular in the Bach category was coffeescup a.k.a. lemonflower

From the “deserving but sadly neglected category” I have chosen:

Newton: The hole through the middle of the Earth – filled with air

Excel:  Faster Ferns

Bach: Danny Thompson & Zoe Rahman …

Most frequent referrers to this site came from:

Referrers to Newton Excel Bach

Thanks to those and all others with links to this site.

Posted in Excel | Tagged | 4 Comments

ALGLIB linear and polynomial fitting functions

As promised here: Fitting high order polynomials this post presents details of four Excel User Defined Functions (UDFs) linking with ALGLIB functions for least squares fitting of linear and polynomial functions.  The new functions are:

AL_Linest:

AL_Linest: Simple linear least squares fit

AL_LinestCW:

AL_LinestCW: Weighted and constrained linear least squares fit.

AL_PolyFit:

AL_FitPoly Simple polynomial least squares fit

and AL_PolyFitCW:

AL_FitPolyCW Weighted and constrained polynomial least squares fit.

These functions have been added to the AL-Spline-Matrix spreadsheet, which may be downloaded from:

AL-Spline-Matrix03.zip or AL-Spline-Matrix07.zip (recommended version for anyone running XL 2007 or later).

Output from the new functions is shown in the screen-shots below:

AL_Linest and AL_LinestCW Output

AL_PolyFit and AL_PolyFitCW Output

The AL_Linest function offers no obvious advantage over the built in Excel Linest function, but the other three functions have significant extra functionality:

  • AL_LinestCW allows weighted and constrained linear least squares fitting.
  • AL_Polyfit and AL_PolyfitCW are specifically written to fit a polynomial function of any order, and appear to be much more stable with high order polynomial functions than any of the built-in methods in Excel.
  • In addition AL_PolyfitCW allows constrained and weighted fitting.

The current VBA version of ALGLIB returns the polynomial data in barycentric form, rather than polynomial coefficients, and the Excel UDF converts this to interpolated function values using the ALGLIB BarycentricCalc function.  The latest version of the library (3.2) will also return the polynomial coefficients, but this version has yet to be released in VBA.  All versions of the ALGLIB library may be freely downloaded from the ALGLIB site.

Posted in AlgLib, Excel, Maths, Newton, UDFs, VBA | Tagged , , , , , , | 5 Comments

Two more letters

I can’t help feeling that I was a little unfair to Douglas Hofstadter in my last post, so as these were hypothetical letters, I will travel back in time, and send two more, enquiring whether a reductionist or a holistic approach is the better way to approach questions of Life, the Universe and Everything. Here is the text of both hypothetical letters:

Most Excellent Wordsmith,

Holism or reductionism?

Master of the (not (just) an) Excel WordPress-blog

On this occaision I suspect that Douglas Adams would have decided to answer the question in the form of a full length book, called The Trout of Indecision, but being unable to decide whether it should be yet another addition to the Hitchhiker’s Trilogy (with the theme of not being able to see the school for all the fish), or a Dirk Gently story (with the theme that there are interconnections between all things, but some interconnections are much more interesting than others), he would miss deadline after deadline, until sadly and finally being swept up by the final deadline that comes to us all.

Douglas Hofstadter on the other hand would reply promptly with a single two letter word:

clearly intended to be read three times, the first and last as a phonetic contraction of the TLA, MEW, but what is the meaning of the body of the letter?

Fortunately my imaginary infinite resolution monitor works well on hypothetical documents, and I am able to zoom in on MU, and sharpen it, revealing:

We can now see that the M is made up of three copies of the word “HOLISM”, and the U by the word “REDUCTIONISM”, and if we zoom still further, we see that:

each letter of “HOLISM” is made up of the word “REDUCTIONISM”, and that:

each letter of reductionism is made up of the word “HOLISM”.  And if we zoom still further we see that:

each letter of the smaller words is made up of multiple copies of the word:

…  and so on.

All images in this post are based on the original from Douglas Hofstadter’s “Godel, Escher Bach

Some nice links about Godel, Escher, Bach, and Godel, Escher, Bach can be found at: Godel, Escher, Bach

Posted in Newton | Tagged , , , , , , | 2 Comments