Installing Python, Scipy and Pyxll

I have recently been experimenting with Pyxll, which after a few initial hiccoughs is proving to be an easy and efficient way to connect Excel to Python based scientific and maths applications, such as linear algebra routines and ODE solvers.  An overview of what the program will do is given in the Youtube link below:

The main thing to watch out for when installing is to make sure that all the software packages required are compatible with your version of Excel, and with each other.  Note that to run Pyxll the version is controlled by the version of Excel (32 bit or 64 bit), regardless of the number of bits of your Windows version.

Finding the version information used to be easy; it was always under Help-About, but Microsoft has apparently decided that this information might frighten the uninformed, so they have decided to hide it:

  • In Excel 2010 it is under File-Help
  • In Excel 2013 it is under File-Account-About Excel

The second choice is the version of Python to install, either the latest 3.x or 2.7.  After a few false starts I ended up installing 2.7, which seems to give a better chance of compatibility with the various add-on packages.

Finally there are numerous different options for the source of the maths and science routines.  I ended up installing Anaconda (one of the packages recommended at the SciPy site); since it seemed the best option for including everything I needed, without being excessively complex.

Having downloaded and installed the Windows 32 bit, Release 2.7 version of Anaconda and Pyxll the next steps were:

  • Add the PyXll path to the system search path; using (in Windows 7) Control Panel- System and Security- System- Advanced system settings- Environment Variables- System variables, then (finally) find and Edit the “Path” variable.
  • Install pyxll as an Excel add-in using File- Options-  Add-ins, select Excel Add-ins in the Manage drop-down box, click Go…, then Browse and find the PyXll add-in file

Having done that you should get a PyXll message when you start Excel.  I then suggest looking at the Examples.xls file in the PyXll/Examples folder.  Have a look at the introduction video linked above, and the Pyxll documentation.

Posted in Excel, Link to dll, Link to Python | Tagged , , , , | 15 Comments

Arrays vs Collections vs Dictionary Objects (and Dictionary help)

A recent comment by Lori Miller at Daily Dose of Excel suggested that (for the particular application being discussed) the VBA Collection object was quicker than using scripting dictionaries.  Since I hardly ever use collections (other than the built-in ones), and recently I have made quite extensive use of dictionaries I thought I would investigate further. In the process I found a comprehensive article on the scripting dictionary, which I recommend to Microsoft to see how to write truly helpful help articles:

Using the Dictionary Class in VBA

Commenting on the differences between dictionaries and collections, this article says:

For relatively simple needs, such as identifying only the distinct items in a list, there is no advantage to using a Dictionary from a feature functionality perspective.  However, if you must:

  • Retrieve keys as well as the items associated with those keys;
  • Handle case-sensitive keys; and/or
  • Be able to accommodate changes in items and/or keys

then using a Dictionary object offers a compelling alternative to a Collection.
However, even for relatively simple needs, a Dictionary may offer a significant performance advantage, as suggested by the analysis at the end of this article.

So which is it? Are collections faster than dictionaries, or are dictionaries faster than collections?

The linked article finishes up with a downloadable benchmark spreadsheet addressing this very question, giving the following results:


suggesting a substantial speed advantage for dictionaries. But that example was using a data set of 10000 rows x 50 columns.  Reducing the number of rows to 10 (and increasing iterations from 20 to 2000) I get:

Collection vs Dictionary (small data set)

Collection vs Dictionary (small data set)

In this case the collection is more than twice as fast as the dictionary, so it seems that it all depends on the size of the data (but for a small data set it probably doesn’t matter anyway, unless you have a huge number of repetitions).

As for arrays, they don’t have the lookup speed of collections and dictionaries, but they do have a number of advantages:

  • They are easy to create in VBA with a single statement:
    (Arrayname = Range(“rangename”).Value2)
  • 2D arrays are in the format required for matrix arithmetic, and are convenient for other arithmetical operations.
  • They are easily transferable to other programming languages.

Update 3rd September 2013

Two links provided by Lori in the comments are well worth a look, including more detailed benchmarking results.

Charles Williams: VBA UDF shootout between Linear Search, Binary Search, Collection and Dictionary

Colin Legg: Count Distinct Or Unique Values – VBA UDF

I think the main conclusions are:

  • Overall the scripting dictionary has the best balance of speed and convenience.
  • For very large data sets collections can have a speed advantage
  • For the best performance with large data sets consider using a C++ xll function.
Posted in Arrays, Excel, VBA | Tagged , , , , | 7 Comments

Running VBA routines from a new workbook

I have had a couple of queries recently which seem to be resulting from trying to run User Defined Functions (UDFs) from a new workbook, so this post will look at the options for using the VBA routines provided here (or anywhere else) with new data.

VBA routines come in two basic types:

  • UDFs, which work in a similar way to the built-in Excel functions.  The function name is entered along with the required arguments, and it displays the results, either a single value or a range of values.
  • Macros, which perform a series of operations, either predefined, or with interaction from the user.  Macros are initiated either by clicking on a button (or other object), or through a custom menu or icon on  a tool bar or ribbon, or by selecting from a list (press Alt-F8, or click Macros on the Developer Ribbon).

In both cases all the VBA code defining the routines is stored with the workbook they were created in.  Once the workbook is closed, the routines are no longer available.  To use the routines with the new data there are several options:

The quickest and easiest is to simply save the original workbook with a new name, and enter (or copy and paste) in the new data.  UDFs may be used anywhere, just select the required data ranges as you would for a built-in function.  Macros often use named ranges, and data should be entered in the ranges used in the examples provided.

It is possible to use UDFs in a new workbook, providing the original file is kept open.  The screenshot below shows the UDF Cubic() being used in a macro-free workbook, by calling it from the Polynomial workbook:

Calling Cubic from the open file, Polynomial.xlsb

Calling Cubic from the open file, Polynomial.xlsb

The “Insert Function” dialog is called by clicking the fx icon immediately to the left of the edit bar, then select “User Defined” in the “select a category”.  This will list all UDFs in all open workbooks; for those not in the current active workbook the function name will be preceded by the workbook name.  After selecting the desired function, click OK, and the function arguments may be selected as usual:

UDF-Cubic2

Selecting the data range for the Cubic function

The third option is to save the workbook containing the UDF as an add-in, and install it as described here: Creating An XLA Add-In For Excel and  Installing An Add-In

After installing and opening the add-in file all the functions it contains will be listed in the Insert Function dialog, but will also appear as a tool-tip, as you type the function name:

UDF-Cubic3

Posted in Excel, UDFs, VBA | Tagged , , , | 1 Comment

Deletionists vs Inclusionists

Port Macquarie is a small Australian coastal town just up the road from me (360 km, according to Google), which has a fairly undistinguished Presbyterian Church:

PortMacQPC

This church used to have its own article in Wikipedia, but the Wikipedia deletionists decided that it was insufficiently distinguished for such an honour, and deleted it, which was recorded by the Belligerati Blog in August 2008.

James Gleick (who does have a Wikipedia entry) came to hear of this, and wrote of it in his 2011 book “The Information”, so now The Port Macquarie Presbyterian Church has the unique quality of being the only Presbyterian Church having been mentioned by a Wikipedia-worthy author as having been deleted from Wikipedia.

This would surely make the church of sufficient note to be included in Wikipedia, but then its one claim to fame as a church deleted from Wikipedia would be gone, and they would have to delete it again.

And so on …

Posted in Bach | Tagged , , , , , | Leave a comment

Lift Humour

I recently received an e-mail link to this comedy sketch set in a voice recognition lift in Scotland:

which reminded me of this Woody Allen sketch from nearly 50 years ago:

It seems that neither voice recognition technology nor humour is advancing as fast as is often claimed.

Posted in Bach | Tagged , , | 1 Comment