Python functools – cache and lru_cache

Update 3rd April 2022: Following the comment from Larry Schuster I have modified the code so that the factorial0 function actually uses cache as intended, rather than lru_cache. I also added code to clear the cache after each timer run, because the uncleared cache was giving misleading results, and added a timer function to call each of the four factorial routines from within Python, rather than calling them as separate UDFs from Excel. The example timer results have all been updated for the new code.

Functools is a built in Python module “for higher-order functions: functions that act on or return other functions”. Of the collection of functions, the two that seemed the most obviously useful for my purposes were cache and lru_cach:

Simple lightweight unbounded function cache. Sometimes called “memoize”.
Returns the same as lru_cache(maxsize=None), creating a thin wrapper around a dictionary lookup for the function arguments. Because it never needs to evict old values, this is smaller and faster than lru_cache() with a size limit.

Decorator to wrap a function with a memoizing callable that saves up to the maxsize most recent calls. It can save time when an expensive or I/O bound function is periodically called with the same arguments.

As a simple example, I have used a recursive function to find the factorial of the passed argument, with alternative versions with the cache and lru_cache decorators, and also the Numba just in time compiler decorator:

def factorial(n):
    return 1 if n <= 1 else n * factorial(n - 1)

def factorial0(n):
    return 1 if n <= 1 else n * factorial0(n - 1)

def factorial1(n):
    return 1 if n <= 1 else n * factorial1(n - 1)

@njit(cache = True)
def factorial2(n):
    return 1 if n <= 1 else n * factorial2(n - 1)

These functions have been called from Excel using pyxll, via timer functions allowing a specified number of repetitions, and returning the factorial result and the execution time for each function. One timer function called a specified factorial function, and was entered separately in Excel for each of the factorial functions. The other called all four factorial functions from within Python.

For 1 repetition with a low input number the cache and Numba functions were all slower than plain Python when called as UDF’s and slightly faster when called from Python:

Increasing the input value to 100 made the cache functions relatively slower, since the cache was updated at each step of the factorial calculation, but never actually used. The Numba function was faster than plain Python, especially when called from within Python, with the speed improvement being due to compilation of the code, rather than use of the cache.

With 10,000 repetitions and input value of 100 the cache functions are over 100 times faster than plain Python, with little difference in the time when called from Excel or Python. The Numba function was only 26 to 27 times faster than plain Python:

In summary the cache functions were substantially faster than plain Python for cases with a high number of repetitions of the factorial function, and for this example were about 4 times faster than the Numba function for 10,000 iterations. When the function was only called once the cache functions were significantly slower.

Examples with more practical applications will be examined in later posts.

Posted in Excel, Link to Python, PyXLL, UDFs | Tagged , , , , , , , | 2 Comments

More Matplotlib problems

Following an upgrade to some Python libraries, I found that problems with plotting from Matplotlib causing Python to crash were recurring (previously reported here). When called from Excel with pyxll this was also causing instant crashes of Excel.

I eventually found the problem was with the latest version of Freetype, and after downgrading to release 2.10.4, and then upgrading Scipy to the latest version, everything went back to working normally again.

The procedure to downgrade to a specific package using Conda is:

conda install freetype==2.10.4
Posted in Excel, Link to Python, PyXLL | Tagged , , , , , | 3 Comments

Transatlantic Sessions and Sarah Jarosz

Transatlantic Sessions (Wikipedia tells me) is the collective title for a series of musical productions by Glasgow-based Pelicula Films Ltd, funded by- and produced for BBC Scotland, BBC Four and RTÉ of Ireland. The productions comprise collaborative live performances by various leading folk, bluegrass and country musicians from both sides of the North Atlantic, playing music from Scotland, Ireland, England and North America, who congregate under the musical direction of Aly Bain and Jerry Douglas to record and film a set of half-hour TV episodes. Wikipedia

Here are some samples featuring Jerry Douglas and Sarah Jarosz (accompanied by Danny Thompson on double bass), followed by a rockier (not Transatlantic) session from Sarah Jarosz:

… and an earlier session with John Martyn and Kathy Mattea plus Jerry Douglas and Danny Thompson:

Posted in Bach | Tagged , , , , , | Leave a comment

VBA procedures for the Numerical Analysis of Tabular Functions

Long time on-line friend Alfred Vachris has recently been converting Fortran code developed through his working career into VBA:

I had the opportunity to work at Grumman while going to Graduate School and they gave me a part-time job as a programmer. The first day on the job, I was handed a copy of A guide to FORTRAN programming by Daniel McCracken. It was love at first sight. I was a Mathematics major from Notre Dame, but all my courses were focused on theoretical mathematics rather than a more engineering focused piratical mathematics. I switched over to Aeronautical Engineering during my senior year. And then followed up by going to Graduate School for a degree in Astronautics.

In the early 1960’s I discovered an article in Design News, a trade magazine, that featured a short article on the Aitken Graphical Construction for generating additional points on a Parabola. This article became the foundation of my efforts to then put together my Tool Kit of codes for Tabular functions.

I am re-factoring the original FORTRAN code to Excel VBA.  I was first posting on WordPress:
But I find it much easier to post PDF documents on LinkedIn.

Here is a list of LinkedIn posts related to the Toolkit:

Aitken Demonstration

Aitken Variation – Hermitian Interpolation

Arc Length

Non-Linear Interpolation

Graphical Twist – Alternate Brent Root Finding

Root Finding Examples – Polynomials

Slope: Numerical Differentiation of Tabular Functions

Tangent Estimates

For the benefit of those (like me) not familiar with how Linkedin displays pdf documents, to display the full text and make links active:

Scroll to the bottom of the displayed page and click the “full-screen” icon at the bottom right:

At the top of the page click on “Accessibility Mode”, or the download icon:

Posted in Coordinate Geometry, Curve fitting, Excel, Fortran, Maths, Newton, UDFs, VBA | Tagged , , , , , , | 1 Comment

Larkin Poe

Larkin Poe is an American roots rock band originally from north Georgia, currently based in Nashville, Tennessee,[1] and fronted by sisters Rebecca Lovell (born January 30, 1991) and Megan Lovell (born May 12, 1989). Featuring strong southern harmonies, heavy electric guitar riffs, and slide guitar, they are often touted as “the little sisters of the Allman Brothers“.[2] The band performed at the 2014 and 2016 Glastonbury Festival and were voted “Best Discovery of Glastonbury 2014” by the UK’s The Observer.

They are great musicians, performing a great variety of musical styles. Here are a few samples.

Posted in Bach | Tagged | Leave a comment