Installing PyPardiso

The PyPardiso package provides an interface to the Intel MKL Pardiso library to solve large sparse linear systems of equations. Trying to install this package with Conda raises the message:

To search for alternate channels that may provide the conda package you’re
looking for, navigate to https://anaconda.org and use the search bar at the top of the page.

This provides the link: https://anaconda.org/haasad/pypardiso

which provides the command line:

conda install -c haasad pypardiso

Ensure that the Intel MKL library is installed before installing pypardiso.

The pypardiso Github site is at:https://github.com/haasad/PyPardisoProject

This provides the following documentation:

I have now modified my py_spsolve function to use the PyPardiso solver by default:

@xl_func(disable_function_wizard_calc=True)
@xl_func(category = "Scipy-Linalg")
@xl_arg('v', 'numpy_array', ndim=1)
@xl_arg('i', 'numpy_array', ndim=1)
@xl_arg('j', 'numpy_array', ndim=1)
@xl_arg('ya', 'numpy_array', ndim=1)
@xl_return('numpy_array')
def py_spsolve(v, i, j, ya, out = 0, use_pardiso = True, permc_spec = 'MMD_AT_PLUS_A'):
    """
    Solve a matrix equation Ax = y in sparse COO format.
:param v: value vector
:param i: row index vector
:param j: column index vector
:param ya: y vector
    """
    timea = np.zeros((1,2))
    stime =time.perf_counter()
    n = ya.size
    A = ssp.coo_matrix((v,(i,j)),shape=(n,n)).tocsc()
    timea[0,0] = time.perf_counter()-stime
    if use_pardiso:
        res = spsolve(A, ya)
    else:
        res = sspla.spsolve(A, ya, permc_spec)
    timea[0,1] = time.perf_counter()-stime
    if out == 1:
        return timea
    else:
        return res

Posted in Excel, Finite Element Analysis, Frame Analysis, Link to Python, Newton, NumPy and SciPy, PyXLL, UDFs | Tagged , , , , , , | Leave a comment

The Chieftains and Paddy Moloney

The Chieftains are a traditional Irish folk band who have been performing for nearly 60 years. Paddy Moloney was one of their founding members, and died in hospital on 12th October. He has received widespread tributes, including from the President of Ireland, Michael D. Higgins:

“The Irish music community, and indeed the much larger community throughout the world who found such inspiration in his work, will have learned with great sadness today of the passing of Paddy Moloney. […] Paddy, with his extraordinary skills as an instrumentalist, notably the uilleann pipes and bodhrán, was at the forefront of the renaissance of interest in Irish music, bringing a greater appreciation of Irish music and culture internationally.”

In my opinion some of their best work was produced with other World renowned artists, of which the performance below with Sinead O’Connor is an exceptional example:

For more on the history of the Chieftains see:

Posted in Bach | Tagged , , | Leave a comment

Making Finite Element Analysis go faster – Update and PyPardiso

The previous post on this topic looked at the performance of alternative Scipy sparse equation solvers. This post updates those results with the current Scipy version (1.7.1), with very different results. It also compares the Scipy solver performance with the PyPardiso package, which provides an interface to the Intel MKL Pardiso library to solve large sparse linear systems of equations.

The sparse solver function spsolve has an optional argument called permc_spec that defines “how to permute the columns of the matrix for sparsity preservation”.  My timings for the previous post found that the argument NATURAL had very much better performance than any other alternative. This time, NATURAL performed as before, but all the other options had substantially improved, so that it had become the second slowest for the matrix used in the previous post, and the slowest for larger matrices.

The solvers have been compared for solving the stiffness matrix for 3D building frames of increasing size. The table below shows the timing results for the four spsolve options, the banded solver, two iterative solver options, and the PyPardiso solver. Each option has been timed for a system with 14742 equations (as in the previous post), 29487 equations, and 56705 equations.

Two times are reported for the Pardiso function: the time for the first calculation, and a much faster time for subsequent calculations, provided the stiffness matrix is unchanged.

  • The MMD_AT_PLUS_A option is now the fastest of the four spsolve options.
  • The Pardiso function is substantially faster than any spsolve option on the first run, especially for the larger equation systems, and of the order of 50 times faster again on subsequent runs with the same stiffness matrix.
  • The banded solver was slower than Pardiso for the two smaller systems, and failed with a memory error for the third.
  • The two iterative solvers were faster than Pardiso, especially for the largest system, but returned solutions with significant differences from the “exact” solver solutions. Also note that if the same stiffness matrix was to be used with multiple load cases then Pardiso would be very much quicker.

In summary the Pardiso package was substantially faster than any spsolve option, especially for very large systems, was also faster than the banded solver, and was able to handle larger systems than the banded solver.

In the next post I will look at procedures for installing the Pardiso package, and calling the function from Excel using pyxll.

Posted in Arrays, Excel, Finite Element Analysis, Frame Analysis, Link to Python, Newton, NumPy and SciPy, PyXLL | Tagged , , , , , , | 1 Comment

Upgrade to Python 3.10?

Python 3.10 was released on October 4th, rapidly followed by a new release of pyxll (5.3.0), but the Anaconda site does not seem to have it yet, so is it time to upgrade or not. A search on that question found:

Python 3.10 is now available–but should you switch to it immediately? And if not now, when?

The short answer is, no, you probably don’t want to switch immediately; quite possibly you can’t switch immediately. To understand why, we need to consider Python packaging, the software development process, and take a look at the history of past releases.

We can then make a guess about when Python 3.10 will actually be usable.

The problems with a new major Python release

I’m writing this article on October 5th, 2021, the day after 3.10 was released… and far too early to start using Python 3.10.

As with many open source projects, Python, Python libraries, and most of the toolchain and packages mentioned in this article are based on volunteer labor. None of this should be read as a complaint towards the people doing the maintenance, they’re doing hugely valuable work for free, and everything takes time. In addition, because upgrades involve so many different groups, coordination and releases take even more time.

With that in mind, let’s consider the problems with using 3.10 today: …

by Itamar Turner-Trauring

Full article

Posted in Excel, Link to Python, PyXLL | Tagged , , | Leave a comment

Steeleye Span Live

Two recent additions to You Tube with live performances from the early days of Steeleye Span:

This has been a fun little restoration project. My friend and music journalist Simon Jones handed me an old VHS tape recently. On it he tells me, “is the holy grail for Steeleye Span fans. A very rare recording of a TV appearance they made in 1970.” His request was simple, “could you copy the programme from the tape?” The transfer process was simple enough, if a little convoluted. I still have a couple of fully functioning JVC S-VHS decks, so that helped a whole lot! The one I plumped for is a HR-S8700EK. A S-VHS deck with a built in digital time base corrector, which I would use to stabilise the video image. The audio and video outputs from this were hooked up to a Pioneer DVR-720H DVD/Hard Disk recorder and the programme was recorded digitally. Normally I would capture the tape directly to my PC with a Terratec Firewire capture card but in this case there was some tape deterioration and the resultant picture disturbance upset the capture card and interrupted the capture. The Pioneer unit was far more tolerant to the signal and recorded it cleanly. The disk recording was burned to a DVD which was transferred to my editing PC. Here it was upscaled to 1080p, pillar-boxed and set in a 16:9 frame to look better on modern widescreen monitors and colour corrected, just a tad. The Music Room programme has the ATV ident and countdown clock at the beginning so its likely the recording came from ATV themselves, perhaps as someone’s archive copy. If anyone has details on that please let me know.

Tim Vickerstaff

Live At My Father’s Place, Roslyn, New York, 20th April 1973

00:00 One Misty Moisty Morning
04:59 The Ups And Downs
09:54 Oak Tree, Pigeon At The Gate
13:58 Sheep-Crook And Black Dog
20:18 Spotted Cow
24:41 Lady Isabelle Barnet’s Jigs
28:11 Rogues In A Nation
33:26 Cam Ye Oer Frae France
38:18 Alison Gross
43:31 Robbery With Violins
47:40 Gaudete
51:36 Royal Forester
57:05 Mason’s Apron

Posted in Bach | Tagged , , | Leave a comment