3D Matplotlib Plots in Excel

As well as Excel, the code shown in this post requires current versions of Python, Numpy, Matplotlib, and pyxll. The required import statements at the head of the Python code are:

import numpy as np

import matplotlib as mpl
import matplotlib.pyplot as plt
import mpl_toolkits.mplot3d.axes3d as axes3d
from matplotlib import colors

import pyxll
from pyxll import xl_func, xl_arg, xl_return, plot

The spreadsheets and Python code described below may be downloaded from Matplotlib3D.zip.

The last post on creating Matplotlib animations in Excel had examples of 3D plots which either plotted a single line, or a single surface defined by points on a regular grid. The code below is a simple example of the latter, using test data included in the Matplotlib library:

@xl_func
@xl_arg('rtn', 'int')
@xl_return("numpy_array")
def PlotWireFrame(val = 0.05, rtn = 2):
    fig = plt.figure()
    ax = fig.add_subplot(111, projection='3d')
    ax.set_title('Wireframe Plot')

    #getting test data
    X, Y, Z = axes3d.get_test_data(val)

    #drawing wireframe plot
    cb = ax.plot_wireframe(X, Y, Z, cstride=10,
                            rstride=10, color="green")

    plot(cb)
    dat = X, Y, Z
    return dat[rtn] 

This generates the graph below when called from Excel:

The Excel PlotWireFrame UDF generates the wireframe graph and also returns the data for the selected axis.

The plot_wireframe function plots a 3D surface, but for my purposes I more frequently need to plot a 3D frame with the following features:

  • The plot should allow for a large number of straight line segments with any orientation and connections.
  • It should be possible to plot different groups of elements with different colours.
  • All three axes should be plotted to the same scale.
  • It should be possible to specify the viewpoint angles and the centre and extent of the plotted image.

The code below performs this task. The input data is specified in two ranges:

  • The lines are specified as separate straight line segments with a “material” number, start node number and end node number.
  • The nodes are specified in a 3 column range with X Y and Z coordinates.

The code combines all the lines of the same material into 3 arrays with X, Y and Z coordinates. Each line is specified with the start and end coordinates, then None, so that lines that are not connected are plotted with a gap between them:

@xl_func
@xl_arg('Nodes', 'numpy_array')
@xl_arg('Beams', 'numpy_array<int>')
@xl_arg('CenXYZ', 'numpy_array', ndim=1)
@xl_arg('ViewAng', 'numpy_array', ndim=1)
@xl_arg('DisplayAx', 'bool')
@xl_arg('Xrange', 'float')
@xl_arg('LineWidth', 'float')
def Plot3D(Nodes, Beams, ViewAng = None, CenXYZ = None, Xrange = 0., DisplayAx = False, LineWidth = 1):
    fig = mpl.figure.Figure()
    ax = axes3d.Axes3D(fig)
    
    ax.set_box_aspect([1, 1, 1])
    ax.set_proj_type('persp')

    # Set viewing angle if specified, or create axes with defaults
    try:
        ax = axes3d.Axes3D(fig, azim = ViewAng[0], elev = ViewAng[1]) # roll = ViewAng[2] to be added in next release
    except:
        ax = axes3d.Axes3D(fig)
    
    # Set axis limits
    if Xrange == 0:
        Xrange = (np.max(Nodes[:,0]) - np.min(Nodes[:,0]))/2
    rng = Xrange/2
    
    try:
        X, Y, Z = CenXYZ[0:3]
    except:
        X = (np.max(Nodes[:,0]) + np.min(Nodes[:,0]))/2
        Y = (np.max(Nodes[:,1]) + np.min(Nodes[:,1]))/2 
        Z = (np.max(Nodes[:,2]) + np.min(Nodes[:,2]))/2    
    
    ax.set_xlim3d(X-rng,X+rng)
    ax.set_ylim3d(Y-rng,Y+rng)
    ax.set_zlim3d(Z-rng, Z+rng)
    
    # Read beams and coordinates
    rows = Beams.shape[0]
    mats = np.max(Beams[:,0])
    nummata = np.zeros(mats, dtype = int)
    for i in range(0, mats):
        nummata[i] = np.count_nonzero(Beams[:,0] == i+1)

    colors =['tab:blue', 'tab:orange', 'tab:green', 'tab:red', 'tab:purple', 'tab:brown', 'tab:pink', 'tab:gray', 'tab:olive', 'tab:cyan']
   
    if DisplayAx == False:
        ax.set_axis_off()

    # Create lines for each material
    for mat in range(0, mats):
        rowsm = nummata[mat]    
        rows2 = rowsm*3
        x_line = np.empty(rows2)
        y_line = np.empty(rows2)
        z_line = np.empty(rows2)
        matcol = np.mod(mat, 10)
        col = colors[matcol]
        j = 0
        for i in range(0, rows):
            if Beams[i, 0] == mat+1:
                n1 = Beams[i,1]-1
                n2 = Beams[i, 2]-1
                x_line[j] = Nodes[n1,0]
                x_line[j+1] = Nodes[n2,0]
                x_line[j+2] = None
                y_line[j] = Nodes[n1,1]
                y_line[j+1] = Nodes[n2,1]
                y_line[j+2] = None
                z_line[j] = Nodes[n1,2]
                z_line[j+1] = Nodes[n2,2]
                z_line[j+2] = None
                j = j+3
        
        ax.plot3D(x_line, y_line, z_line, col, linewidth=LineWidth)
    plot(ax)

The code was checked by plotting three circles centred at the origin and with radius 10, in the XY, XZ, and YZ planes:

The default plot looks along the X axis with the Y axis to the right, and the Z axis vertical. In the next plot the view point is rotated 45 degrees about the Z axis (azimuth), with a vertical deflection of 30 degrees:

The next example shows a much more complex plot; a 3D image of the Sydney Harbour Bridge with the axes display turned off and the line width reduced to 1:

The data for this image was taken from a Strand7 file, available from the Strand7 web site to licenced Strand7 users. The top of the data range is shown below. In all there are 11,473 beams and 7737 nodes.

The next example shows the same data with a different view angle and centre coordinates:

Finally the same data viewed in the Strand7 FEA program, showing a very similar image to the Matplotlib/Excel plot with the same view angles:

The current Matplotlib code does not allow for rotation about the line of sight (“roll”), but this feature is under development and is expected to be included in the next release.

Posted in Charts, Charts, Coordinate Geometry, Drawing, Excel, Link to Python, Newton, NumPy and SciPy, PyXLL, UDFs | Tagged , , , , , , | 2 Comments

Free and Simple Tools for Editing Videos

I recently needed to extract 15 minutes from a 45 minute long video, in 5 separate clips. I discovered that there are free tools to do this simply and efficiently, although they are not widely advertised. Here’s how:

By default, videos on my computer open with the Windows 10 “Film & TV” app, which has an icon in the bottom right corner to “Edit with Photos”. Clicking that lists several options, including Trim:

Alternatively the video can be opened directly in Photos, which has an Edit-Trim icon at top right:

Either option opens the video in the Photos Editor, where the section of the video to be trimmed can be selected by moving the start and end sliders at the bottom of the window. Then click Save-as at the top right:

This process is repeated for each clip by re-opening the original video, in my case producing 5 video clips. These can be simply combined into a single video with the Adobe online Merge Videos page:

https://www.adobe.com/express/feature/video/merge

Posted in Bach, Computing - general, Newton | Tagged , , , | 2 Comments

Python functools – cache and lru_cache

Update 3rd April 2022: Following the comment from Larry Schuster I have modified the code so that the factorial0 function actually uses cache as intended, rather than lru_cache. I also added code to clear the cache after each timer run, because the uncleared cache was giving misleading results, and added a timer function to call each of the four factorial routines from within Python, rather than calling them as separate UDFs from Excel. The example timer results have all been updated for the new code.

Functools is a built in Python module “for higher-order functions: functions that act on or return other functions”. Of the collection of functions, the two that seemed the most obviously useful for my purposes were cache and lru_cach:

@functools.cache(user_function)
Simple lightweight unbounded function cache. Sometimes called “memoize”.
Returns the same as lru_cache(maxsize=None), creating a thin wrapper around a dictionary lookup for the function arguments. Because it never needs to evict old values, this is smaller and faster than lru_cache() with a size limit.

@functools.lru_cache(user_function)
@functools.lru_cache(maxsize=128typed=False)
Decorator to wrap a function with a memoizing callable that saves up to the maxsize most recent calls. It can save time when an expensive or I/O bound function is periodically called with the same arguments.

As a simple example, I have used a recursive function to find the factorial of the passed argument, with alternative versions with the cache and lru_cache decorators, and also the Numba just in time compiler decorator:

def factorial(n):
    return 1 if n <= 1 else n * factorial(n - 1)

@cache
def factorial0(n):
    return 1 if n <= 1 else n * factorial0(n - 1)

@lru_cache()
def factorial1(n):
    return 1 if n <= 1 else n * factorial1(n - 1)

@njit(cache = True)
def factorial2(n):
    return 1 if n <= 1 else n * factorial2(n - 1)

These functions have been called from Excel using pyxll, via timer functions allowing a specified number of repetitions, and returning the factorial result and the execution time for each function. One timer function called a specified factorial function, and was entered separately in Excel for each of the factorial functions. The other called all four factorial functions from within Python.

For 1 repetition with a low input number the cache and Numba functions were all slower than plain Python when called as UDF’s and slightly faster when called from Python:

Increasing the input value to 100 made the cache functions relatively slower, since the cache was updated at each step of the factorial calculation, but never actually used. The Numba function was faster than plain Python, especially when called from within Python, with the speed improvement being due to compilation of the code, rather than use of the cache.

With 10,000 repetitions and input value of 100 the cache functions are over 100 times faster than plain Python, with little difference in the time when called from Excel or Python. The Numba function was only 26 to 27 times faster than plain Python:

In summary the cache functions were substantially faster than plain Python for cases with a high number of repetitions of the factorial function, and for this example were about 4 times faster than the Numba function for 10,000 iterations. When the function was only called once the cache functions were significantly slower.

Examples with more practical applications will be examined in later posts.

Posted in Excel, Link to Python, PyXLL, UDFs | Tagged , , , , , , , | 3 Comments

More Matplotlib problems

Following an upgrade to some Python libraries, I found that problems with plotting from Matplotlib causing Python to crash were recurring (previously reported here). When called from Excel with pyxll this was also causing instant crashes of Excel.

I eventually found the problem was with the latest version of Freetype, and after downgrading to release 2.10.4, and then upgrading Scipy to the latest version, everything went back to working normally again.

The procedure to downgrade to a specific package using Conda is:

conda install freetype==2.10.4
Posted in Excel, Link to Python, PyXLL | Tagged , , , , , | 3 Comments

Transatlantic Sessions and Sarah Jarosz

Transatlantic Sessions (Wikipedia tells me) is the collective title for a series of musical productions by Glasgow-based Pelicula Films Ltd, funded by- and produced for BBC Scotland, BBC Four and RTÉ of Ireland. The productions comprise collaborative live performances by various leading folk, bluegrass and country musicians from both sides of the North Atlantic, playing music from Scotland, Ireland, England and North America, who congregate under the musical direction of Aly Bain and Jerry Douglas to record and film a set of half-hour TV episodes. Wikipedia

Here are some samples featuring Jerry Douglas and Sarah Jarosz (accompanied by Danny Thompson on double bass), followed by a rockier (not Transatlantic) session from Sarah Jarosz:

… and an earlier session with John Martyn and Kathy Mattea plus Jerry Douglas and Danny Thompson:

Posted in Bach | Tagged , , , , , | Leave a comment