PdfLink.zip (free download with open source code)

See the link above for more details of the spreadsheet, including the link to pdf function. Below is some more information on using the search function.

Entering text in the Search Text cell, a list of words separated by spaces will find all the files that contain all the listed words.

A list of phrases enclosed in quotes and separated by spaces will find all the files that contain all the listed exact phrases. On my computer the list below shows the search terms and the number of files found:

crack 3360

crack width 2127

crack width slabs 1083

“crack width” 750

“crack width” “slabs” 475

“crack width” “paving slabs” 2

All the searches worked almost instantaneously.

]]>The Numba docs include a series of short code examples illustrating the main options. I have re-run these with pyxll based interface functions, so the routines can be called from Excel as a user defined function (UDF), and return the execution time for any specified number of iterations.

Typical code for timing a function is:

```
@xl_func
def time_ident_npj(n):
x = np.arange(n)
stime = time.perf_counter()
y = ident_npj(x)
return time.perf_counter()-stime
```

Note that the calls to the time function must be outside the function being timed, since Numba does not support the time function.

The fist example from the Numpy docs compared evaluation of an array function using Numpy arrays and Python loops, with or without Numba:

@njitdefident_np(x):returnnp.cos(x) ** 2 + np.sin(x) ** 2@njitdefident_loops(x): r = np.empty_like(x) n = len(x)foriinrange(n): r[i] = np.cos(x[i]) ** 2 + np.sin(x[i]) ** 2returnr

Results from these functions are shown below, with times as reported in the Numba article, and as found with my code:

Using Numpy arrays, the Numba function was only slightly faster for me, and was slightly slower as shown in the Numba article. This is not surprising since the Python code had only a single call to the Numpy function, which is already C compiled code, so there is little scope for improving performance.

The function using Python loops was very much slower, and my results were slightly slower than the time in the Numba article. Presumably this is related to using different versions of Python. Adding the Numba decorator reduced the execution time for my code by a factor of 170, and the result was slightly faster than the Numpy function with Numba decorator.

The next examples looks at the effect of the Numba “fastmath = True” and “parallel – True” options:

@njit(fastmath=False)defdo_sum(A): acc = 0.# without fastmath, this loop must accumulate in strict orderforxinA: acc += np.sqrt(x)returnacc@njit(fastmath=True)defdo_sum_fast(A): acc = 0.# with fastmath, the reduction can be vectorized as floating point# reassociation is permitted.forxinA: acc += np.sqrt(x)returnacc

@njit(parallel=True)defdo_sum_parallel(A):# each thread can accumulate its own partial sum, and then a cross# thread reduction is performed to obtain the result to returnn = len(A) acc = 0.foriinprange(n): acc += np.sqrt(A[i])returnacc@njit(parallel=True, fastmath=True)defdo_sum_parallel_fast(A): n = len(A) acc = 0.foriinprange(n): acc += np.sqrt(A[i])returnacc

Results for these functions were:

For this case the Numba compiled code was over 300 times faster than plain Python. The “fastmath = True” option was only of limited benefit in my case, although the Numba article results show a speed up of more than two times. Setting “parallel = True” increased performance by more than 10 times, with “fastmath = True ” again only providing a small further gain. With both options applied, the Numba compiled code was almost 4000 times faster than the plain Python for this case.

This raises the question as to why with more complex code the speed gain from using Numba is often much smaller. This will be examined in more detail in a later post, but the main reason is that if Numba is set to revert to Python mode if there is code it cannot compile (nopython = False), then the resulting code can easily be almost all Python based. The same effect is found using the alternative @jit or @njit decorators. The @njit decorator result in all the Python code being compiled, and will raise an error if any of the code cannot be compiled by Numba. The alternative @jit decorator will switch to Python mode if any code cannot be compiled, but with much reduced (if any) speed improvement. Examples from my own code that will raise an error with @njit, or will not be fully compiled with @jit include:

- Use of the time function
- Checking the data type of a variable, such as: if type(x) == tuple: …
- Statements such as “StartSlope = EndSlope”, where EndSlope has not yet been defined at compile time.

Concrete 2021 (the 30th Biennial Conference of the Concrete Institute of Australia) is just around the corner, and this year will be all on-line:

The virtual conference format not only provides greatly reduced travel and accommodation costs, but also all presentations will be downloadable on demand for 30days after the conference. Early-bird registrations are available to midnight Saturday 31st July, East-Australia time (2:00 pm GMT), so click the link above for more information and to secure low-cost tickets for the four day conference.

]]>Copy Chart to New Sheet and Link to Data on New Sheet at Jon Peltier’s blog

The procedure is:

- Right click on the tab of the worksheet to be copied
- Select “Move or Copy …” then in the dialog box select “New book” under “To book”, and select the “Create a copy” check box
- Click OK
- The worksheet will be copied to a new file (called book1.xlsx), including any charts, with the charts linking to the data in the new file.
- Save the new file with a new name, remembering to save as .xlsb or .xlsm if you want to add any VBA code.

Note that the worksheet may also be copied to an existing file, or to a new position in its current file. Also it is possible to copy more than 1 worksheet by selecting the sheets to be copied before right-clicking on one of the selected tabs.

Copying the worksheet:

The resulting new workbook, with chart linked to the data in the new file:

]]>As a check that the functions were working correctly, the Python functions were modified to return the sum of the largest array in the first row, revealing that the Numpy code was returning the wrong results! …

It seems that the Numpy arange function uses 32 bit integers, even if the range extends outside the 32 bit range!

That’s not quite right. In fact the range of Numpy 32 bit integers is -2,147,483,648 to 2,147,483,647 (which is the same as VBA Longs), and the largest value generated by the quoted arange function was only 100,000. The code generating an incorrect result (without generating an error message) was:

```
@xl_func
def numpysumn(k):
a = np.arange(k)
return a.sum()
```

With a k value of 100,000 the Numpy arange function generates a sequence from 1 to 99,000, so there is no problem generating the array, but the sum of the array members is 4,995,000,000, which exceeds the 32 bit integer limit.

Alternative solutions to this problem are:

- Declare the arange function as a 64 bit integer:

a = np.arange(k, dtype=np.int64) - Declare k as a 64 bit integer:

k = np.int64(k)

In both cases the array a will have a 64 bit integer datatype, and a.sum will return the correct result.

]]>Starting with a string consisting of numbers with a single central group of letters, how can this string be truncated at the end of the letters, so that for instance 3L481 becomes 3L?

This prompted a lengthy discussion, with many twists and turns, and even some useful answers to the original question.

The original question was looking for an on-sheet solution, rather than VBA, and the first working formula is shown below (all 420 characters of it):

A much more practical alternative (in my opinion) is a VBA user defined function (UDF), which requires just the one argument of the cell address. Two examples are shown below:

```
Public Function LeftToString(InpStr As String) As String
'
' Removes trailing digits from a string that comprises
' a mixture of UPPERCASE letters and digits.
'
' Elaborations might be required for strings that contain:
' Lower case letters
' Only letters
' Characters that are neither digits not letters.
'
Dim L As Long 'Length of input string
Dim i As Long 'General purpose integer
L = Len(InpStr)
If L <= 0 Then
LeftToString = ""
Exit Function
End If
'
' Loop backwards from the input string's end until hit an uppercase letter.
'
For i = L To 1 Step -1
If UCase(Mid(InpStr, i, 1)) >= "A" And UCase(Mid(InpStr, i, 1)) <= "Z" Then
LeftToString = Left(InpStr, i)
Exit Function
End If
Next i
'
' Input string contains no letters.
'
LeftToString = InpStr
End Function
```

And a shorter version:

```
Function LeftToString2(X As String) As String
Dim i As Long
For i = Len(X) To 1 Step -1
If (InStr("0123456789", Mid(X, i, 1))) = 0 Then
LeftToString2 = Left(X, i)
Exit Function
End If
Next i
End Function
```

Results of the two UDFs are shown in the screen-shot above. Note that the first UDF checks for upper-case text, between A and Z, so the extracted character must be converted to upper case. The second avoids the problem by checking if the character is a number.

A much shorter on-sheet formula was then supplied:

Even with the shorter formula, it is not immediately obvious how it works, so I have split it up into its constituent parts. With the input string in Cell B30:

- =RIGHT(B30,ROW(INDIRECT(“1:”&LEN(B30)))) returns an array of progressively longer strings, starting from the right hand end.
- =VALUE() converts each member of that array either into a a number or #VALUE!.
- =ISNUMBER() returns TRUE or FALSE for each of those values.
- =SUM(–array) or =SUM(array*1) returns the number of TRUE values. The — or *1 operators are required for the SUM function to treat TRUE as a value of 1, otherwise the SUM will always return 0
- Finally =LEFT(B30, LEN(B30)-K30) extracts the string up to the last text character.

In the latest version of Excel with “dynamic arrays” the second function will work when entered with the enter key, as usual. In older versions it must be entered as an array function, by pressing Ctrl-Shift-Enter.

Even with the second simpler formula, to my mind the VBA function is the better alternative, both in terms of application in a new spreadsheet, and understanding how the thing works. For those not familiar with VBA, the process of creating a new function is quite simple:

- Open a new spreadsheet (or an existing one you want to use the function in) and save with a chosen name.
- Press Alt-F11 to open the VBA editor.
- Right-click on VBAProject(spreadsheet name) in the list of open files on the left, and Insert-Module (see screenshot below)
- Copy the VBA code and paste it in the new module.

Finally I recommend having a look at the full EngTips thread linked above for discussion on a variety of topics, including whether to declare integer variables as Integer or as Long, how the first long formula works, the quick way to review long formulas, without splitting them up into parts, and VBA code for highlighting the active cell.

]]>The changes in the new version are:

- Stress block type 3 will now use the Eurocode full parabolic stress block.
- In previous versions entering a concrete yield stress of 600 MPa resulted in a divide by zero error. This has now been fixed.
- Using the AS 3600 rectangular stress block, there has been a minor correction to the calculation of section capacity when the neutral axis is at or beyond the “tension” face, so that linear interpolation is used, as required by the code.

The Eurocode parabolic stress block may be used with AS 3600 and AS 5100, as well as Eurocode 2:

Interaction diagrams to AS 3600 are shown below for the rectangular stress block (including 10% reduction in concrete stress for a non-rectangular section, the Eurocode parabolic-rectangular stress block, and the full parabolic stress block:

]]>A young band from Coonamble aged 11-13, they’ve been performing together for over 4 years

11 July 21: For some reason the Facebook video link isn’t working any more, so here is another one from Youtube:

]]>As promised in the previous post in this series, I will be looking at speeding up the formation and solving of matrix functions for the analysis of structural frames, using Excel and pyxll to call Python routines with the Numba jit compiler, and Scipy sparse matrices and solvers.

For an example I have used the 3D analysis of a concrete building frame with 7065 beams and 2730 nodes. This has 14,742 degrees of freedom, which if analysed using the complete stiffness matrix would require creating and solving a matrix with 217,326,564 elements. The actual matrix has only 176,904 non-zero elements, so the use of a matrix in sparse format saves an enormous amount of memory resources and computing time. The use of alternative Scipy solvers, and options within each solver, also made a huge difference to the solution time.

The table below shows times for reading the data from Excel and setting up the stiffness matrix:

The time saving from using Numba is fairly modest (presumably because most of the work involves manipulating data within Numpy arrays), nonetheless the creation of the final stiffness array shows a useful time saving when using Numba.

I have used two matrix solver routines from the Scipy linalg and sparse.linalg libraries, and each of the 9 alternative iterative sparse solvers:

- solveh_banded: Solve equation a x = b. a is Hermitian positive-definite banded matrix.
- spsolve: Solve the sparse linear system Ax=b, where b may be a vector or a matrix.
- Iterative solvers:
- bicg: Use BIConjugate Gradient iteration to solve Ax = b.
- bicgstab: Use BIConjugate Gradient STABilized iteration to solve Ax = b.
- cg: Use Conjugate Gradient iteration to solve Ax = b.
- cgs: Use Conjugate Gradient Squared iteration to solve Ax = b.
- gmres: Use Generalized Minimal RESidual iteration to solve Ax = b.
- lgmres: Solve a matrix equation using the LGMRES algorithm
- minres: Use MINimum RESidual iteration to solve Ax=b
- qmr: Use Quasi-Minimal Residual iteration to solve Ax = b.
- gcrotmk: Solve a matrix equation using flexible GCROT(m,k) algorithm.

For all cases the initial “A” matrix (i.e. the square stiffness matrix) is prepared in COO format, which allows the matrix to be compiled with the minimum of calculation. The COO format consists of 3 columns listing each non-zero value of the array, and the row and column number of each value.

The solveh_banded function requires the A array to be a Hermitian banded array. Scipy does not seem to have a function to convert from COO to banded, so I wrote one:

```
def COO2BandedVect(kv, ki, kj):
"""
Convert a COO matrix to symmetric banded upper triangle format.
kv ki kj: COO matrix vectors
"""
rows = kv.shape[0]
n1 = 0
cols = 0
for k in range(0, rows):
i = int(ki[k])
j = int(kj[k])
if i > cols: cols = i
if (i-j) > n1:
n1= i-j
cols = cols+1
bandwidth = n1+1
BandMat= np.zeros((bandwidth, cols))
for k in range(0, rows):
i = int(ki[k])
j = int(kj[k])
if i - j <= 0: BandMat[ i + (n1-j), j] = BandMat[i + (n1-j), j] + kv[k]
return BandMat
```

For the other functions the three vectors, v, i and j, must be converted to a COO matrix then to csc or csr format, which may de done with a single line of code:

```
A = ssp.coo_matrix((v,(i,j)),shape=(n,n)).tocsc()
```

The solveh_banded function performed much better than the other analytic solver (spsolve) with default arguments, with solution times of 0.70 and 18.15 seconds respectively. Spsolve has an optional argument called permc_spec that defines “how to permute the columns of the matrix for sparsity preservation”. The options with solution times are shown below:

For this case the “Natural” option is thus almost 6 times faster than the default “Colamd” option, and 11 times faster than the slowest option! The solveh_banded function, with default arguments, however remains substantially faster.

The 9 iterative solvers also had a huge range of performance, with solution times ranging from 0.5 seconds to 69 seconds. The gmres solver was by far the slowest, and gcrotmk was also very slow. The cgs solver returned an error. The performance of the other 6 is shown in the graphs below, with the first plotting solution time against the input tolerance value (log scale), and the second solution time against maximum error, compared with the results from the analytic solvers.

The minres solver was substantially faster than the others, but also generated greater errors with the default tolerance. The errors from the other solvers were all close to the minumum when run with the default tolerance of 1E-08. To reduce the minres error to this level it was necessary to reduce the specified tolerance to 1E-11. With this reduced tolerance minres remained the fastest of the iterative solvers, with a solution time of 0.75 seconds.

In Summary:

- The banded solver (solveh_banded) was the fastest overall for high precision results, when run with default input.
- The spsolve function performed much better with the permc_spec argument set to “NATURAL”, rather than the default, but was still substantially slower than solveh_banded.
- The minres function was the fastest of the iterative solvers, but required the tolerance value to be set to 1E-11 to minimise the maximum error in the results.
- All results are based on solving one large, very sparse matrix. Different problems are likely to have different functions giving the best results.