Python Traps

When converting existing VBA code to Python there are a number of obvious changes to the syntax that need to be made for the code to run at all, but there are also a number of not so obvious traps that may result in the code running, but giving incorrect results.  Three that regularly catch me unawares are illustrated in the screen shots below:

The first is the value of the counter at the completion of a for loop.  In VBA the counter runs from the specified start to stop values inclusive, then after the final loop the counter is incremented again.  In Python the counter runs until the last value lower than the specified stop value, and is not incremented after the last step.  The result is that the examples below generate the same number of steps (n), and the same value in the calculated quantity (j), but the final counter value (i) is one higher in the VBA code than in the Python:

 

Differences in application of the = operator when applied to arrays have been discussed previously (The meaning of = in Python).  Similar differences occur when variables are passed to other functions, with results that can be confusing and unintuitive (at least to me).  In the examples below:

  • A variable, x, is passes to a function.
  • A local variable y is assigned the value of x, then multiplied by n (sometimes in one operation).
  • x is multiplied by n.
  • y is returned to the calling function, which then returns x and y to the spreadsheet.

The returned results depend on the variable type, how the multiplication is carried out, and the language.  In the examples below the variable x is passed in Python first as a float then as a Numpy array, and in both cases the multiplication is carried out with x = x*n for x and y = x*n for y.  In VBA the multiplication was carried out with a for loop.

  • In the Python code y returns the multiplied values, but the operations on x are treated as local, and the values in the calling function remain unchanged, both for the single value x and the Numpy array.
  • in the VBA code in both cases the x variable is passed by reference, so any operations on x are reflected in the calling routine:

In the next 3 examples x is passed as a Numpy array in Python, and y is created with y = x*n.

  • The modified value of y is always returned to the calling routine.
  • When x is multiplied by directly modifying the value of the array elements the modified values are reflected in the calling routine, but using for a in x: creates multiplied values of a, but does not modify the values in the x array, so the values in the calling routine are also unchanged.
  • In VBA, options 3 and 5 are essentially the same as Python option 5, and return the same results.  VBA Option 4 uses a for each loop for both x and y, which does not modify either array, locally or in the calling routine.

In the final 3 Python examples x is passed as a list and y is set equal to x.

  • Where the multiplications are applied to the individual elements of either x or y, both arrays are modified, and the modified values are transferred to the calling array.
  • Where a for a in y loop is used neither array is modified, either locally or in the calling routine.

in the VBA examples:

  • In Option 6 y is created as a Range object and x is then converted to a variant array.  The multiplication of x is reflected in the calling routine, but y is unchanged.
  • Use of “for each a in y” in Option 7 does not change the values in either x or y.
  • In Option 8 y is set equal to x, then converted to a variant array and multiplied by n.  The changes in y are returned to the calling array, but x is unchanged.

The code for all the routines is listed below:

Python Code:

@xl_func
@xl_arg('start', 'int')
@xl_arg('stop', 'int')
def CountLoop(start, stop):
    n = 0
    j = start
    for i in range(start, stop):
        n = n+1
        j = j + 1
    return [[i, j, n]]

@xl_func
@xl_arg('val1')
@xl_arg('val2')
@xl_arg('rng1', 'numpy_array', ndim=1)
@xl_arg('rng2', 'float[]')
@xl_arg('out', 'int')
#@xl_return('numpy_array')
def PassArgs(val1, val2, rng1, rng2, out):
    if out == 1:
        rtn = mult1(val1, val2)
        return [rtn, val1]
    elif out == 2:
        rtn = mult1(rng1, val2)
        return [rtn, rng1]
    elif out == 3:
        rtn = mult2(rng1, val2)
        return [rtn, rng1]
    elif out == 4:
        rtn = mult3(rng1, val2)
        return [rtn, rng1]
    elif out == 5:
        rtn = mult4(rng1, val2)
        return [rtn, rng1]
    elif out == 6:
        rtn = mult5(rng2, val2)
        return [rtn, rng2]
    elif out == 7:
        rtn = mult6(rng2, val2)
        return [rtn, rng2]
    else:
        rtn = mult7(rng2, val2)
        return [rtn, rng2]

def mult1(x, n):
    y = x*n
    x = x*n
    return y

def mult2(x, n):
    y = x*n
    x[:] = x[:]*n
    return y

def mult3(x, n):
    y = x*n
    for a in x:
        a = a*n
    return y

def mult4(x, n):
    y = x*n
    for i in range(0, 2):
        x[i] = x[i]*n
    return y

def mult5(x, n):
    y = x
    for i in range(0, 2):
        x[i] = x[i]*n
    return y

def mult6(x, n):
    y = x
    for a in y:
        a = a*n
    return y

def mult7(x, n):
    y = x
    for i in range(0, 2):
        y[i] = y[i]*n
    return y

VBA Code:

Function vbCountLoop(start, last)

    n = 0
    j = start
    For i = start To last
        n = n + 1
        j = j + 1
    Next i
    vbCountLoop = Array(i, j, n)

End Function

Function vbPassArgs(val1, val2, rng1, rng2, out)
    If out = 1 Then
        rtn = mult1(val1, val2)
        vbPassArgs = WorksheetFunction.Transpose(Array(rtn, val1))
    ElseIf out = 2 Then
        rtn = mult1a(rng1, val2)
        vbPassArgs = (Array(rtn, rng1))
    ElseIf out = 3 Then
        rtn = mult2(rng1, val2)
        vbPassArgs = (Array(rtn, rng1))
    ElseIf out = 4 Then
        rtn = mult3(rng1, val2)
        vbPassArgs = (Array(rtn, rng1))
    ElseIf out = 5 Then
        rtn = mult4(rng1, val2)
        vbPassArgs = (Array(rtn, rng1))
    ElseIf out = 6 Then
        rtn = mult5(rng2, val2)
        vbPassArgs = (Array(rtn, rng2))
    ElseIf out = 7 Then
        rtn = mult6(rng2, val2)
        vbPassArgs = (Array(rtn, rng2))
    Else
        rtn = mult7(rng2, val2)
        vbPassArgs = (Array(rtn, rng2))
    End If

End Function


Function mult1(x, n)
    y = x * n
    x = x * n
    mult1 = y
    End Function


Function mult1a(x, n)
    x = x.Value2
    y = x
    For i = 1 To 2
    y(1, i) = x(1, i) * n
    x(1, i) = x(1, i) * n
    Next i
    mult1a = y
    End Function


Function mult2(x, n)
    x = x.Value2
    y = x
    For i = 1 To 2
    y(1, i) = x(1, i) * n
    x(1, i) = x(1, i) * n
    Next i
    mult2 = y
    End Function
    

Function mult3(x, n)
    x = x.Value2
    y = x
    For Each a In y
        a = a * n
    Next a
    For Each a In x
        a = a * n
    Next a
    mult3 = y
    End Function
    
Function mult4(x, n)
    x = x.Value2
    y = x
    For i = 1 To 2
        y(1, i) = y(1, i) * n
        x(1, i) = x(1, i) * n
    Next i
    mult4 = y
    End Function

Function mult5(x, n)
    Set y = x
    x = x.Value2
    For i = 1 To 2
        x(1, i) = x(1, i) * n
    Next i
    mult5 = y
End Function

Function mult6(x, n)
    Set y = x
    For Each a In y
        a = a * n
    Next a
    mult6 = y.Value2
End Function

Function mult7(x, n)
    Set y = x
    y = y.Value2
    For i = 1 To 2
        y(1, i) = y(1, i) * n
    Next i
    mult7 = y
End Function
Posted in Arrays, Excel, Link to Python, Newton, NumPy and SciPy, PyXLL, UDFs, VBA | Tagged , , , , , , , , | 1 Comment

Excel Dynamic Arrays

Previews of dynamic arrays in Excel have been available for some time through the “Insider” programme.  They are now being published to general subscribers through Office 365 monthly updates:

Further information is given at:

Easier array formulas
and
Dynamic array formulas vs. legacy CSE array formulas

Most of my spreadsheets make extensive use of user defined functions (UDFs) that often return arrays of data, that up till now needed to be entered by selecting the output range and pressing Ctrl-Shift-Enter.  These “legacy” CSE functions continue to work as they did before, but if you want to change to using the new dynamic arrays the procedure is:

Select the entire output range of the CSE array.  Note that it displays in the edit line surrounded by {} :

Press F2 to edit, then convert the function to text by entering ‘ before the = sign, then enter with Ctrl-Shift-Enter:

Now delete the text from all the cells other than the top left:

Finally press F2, delete the ‘ to return the cell to an active function, and press enter (not Ctrl-Shift-Enter).  The entire array will now display:

Note that the function only occupies the top left cell.  You can enter data in any other cell of the array output range, but if you do the function will display as SPILL!:

As far as I know, there is no way to return part of an array using the new functionality, other than the top left cell, that can be returned with the Single() function.  Fortunately the old Ctrl-Shift-Enter method still works.

Posted in Arrays, Excel, UDFs | Tagged , , , , , | Leave a comment

Clive James and Pete Atkin

Following The Barrow Poets, more poetic music from the early 70’s:

Clive James finally succumbed to a long illness this week.  He was mainly known for his humorous work on TV and in writing, but he was also a prolific poet, and in his early career teamed up with Pete Atkin to produce musical versions of his work.  The sample below is from their first album, Beware the Beautiful Stranger:

Posted in Bach | Tagged , , | Leave a comment

Biaxial bending update

The reinforced concrete biaxial bending spreadsheet has now been updated:

  • Stress block and capacity reduction factors updated for AS 3600-2018
  • ACI factors for US customary units used when stress is entered in ksi or psi units
  • Example data updated

The revised spreadsheet is included in the file:

ULS Design Functions.zip

I have checked the spreadsheet results against a paper published by Enercalc:

Reinforced Concrete Columns in Biaxial Bending

Enercalc results:

ULS Design Functions-biax results:

After adjusting the Enercalc moments to the centroid of the uncracked concrete section, the results are in near exact agreement.

The results below compare the spreadsheet results with the simplified procedures given in AS 3600 and Eurocode 2, using a rectangular section and plotting moment capacity results under different axial loads:

The AS 3600 and Eurocode provisions for biaxial bending take exactly the same form:

The two codes give significantly different results however because:

  • In AS 3600 the Phi factor is defined as 0.65 for all axial loads, whereas the Eurocode uses the design Mux and Muy values for the applied axial load.
  • The definition of the exponent alpha is different, resulting in AS 3600 having a lower value for low axial loads, but a higher value for high axial loads.

For the purposes of comparison, the Eurocode Mx and My values used the design moment capacities under the applicable axial load to the AS 3600 rules.

For an axial load well below the balance load both codes have an alpha value of 1, resulting in conservative capacity values under biaxial loading.  The AS 3600 requirement to use a Phi factor of 0.65 results in a further substantial conservatism:

Increasing the axial load to close to the balance load increases the alpha values above 1, and brings the AS 3600 Phi factor down close to 0.65, reducing the conservatism of the simplified approach:

A further increase in the axial load increases the alpha value for AS 3600 to close to 2, so that the AS 3600 results are no longer conservative.  The Eurocode alpha value is lower, and it is still conservative:

A further increase in the axial load shows the same trend, with the AS 3600 results increasingly unconservative, and the Eurocode results closer to the detailed calculation, but still conservative:

For very high axial loads, where the base of the rectangular stress block is outside the concrete section, there is a large reduction in the moment capacity from the detailed calculation.  This is not reflected in the simplified method from either code, which both have an alpha value of 2 at these loads:

Further complications arise with non-rectangular sections:

In this case the simplified method from both codes gives results that are highly unconservative for some moment combinations, even for axial loads well below the design limit:

It should be noted that all calculations assumed a rectangular stress block to the AS 3600 code.  Use of a parabolic linear stress block would result in further significant reductions in the design moment capacity when the concrete compression zone was triangular.

Posted in Beam Bending, Concrete, Newton | Tagged , , , , , , , , | 35 Comments

International Keyboard Shortcut Day

Daily Dose of Excel is increasingly inaccurately named these days, but it still contains a wealth of information, and we get a new post on special occasions, such as:

International Keyboard Shortcut Day

Have a look for all the keyboard shortcuts you didn’t know/ had forgotten about.

 

Posted in Computing - general | Tagged , | Leave a comment