Extracting and counting factors with Python and Numba

This post features code using the pyxll add-in to call Python code from Excel. See pyxll for more details and downlod.

A recent issue of New Scientist had a series of puzzles finishing with: how many integers between 1 and 1 million have the property that 1/3 of their factors are divisible by 3.

This can actually be solved without factorising 1 million numbers, which is the point of the puzzle, but it is an interesting exercise to write the code to do it the brute force way in a reasonable time.

First a function to list all the factors of any input integer:

def py_factors1(n):
    factlist = reduce(list.__add__, ([i, n//i] for i in range(1, int(n**0.5) + 1) if n % i == 0))
    return list(set(factlist))

Then count the total number of factors, and that number that are exactly divisible by an input value:

def py_countfact1(factlist, n):    
    numfact = len(factlist)
    numdiv = 0
    for i in range(0, numfact):
        if factlist[i] % n == 0: numdiv = numdiv+1
    return(numfact, numdiv)

Then call each function over the given range, and count the number that satisfy the required factor ratio.:

@xl_func
@xl_arg('factrange', 'int[]')
@xl_arg('n', 'int')
@xl_arg('ratio', float)
@xl_arg('step', 'int')
def py_countfactrange1(factrange, n, ratio, step = 1):
    stime = time.perf_counter()
    start, stop = factrange[:]
    rtnnum = 0
    for i in range(start, stop, step):
        fact = py_factors1(i)
        countfact = py_countfact1(fact, n)
        if countfact[1]/countfact[0] == ratio: rtnnum = rtnnum + 1
    return rtnnum, time.perf_counter() - stime

The New Scientist puzzle is solved by this code in about 9 seconds, finding that there are no numbers satisfying the required factor ratio:

The solution time can be greatly reduced with the Numba just-in-time compiler. For the second function it is only necessary to add the Numba decorator at the top of the code. I have also added pyxll decorators, so it can be called from Excel:

@xl_func
@xl_arg('factlist', 'int[]')
@xl_arg('n', 'int')
@njit
def py_countfact(factlist, n):    
    numfact = len(factlist)
    numdiv = 0
    for i in range(0, numfact):
        if factlist[i] % n == 0: numdiv = numdiv+1
    return(numfact, numdiv)

The first function is not so simple. Numba does not work with set objects, or the reduce function, so it must be re-written:

@xl_func
@xl_arg('n', 'int')
@njit
def py_factors(n):    
    maxn = int(n**0.5) + 1
    factlist = [1]
    for i in range(2, maxn):
        if n % i == 0:
            if not i in factlist:
                factlist.append(i)
                if i != n//i: factlist.append(n//i)
    factlist.append(n)
    return factlist

The calling function only requires the Numba decorator, and editing to call the new functions:

@xl_func
@xl_arg('factrange', 'int[]')
@xl_arg('n', 'int')
@xl_arg('ratio', float)
@xl_arg('step', 'int')
@jit
def py_countfactrange(factrange, n, ratio, step = 1):
    stime = time.perf_counter()
    start, stop = factrange[:]
    rtnnum = 0
    for i in range(start, stop, step):
        fact = py_factors(i)
        countfact = py_countfact(fact, n)
        if countfact[1]/countfact[0] == ratio: rtnnum = rtnnum + 1
    return rtnnum, time.perf_counter() - stime

The new code reduces the execution time to 0.47 seconds:

The functions can be called from Excel with any desired range, divisor and target ratio:

I have added the functions described above and the associated code to the pyxll examples.xlsx and worksheetfuncs.py files, which can be downloaded from: FactorUDFs.zip

Posted in Excel, Link to Python, Maths, Newton, PyXLL, UDFs | Tagged , , , , , , , | Leave a comment

ULS Concrete design with Python

The py_UMom and py_ULS Design Functions have now been updated and can be downloaded from:

py_UMom.zip

py_ULS Design Functions

More details of the two spreadsheets can be found at Python Code Updates – py_UMom and ULS Design Functions – Python version.

The most significant updates are:

  • The calculation of the effective depth for longitudinal force due to shear has been updated to be consistent with the Australian code requirements.
  • An example of the py_UMomR function has been added to the py_UMom spreadsheet.

The py_UMomR function has the same simplified input as py_UMom, specifying a rectangular section with 2 layers of reinforcement, with the addition of an optional range for prestress details. This allows a prestress force to be applied to one or both layers of reinforcement, and/or an additional prestressed tendon to be specified.

Posted in Beam Bending, Concrete, Excel, Link to Python, Newton, PyXLL, UDFs | Tagged , , , , , , , , | Leave a comment

More from Anne Briggs

A (comparatively) recent YouTube of Anne Briggs from her brief return to public performances in 1990-91:

Anne Briggs – The Jim Lloyd Sessions, 1990-91

Anne Briggs made two appearances on Jim Lloyd’s ‘Folk on Two’ during her brief 1990-91 return to live performance. On 7/11/90, the show broadcast ‘Martinmas Time’ from Salisbury Arts Centre, recorded during a short tour with Martin Carthy & Dave Swarbrick. On 20/2/91 she was live in the studio, interviewed by Jim and performing four songs. The photos here include: three taken by Eamonn O’Docherty in Ireland 1967; one taken by me in Lincolnshire 1991; several taken by me in Edinburgh 1992 (during the filming of ‘Acoustic Routes’); two taken near Oban in 1997; one taken by Brian Shuel in 1962; and a couple of others from the 60s.

… and an interview from April 2019, reposted this month:

I don’t know who I am, I don’t know what I am… but I am” Anne Briggs interviewed

Posted in Bach | Tagged , , , | Leave a comment

Close to it all ..

The death of Melanie Safka was reported yesterday.

She was best known for her work in the late 1960’s and 1970’s, but she continued performing throughout her life. Here is a small selection from her live performances over the years.

MELANIE 26 minutes of bliss – 1971:

When I was a young man living in England there was a series of “In Concert” TV programmes broadcast with singer / songwriters of the day including Neil Young, Cat Stevens, Joni Mitchell and of course Melanie Safka. This particular short “concert” was recorded in London on 14/7/1971 and broadcast 8/1/1972. I remember being mesmerised by Melanie’s performance, and its power has not diminished over the intervening years, although sadly the “peace and love” message, so prominent in Melanie’s lyrics, never really got much further than 1971, we all hoped it would after the short lived “halcyon” years of hope from 1967 – 1970. As Joni Mitchell sang in 1972 on California ” Sitting in a park in Paris, France, Reading the news and it sure looks bad, They won’t give peace a chance, That was just a dream some of us had …. “. I’ve remastered this as best I can from the original tape I had – I hope you enjoy it. Peace!

Lay Down / Candles In The Rain MELANIE & DAUGHTERS Live ’91:

MELANIE & MILEY CYRUS Look What They’ve Done To My Song, Ma (2015)

Posted in Bach | Tagged , , , , , | 2 Comments

New GROUPBY and PIVOTBY functions

Microsoft recently announced two new functions that provide similar functionality to pivot tables, but will update automatically for any change in the referenced data.

These functions are currently rolling out to users enrolled in the beta channel for Windows Excel and Mac Excel.

I’m excited to announce Excel’s new GROUPBY and PIVOTBY functions. These functions allow you to perform data aggregations using a single formula. And while these functions are extremely powerful, they are also simple to get started with. In fact, you can do a data aggregation with just 3 arguments, the same number as a simple XLOOKUP. 

Note: These are preview functions, their signature and results may change substantially before final release based on user feedback. Until final, we do not recommend using them in important workbooks.

https://techcommunity.microsoft.com/t5/excel-blog/new-aggregation-functions-groupby-and-pivotby/ba-p/3965765

See the link above for more details and also the article below from myOnlineTraininghub. Both include full descriptions and examples of the new functions:

The GROUPBY and PIVOTBY functions are a significant breakthrough in Excel’s toolbox of functions.

They let you easily group or aggregate data, a concept that’s been around in Excel since the days of PivotTables in 1993 and Power Query since 2010.

But what’s exciting about GROUPBY and PIVOTBY is that they simplify this process down to a formula.

This means any changes to the source data are instantly updated in your reports, unlike PivotTables which require a click of the Refresh button to update.

But what about Slicers, which are one of the best features of PivotTables?

Don’t worry, I’ll share a clever trick so you can still use Slicers with GROUPBY and PIVOTBY

https://www.myonlinetraininghub.com/excel-groupby-and-pivotby-functions
Posted in Computing - general, Excel | Tagged , , , , , , | Leave a comment