My lifelong love affair with Microsoft Excel
I love Microsoft Excel. There, I said it. Excel has been a constant ever since I was a student using an Apple MacIntosh. My university’s motto, translated into English, is “Ever to Excel”. I took that hint and failed to apply it as my parents would have intended. Instead I became a daily user of Microsoft Excel.
For a few years I broke the habit, and became not bad at Lotus-1–2–3, but then Excel lured me back. It lured me back with all those fantastic formulas. And it kept improving. Did you know that Excel was originally just one sheet per file? I used to create links from one different file to another. It got quite messy on that PC of mine, but it was my mess, for my eyes only. Then, Microsoft invented tabs, and like chapters in a book, I organized my work that way. It was as if Microsoft had been listening to my prayers. Or tapping into my dreams. Over the years, I’ve had many an Excel dream. It usually involves the very slow typing of some really long formula, over and over again.
When Excel invented the pivottable I practically cried tears of joy. You could link a pivottable to a chart. You could give the datasource to the pivottable a general enough formula that you could just plonk new data in your worksheet, refresh the pivottable, and, bingo, you had a new chart! And with some know-how you could really work those charts into a thing of beauty. Excel did not have bubble charts or scatter charts at the time, but that did not stop me.
(At this point, I’d like to give some advice. Never ever look back at your old workbooks if you don’t have to. It is better to keep the fantasy alive.)
Around the same time that I was rocking Excel charts, I became good at macros (Excel VBA). At work, we put together enormous business intelligence dashboards, all in Excel. We also had some pretty complicated workbooks to do simulation and optimization tasks.
Then things started to go wrong. Someone would ask for a change in the reports, and it would become a mammoth task. Or, we would find errors buried deep in the workbook — maybe just one formula that had a domino effect. Or, processing a lot of data really tested my patience. My whole computer would be tied up running some calculation or other. Or, nasty macro viruses appeared and then no one wanted macros anymore, even nice ones I’d written. Reality and bad dreams merged into one.
A few years later, Microsoft launched the ribbon interface. This was a betrayal. Before the ribbon interface, I was an Excel master. Now I had no idea where anything was anymore. Arguably, I have never recovered from that event.
I went through a mini version of the ribbon episode more recently when I had to use Excel for the Mac at work. I know that Apple and Microsoft have not always been friends, but why punish the poor Apple user this way?? Then one of my children who has a Mac asked for help in Excel. Bear in mind that teenagers are already questioning their parents’ abilities in many areas. So, it is deeply embarrassing to fumble at something “I thought you said you were good at, Mum?”
It’s been a decades long realization that Excel is not suitable for every task going. Of course, deep down, I knew that, having been using statistical software all this time also. But it still hurt, like discovering your partner will not move house again without calling the moving company. It’s time to grow up, and call the moving company.
Recently I have been working on outlier detection problems for my company, Penny Analytics. This is an area where Excel appears to be useful, but in fact is not. Unless your data consists of one column only, most of the advice to Excel users online is totally unhelpful. Business users who have Excel skills and an outlier detection problem can search the internet, but what they will find is either inadequate recipes in Excel or material that is far too deep into data science and academic theory.
Outlier detection (or anomaly detection, if you prefer) is one of the thornier data science problems. It typically requires multiple algorithms, different approaches depending on whether the data is multivariate or time series, and serious computing power. No Excel user would want to wait for their computer to do the number crunching required. So, what makes a lot more sense is to offload all that work, and that is our operating model. We serve the Excel user, by literally serving the Excel user results rather than headaches.
But I still love Excel. Opening a blank Excel workbook fills me with hope, whereas opening a blank Word document somehow fills me with fear. I love the general nature of the Excel, the fact that sometimes when you select a block of data and press a button, it does exactly what you need it to do. The joke in my family is when asked a question, my answer is often “I have a spreadsheet for that”. And I do.
Anna Turner is Data scientist at Penny Analytics
