This article is also published on the GRID Blog.
One of my biggest inspirations when doing background research before starting GRID was Felienne Hermans. I met Felienne first at Strata in London in 2012, and we’ve kept in touch periodically since. At the time she was studying spreadsheets for her PhD, and I immediately realized that she had a deeper understanding of what spreadsheets are and how they are used than anyone else I’d met.
One of her favorite lines was, “Spreadsheets are code.” (She has a great presentation on this: video, deck.) This quickly rang true for me, but as she so elegantly explains in her recent keynote from rstudio::conf 2019, that was not the case for everyone!
For those of you that are not there yet, I’ll say this: When you understand that spreadsheets are not only code but programs written by “non-programmers,” everything about spreadsheets and how they are used (and abused) will make sense. So give it a deep, long think!
In this keynote she also shares a story about how the research subject for her PhD evolved. It’s perfectly explained in these three cartoon frames from her talk. DSL is short for “Domain Specific (programming) Language”:
I’ve heard Felienne’s story before, but when I saw it again it rang even louder this time around because I’d recently heard my friend Ziggy say exactly the same — just not quite as elegantly.
The Dilemma of Quant Freedom
Ziggy works for a company that is building out a specialized tool for quantitative analysts (quants) on Wall Street. The overall tool calculates weighted risk, running Monte Carlo simulations on thousands of scenarios over millions of assets.
As a part of the tool they had to allow the quants to define their own risk evaluations and variations. And the solution they picked was to give quants the freedom to write their own script snippets to run as a part of the simulations.
But freedom comes at a cost. They quickly realized that giving the quants this flexibility allowed them to shoot themselves in the foot. There are unintended consequences to enabling quants to write custom loops — and even embed entire libraries — in code that runs billions of times to complete a single simulation. In fact, when running a script billions of times, each millisecond in execution slows things down beyond acceptable. A change that adds 1 millisecond to the execution time, will cost almost 12 days to run 1 billion times without parallelization!
By adding functionality that was meant to give the quants more freedom, Ziggy and his colleagues created a new set of challenges for themselves. They now found themselves deep in problem solving mode, helping their clients optimize and simplify scripts.
Ziggy’s team started wondering if there was a simpler solution. Maybe it would be better to allow customers to express their “mathematical desires” in a declarative way rather than giving them the freedom to write scripts from scratch.
And they realized there is a simple solution: Spreadsheet models are perfectly declarative and allow people to describe their intentions without thinking about the exact method. Optimization can be done on the spreadsheet engine and the implementation of the individual spreadsheet functions, rather than on each and every custom model.
And as a major bonus point: Every single person in finance knows this language by heart!
Excel is indeed a Domain Specific Language for finance (and beyond), and a pretty darn good one at that.