DORA’s Journey: An Exploration
DevOps Research and Assessment (DORA) recently joined Google Cloud. The following post gives you an insight into our journey: how we went from our first customer to a successful exit in under three years with no investors and no debt by helping our customers become more awesome at software delivery.
The Beginning: Researching Software Delivery Performance
Our story begins with a collaboration between Puppet and the team that later co-founded DORA: Dr Nicole Forsgren, Gene Kim, and me. Together, we worked on the annual State of DevOps Report from 2014–2017 (shout out to our awesome co-authors Nigel Kersten and Alanna Brown). As part of this program we developed a valid and reliable way to measure software delivery performance, investigated the capabilities that predict it, and showed that it drives both commercial and non-commercial business outcomes. (You can check out our research and our book, Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations for more on our research program.)
Not only did we discover and define a scientific method to measure software delivery performance and the capabilities that impact it, we also had a large dataset from companies of all different types and sizes. Our State of DevOps Reports had given organizations insights to improve software development and delivery. But it is still very hard for organizations to implement change.
From Research to Helping Companies Improve
One common failure mode occurs when organizations attempt big-bang transformations. In this model, organizations try to deploy a new software delivery paradigm in one shot using a one-size-fits-all template, usually involving the implementation of an Agile Methodology and some kind of maturity model. We knew from our experience and the experiences of others that this approach was extremely prone to failure. We wanted to help companies adopt a continuous model in which implementing the principles and practices that lead to improvement is driven through experimentation by practitioners as a core part of their daily work.
We also wanted to provide companies with personalized recommendations. While the State of DevOps Reports described broad industry-level benchmarks in software development and delivery performance and described the key capabilities teams could leverage to improve software delivery performance, they didn’t offer customized measurement and benchmarking. Companies often bring in consultants to help them assess where they are and what they should do, but this approach has drawbacks. Consulting assessments can only hope to talk to a handful of the people on the ground who are doing the work, and they rely a great deal on the individual expertise of the consultants. This makes it hard to baseline and track progress over time in an objective way. It’s also not a particularly good business model for consultancies — the consulting approach doesn’t scale, and you tie up your most senior people doing assessments with no guarantee of follow-on work actually helping to execute change. While consulting firms can be incredibly helpful in organizational change, I had done consulting for years and was more interested in building a product.
Adoption Through the Lens of an Algorithm
So Nicole and I started chatting. Building a tool to measure teams’ software delivery performance outcomes and key capabilities was pretty straightforward, since we had already done the work of building a rigorous, scientific measurement instrument. And with our industry-wide data, we could provide benchmarking against the industry — something no one else had. But what about the most important (and hard) question that people always ask next: “Where should I start?”
When this came up, Nicole’s eyes lit up, “You know, if I have data from dozens of people throughout the software delivery process, I can answer that. I know which algorithm to use, and how I would modify it. This is just a Theory of Constraints problem! We can tell them how to set their strategy in the smartest, most efficient way.” I paused for a minute and said, “Wait, really? Well then let’s build it!” And with that, DORA was born. We sat down and started on mockups the next day.
Our First Customers
At DevOps Enterprise Summit 2016 we met up with Dr Tapabrata (“Topo”) Pal of Capital One. Armed with a slide deck with some prototypes we had mocked out, we pitched our idea: we could measure the key capabilities of their teams and benchmark them against the industry using capabilities that were predictive of performance outcomes, rooted in our research program with the State of DevOps Reports. Furthermore, we could algorithmically identify which areas they should focus their efforts to get the most impact as they continued their transformation. Topo liked the idea so much that Capital One became our first customer for the assessment, thus funding the first release of our platform¹, which allowed us to assess, benchmark, and help Capital One with their transformation strategy.
Capital One ran the assessment across more than a dozen lines of business. The analysis approach Nicole developed identified two capabilities that were a common constraint in almost every one of these lines of business — that is, where they should focus on for the most impact: improving their change approval process and moving to trunk-based development. The first was anticipated by the leadership, but the second was controversial and a surprise to the leadership team. After convincing them to dig into it, the results were staggering. In just two months, the teams saw a 20x improvement in release frequency with zero increase in incidents. They were so impressed with the results that they approved a logo and case study, and talked about DORA to friends in industry — including in a conference talk from main stage at DevOps Enterprise Summit (DOES) later that year.
During our initial development in 2016, Nicole was working at Chef and I was at 18F². After working with two customers and seeing strong results, and with more interest incoming, Gene and I convinced Nicole to run the company full time starting in October 2016 — just in time for DOES 2016 in San Francisco. The conference became DORA’s “coming out party,” with Capital One and Verizon both talking about their success with assessment during their conference presentations.
Fast forward two years, DORA had strong organic growth driven by inbound sales leads through referrals, word-of-mouth, conference talks, and our world-renowned research program. We also created a partner program to extend our reach and drive growth; this enabled consultancies and vendors to resell our assessment and combine it with their own products and services. Our year-one revenues were on par with several VC-backed SaaS companies, and our revenues in year two were more than double that.
Google Cloud was our diamond sponsor for the 2018 Accelerate State of DevOps Report, and we worked closely with domain experts in Google Cloud to test hypotheses in key areas like monitoring and observability, blameless post-mortems, reliability, and the impact of adopting cloud infrastructure on performance. While several companies expressed an interest in acquiring DORA, the relationship we developed with Google Cloud fostered collaboration and discussions between the companies that ultimately led to our acquisition in December 2018.
Lessons Learned
How did we do it? I think there were several contributing factors. First, we were ruthless about applying ideas from the Lean Startup playbook: we used lightweight prototyping (our MVP was a slide deck) and only delivered content that customers would pay for from day one. (As Don Reinertsen says, “the way people tell you what you’re doing is valuable is they send you money.”) Second, we were very careful with our cashflow: we never had an office or any employees, and our infrastructure spend was consistently under $1,000 per year³. While this occasionally led to disagreements over hiring and expansion (Nicole was ruthless about cash flow projections, which saved us on more than one occasion), not accepting investment was an advantage. Why is that? Because it meant we controlled our own destiny: it was our company, and our decisions. We weren’t beholden to others and what they wanted us to do — or what types of returns they felt entitled to. When a great opportunity came along (like the one to consider acquisition and join Google Cloud), we could decide as a team what we wanted to do. That would have been much harder with investors to answer to.
Being frugal and bootstrapping comes with a downside: burnout. Startups have a reputation for driving founders into the ground, and we were no exception. By far the biggest burden was on Nicole. As CEO, she was responsible for the company’s strategy, financials, and overall operations. Not only did Nicole almost single-handedly drive and execute our go-to-market sales and strategy as CEO, she was also principal investigator on the State of DevOps Reports, lead author of Accelerate, and lead on the acquisition. As she says, all of these outcomes were years in the making, but they would certainly not all have come to pass without long hours of work (despite her astonishingly high throughput). This was the downside of not taking investment — we couldn’t hire a larger team to take on more of this burden.
Fortunately we did have an amazing team, to whom both Nicole and I are extremely grateful: Soo Choi, who joined us as Chief Commercial Officer and helped expand our partner program; Xavier Velasquez, our Chief of Staff; Dianna Wiseman, our controller; Kelly Cancialosi, our legal counsel; and Todd Sattersten, our original company manager and co-founder. Below is a picture of our team, including Gene Kim, our co-founder and general partner, at DOES 2018 in Las Vegas (sadly, Kelly and Todd couldn’t join us). Thank you all for the amazing ride.
Finally, a huge thank you from Nicole and I to our customers and partners, and to the DevOps community, which has supported our work for many years. We’re looking forward to working with you to understand how to grow happier and more productive teams from within our new roles at Google Cloud!
1. Our platform also powered data collection for the State of DevOps Report from 2016 onwards.
2. When you’re thinking about doing a startup on the side, make sure you do what we did and verify you can own any IP you create in your own time on your own equipment!
3. Gene made fun of me on several occasions for running our production webservers on 256Mb of RAM.
