How We Design and Plan at DataXu: “Gruhl’s Rule”
(This post orginally appeared as a post on the DataXu Blog.)
When I was young, I wanted superpowers. I didn’t want the kind of superpowers you are probably thinking of. I didn’t want to be able to fly, see through walls, or move objects with my mind. I wanted to be able to solve problems that no one else had been able to solve in ways that no one had ever thought to solve them. To me, this was a superpower.
My fascination with this kind of challenge drew me towards space flight. I was captivated by it. Thousands of tiny parts, all assembled into one mission, carefully orchestrated together, somehow worked seamlessly to put a human on the moon and return him safely to Earth. How did someone lead this project to success? How would one even get started on a project like this?
The desire to find answers to questions like this drew me into the field of aerospace engineering. A field that is fundamentally riddled with the biggest, hairiest engineering problems you can imagine. Just about every problem you face starts with something like this:
“We need you to do something extremely complex that’s never been done before using technology that’s not yet invented. OK? Go!”
Obtaining my PhD from MIT in Aeronautics and Astronautics was as close to superpowers as I could envision. I was working on engineering problems that no one else was able to solve in ways that no one else had invented. I was working with a brilliant team to plan for success of projects that went far beyond our lab.
Here at DataXu, I also get to work with brilliant people and tackle tough problems. These problems just happen to be in marketing instead of aerospace engineering. We aim to solve CMOs’ most difficult challenges by building brand-new solutions using cutting-edge technology . That’s why I love my job. As Chief Technology Officer, I am involved in the conception of these never-before-thought-of solutions and the new technology that makes them possible, all while working cohesively with other exceptional people.
As I continue to solve tough problems in my daily work, I can apply lessons I learned during my studies in aerospace engineering to software engineering, and business in general. One such lesson is how to plan large projects. A situation that happens regularly at DataXu.
The Werner Gruhl Study
While studying at MIT I came across a study by Werner Gruhl, the retired Chief of NASA Headquarters’ Cost and Economic Analysis Branch. In his 26-year career at NASA, Gruhl was responsible for the in-depth collection, study and analysis of spacecraft project management, technical and statistical history to effectively estimate new spacecraft design costs.
In his study, Gruhl observed a project’s cost during the initial cost and planning phases of a space mission, commonly referred to as “Phases A and B”. His goal was to determine how much NASA should allocate to these phases for a typical space mission, and the success rate of those projects based on the allocation of this spend.
Gruhl first completed the study in the 1980s and repeated it in the 1990s. Both rounds had similar results, which Gruhl presented at the International Council of System Engineering (INCOSE) System Engineering Seminar in 1998.
Source: “Effect of Requirements Definition Investment on Program Costs.” Werner M. Gruhl, Chief Cost & Economic Analysis Branch, NASA Headquarters.
This chart shows the correlation between the amount spent in Phases A and B and the total percent of cost overrun of a project. The Y-axis of the chart shows cost overruns and the X-axis accounts for how much of the project budget was spent on design and planning. Sadly, only three projects hit their budget during the entirety of Gruhl’s study. Most projects had at least a 70% cost overrun.
Your conclusions may vary, but when I look at this chart I interpret the data as showing that the optimal amount of design and planning allocation is about 10%. After that point, you’ll see diminishing returns on your investment. Essentially, if you spend more than 10% on design and planning, you will not have a noticeable change in your ability to hit your budget.
On the flip side, spending less than 10% is asking for trouble. The worst cost overruns are incurred by projects where design and planning was underfunded.
How does Gruhl’s Chart apply to software engineering?
One may argue that you can’t simply take lessons gleaned from billion dollar government space missions and apply them to software engineering for digital marketing. However, in my experience there are a lot of similarities in the types of problems we aim to solve. Most of these problems need original solutions and new technologies. I’ve found that following this rule of thumb works well in practice at DataXu.
I use the Gruhl rule when staffing our team to help ensure that we are as effective as possible in completing these kinds of difficult projects. About 10% of the engineering team’s time should be allocated to design and architecture tasks. Our agile sprints for development are on a two-week cycle, so about one-day’s worth of team time is spent on managing, planning and wrapping up those sprints. The rest of our time is spent on actually working together and solving the problem.
This rule of thumb can also be applied to the broader company’s hiring strategy, where about 10% of the staff should be in general and administrative (G&A) roles, working on planning and resource allocation. I even use Gruhl’s results as an inspiration in planning my personal allocation of time in a given week, where about 10% of my time is devoted to planning and designing that week’s work and activities.
More allocation than that may be a waste of time. Less than that is a recipe for drastic underperformance.