Modern portfolio theory of management: variance maximization in engineering

I’ve spent much of my career as an engineering manager in high-growth environments[0], first at Facebook and now at Stripe, where I am responsible for scaling our core payments team. I love applying vaguely mathematical concepts to management[1] and in a recent one-on-one, the topic of variance maximization came up.

By applying techniques from modern portfolio theory to manage variance across a portfolio of people and systems, engineering managers can help make their teams more successful.

What is variance maximization?

Variance maximization appears in fields such as portfolio management theory[2] and machine learning[3]. In applying it to software engineering, let’s use variance to define the range of expected output for a given system, from worst-case to best-case scenarios.

Software engineering is typically a variance minimization game[4] — every bug or system failure causes undesirable variance in quality. For each component, you minimize variance as much as possible; to increase overall performance, you combine reliable components. Automatic failovers, rate-limiting, and predictive modeling are all variance minimization techniques. Development experience and skill are demonstrated by how quickly variance is minimized, and by how much quality increases with each change. It’s hard to imagine a software system that is simultaneously high quality and high variance.

Increasing software performance using components that individually minimize variance

Management is different — you’re not only required to tolerate variance at all times (people are much less predictable than software systems!) but you may even encourage periods of high variance. Great results come from skillfully managing the natural variance across a portfolio of teams and projects, while steadily increasing the quality bar over time. With engineering management, there’s the added challenge of aiming for high variance only in the positive direction[5] — mistakes (system outages, user churn, team burnout) can be costly or even irrecoverable.

Increasing quality while managing through constant variance

How do you manage variance in your work?

While there are many different ways people can manage variance, here are some common techniques I’ve employed in my personal experience.

Process can minimize variance by standardizing results. Design reviews give products a consistent look and feel. Incident reviews help teams learn from mistakes and identify remediations. Strategy reviews can create a variance-minimizing effect by filtering ideas through the same set of minds[6].

Ambitious goals incentivize (and sometimes force teams) to maximize variance by changing behaviors and thinking outside the box. This could lead to innovation and unexpected wins that may not otherwise be realized using a more conservative variance minimization approach. Used incorrectly, overly aggressive goals can have the reverse effect, causing burnout or demoralization.

Experimentation frameworks can selectively control for variance by testing a range of approaches across the variance spectrum. For example, you may want to maximize uptake of a feature without affecting revenue. Experimentation allows you try less and more aggressive approaches until you’ve found the right balance.

How does variance maximization apply to managing people?

People come in all forms — some might naturally be variance minimizers or maximizers, while others adapt easily to different circumstances and roles. All are motivated to do great work, but may go about it in entirely different and seemingly incompatible ways.

Minimizers often work in a predictable manner, are less likely to take risks, and gravitate towards well-structured work or process. Minimizing managers may have an established playbook they’ve found works well, and focus on helping their teams adopt a consistent development process.

Maximizers on the other hand, often love to pursue crazy ideas and seek more freedom, autonomy and variety in their work. Maximizing managers might look at each person individually and give them space to use what works best for them — even if it means supporting styles that differ greatly from their own.

Teams are complex; mixes of people can cause friction or amplifying effects, and be suited for different problems. Teams of minimizers might develop mastery in an area and support long periods of stability, but suffer from a lack of new ideas or be slow to evolve their long-term strategy. Teams of maximizers might take on big bets and have a high production rate, only to have trouble scaling or prioritizing projects.

Organizations can benefit from a healthy mix of both types. Great leadership involves putting people in the right places and coaching them to flex in different directions — helping minimizers take bigger risks and getting maximizers to raise their quality bar.

How can managing variance help you maximize quality?

Both variance maximization and minimization have their place, and knowing when to use each can produce the best results.

If your team’s mandate is to go and build the next big thing, you may be trying to maximize variance while limiting downside risk. Set aggressive goals with maximizer leaders to encourage bold new ideas, but employ constraints like fixed budgets or time horizons to limit losses.

As part of the original Timeline team at Facebook, our goal was to meaningfully improve how people used their profiles: giving them better ways to personalize (cover photos and life events) and access their memories (profile history). To accomplish this, we had to overhaul both the underlying infrastructure and user-facing product — a big and risky undertaking. We formed a new parallel team to work on the project without interruption, and transferred in people from News Feed (including myself) to help design the Timeline backend and minimize infrastructure risk. We launched in 2011[7] and were subsequently able to deprecate the old profile entirely.

If you’re operating at massive scale with low tolerance for error, you may be trying to minimize variance. Automated dashboards and alerting enable you to address unexpected errors. Bringing in people with deep domain expertise helps teams see around corners more quickly.

Since joining Stripe in 2015, I’ve spent a significant portion of my time leading efforts to improve our engineering recruiting process. We’ve transitioned from generalist to specialist pipelines, improved our candidate experience with closing calls and surveys, and increased our offer acceptance rate — all while growing from 80 to 300+ engineers.

We couldn’t afford to take big risks because we needed to continue hiring great people and keep pace with Stripe’s business growth. We relied heavily on variance minimization to do so; before introducing each change, we reviewed its impact and built a repeatable process for it. As a result, we were able to improve outcomes without sacrificing consistency.

What do you think?

Does this framework resonate with you? How do you manage variance in your own work and with your teams?

Thank you to Michael Manapat for the original concept and inspiration & Aditya Koolwal, D L Schafer, Tiffany Fenster, Jackson Gabbard, Jackson Gates, kortina, Mariya Yao, and Oana Carja for additional help with this post.


Footnotes

[0] I was at Facebook from 2009 (~700 employees) until 2015 (over 10,000 employees). At Stripe, I’ve seen the company grow from just over 200 to over a thousand today.

[1] Here’s a graph I use to model switching from an individual contributor to engineering manager. As an IC, you make steady progress on each project until you ship and start over again. As an EM, you can experience wild productivity-mood swings, and the lows (e.g. unhappy teammates) can feel a lot lower.

The productivity journey of engineering life, IC vs. EM

[2] Modern portfolio theory describes maximizing returns for a given amount of risk (variance) by decreasing the correlation of components. There’s also a lot out there on using variance minimization in portfolio management.

[3] In principal component analysis (PCA), you select components that maximize variance in order to capture as much of the original data variability you can, while collapsing into fewer dimensions.

[4] Note that this doesn’t apply as well to building products, where you may aim to maximize variance in some aspects of the user experience in order to surprise and delight. Product development and prioritization feel closer to “management” than to “software engineering”.

[5] Publishing or venture capital may rely heavily on a “big hits” approach to variance maximization, while accounting for expected failures in their business models.

[6] This relates to Jeff Bezos’ philosophy of looking for leaders with great judgement, who are “right most of the time”.

[7] For a trip down memory lane, you can read about the original FB timeline launch and a very old Q&A with me