The Engineering Manager’s Dilemma

In my previous blog post, I wrote about how software organizations grow to need a platform team. I love how Peter Seibel’s model yields information about the growing need for a platform team as organization get bigger. However, his model is a static analysis.

One of the pieces missing from Peter’s model is the effect of time. The increased effectiveness of a single platform project doesn’t occur overnight. If a 1000 person organization creates a 250 person platform team, it will take months (if not years) to see the organization’s effectiveness increase by ~46%.

Let’s add time to Peter’s model. Ideally, most platform projects will follow a sigmoidal curve (growth is slow first, then fast, and then slow again) in their increase in effectiveness:

In this example, the platform project takes ~6 months to complete and ~12 months to roll out and be adopted by the organization. Of course this is an average number. Some projects will be completed and adopted in a few months, while some other projects will take over a year to complete. From what I’ve seen at Adobe, 6 months to complete a project followed by a 12 month roll out seems about right for most projects.

So let’s take the case of the 1000 person engineering organization. Let’s say an engineering manager buys into the idea of engineering effectiveness and reallocates 256 people to a platform team, as the model suggests. As the projects being worked on by the 256 engineers mature and become adopted, the total effectiveness of the organization will rise — over time. The organization’s total effectiveness as a function of time would look something like this:

The organization’s total effectiveness will drop in the first several months. The drop is equal to 256 engineers. After 18 months, the organization’s total effective will rise to ~1400 (as suggested by the model). The advantages of the reallocation of resources to platform are realized, but the organization had to go through a painful year of significantly (~25%) reduced effectiveness. That is not a year any engineering manager would want to live through. Around month 9, the executives of the company would revolt and fire the engineering manager in an attempt to save the company from chaos caused by losing 25% of engineering effectiveness.

Thus, the Engineering Manager’s Dilemma. The shaded region represents the total loss in effectiveness cause by staff reallocation. This deficit in effectiveness will not be made up until around month 18. So, something that was suppose to deliver greater effectiveness, has instead created a tough 18 month transition problem for the organization. This creates hard choices for the engineering manager.

Is there a way to phase in a platform team without destroying the company?We want the effectiveness curve to look more like the green curve rather than the blue:

Unfortunately, for a large organization, there is no easy way to avoid the Engineering Manager’s Dilemma. The most common way to build a platform team is to reallocate engineers from existing product teams. As we saw above, this will result in an immediate drop in the organizations effectiveness. What if the organization hires new people to create the platform team? Again, the organizations total effectiveness would initially drop. The organization total effectiveness will increase once the new hires begin to complete projects. But this will take time. Again, the organization will face the same “Engineering Manager’s Dilemma” as above.

What if the organization slowly reallocates engineers to the platform team? Rather than moving 256 engineers all at once, perhaps we start with 10 engineers. Once the increased effectiveness of these 10 engineers are realized, we move over 10 more engineers, and so on. Can we mitigate the pain by slowly creating the platform team? On the surface, this sounds like a prudent strategy. Why suffer a 25% lose in effectiveness, when we can manage through a series of 1% loses in effectiveness? This strategy will result in the slow creation of a platform team. It will also result in a perpetual state of ineffectiveness. Each time effectiveness increases, we move more engineers over to the platform team. So every time we see an increase, we reduce the effectiveness by moving more engineers to the platform team. By stretching out the reallocation time frame, it will take about 25 years before the organization will finally see a true increase in effectiveness.

Once a organization finds itself in the state of needing to move 25% of its engineers to a platform team, the engineering manager’s dilemma becomes difficult to avoid. The total lose in effectiveness (the shaded region) will remain the same no matter how you incrementally grow your platform team.

The best way to avoid the Engineering Manager’s Dilemma is to incrementally grow your platform team as your organization grows.

The best strategy is to plan ahead. Start growing the platform team when the organization is around 100 engineers. As the company grows, grow the platform team. Follow the optimal curve from “Why Do Organizations Need a Platform Team?”. The organization will still experience the initial reduced effectiveness of each new platform team, but the pain will distributed over time.

Mitigating the Engineering Manager’s Dilemma: Avoiding the Engineering Manager’s Dilemma takes a tremendous amount of forethought for company leaders. Most companies find themselves in the problem of creating a platform team once they are big. The only solution is strong leadership to ride through the time period of reduced effectiveness. For companies that have the fortitude to push forward, they will be rewarded with a huge effectiveness boost.