Beware SAFe (the Scaled Agile Framework for Enterprise), an Unholy Incarnation of Darkness
“And, in the next place, since evil is specially characterized by its diffusion, and attains its greatest height when it simulates the appearance of the good, for that reason are signs, and marvels, and lying miracles found to accompany evil, through the cooperation of its father the devil.”
— The Theologian Origen (185–254) on recognizing the Antichrist
If you’ve never heard of it before, the Scaled Agile Framework for Enterprise is a collection of principles and practices assembled with the goal of offering a way to “scale up” an Agile working model for large companies.
If you need it, I have a refresher on what Agile is and isn’t here.
Since its inception in 2011, SAFe has experienced enormous growth. Almost half a million people worldwide have become certified SAFe practitioners. If you expect to work in product, design, or engineering at a large company — or even a growing startup — there’s a chance you’ll encounter SAFe at some point in your career.
Spoiler alert: I’m not a fan.
In fact, as you may have guessed from the title of this article, I actually think SAFe is very bad. It proclaims values that seem sensible and then turns around and imposes processes and structure that stifle real agility. There are a number of “scaling frameworks” out there of varying quality, but it’s SAFe in particular that exemplifies the worst of misunderstood and poorly applied Agile thinking. It’s worth paying attention to — even just as an example of what to avoid.
You should treat this stance for what it is: one Product Designer’s individual perspective. I can’t conclusively prove that SAFe is the wrong choice for all situations. What I can do is raise awareness about the commonly expressed criticisms that corroborate with what I’ve learned myself and heard from others in my network.
I also hope to draw a clear connection between the high level criticisms and the low level structural details of the framework. This will require some explanation of those details. The good news is, there’s a graphic that explains everything in a simple, intuitive way:
…or maybe not.
If you’d rather SAFe speak for itself you can use the link above. There’s hundreds of pages of documentation, hours of videos, 15 different training courses, and more. If that sounds like a little much, I’ll do my best to provide some explanation of how SAFe works as I go through each point.
SAFe enables bureaucratic, top-down control
At the highest level, called the “Portfolio”, SAFe does advocate funding indefinite “Value Streams” — which usually represent a product, product line, or customer type. However, the Lean Portfolio Management team (LPM) that controls funding (likely the same people previously responsible for waterfall-style project budgets), are given sole authority to approve which Portfolio Epics (large initiatives) move into each stream. Epics are not explanations about a problem that needs to be solved. They are pre-formed ideas about how best to solve those problems.
Right away we can see signs of the old-school mindset of viewing teams as a “delivery” function instead of a strategic one. The high level thinkers come up with ideas, and the low level doers execute on those ideas. Ignored is the possibility that those closest to the work might be best equipped to make decisions about it. Escaping from this misguided mindset is a core goal of Agile thinking that SAFe fails to remotely accomplish.
A similar issue pops up again when we look at a slightly lower level.
SAFe collects small product teams (often Scrum teams) into “Agile Release Trains” — groups of teams with an additional layer of management roles spanning each group at what is called the “Program level”.
Generally these roles impede the autonomy of teams. They add process and communication overhead out of proportion with the value they provide.
These roles include a Product Manager (PM), a System Architect (SA), and a Release Train Engineer (RTE).
The SA and PM define and break up larger pieces of work (often inherited from the portfolio process above) and then pass the pieces into the teams.
The Product Owner and team might theoretically be able to prioritize other, smaller pieces of work against the work imposed on them, but these efforts have limited visibility and buy-in from above. There will be a natural pressure to treat work coming from the highest point as most important — even if every single team member believes other items would be more valuable. Because of this, the role of the Product Owner is reduced to writing stories and checking acceptance criteria, instead of being the single point of accountability for product leadership that was the original intention for the role.
The System Architect is not in a position to be close to the actual engineering work, so the architectural plan they pass on may be disconnected from the reality of the work. These sorts of roles were a hallmark of big-design-up-front waterfall projects and are well preserved in SAFe.
Release Train Engineers define consistent cross-team process and cadence, and handle many operational tasks. Ultimately this leaves less room for individual teams to modify, improve, or experiment with their own practices in any way that deviates from established standards.
Sometimes all of these problems are repeated again with the addition of a “Large Solution Level” — groups of groups of teams with analogous roles to those at the program level, but spanning Release Trains. When this occurs the same problems are likely to be repeated, but the Solution Level does seem to be less commonly put in place.
SAFe comes bundled with Rally — FURTHER limiting team autonomy
SAFe is often a package deal with the project management software Rally. Rally is the sort of software you’d expect a company using SAFe to make — it’s overloaded with features, unfocused, confusing, unstable, and consequently difficult to learn and use.
The value proposition of adopting Rally for leadership is that they’ll be able to see reporting that gives them a simple, unified picture of the progress and problems across the entire enterprise.
Of course, this means everyone in the enterprise not only needs to use Rally, but they need to use it in a consistent way. Again, this can only be achieved through top-down dictation. Rally and the way it thinks about work are forced onto all teams regardless of their preference, context, or buy-in. To actually have all the information roll up requires incredible overhead in estimating and tracking that slows everything down and can be extremely disruptive to actually getting work done.
Additionally, the top level reporting isn’t even particularly useful. It focuses on the things Rally tracks— metrics like the volume of story points delivered (literally made up) or the quantity of bugs addressed (a terrible measure for “product quality”). If management ignores these stats, the overhead they add will be for nothing. If they manage to the metrics, the stats will be fudged so that teams look good and are left alone. Estimates will be over-padded, small pieces of work will be exaggerated, and this can easily lead to a breakdown of trust.
SAFe takes the worst possible approach to managing dependencies
A dependency is an instance where a team needs to wait for something to be done elsewhere before they can complete their own work. SAFe is structured around a backwards attitude that dependencies are an immutable fact of life that should be accepted and managed around. It even sometimes refers to them as a strategic advantage. In reality, when you think of dependencies as a bad thing — to be minimized at every opportunity — you may find that those opportunities are plentiful and that taking advantage of them results primarily in benefits. Here are just a few suggestions that SAFe under-emphasizes or entirely ignores:
- Encourage a culture of inner-sourcing where one team can submit work to another team’s code base without having to depend on them beyond accepting the merge
- Make it easy for team members to temporarily or permanently swap teams when it makes sense to do so
- Focus on hiring, structuring, and training teams to handle their own needs without outside help
How SAFe actually chooses to manage dependencies is by increasing focus on planning, process, hierarchy, and standardization. Predictably, this results in lots of meetings that interfere with the ability to get work done. It imposes this approach through a universal roll-out that affects the entire organization at once. No consideration is given to which areas could have been self reliant without the additional burdensome structure.
SAFe is excessively oriented around planning
A core feature of SAFe is the idea of ~10 week Program Increments (PIs). You can think of these sort of like huge sprints that contain all of your normal sprints.
Each PI begins with a PI planning event that runs for about two days. PI planning also requires pre-planning and post-planning activities at the Program level.
There is definitely some value in having people get together in person to build relationships, share information, and orient around goals. On the other hand, using that limited time window to make 10 week plans of specific user stories based on pre-defined features, and then requiring commitment to those plans is much less valuable.
As soon as PI planning ends, those plans created based on limited understanding and numerous assumptions will become obsolete as soon as anything new is learned. Teams will be continuously torn between sticking to the plan they’ve learned doesn’t make sense and and reorienting expectations for reasons those above them may not be in a position to understand.
SAFe is oriented around volume, not value
In all this focus on volume metrics, estimation, and churning work through the pipeline, the concept of what’s actually valuable or successful is easily lost. It’s often assumed that more work shipped out the door must be “value”, even if the experience of the product is actually suffering and users are not benefiting from the additional features.
SAFe is aware of the criticism that it is not value focused and has recently added “design thinking”, “customer centricity”, and other concepts to its documentation to compensate. So far I’m not convinced it makes any significant changes in its process that actually make room for those things, and it fumbles in even understanding them in the first place.
For example, SAFe’s definition of “customers” leaves the door open to defining business stakeholders or those funding the value streams as being “customers”. This is very different from reorienting everyone (including those providing the funding) to focus on the needs of the end-users actually using the software, a core element of design thinking.
SAFe is unnecessarily complex
SAFe has a natural advantage that protects it from criticism; It is so complicated that it’s difficult to fully comprehend. Despite the additional time I’ve spent researching the framework, I’m probably still likely to get some things wrong or miss some important points. However, an abundance of complexity is itself a legitimate concern. SAFe has so many meetings, checkpoints, values, and methods that it’s basically impossible to get everyone on the same page.
SAFe limits retrospectives and continuous improvement
Roles at the program level and above cannot possibly attend all team retrospectives. This means retrospectives will not be directly heard by the people who can actually change many of the things being discussed.
The way SAFe tries to compensate for this is not sufficient.
PI planning is the only time where everyone in a Release Train is guaranteed to be together in-person. Unfortunately it contains a short retrospective related only to the success of the planning event itself.
SAFe does include an “Inspect and Adapt” event towards the end of each PI, but it seems almost designed to be skipped based on how hard it is to coordinate for the RTE. When held, the event is primed by reviewing — of course — volume metrics from the last PI. Additionally only 30 minutes of the event are allotted for an actual retrospective where issues are surfaced and agreed upon. If this time is split evenly among all the members of a 100 person Release Train, then each participant will get about 18 seconds every ~10 weeks to bring up issues in a context where they might actually be heard.
SAFe also has some other elements like “assessment charts” where teams are surveyed more broadly, but these lean heavily towards confirming that SAFe practices are being followed, not whether or not they’re effective.
SAFe degrades the concepts it aggregates
A key part of SAFe that I have not yet touched on is the aggregation of existing concepts like Scrum, Kanban, Lean Product, Lean UX, and DevOPs.
If you’re unfamiliar, I’d suggest exploring each concept independently over time rather than all at once. Many are valuable themselves, but SAFe doesn’t do a great job at actually synthesizing them and can sometimes add confusion. Case in point:
SAFe routinely espouses how its practices follow “Lean-Agile” principles. The trick is that “Lean-Agile” doesn’t actually refer to “Lean” or “Agile” principles.
Instead “Lean-Agile Principles” are a creation of SAFe’s own design. This new set of “principles” warps the concepts it assimilates. For example, the page about decentralizing decisions (ranked the lowest of all principles) subtly subverts itself by emphasizing the importance of centralizing decision-making for many use-cases. Other listed principles have similar problems. For a person being introduced to all of this information at once, the actual meaning of the original concepts may be lost.
SAFe is not Agile
By now many of the ways SAFe is inconsistent with an Agile mindset should be pretty clear. It’s plan focused, bureaucratic, complicated, includes a lot of often unnecessary process, dis-empowers team autonomy, and more.
But does it matter whether or not SAFe is Agile when what we really care about is effectiveness?
Yes. It does.
Agile principles are only Agile principles because they were generally recognized and agreed on as being effective. That thinking resonated with a broader community of practitioners which is why the movement originally took off. If you think agility is a predictor of effectiveness, (a belief presumably behind some of the interest in SAFe) then it matters quite a bit if SAFe is consistent with Agile thinking. Misleading people about what they’re buying is a bad thing and it isn’t wrong to call that out.
What about all the case studies where SAFe is shown to have worked?
There are about 40 case studies on successful SAFe adoption on scaledagileframework.com. The case studies for failures are conspicuously absent. Contrary to the acronym, SAFe involves simultaneous changes to a huge new process across an enormous ecosystem— a massive risk. Unless each of those 40 successful companies each had 11,250 certified SAFe practitioners there are quite a few companies we’re not hearing from.
Also, I don’t put much too much stock in the case studies that do exist. Even if the volume metrics they focus on really were valuable it’s easy to cherry pick a few stats that make it seem like things went well.
To focus in on a specific example, let’s take a look at an excerpt from the case study about SAFe implementation at Fitbit:
In 2016, consumer technology company, Fitbit, released four new products to the market that were positively received by consumers, and shipped over 22 million devices. Delivering its highest number of products in a year is due in part to the company’s commitment to, and success in adopting SAFe® (Scaled Agile Framework®) as a way to scale the team to meet target dates.
© Scaled Agile, Inc.
Here’s Fitbit’s stock price over the last five years:
Yeah… 2016 looks like a great year for Fitbit.
Now, we can’t really attribute the downfall of Fitbit to SAFe adoption with the amount of information we have. But if a company can struggle this much and still hail SAFe adoption as improving the responsiveness of their product decisions, I’m going to narrow my eyes a few degrees more when checking out the rest of the case studies.
It’s OK as a transitional step though, right?
If you’re someone who would make this argument then you must think that SAFe needs to be transitioned away from. On this point we agree!
To make that transition we will of course have to spend some time pointing out the parts that don’t work well.
Is that something you’ve been spending a lot of time on?
Even if you are, SAFe is set up in a way that prevents you from actually transitioning. The authority it places in the hands of management acts as a legitimization of the top-down control mentality. This pairs poorly with the lacking methods for process improvement we’ve already covered. The authority to “customize” SAFe to the appropriate context is mostly unavailable to the actual teams best positioned to recognize what isn’t working.
Nowhere in the SAFe road-map does it actually refer to any of its core practices as transitional. If it’s a transitional framework, then it does a pretty poor job of providing any kind of transition plan.
The hyper-focus on certifications also hurts the framework’s flexibility. If you spend hours and hours working to become certified, the certification become a part of your own value. A transition to something that looks less like SAFe may seem like it threatens that value. I can’t blame anyone for getting certified in a job market where certifications matter. But those individuals have a particular responsibility to pay attention to the downsides of SAFe and possible ways to improve or deviate from it.
These problems are broadly experienced
I‘m not alone in expressing many of these criticisms — though I hope I’ve gone a bit further in connecting them to the specific practices in the framework. If you’re looking for corroborating testimonies, here are a few:
Ken Schwaber, co-developer of Scrum expressed serious concerns about SAFe as early as 2013.
Nicholas M. Chaillan, U.S. Air Force Chief Software Officer, discourages “using rigid, prescriptive frameworks such as the Scaled Agile Framework (SAFe).”
Forbes contributor Steve Denning calls out SAFe for being “ particularly worrying” in his article on how to spot fake Agile.
Marty Cagan of the Silicon Valley Product Group explains how SAFe is antithetical to the mindset of Silicon Valley companies.
Stay safe out there (not SAFe)
While SAFe has certainly received a good deal of criticism, my personal sense is that this criticism may not have reached a large audience beyond certain online circles. This is especially likely when considered alongside the increasing number of companies adopting the framework every day.
For many people, the idea that there’s anything wrong with SAFe— or that it conflicts with Agile principles at all — may be completely novel.
Increased discussion aimed at a broader audience can probably help. I hope to touch on this topic more in the future and to detail what other alternatives are out there.
For now, even if you aren’t working in a SAFe environment, that’s not an excuse to get complacent. The concerns I’ve laid out here should be taken as a broader warning — even the most sensible of ideas can easily be co-opted, corrupted, and confused if you’re not paying close attention.