Managing the Cultural Changes of Agile Procurement
Authors: Michael Wilkening, Special Advisor on Innovation & Digital Services to California Governor Gavin Newsom; Nagela Nukuna, Master in Public Policy, Harvard Kennedy School and Master in Business Administration, MIT Sloan; James Hutt, Special Student, Harvard Graduate School of Arts and Sciences
Introduction
Much has been written about the benefits of agile procurement as a means of avoiding high-profile technology procurement mishaps. Beyond cost savings and improved reliability, there are other benefits to governments taking on central integration roles in technology projects. The government is better able to retain control of personal data and — over time — advise vendors based on deep historical knowledge. The government also has purview over its internal system architecture, creating the opportunity to connect otherwise siloed systems. For these reasons, the idea of government institutions embracing and leading agile approaches to procurement has become increasingly attractive.
However, this approach requires a cultural shift — a willingness to implement new practices and skills, as well as to work with vendors who are themselves adopting an agile working style. While digital service groups often seek to achieve this cultural change within their organization, relatively little has been said about the challenges of shifting away from a “waterfall” approach toward an “agile” approach. Identifying the costs of switching to agile procurement is a first step toward overcoming these barriers and creating new best practices.
Long-held working practices are just as big of an obstacle — if not bigger — than legal or technical requirements. While teams can sometimes get one-off approval for agile projects, it is bureaucratically difficult to deviate from the normatively (if not necessarily legally) defined standard operating procedure. Teams also face costs in the form of unassessed risks, insecure funding, and new forms of personal accountability for failure.
It may be tempting to focus on delivery as a strategy: As long as you deliver good products and services, you needn’t worry about consequences. Instead, we advocate for identifying potential risks and consequences up front. Only by resolving the difficult organizational and cultural questions that governments face in this area will lasting change be achieved.
California’s Child Welfare Services Case Management System
In 2017, David Eaves documented the way that California’s Department of Social Services made use of a modular, agile procurement approach. The case study outlines how the state’s Division of Child Welfare Services (CWS) needed a technologically up-to-date case management system that matched the needs of social workers handling reported cases of child abuse and neglect. In 2015 alone, when the Request for Proposals (RFP) was announced, CWS received approximately 500,000 cases.
With the stakes so high, after a lengthy examination of the case management system, California decided to shift from a waterfall-style, vertically integrated solution to a modular, agile approach. This allowed for a more user-centric approach that elicited and incorporated feedback for each portion of the system.
CWS faced many obstacles in its transition to an agile approach for case management, largely because traditional government systems were not designed to match agile process needs. Expectations surrounding procurement, from budgeting to timelines, had to evolve. Given its large scale, the CWS project required a cultural change that would incentivize various stakeholders in different ways.
Challenges To Agile Procurement
The following questions build on CWS’s experience with modular, agile procurement. They highlight challenges that a team should feel comfortable discussing, if not fully resolving, with political sponsors and senior civil servants.
1. How do we build new partnerships?
As with employees, a change in working style will be unsettling for traditional contractors. What scope is there for growing this pool of partners? Who needs to be involved in the design process who hasn’t been before? Where do we meet them? Do they come to government meetings or prepare pitches in the way that our agency is used to?
The CWS team adopted an iterative, user-centric approach that required multiple partners to work with the government, who ultimately remained accountable for the project’s success. This diverged from the previous practice of bidding out the project to a single vendor or aligning the project’s needs to match off-the-shelf software. To do this, the CWS team needed to find partners who were willing to build out smaller pieces of the case management system and use open-source software. This required government staff to change how they executed on projects and to start running scrum processes to better support their agile procurement approach.
2. How do we get support and approval for a project?
Agile projects cause uncertain timelines and budgets, which are traditionally used to justify a project in advance. Which oversight bodies will need to become comfortable with new ways of working? How will the project justify the value it provides for the money spent? Without a timeline to completion, what can we promise officials and the public will be delivered and when?
For a traditional waterfall project, outputs typically aren’t expected until the final stage. Therefore, allocated budgets can be used closer to those later stages, when the tool is typically built and implemented, and the government can meet the incremental costs in an individual year. The project scope can be adjusted on a month-by-month or year-by-year basis in order to meet predetermined success metrics. This all obscures the impact of any initial investments and reduces clarity around an expected output in a given year. However, this is also more politically feasible — the impact of the initial investment is inextricably tied to the final output (and how successful or impactful the generated final product is).
Under the new procurement approach, separating work into modules carries inherent risk. In a modular approach, stakeholders can more easily exit at certain points within the project. Additionally, funding may be required at each stage, rather than just towards the end when a solution is implemented. This is a requirement that the annual legislative budget process in California, and across most levels of government, is not designed to meet. Finally, the cost at each stage depends on the work done in the previous stage, so costs are somewhat indeterminate. Due to this increased uncertainty, a modular approach requires more political will.
3. Who is going to have new demands on their time?
Agile projects require hands-on involvement after the RFP is awarded. The work is distributed, rather than front-loaded. Who will manage these ongoing exchanges? Would they normally have moved on to managing a new procurement? If so, who will take on that work? How will user testing be arranged? Are front-line staff motivated to be involved? Will they believe that feedback will be taken seriously and incorporated into the product?
For the CWS project, decision-making needed to be pushed down to those most proximate to system issues. As such, workstreams needed to reflect a bottom-up approach and staff needed to feel empowered to make decisions or course-corrections quickly. Given the traditional orientation of most government IT functions, this type of decision-making culture required executive sponsorship. Many government executives are unable or unwilling to embrace this kind of decision-making structure, and are therefore hesitant or to endorse agile projects.. Furthermore, when the project gets tough, this sponsorship may not be enough and decision-making may fall back to a more familiar and traditional approach. It is crucial to adopt a risk-friendly mindset and to continuously engage the team making decisions to ensure proper project management.
Overall, the modular approach is time-consuming. From cultivating an iterative culture early on, to ensuring that the organization has the right talent to be successful, the entire process requires a level of engagement that governments must prepare for.
4. What does success look like?
Agile modular procurement redefines what it means to be “on time and on budget.” Without planning milestones up front, how will we assess the progress of the project? What are we expecting from the first delivery? And is everyone expecting the same? How do we know if contractors are working quickly or slowly? How much are we prepared to spend on discovery to avoid a bad outcome that we can’t change? How do we reward experimentation?
Laying out new success metrics is crucial for an institution embarking on a modular procurement project. Each stakeholder (or subset of stakeholders) may have different strategic goals and metrics that determine whether the project was successful or not. Developing success metrics for agile projects can also be challenging, as typically project leaders and sponsors may not have an aligned — or any — definition of what success looks like. Without this, the definition of success may become “don’t make mistakes,” which is counter to the principle of trial-and-iterate. Any instituted metrics must be meaningful and move the project forward while allowing for continuous trials and iterations.
For the CWS case management system, success, at the highest level, was defined as a well-functioning product. Within this, it was important to identify milestones for each phase and across categories to ensure steady progress was made. For project leadership, success included developing lessons learned and finding best practices to replicate elsewhere. At a supervisory level, for the California Department of Social Services, some indicators included lower turnover of senior team members and more efficient talent recruitment and retention practices.
Some of the success metrics don’t change with the transition from waterfall to agile — namely, budgetary and time concerns. Efficient spending and timely results remain critical; the difference is the prioritization of these factors in relation to other goals and metrics. It is important to develop a clear roadmap of these expected metrics, particularly in the public sector, because the use of public funds for a project creates an expectation that there will be specific results. Setting expectations through a clear set of potential outcomes will allow the public to maintain trust in the project.
5. Are agile projects more susceptible to changing external pressures?
Under a waterfall approach, testing is back-loaded, which removes opportunities for the team to improve the product, but also for opponents to intervene or for political winds to shift. With frequent testing, feedback, and iteration cycles, will the team have to spend more energy defending the project from outside? Will regular testing make it hard for the project to make progress out of the spotlight? Although sunk costs are low compared to a waterfall RFP, is there a risk of the whole project losing political support?
In CWS’s case, product clients — social workers, municipal governments, and other key consumers of the system — were very supportive of an agile methodology, as it allowed them to be more involved in the new system’s creation. Because of the high visibility of the project, as well as its scale and focus on helping vulnerable children, some political pressure arose organically. However, that was largely mitigated by obtaining stakeholder support early on, before beginning the project.
Regardless of whether a team proceeds with a waterfall or an agile process, obtaining external support and political buy-in is essential. In the CWS project, the team adapted its process to match the annual legislative budgetary appropriation cycle. This allowed external stakeholders to receive a project update once a year — at a minimum — to check on progress relative to the modular timeline, similar to a traditional waterfall process. In addition, the adaptation provided the project with ongoing momentum: Because the state had attempted the waterfall method many times in the past, clients or external stakeholders remained interested in trying something new and potentially more effective. California’s CWS project continues to serve as a model, given its complexity, scale, and length of implementation.
6. How do we manage risk?
An agile approach removes much of the detailed planning from an RFP and allows flexibility during the project. What safeguards are in place to prevent scope creep? How can someone not involved in the project assess its performance? What steps are in place to ensure problems do not escalate? And are they written in terms that civil servants unfamiliar with agile processes will understand?
When thinking about managing risk and ensuring accountability, the two questions to keep top of mind are: (1) What is left to do? and (2) How are we performing against those deliverables?
In the modular approach used for the CWS project, the state government performed the role of the systems integrator. This necessarily meant assuming most of the project risk. To decrease some of that risk, California brought on Salesforce to ensure that each of the modular units worked in that environment.
The modular approach involves examining and designing each part of the system — intake, licensing, etc. — discretely. As such, the CWS team regularly had to decide whether it could build and integrate a particular portion in-house or whether to bid out that module to a vendor, to meet both client needs and agreed-upon performance metrics. There are pros and cons to this discrete approach: It may take additional time to arrive at a final product, but it also allows each module to be built to the specifications of the user, and permits increased testing and iteration. This tradeoff may dissuade teams from proceeding with a more modular or agile approach. However, such an approach allows the teams to pivot in a way that is impossible with a waterfall approach, including bidding modules out to the marketplace to take advantage of the most updated technology. Conversely, the waterfall approach might require early commitment to a technology that has become outdated by the time implementation commences.
Ultimately, any team moving forward with modular or agile processes must consider who will own the architecture of the project (i.e., the mapping and timeline) versus the development of the project (i.e., implementing the modules and ensuring that they combine seamlessly as one end-to-end system). There will likely be overlap between these roles, but keeping them distinct will be essential to success. In the CWS case, the state played more of the role of systems architect and brought on Salesforce to lead on the technical systems integrator role.
7. Are we prepared for additional accountability?
Waterfall projects frequently run for several years before a course of action is proved right or wrong, rendering accountability for the outcome shared or diffused. Conversely, agile methods quickly reveal whether assumptions or products have the desired outcome, placing accountability on the current team, right now. Can you still be in your role when a project goes wrong — rather than in five years’ time, as in a traditional waterfall project — and not be personally penalized? How does this differ for staff and leadership? What if testing rapidly shows that a technology product cannot solve the problem that was initially identified? Where does the team go next?
Similar to waterfall projects, each modular project has a management structure. In the case of CWS, the project director oversaw day-to-day responsibilities and the governing body for the project (i.e., the director and chief deputy of office and systems integration, the director of social services, and other political or governmental leadership). They also served in a leadership capacity to make decisions. When a project veers off course, the chain of command is contacted to identify gaps or failures. Once bottlenecks are identified, they can be addressed appropriately.
Stakeholder accountability in a modular or agile project structure differs from a traditional waterfall operation. Under the waterfall method, the client is not as involved in the progress being made. However, there is an additional accountability to the final user with modular projects because of the continuous exchange of ideas and information from the user to the project team. This increases the complexity of the project
8. How do we make space for risk-taking?
Agile methods force teams to accept that they won’t always get the right answer the first time — this is why they test and iterate. Accepting small failures is necessary. Nevertheless, nobody wants to see government resources wasted, which gives rise to strong mechanisms of accountability — individually and for teams — when projects go wrong. What sorts of incentive structures set up the right balance between prior planning (whatever the cost) and trial-and-iteration? How and when should teams deviate from plans? Does the organization value an experimentally )rather than policy-based or theoretically) created answer? Is it okay to say “I don’t know — let’s build a cheap prototype to test it”?
It is essential to create a culture in which learning is expected. In this environment, decision-making moves closer to the user and choices based on in-project learning can be made, even where this means changing course from a previous decision.
The CWS leadership team agreed from the start that bumps in the road would be taken in the spirit of learning and that this flagship project would provide lessons for other software developments in the future. Early consensus on this outlook was essential, allowing the culture to persist even in the face of leadership and staff changes. The team recognized the need to be understanding of delays and be willing to spend time and money on prototypes or paths that would ultimately not be pursued.
When moving to an agile or modular process, it is easy to be dogmatic about recommended practices, such as “working in the open” or “daily stand-ups”, while losing sight of the purpose of agile processes — to find the truth quickly. For example, the CWS team developed a module in-house just to say it used open-source, rather than purchase a readily available commercial option that would have better met its needs. However, frequent reassessment allowed the team to backtrack on this decision and in other instances where its plans (even if they matched the by-the-book agile methodology) needed to change course.
Conclusion
The success of new procurement models in the private sector, and, increasingly, in the public sector, is giving thousands of digital professionals and teams the evidence, political buy-in, and enthusiasm to try something new. Across types and sizes of organizations, the way we build technology for governments, and ultimately citizens, is starting to improve. But in each government, often within each department or team, someone must be the first to say “Hey, there’s a better way we can do this.” The nature of challenges these people face is as varied as the personalities, histories and cultures of the places they work — but many of the themes are the same.
Government staff members are regularly busy and stressed doing things the way they have always been done. This alone may explain why it has been easier to apply agile methods to brand-new products or services, rather than replacement of legacy systems. Unfortunately, it is exactly these complex system overhauls that most need an agile, modular mindset. Moving agile from new practice to best practice will take work. Many in government do not have the bandwidth to assess whether a given new practice — which pushes to the limit existing processes and safeguards — will make services better for the citizens they serve. This creates resistance.
It is often said — wrongly — that people are resistant to change. More accurately, people are resistant to change that they do not own or understand. As we transition to a more agile public sector, we must make sure that public servants are part of the change, rather than having it forced upon them. Though we may not have all the answers, we should at least start by engaging with these public servants’ questions, which are frequently challenging, but always sincere.
