Ideas for Overcoming the Federal Government’s Challenges with IT Share-in-Savings Contracting

— by Chris Cairns and Robert L. Read, PhD

This is the third post in a four-part series. In the first post, we laid out reasons for why the federal government should consider revisiting share-in-savings (SiS) contracting as a means to modernize their legacy information technology (IT) systems. In the second post, we gave a brief history lesson on the federal government’s experience with SiS IT. Now we identify 16 challenges encountered and discuss ideas for handling them. Many of these ideas form the basis of our Agile Share-in-Savings Model, which we’ll discuss in the next post.

There isn’t any one dominant factor that you can point to as the reason for the federal government’s initial failure with SiS IT contracting. Some of the challenges were inherent to SiS IT contracting itself, while others inherent to institutionalizing new policies and practices in a large bureaucratic environment. Some challenges even came in the form of political opposition, with watchdog groups going so far as to say SiS contracting is “unconstitutional,” which isn’t surprising given the game-changing nature of SiS. Let’s walk through these challenges and discuss possible solutions within the context of modernizing the federal government’s legacy systems. But first, if you haven’t already, be sure to read our second post for important background information.

Overcoming the upfront-funding-of-termination-liabilities hurdle.

Upfront funding of termination liabilities, which can be a sizable sum, has frequently been cited as a major deterrent to the adoption of SiS. Obviously one solution is to eliminate or soften the existing statutory requirement. That would take act of Congress, which isn’t infeasible given that they’ve temporarily tried to remove it as a blocker before with the passage of the E-Government Act in 2002. Another solution is to negotiate with the contractor to waive the contingency liability clause from the contract, assuming it’s multi-year, which might be unlikely given the risk involved. Recall from our second post, though: termination liability doesn’t apply if payments to the contractor can be funded out of in-year savings. And therein lies the key, we believe: chunk the modernization of the legacy system into bite-sized pieces where each piece independently represents a sufficient return on investment and generates in-year savings. Incidentally, the chunking approach aligns nicely with modern agile software methodology principles.

Clearly specifying the expected outcome.

For SiS to work, you need to be able to identify, effectuate, and track savings or revenues. That initial step requires both the government and contractor to define and develop a shared understanding of the desired outcome, so resources can be directed appropriately. With legacy systems, desired outcomes often revolve around reducing the cost to change and maintain the system. Some exploration is required to determine why the system is bad, how to fix it (refactoring code, eliminating unused features, re-architecting, etc.), and whether the opportunity exists to generate hard benefits.

Those are important questions in which the government has typically relied on external third parties to answer. Through the use of in-house, trusted technical agents such as 18F or the U.S. Digital Service (USDS), agencies can develop much crisper definitions of the expected outcome, particularly from a technical standpoint. By no means are we suggesting to exclude contractors in the process. They should be collaborative participants as well since they’re assuming the most risk. But what we are saying is that the process of defining expected outcomes could be improved when some of the smartest technical people “in the room” are government employees. The technical services of federal employees can be obtained with interagency agreements without a competitive bidding process for those services, greatly easing the complexity of using third-party technical expertise in procurement.

Defining mutual incentives.

Assuming an opportunity to use SiS exists, the government and the contractor need to strike the right balance between the level of risk and reward they are willing to pursue. In some cases, the government may not wish to invest any funds upfront in exchange for giving the contractor bigger returns. In other cases, the government may wish to retain a larger portion of the generated benefits by investing some money upfront. As for the contractor, there are real risks involved if the benefits aren’t realized as anticipated (magnitude and timing), so they may be unwilling to accept a pure SiS arrangement. Both parties need to work through the issues and decide how far to take the arrangement.

Establishing performance measures for making payments.

How do you determine what portion of the savings or revenues should go to a contractor? In general, you need a baseline of the system’s current costs or revenues, and good performance measures for gauging what financial results are being achieved by the contractor against the baseline. For services such as energy management, this is relatively easy, but for IT, it’s much more complex. You need a good cost accounting system, an area where federal agencies struggle. This is particularly true for tracking internal-type costs such as how government employees are allocating their labor across various activities. For external-type costs such as contractor labor, there are definitely more reliable accounting systems in place. (Vendors got to get paid somehow!) Systems that rely on revenues through end-user fees typically have better accounting mechanisms as well, based on our experience.

While the government should definitely take measures to improve accounting for internal costs, we don’t see it as a blocker to the immediate usage of SiS, as some have suggested. The main reason for this is that because improving government labor cost efficiency doesn’t actually translate into hard savings payable to a vendor, a world-class cost accounting system isn’t required. Of course, elimination of whole positions is relatively easy to account for, but that’s not an option that agencies are going to consider (Congress explicitly ruled it out under the E-Government Act share-in-savings pilot). One exception to this “don’t-need-no-good-internal-cost-accounting-system” argument might arise when there’s an opportunity to reallocate a government employee, whose labor is spread across multiple activities, to a new role to avoid hiring someone else. In this case, you’ll want accurate accounting of that person’s time. Ultimately, all this discussion leads to the suggestion that the best place to focus SiS efforts is on systems that are either revenue based or have pre-existing operations and maintenance (O&M) budgets with a high proportion of external costs. The baselines are more easily quantifiable.

So what measures do you use to link a contractor’s intervention to savings or revenue produced? The answer to this question is one of the core features of our Agile Share-in-Savings Model. Measurements and payments are based on the use of highly technical judges who are aided by decision-support tools (for example, automated code quality measurements), verifiable savings or revenue, and the application of economic principles in which a contractor’s payout depends not just on their individual performance, but other vendors as well.

Generating a large enough pool of benefits relative to the costs.

The SiS model completely breaks down if the ratio of benefits to costs is low. You want to target systems or components thereof that could generate a high return on investment. Systems that are fee-driven and could generate even more revenue through modernization are perfect targets. For example, Electronic Immigration System and USAJOBS both charge and are funded out of end-user fees. Tax collection systems generally fall into the category of revenue-based systems as well. Another place to target is systems with pre-existing O&M budgets that have one or more of the following characteristics: high annual sustainment costs relative to the total number of source lines of code, expensive components that could be replaced with cheaper commodity solutions (think open-source and cloud alternatives), or a high technical debt ratio.

Technical debt ratio is a metric that expresses the number of estimated person days or costs to pay down accumulated technical debt relative to the size of the system (in terms of source lines of code). This metric allows you to compare the quality of systems across the enterprise, giving you metric-based visibility into systems with the greatest potential to generate high returns. Technical debt ratio can be calculated manually, but thankfully there are a few tools that do this automatically. SonarQube is one example, which is open source, free, and supports over 20 programming languages.

Beating the learning curve.

Lack of policy guidance and training were key reasons why SiS contracting didn’t take off under the E-Government Act pilot program. It’s a complex and unfamiliar technique to the federal IT acquisition community. So educating them on how to use it is critical to overcoming the inevitable forces of cultural resistance. We think that the government had the right idea in creating a center of excellence for share-in-savings within GSA back in 2003. To tackle the aging-systems crisis using SiS, the government should establish a similar program with the purpose of providing outreach, guidance, training, tools, and on-the-ground expertise to agencies. In addition, the program office, which could be housed somewhere like 18F, should pair both acquisition professionals and legacy system engineers together.

Forming a collaborative partnership between the government and contractor.

As we discussed in our second post, one of the major reasons that the Department of Education succeeded with the government’s only attempt at SiS IT was because they forged an equal partnership with the contractor. The government often has the tendency to treat contracts with vendors as transactional relationships, which is a formula for sure-fire failure with SiS. In fact, modern agile software methodologies in general have shown success by moving away from a hands-off, transactional approach. Success requires high-functioning collaboration, featuring a fully integrated and empowered team, shared vision of the project, open knowledge and information sharing, and trust. There are a number of things that the government can do to form a collaborative partnership. This includes, but certainly not limited to:

• Involving the contractor in defining the desired outcome and mutual incentives, establishing a cost/revenue baseline, and estimating the projected savings/revenue
• Providing access to system-related information and assets such as budget data, source code, and architecture diagrams
• Including the contractor as a full member of the integrated project team
• Giving the contract an equal seat at the leadership table, which includes decision making (after all, they’re the ones assuming the greatest financial risk)
• Setting up collaboration tools

Avoiding an increase in direct spending.

Citing a Congressional Budget Office study, opponents of SiS argue that it would actually lead to an increase in the government’s direct spending. For this to be true, that would mean that the contracted effort either only produces soft benefits (for example, a process improvement with no corresponding budget reduction or an increase in revenue) or fails to generate any hard benefits whatsoever. The solution to these two scenarios is simple: only undertake projects with a good probability of generating hard benefits. Legacy systems with pre-existing operations and maintenance budgets are perfect targets. In addition, only tie payments to verifiable, not estimated, benefits. In other words, the government should only be paying a contractor for demonstrated results. In terms of agile software development, that means code which is “done,” shipped, and delivering value.

Affording opportunities for small business participation.

Opponents of SiS have argued that small businesses lack the financial wherewithal to risk investing their own capital in “speculative work” and therefore only large businesses can compete for contract awards. In other words, it creates a non-competitive environment. We absolutely agree with that point in a scenario where the government competes a huge, monolithic, single-award contract. But the government has surely learned the ill-effects of such a strategy. Instead, the totality of the work required to modernize a legacy system should be broken into small, independent, and modular chunks — both architecturally and contractually. This has the effect of leveling the playing field for businesses of all sizes. Of course, the government can always use a “cost plus profit” contract structure, as opposed to “profit” only, to keep the doors open to smaller firms. Other solutions exist as well.

Ensuring payments to contractor adequately reflect risk-reward.

One of the potential issues that can arise with gain-sharing contracts is if the contracted effort produces a larger-than-anticipated amount of savings or revenue, allowing the vendor to receive too large a windfall. “Watchdog” groups have argued that the possibility of such an inequitable outcome is yet another reason SiS is bad. This risk can largely be avoided, however, by adhering to the principle that firms should only be compensated based on decreasing financial risk to the government — within reason. One way to put this principle into practice is by developing risk/reward models and/or involving experts to evaluate the risk/reward reasonableness of vendor proposals as part of the award process. Another way is to employ the use of reverse auctions and let the “market” determine what the appropriate risk/reward ratio is. A final way is to cap the amount of compensation that’s paid out to a vendor, which is the path that we chose in designing our Agile Share-in-Savings Model. Of course, some combination of these options — and certainly others we haven’t thought of — could be used together.

Complexity of setting up and administering SiS contracts.

In comparison to run-of-the-mill techniques that acquisition professionals typically use, SiS IT contracting is more complex. Not only does it require acquisition and technology expertise, but finance as well. (Finance expertise, for example, is needed for developing procedures that allow for the collection and distribution of any accrued savings.) Government has long considered “integrated project teams,” which are focused, cohesive, and multidisciplinary, an essential practice to managing acquisitions effectively. We absolutely agree, but don’t see it put into actual practice enough. Without fail, every SiS IT acquisition should include a fully integrated team of cross-functional specialists from acquisition, technology, finance, and other areas as needed. If this team cannot be staffed from within an agency, the agency should consider going outside for additional specialists, such as those offered by the current innovation trifecta of 18F, the USDS, and the Presidential Innovation Fellows.

Accessing qualified vendors.

Vendors willing to participate in SiS arrangements on legacy systems need to be qualified for their agility, ability to modernize legacy systems, and financial wherewithal. In combination, these are difficult qualifications to meet and too important to risk misevaluating. Creating a curated marketplace of vendors who are pre-vetted by in-house technical experts such as 18F or USDS, and who are accessible to all federal agencies via a government-wide contract vehicle such as IT Schedule 70, is a sure-fire approach.

Minimizing contractor risk.

While it’s true that under a SiS model vendors should be compensated for decreasing financial risk to the government, that doesn’t mean the government shouldn’t do what they can to set vendors up for success. SiS is a two-sided market, and it needs to work for both parties. There are a number of things the government should do to minimize contractor risk:

• Aim to develop collaborative partnerships with vendors, which we touched on above.
• Insist on the use of modern practices such as agile software development, which is an inherent risk reducer in comparison to its evil predecessor methodology — waterfall.
• Internally employ highly-skilled technical resources who can help, among other things, break the modernization of legacy systems into bite-sized chunks and write automated tests for those chunks. This would allow vendors to evaluate how difficult rewriting the chunk would be and for them to be sure when they had done a good job.
• Encourage the use of cloud and open-source technologies. There are many uses cases now in which these technologies are proven, low-risk commodity alternatives. Not only could they greatly increase the pool of benefits paid out, the use of open source could increase “transportability” and decrease vendor lock-in, thus making “within year” savings more reasonably rollable into competitive contracts for coming years.
• Apply economic principles to the design of acquisition structures in order to incentivize good cooperation between vendors, which isn’t always the case now. Imagine what could be achieved if you turned self-interested vendors into altruistic vendors because they had more to gain by doing so. As you will learn in the next post, this is a key feature of our Agile Share-in-Savings Model.

Creating incentives for agencies.

The E-Government Act included a temporary provision that allowed agencies to retain a portion of any savings generated through use of SiS contracting for reinvestment in other programs. In addition, the retained savings functioned as “no-year” money. Typically, agencies only get the funds they need through the annual appropriations process. And those funds have to be spent in the same fiscal year (“use it or lose it”). In general, the Congressional budgeting process is inflexible and not always guaranteed. So the option of self-generating funds that can be spent on whatever and whenever is a powerful incentive. Unfortunately, the provision wasn’t enough to overcome other adoption inhibitors, namely cultural resistance, and it expired in 2005. To give SiS the greatest chance of success, this provision should be resurrected but with an important self-reinforcing constraint: some of the retained savings (or revenue) must be placed in an escrow that is reserved exclusively for paying out vendors. (More to come on that in the next post.) Of course, sometimes just modernizing an old, crusty, snarling system will be motivation enough, regardless of any retained-savings carrot.

Leadership support.

In addition to an organization’s culture, leadership support is a critical factor in determining whether federal agencies will embrace disruptive change. That leadership support must extend from the administrator of the White House’s Office of Federal Procurement Policy down to the executives at an agency level. Those executives must empower integrated project teams to operate independently and to make tough decisions. There also needs to be strong ambassadors for change, and a SiS center of excellence is well suited to playing that role and enlisting others throughout the government to take up the charge.

Violating the Constitution.

Detractors of SiS have argued that it undermines Congress’ constitutional authority over appropriations and its oversight function by giving agencies the power to enter into contracts with unappropriated funds. This is a moot point because Congress has already set the precedence for giving agencies flexibility to manage funds on their own in the form of revolving funds such as the General Services Administration’s Acquisition Services Fund. And there’s plenty of oversight on how these types of funds are used. But even if we assume it is unconstitutional, surely a lightweight process could be put in place to approve how agencies plan to expend “unappropriated” funds on SiS contracts.

Wrap Up

If you made it this far, give yourself a fist bump 👊. We covered a lot of ground, discussing how challenges surrounding the usage of SiS IT in government could be surmounted. There isn’t a one-to-one correlation between all the ideas discussed here and our proposed Agile Share-in-Savings Model, but you’ll certainly see many of them reflected.