The Commercial Scrum Conundrum
Having worked both client and vendor side, I want to write up some of my experiences and thoughts on the commercial challenges involved with Scrum. I’d like the community to jump in.
This post does not cover in-house development, only cases where clients work with vendors applying Scrum.
It feels counter intuitive to be talking about contract negotiations given the Manifesto, but I’d love to hear your take on some of the notions put forth.
How can clients and vendors, find a perfect contractual fit to Scrum?
Tricky, nay. Scrum doesn’t prescribe any contracting model or approach. The Agile Manifesto holistically tells us to value customer collaboration over contract negotiation. So how are agencies and clients supposed to approach this conundrum? Surely, it is very painstaking to convince clients of new contract models during acquisition. Surely you don’t want to get bogged down in contract negotiations, or scare your potential client away with the trouble of comprehending these non-traditional models.
All this is just a train of thought. By no means I want to promote any suggested approaches as a recommended practise, but I would love to exchange my thoughts and experiences with you, the community.
Pay per Sprint.
Agencies generally have ‘hour/effort’, ‘scope’ and time commitments to clients. The Iron Triangle cliché. Poor souls. With Scrum, this is often broken down into Sprints.
The agencies explained to me that they generally find it ‘impossible’ to convince clients to agree to open ended Sprints. Sure enough, when I put forth the suggestion the response is often a blank stare.
Even bringing it up is often out of the question by default. According to them, clients want to know upfront what to expect. Developers would too.
So here is what commonly happens: Per Sprint, the client pays x amount for y Product Backlog Items or Stories (Forecast) to be delivered in its timebox. Each iteration is checked for timely delivery of the negotiated scope. This is, as both client and vendor will argue, because naturally the client will need these guarantees for it to commit to its investment. The number of Sprints is generally predetermined up front and progress is measured carefully through a ‘release burn down chart’ as scope is realised and gets renegotiated over time. The forecast effectively becomes a commitment, and the Sprint’s timebox a deadline. This should be avoided as this makes the team rigid, not adaptable. It is healthy if there is still work remaining on a Product Backlog item (PBI) at the end of a Sprint. You could perhaps introduce the concept of: 90–100% chance team completes x PBIs, 75–90% for y PBIs, 50–75% for z PBIs etc.
I’ve come across contracts that contained a backlog of requirements for over fifteen pre-defined Sprints, without hardly allowing any room for flexibility in the plan. In one particular project, the team (valuing customer collaboration over contract negotiation) responded to change during the series of Sprints following instructions from the Product Owner. All great and all for the client, until, near the very end of the series, the Product Owner’s CEO started referring to all the work that wasn’t delivered according to the (dated) backlog mentioned in the contract… The CEO was still expecting the delivery of pending features mentioned in the original backlog, including all the changes, for free. Trust went down the drain and painful negotiations and delay of payment ensued. The contract didn’t have a clause about changing scope. The team, or the Product Owner, had never seen the contract.
Some agencies told me that it’s rather common for clients to withhold payment until all items for a Sprint had been accepted by them. This often involved numerous revisions and energy draining scope/bug discussions. None of the players generally enjoy this exercise.
I also witnessed cases where agencies, when completing work sooner, would consider the remaining hours to be credit to the client (the client isn’t charged for effort not spent), where, when finishing later, the remaining effort had to be made at zero charge (the client isn’t charged for effort spent). Makes sense from the client’s perspective, but it doesn’t exactly motive the vendor to deliver anything ahead of schedule does it? Sure, some additional clauses might resolve that.
Estimates or #noestimates?
Should each Product Backlog Item be estimated for complexity and value, so cost can be predetermined? This is a seriously hot topic when it comes to Scrum.
The Scrum Guide gives us this.
“Product Backlog items have the attributes of a description, order, estimate, and value.” — The Scrum Guide
And that
“More precise estimates are made based on the greater clarity and increased detail” — The Scrum Guide
But it also tells us that
“In complex environments, what will happen is unknown. Only what has already happened may be used for forward-looking decision-making.” — The Scrum Guide
It may be very tempting to base commercial agreements on these estimates. But, as mentioned, this effectively turns the estimate into a series of smaller commitments. One might argue this to be a good thing (certainly from the client’s perspective), as “Risk of investment would be limited to a single Item”.
An estimate is not a deadline and are by no means meant to be applied as timeboxes (maximum amount of time to be spent on an item). What happens if it’s more or less complex than expected?
Developers (and Product Owners) will likely relate to ‘Charlie’s sausages’ scene from Snatch, thus I’ll cheekily dub this phenomena the Charlie’s sausage effect.
I witnessed workspaces where estimations are discussed and (re-)negotiated daily. I’ve witnessed developers suffer reprisals for ‘bad’ estimates. Developers had to report and justify, in detail, how they spend each hour (or fifteen minutes even). Underestimation generally results in overtime. Talk about pressure cooker environments.
Estimating, re-estimating and constant commercial negotiation with the whole team and client on item level is an intense effort and may even be wasteful and painful. This is a strain on any relationship.
What personally baffles me is why Software/Product Development is so obsessed with estimating work (almost to task level) compared to other professional services. Timetracking is often only expected from Software/Product Developers in a team or organisation. Why this discrimination?
In one case I actually even witnessed a Scrum Master setting a timebox for a developer to complete a ‘ticket’ in. The ‘Scrum Master’ would sit behind the developer, with his mobile set to stopwatch, monitoring his performance, licking the sweat from his neck [exaggeration].
One might argue that a professional developer should be able to explain how much it costs to build something like a carpenter might. Software developers are not creating something they already created in the past. This is not saying they can’t make good predictions. The challenge is that these predictions can, and will be used against them, no matter how jolly, fun, agile and ‘safe’ the organisation pretends to be.
Even using relative estimation using a Fibonacci sequence through Planning Poker, becomes somewhat debatable if a commercial figure is applied to it, even more so when time spent on it is being tracked.
The practise of asking developers for estimating Product Backlog Items indirectly makes them somewhat accountable. Developers will fear giving estimates. Wrong estimates might cause pressure to the team and frustrate the client. At times a developer overestimates themselves and thus underestimates the work. Developers in this case, might also, indirectly, estimate what a client might find reasonable. In many occasions developers are asked to justify their estimate in terms comprehensible to those not familiar with Software Development shenanigans. Then ultimately, when asked to explain their mis-estimations, we can expect a series of “yeah.. but’s” from Developers trying to put into layman’s terms the complexities they couldn’t foresee themselves. It almost feels like they are justifying a mistake, rather then celebrating valuable new discoveries. Developers are not motivated to trying out better approaches they might discover. They will take shortcuts. Shortcuts make long delays. When discovering a better approach they could have taken, rather than going back, and doing it right, they’ll continue to make due with what they wrote. Certainly in Software Development it is known that this attempt at maintaining progress, will slow them down. The scrumundrums drum on and on.
Operational expenses.
Products involve both new development and maintenance (which really is also new development) and upkeep too. Products emerge and either evolve with the market, influence markets, or otherwise fade from it.
Clients I worked with generally love to get securities up front and sign away the risk and cost of failure to the vendor. They love using ‘car’ analogies (I worked with several clients from the automotive industry). For example, “When I buy a car I should be able to know the costs of a car with y tralalas upfront”. The problem is however, with software development, the ‘product’ (car) isn’t already there. Correcting the analogy, the client is asking for a complete ‘custom dream car’ to be constructed based on vague requirements/preferences.
The challenge with clearly specified requirements, on the other hand, is that they create the illusion that all is known. “Everything we expect to have is in there”. This is an assumption that might get both client and vendor off on the wrong foot. This might make them blind or non-adaptable to the realities that development reveals over time. When reality conflicts with expectations this causes friction (after all, we all agreed the plan was perfect right!). Friction (‘a force that resists relative motion’) incurs and slows down development. Friction too (‘as in a clashing between parties’) tenses the relationship. So perhaps ironically, I’ll argue that the more requirements appear to be clear, the bigger risk of rigidness (inadaptability).
Clients are generally paying for the effort to having software developed and for the cost of consumption/utilisation and upkeep. However, many clients will persist to models that are meant for products that just need assembly and configuration.
The fixed time-scope-cost model based on estimates is just a fantasy that buys none of the guarantees that it pretends to. Not even when narrowed down to a Sprint or even a Product Backlog Item.
Money for nothing, change for free.
Scrum Inc. shares the model ‘money for nothing, change for free’. This is based on predetermined definitions of value and estimations of effort with a 20% room to wiggle. Clauses include active participation of the client, but more interestingly: The customer may terminate the contract at the end of any Sprint in return for paying the vendor 20% of the value of the remaining scope defined. It also includes the clause that enables to client to make changes to the scope defined in the contract, if the total scope of contracted work is not changed. Valuable changes are enabled for free and the risk of early termination is shared (to an extend).
This model relies on the reliability of estimates. It requires a good sense of whether or not the total scope defined, will in the end deliver a satisfactory result for the client. It requires a predefined (estimated) total scope figure, which reveals it is likely not taking maintenance and continued development into consideration. Does the Product, and thus all activities on it, cease to exist when that guessed number of scope is delivered?! Nope, but surely, one might argue, that is something to consider at that point in time, right? … hmm.
What is scope?
Often clients do not know the extend of the scope themselves, yet still expect the vendor to accept the risk of committing to a budget. Clients want to set a total budget for a total package, even if the contents of that package isn’t clear at the time. They’ll ask for ballpark figures or ‘guestimates’, but once given, they are not treated as such by the client; they are now treated as commitments. They’ll expect all of that undefined and unrefined scope (that big mystery box) to be delivered for that given figure. Vendors all too eagerly accept this risk, in order to reel the client in. In a short time a Development Team will run through unrefined requirements with the client and make due with what is available to try and determine its estimates (which will consequently be treated as deadlines).
Now when development is in full swing, this client is suddenly able to become specific. It’ll cramp as much requirements it can under that vague ‘epic’ in the backlog. As the backlog refines, the work suddenly piles up. Yup…scope creep.
I’ll dub this phenomena the Mandlebrot effect of software development.
Emergence.
At the very heart of this conundrum, that which sets both the client and vendor off on the wrong foot, is the whole idea of setting a fixed budget and timeframe (even for Sprints) for efforts that are supposed to result in an emerging product. Product Development, in my opinion, should (nearly always) be considered as ‘operational expenses’. Investments that continue to flow as long as the product lives, which will return value as long as it lives. Investment may flux, such as the return of value may flux over time. Only time can tell how this plays out.
So how can a vendor and client get off on the right foot?
A flexible commercial strategy?
Hmm, now this sounds risky to a client. Surely clients generally desires certainties upfront (who wouldn’t), even if this is only fantasy. Clients also don’t respond well to what they’d argue to be ‘creative commercial models’; even more so, if there is a purchasing department involved. When confronted with all this new Agile stuff with strange new commercial concepts during acquisition, the client might again… make a run for it and try their luck with the next vendor.
Let’s back up a bit and return the the Scrum Guide. It can give us some good guidelines to help us approach this conundrum.
“Sprints also limit risk to max. one calendar month of cost.” — The Scrum Guide
This concept is great for the client. This prevents the client from having to take a big risk with a large budget over and extended period of time. This is especially great if the overall scope isn’t really all that clear. The client also stays in control of the spending throughout. Kinda like backpacking. This way spending can be validated throughout.
Also we may argue to the client, that with traditional projects, changes are a risk. Though with Product Development, adaptability is valuable.
In practice (and this is only based on my personal experience), there is a ‘poker’ conundrum in effect. As the client is already invested, folding might not feel like an option as it equals to failure. They might not be inclined to pass or fold and try their luck with a fresh new hand. They’ll continue to place their bets even with a shit hand. Hey, after all, perhaps their bluff will prevail.
Let’s return to the following stance in Scrum:
“ Only what has already happened may be used for forward-looking decision-making.” — The Scrum Guide.
This is tricky for both the client and vendor. The vendor hasn’t in all likelihood developed the exact same product the client envisions. So how can a decision be made about expected cost? Now a Scrum organisation may already have a history of teams running Sprints. This past performance can help. A team may even demonstrate what is has delivered in Sprints in the past. This could even be demonstrated in more abstract forms like an x number of backlog items, or x value or effort.
A vendor could share a projection given the scope that is clear (which to me is a myth), and given the known past performance of a team, disclaiming that this should be re-assessed this each Sprint. I personally however, have not experienced very stable teams with stable velocities. Is it me? or is Software Development complex?
This is often already a bit too much for a client to comprehend in the early stages of the relationship and the client will likely care little for it.
Here is what the Scrum Guide says on decision making and risk:
“Scrum relies on transparency. Decisions to optimize value and control risk are made based on the perceived state of the artifacts. To the extent that transparency is complete, these decisions have a sound basis. To the extent that the artifacts are incompletely transparent, these decisions can be flawed, value may diminish and risk may increase.” — The Scrum Guide.
Given this, it might be best to just be honest with the client that Product Development, by nature, is complex. It requires continuous alignment and each component that is to be developed will likely (or perhaps surely) require a few revisions.
So why not agree with the client to track the investment as well as return of each component over time. This increases transparency and decision making over time, thus reducing risk.
This also could make the client aware that, with Sprints, features could be released early and start providing return. It might help to set priorities straight and get the client onboard with MVP or RAT thinking. It might make the customer more aware of ‘the cost of delay’ which would be traditionally the case when “done” features just sit and wait for the total sum of them to be complete in order to be releasable at the very end of a lengthy project.
Now I didn’t just make this up. This feature/component driven ROI model is fairly popular for digital services. This does however generally require upfront definition of how that return value (conversion) is to be measured. With digital products this is fairly easily done. The partners could agree on a relative unit of value per conversion in a similar way as a development team estimates effort and complexity in relative units. This doesn’t change much of what we previously talked about, apart from that we are now approaching Evidence Based Management. Hurray! Now this isn’t to say that EBM (and all its number crunching) doesn’t tempt organisations to it’s own world of potential dark, micromanagement practises and artifacts, but at least now the pillar of ‘Inspection’ is somewhat upheld.
Shared risk, shared return.
Both vendor and client might opt to share in risk and success. In the extreme case, a client may not even pay for development at all, but only pay for utilisation (pay x amount per unit of value returned). Both client and vendor will have to develop a somewhat equal level of trust in the endeavour, but in this extreme case, the risk nearly entirely rests with the vendor. The client only shares in its success, while the vendor invests. This is generally a no-go for the vendor. So perhaps it is not strange to suggest an x% of the risk of development is shared by the client too. This can only work with a termination clause.
Are we getting any closer?
If you want to experience the advantages of Scrum, you cannot expect to achieve this by not adapting the way you approach the relationship defined in a contract. It requires an open and transparent talk.
The concepts I want to talk and exchange thoughts about are these:
- The client agrees that this endeavour involves development, they are not buying a pre-existing product at a fixed price, but an emerging product at fluxing investment and return. Both client and vendor agree that costs of effort that will be made throughout the product’s lifecycle and that those costs may flux.
- Each component of the product will be carefully measured to enable better future decision making and iterated to enhance its performance.
- Both client and vendor, as partners, agree to sharing in the success of the product as well as in the risk of development.
- Client and vendor agree to terms of early exit.
- Client and vendor agree to where the ownership of the product ultimately resides, and the value (and cost) of that ownership.
Pay Per PBI (or Pay Per Story)
Now, a time and materials contract will not work with client who insist on getting fixed price scope commitments. So when this is becomes a strain on the relationship try a Pay Per PBI (Product Backlog Item) model. A fixed price for a (small) item to be charged once it meets its test definitions and definitions of “done” (regardless on time spent). True, the vendor assumes the risk of investment regarding effort spent, but this risk is contained to a single item and could work in the vendor’s favour too. Win some lose some. Scrum’s empirical insights will make effort spent vs value delivered transparent and these insights could be used by the vendor in its calculations of future Product Backlog Items.
What could be valuable to the client?
- No big risky budget commitments up front.
- Early and continuous validation.
- Increased ability to adapt to market throughout.
- Valuable changes are possible throughout, without the need for constant renegotiation.
- Earlier releases/time to market (reduces cost of delay).
- Vendor‘s commitment to product success as they’ll ultimately reap from it too.
What’s could be it for the vendor?
- No fixed time/budget/scope commitment.
- And thus less constant bickering about time/budget/scope/bugs.
- Hands-on involved client.
- More control over what gets built.
- Will share in product success.
All this is still indeed a lot to process.
With all this I haven’t even touched the area of Service Level Agreements to upkeep and maintenance.
Trust and Partnership
All in all, perhaps we should return to the the basic concept of trust and partnership. Let’s make it simple: a preset rate per sprint, not based on either hours nor scope, but tied (to an extend) to the product’s ROI. Alternatively, go for Pay Per PBI limiting the risk to a small defined container of Scope. In any case it has to be based on a flexible operational expenses, not project based with a fixed priced budget.
Less hassle. Less administration. Less negotiation. The client will work with the team to improve their output over time, validating ROI continuously. Scrum’s pillars on transparency, inspection and adaptation come to life. Sure it takes patience.
Players from both vendor and client will work together as a joint-team, on a daily basis. This leaves us with the question on how to gain that trust early on. But perhaps trust is not earned… it’s granted. As a client, in order to grant it, risk needs to be low and shared. The only way to keep it low is to continuously inspect and adapt. So as a vendor, make the first Sprint worth it, by solving the clients toughest concern and their riskiest assumption first… by doing it, not by negotiating it. Send developers, not negotiators.
Here is a webinar on how Scrum.org approached the client-vendor relationship for its own digital platform.
Check out this great follow up read on the conundrums involved the Product Owner resides in this relationship. By João Craveiro
As mentioned, all this is just a train of thought. By no means I want to promote this as a recommended practise, but I would love to exchange idea’s with you, the community.
