Government procurement needs to change — and so does the Digital Marketplace
I’m a buyer on the Digital Marketplace
In the year I’ve worked at the Ministry of Housing, Communities and Local Government (MHCLG), I’ve run more than 10 procurements using the Digital Marketplace. Most of these are on the Digital Outcomes and Specialists (DOS) framework because I usually want to ‘find a team to provide an outcome’.
A quick note about the word framework: it means an ‘agreement between the government and suppliers to supply certain types of services under specific terms’. I thought about removing the word from this post, but I think it’s important so instead I’m going to use it repeatedly.
I’m what the Digital Marketplace team calls a ‘buyer’. This user group also includes the commercial colleagues I work with regularly when we run these procurements and the relevant policy/operational colleagues who need digital support for their policy. My main job is to deliver great digital services that meet users’ needs and are good value for money for the taxpayer.
My needs as a buyer on the Marketplace are therefore:
- to access at a competitive price either a) talent I can’t recruit into the Civil Service or b) specific products or services, so that I can deliver services and have them be good value for money
- for the procurement to take as little time as possible, so that I can meet my deadlines and not have to spend too much of my time doing admin
The Digital Marketplace frameworks, particularly GCloud, are a massive step change towards meeting these needs when you compare them to the way government has traditionally bought things. They have made a significant difference in the way we deliver digital services in government and I’ve personally hired some great teams, which I couldn’t have done without the frameworks.
DOS in particular hasn’t gone far enough — yet.
This is fine.
Changing government procurement is a big job. If we’re to be agile, we understand that we need to change things in small steps. It’s OK as long as we keep changing the service to respond to user needs and to remove pain.
Obviously, I understand some pain can never be removed. Suppliers have different (sometimes opposing) needs to me and the government has rules to try and make sure procurement is fair. However, it feels like it should still be possible to make a buying process so good people prefer to use it. It should be possible (and desirable) to make competition beneficial for the buyer as well as the supplier, after all I want the best price and the best supplier.
I’m not sure that we are continuing to change things enough, though. As a buyer, I still feel a lot of pain.
My pain points as a DOS buyer
Procurement as a team sport
I’m not going to go into this too much as I know the Marketplace team are fully aware of it.
Each Marketplace account is linked to a single username and password, which is a Minimum Viable solution to the need for all opportunities (adverts) to be posted by someone who legitimately works in the public sector. Only that person may post the opportunity and — worse — only that person may answer any supplier questions. The user researchers on the Marketplace know that users in government work in teams (commercial, policy and digital) who draft opportunities together, but implementing team functionality has never been a high enough priority for it to be built.
In some cases I gather all specialisms to sit at a computer and we spend two hours drafting the opportunity together directly into the Digital Marketplace — this works well for an uncomplicated requirement like a discovery. (Co-located multidisciplinary team! Success!) More typically, we pass a Word template around via email, people work on it in track changes, and our commercial lead copies and pastes the text into the Marketplace from his single account.
This lack of functionality could potentially also encourage poor user behaviour such as sharing passwords or creating an account with a joint email address.
The Goldilocks effect
Previously invitation-to-tender documents were very long. A DOS opportunity is very short and focused. This helps me spend less time doing procurements, which is good. However, there is a significant difference between hiring a single person, buying a discovery, and buying a beta development team––both in terms of what’s expected from the supplier as well as how much information I can provide as a buyer. The Marketplace treats a person, a discovery and a beta as essentially the same thing.
What we’ve got at the moment is about right for a discovery. If I’m hiring a person with specialist skills, personally I’d prefer to get their CV (which might also be easier for them). If I’m hiring a team for a beta, I probably have to link off the Marketplace to lots of different pieces of information hosted elsewhere.
CSV download
Your opportunity is live for 2 weeks. Suppliers apply. Then the single person who has access to your opportunity is able to download a CSV of all the applications. The CSV is not easy to edit and has not been formatted in a way that helps buyers rate supplier responses.
After this point, you’re essentially on your own. The Marketplace won’t help you again until it asks you who won, but your pain as a buyer has just begun.
Sifting problems
There are so many of these that I’ve broken them down.
Sift: Lots to read
My primary need as a user is to get a good supplier, so I should be happy that there are so many suppliers on the framework and so many of them applied for my work. Yes — and at the same time, no.
If lots of suppliers apply, then my secondary need (for this not to take too much of my time) will not be met.
For discoveries we often get 30–50 interested suppliers. Each of those suppliers will write a 100 word response to each of the 5–10 criteria buyers have selected. For each procurement, at least three people sit down and rate each of those responses on a scale of 0–3. (Up to a day of work for each person.) You could argue that it’s our job to spend time making the best decision, which is true. (I also don’t want to put anyone off applying for our work! While this is the system we have, I want to use it to its maximum potential and I will put the time in.)
Does spending time sifting these responses necessarily generate the best decision, though?
Sift: Quality responses = quality?
Suppliers have lots of opportunities to apply to. Some have teams of people who are specially focused on responding to Digital Marketplace opportunities — they may well write a better, more tailored response than a smaller company. Some suppliers seem as though they are copying and pasting stock answers, perhaps because they’re so busy building great digital services for someone else that they didn’t have time to craft a beautiful response for me.
Sift: Evidence isn’t even helpful
The Marketplace guidance for suppliers states: “Buyers find it easier to understand how your skills and experience match their needs if you provide evidence that includes specific examples of work you’ve done or skills you’ve used in the past.” It says you should give only one example and suggests a Situation-Task-Action-Result (STAR) format, which was until recently also used for Civil Service competency-based job applications.
If, as a buyer, you’ve made the mistake of thinking the Marketplace should be an easy-to-use service and you don’t need to read the guidance, you may not even know this. You may think, ‘the supplier has only done X once!’ This means many buyers may be marking suppliers unfairly.
While the STAR format makes some sense for individuals applying for a job in the Civil Service, it’s problematic for outsiders who aren’t used to answering in this way. I’m dubious it makes any sense for companies at all.
I ask whether you as a company have experience of working in the NHS. You tell me that, as a company, you have provided a team who worked in the NHS and did a fantastic job, so I sift you into the shortlist stage. You may then provide me with a team of people who did not work in the NHS (although your company has that knowledge).
Because I’m desperately trying to differentiate between 40 suppliers somehow, I’m probably going to use nice-to-have criteria, which means they stop becoming nice to have and start being essential. This means that if you are a supplier and you don’t have that experience, you shouldn’t apply — even if you are overall a better fit for my work.
Sift: No visibility of cost or team structure
This means I can’t choose to focus on suppliers who are very inexpensive but might not meet my requirements, which I understand. I’d like to have some rough idea whether I’m sifting in only the most expensive suppliers, though. It doesn’t seem value for money for this not to be a consideration at this stage.
Presentation doesn’t meet needs
Most buyers like to meet the supplier team who we might be working with — problems are solved by people after all.
DOS gives you the opportunity to ask for a presentation as part of the evaluation process, but states the presentation should be focused exclusively on cultural fit. This means that you often end up listening to a presentation about how much suppliers like agile, when really what you want to know is who they’re going to put on your project, how they’re going to do your work, and whether those people are likeable and any good at their jobs.
I know I’m not the only buyer who actively requests that the proposed supplier team come to this presentation, rather than an account manager or salesperson. However, I also know (now) that this desire directly conflicts with supplier needs and may not even be possible. The people they want to put onto the work are likely to be working on another client’s project during the presentation and may not be available. Given that the work has not yet been won, suppliers may not even know who will be in their team.
Time
I mentioned this in the sift section, but I’m pulling it out as a separate pain all of its own because it’s so significant.
Reading supplier responses takes a day. Watching supplier presentations takes a day. That’s at least two days for at least three people. (We often have five — two from digital, two from the relevant policy team, one from commercial.) The commercial team also do a lot of work on the contract and setting up purchase orders.
In terms of actual elapsed time, we often say a procurement takes 6–8 weeks to run between posting an opportunity and a supplier being on site. Realistically this is a best case. While it’s much quicker than traditional procurements, when you’re building a digital service, you don’t want to wait at least 6–8 weeks to get in a supplier to do your next phase of development.
In its current form, the Marketplace therefore disincentives buyers from disaggregating their requirements into smaller procurements. Buyers are likely to (at best) group phases of delivery together within a single opportunity. At worst we see Departments publishing opportunities that span multiple digital services, which means we might as well have stayed with the large IT contracts of the past.
We need to iterate the framework
When I started thinking about my pain points, I realised that most of them can’t be fixed by simply changing “the website”. They’re problems with the way the procurement framework is set up.
The 100 words don’t help me as a buyer and don’t show suppliers in their best light. We could add more help text for both buyers and suppliers, but it might be more effective to do something else entirely. How about a much briefer longlist stage (one week?) where suppliers answer one question, whatever’s the most important to the buyer? Maybe they provide a rough estimate of costs as well at this stage. Buyers could then select a smaller set of suppliers and ask for more information from those suppliers only. (N.B. This is one idea, which is based on the hypothesis that posting an advert that suppliers apply to is indeed the best solution. I’m not saying we should do it, but we could test it alongside other ideas.)
The framework is a part of the digital service — it should also be iterated and improved.
The first version of the Digital Outcomes and Specialists framework went live in 2016. The fourth version goes live this year.
I think it is fair to say that the newest versions of the framework have primarily allowed new suppliers to join and all suppliers to change their prices. The framework has not been significantly iterated.
Why haven’t we iterated the framework?
I don’t know everything about why decisions were made. What I do know is that, while I was at GDS (May 2017––June 2018), we asked Crown Commercial Service (CCS) not to make changes to the frameworks ahead of launching GCloud 10 and DOS3. We did this even though many of the things CCS wanted us to do would have helped users, albeit primarily supplier users.
Launching a new version of GCloud or DOS is very labour-intensive and in order to release a new version of both frameworks to an agreed deadline, we did not have capacity to also make changes to the way the framework operated.
It is also impossible on the Marketplace (as it is at the moment) to make changes that materially affect the experience that a supplier has while applying for an opportunity. This is because if an opportunity changes while it’s live, suppliers who apply at different points will have a different experience to each other, which would be unfair. The Product Managers I worked with suggested functionality that would enable all opportunities that had already been posted to continue using the rules that were in force when they went live, while new opportunities would use whatever new functionality had been cooked up. This functionality was not prioritised while I worked in that team. In effect, this means that the Digital Marketplace can only make any changes to the way DOS works by launching a new framework.
Obviously this is a bad situation to be in. The Digital Marketplace team were aware of this. They negotiated an extension of GCloud 9 and DOS2 with CCS in order to make improvements to the website that would make it easier for changes to be made before the next iterations of the framework were published. The following statement was made publicly:
“Previously, we have undertaken continuous and regular refreshes for each of the individual agreements. However, this hasn’t always given us adequate time for the Digital Marketplace to be developed beyond simply the refresh of these agreements, to meet identified user needs.” (CCS statement in Computer Weekly)
When Tom Loosemore made his otherwise excellent statement to the Science and Technology Committee about digital government, he suggested that questions should be asked about why GCloud 10 had been delayed. He intimated that not releasing GCloud 10 on schedule was a failure of GDS, whereas in fact both GDS and CCS had been transparent about the reasoning from the start.
We should be asking questions about why the extension was cut short, given that the product team had made a clear recommendation that time was needed to improve the service.
Is the website holding the framework back?
Early into my time at GDS, one of the Product Managers asked me what I thought would happen if we closed the Marketplace completely.
When you think about it, GCloud is essentially a big, searchable list. You could do it in a spreadsheet if you had to. DOS is a posting platform — I could use Medium or any Content Management Solution to advertise my work and suppliers could respond to me by email. The Marketplace does these things better, of course, but does that justify its existence? I think it does, but we should at least ask the question if the alternative is never iterating the frameworks because we never have time to change the website.
The important thing was to create a framework that lots of suppliers could quickly apply to with lots of very specific products, rather than a framework that only suppliers who could offer all or most products could join, which would have limited competition and driven prices up.(That means, arguably, the most important important thing the Marketplace does is allow lots of suppliers to apply to the framework and sign a contract.) The framework should also be renewed with new services regularly and allow much quicker procurements. Get the right thing at the best price, get it quickly — my two main needs as a buyer.
While I was at GDS, there was a big push for all CCS frameworks to be online, either on the Digital Marketplace or on CCS’s Crown Marketplace. I understand there still is. In my opinion this misunderstands what is successful about the Marketplace.
It wasn’t the website that mattered. It was always the frameworks that were the revolution. The frameworks were the step change in procurement.
GDS built the Digital Marketplace because it’s the government digital service and when you’re a hammer, all problems look like a nail. It’s also possible that they knew they needed some sort of leverage over the procurement question if they were to get the result they wanted. But what they wanted wasn’t a website. What they wanted was to meet the needs they and other government buyers had as users.
To do that, you need to iterate the whole service.
To do that, you need to iterate the frameworks.
