Exploring portable ratings for gig workers

Traditional labour economics textbooks teach you that if an employer cuts wages, workers who can do the same work somewhere else will move. In a ‘frictionless’ labour market, people will get paid what they are worth.

But the real world isn’t frictionless. A parent might stay with a stingy employer because it’s near his children’s school. An office worker might not apply for a new job because she likes her colleagues too much. In short, theory isn’t the same as practice.

So what about the gig economy?

In theory, this is the place in which people can flit between employment at will — turning apps on and off, even having more than one on at once. But in practice, there’s one big spot of friction: the ability to take your feedback with you wherever you go.

Unlike the traditional economy, the gig economy doesn’t rely on CVs or letters of recommendation. You build your reputation on one platform at a time — and your reputation is often the route to higher earnings (A service user is more likely to choose someone with 100 five-star ratings than just one or two). Platforms don’t want people to leave, so they don’t let workers have ownership over their own ratings. Leaving a service means starting over.

The Resolution Trust has been researching ratings portability — Gavin Kelly and Daniel Tomlinson’s article “Give me my reputation back” is an excellent read for framing these issues.

“ From freelancers offering their coding services through platforms like Upwork or PeoplePerHour, to drivers contemplating moving from Uber to Lyft, this is an issue that will rise in salience. And the more important ratings are to a worker’s prospects, the more tied they are likely to be to the platform where they first got established.”

More recently, we’ve been exploring the “how” of ratings portability: what technology, data, user experience and investment might be needed to make this real.

Our design team, along with our policy intern and developer James Darling, have been conducting user research and prototyping possible technical solutions for ratings portability. Here’s where we’ve got to so far.

User research

Our team immediately got to work — ordering cabs, meals and Ikea furniture assembly so they could interview some workers. We wanted to understand the behaviours workers have around their ratings, how they use them and how they understand them. We also wanted to get a sense of how important reputation is in their line of work, and how digital platforms have affected this. We also wanted to know whether people felt locked in to their current platforms because of the reputations they’d already built.

“Cab” drivers didn’t have visible habits around their ratings, weren’t checking them frequently and when we spoke about them, they told us that this wasn’t something they’d considered before or something they were particularly concerned about. They were confident in their skills and ability to find work outside of their platforms, and viewed ratings more as performance indicators for their platform owners — the main fear being a drop below 3.5 stars, where they might be dropped from the platform completely.

This “performance indicator over ratings” feeling was even stronger with food delivery workers. They expressed even less concern about the issue, focussing more on their delivery metrics such as attendance and cancellations. The rider app screens we were shown support this.

This makes sense for both food delivery and transit: the customer has little to no ability to use workers’ reputation data to inform their purchase decision. (When we press a button to order a cab or for food to be delivered, speed is the primary factor and platforms emphasise that in their design.)

It was a radically different story for tradespeople. Their reputation data feels important to them, and they prefer to keep control over it. They preferred word of mouth reputation and recommendations, as there was no middleman who could take that away from them. Online platforms were seen as something to graduate away from once you had a sufficient “real world” presence.

We heard stories from tradespeople about how their accounts, along with their online reputation data, had been deleted without warning, explanation or ability to appeal. “Nick” spent a year and a half building his reputation via one of these platforms until his account was closed without warning or reason.

‘I tried to reach them and I was really upset because I had built that profile…I couldn’t take my profile and move it. I lost everything but they didn’t really care because it says in their terms and conditions they can close any account without reason.” — Nick

We also talked to tradespeople who wanted to move into a more stable working environment but found it impossible to get references from companies they’d worked with, even after more than a year of service.

After our user research we felt like we had a sense that reputation portability was a real issue for some gig economy workers, and thus it was worth beginning to explore solutions.

Technical analysis

Alongside our user research, James Darling looked at the technical possibilities, drawing on the Resolution Trust’s initial work and the research that our policy intern did. They came up with five possible solutions and gave them names and some logos. They are in increasing order of complexity.

Personal reference

This is the status quo: when approaching a new employer, workers create their own CVs, loosely standardised by convention. To verify, reference contact details are often given that the new employer can contact to verify details given in the CV and/or make additional enquiries.

Is this system appropriate for gig economy workers? Does new technology offer opportunities to improve this system? Do new technologies create new problems, such as the inherent reduction in HR people-power to keep such systems effective?

Publicly hosted reputations

What feels like a technical quick win is to ensure that a platform hosts a publicly accessible web archive of all worker reputation data, including for profiles which have been disabled. This would allow workers to provide a URL to anyone they wish to provide their reputation data. How would this be encouraged/enforced?

Profile verification

How does a worker prove that they are the owner of a publicly hosted reputation profile? There are a few technical solutions that could be explored here, like a public/private key verification or explorations around OAuth. Is it possible to create something that is secure, but also usable?

Decentralised open data standard

A data standard for reputation data could be created, allowing automated transfer and use of reputation data by competing platforms or external services. Creating the standard would be the trickiest part here: is it possible to translate between both technical differences of different platforms (eg 5 stars versus 80%), but also the values inherent in them. For example, a community on one platform may value cleanliness over promptness, or another may culturally be more generous with their ratings. Additionally, how do we encourage or enforce that platforms implement these standards?

If we were able to overcome this, then this standard would allow users to quickly transfer all reputation data in a few clicks.

Centralised data holder

Perhaps one way to help standardise and enforce this easy transfer of reputation data is to create some sort of legal entity responsible for holding and transferring this reputation data. A lot of discussion would have to be had about the legal framework for this: is it a government department, a charity, a de facto monopoly?

Prototyping

Exploring different technical approaches led us to clarity in terms of what we wanted to prototype. Based on all the factors, we were able to start moving towards a solution that, in a world where ratings portability was possible, would make it as frictionless as possible for people to export and transfer their own data.

Over the course of about half a day, James Darling created an example of an online archived reputation portfolio. This helped us think about what information would be needed to be displayed, and how it would be discovered — for instance, how to futureproof and standardise the data for use in any platform.

We also thought about ways to verify identity (by including an RSA public key), what a best practice data standard might look like (here’s an example in JSON), and what the import process might look like (via a mock competitor site). The code for all this is on Github, and everything above is available in a slide deck here.

What’s next

We’ve managed to get a lot done in a short amount of time, poking a reasonably abstract concept and starting the process of nailing it down using several digital techniques. But at the same time, we also know this is a small part of a much wider set of issues around workers’ rights and the institutions that are needed to uphold them. Whilst we discovered that rating portability might not be as important to some workers as we’d first anticipated, we think it is potentially still important to society.

It links to my blog post on moral needs, not just user needs. I worry that the concept of “owning” people’s ratings reflects some deeper, more systemic issues around who “owns” things more generally in society. In the coming months, we’d like to keep working with like minded organisations to explore that idea more, as well as how the cumulative effects of those systems affect us all.

Through doing the user research, we also uncovered some price discrimination. To find willing user research participants on some of the “builder” platforms, we advertised a job, and people that were suitable to do the work were then shortlisted. Those that were shortlisted then all get charged to have access to the clients details. We were surprised that a platform thinks it’s okay to charge workers money even before they’ve necessarily got the work, and even more surprised to discover that the 5 shortlisted candidates were charged different amounts to contact us, based on their loyalty to the site. Some were charged £3 for our phone number, others as much as £7.50, just to get the access without any guarantee of getting the job. We want to uncover more of this kind of discrimination that workers face and link it to the price discrimination work we are doing with Citizens Advice.

The user research also highlighted the growing affects of how technologies are being used to monitor “performance” — Uber drivers are watched for their speed (if they go over 70 their ratings are lowered) and they are even called out for breaking too hard or too often. The Uber app is not only tracking the journey and its cost, but every small detail of the drivers behaviour. The sense of constantly being monitored and watched puts a huge amount of strain on people, who are already facing the pressures of precarious, low-paid work and the cumulative effects of this could be catastrophic for things like mental health.

Lastly, we know there are others doing good work in this space, like Traity and Deemly. We’d like to know what they are basing their rating decisions on and to hear more about their plans to design for users to move their ratings between multiple services.

If you have any questions about our work, or if you want to talk more about what we’re doing, send us a note at hello@doteveryone.org.uk.