Beyond the Metrics: What the Job Success Score Can’t Capture
To efficiently mediate their digital marketplace, Upwork generates metrics about freelancers based off of their completed contracts. Metrics attempt to display a freelancer’s work history on the platform. Badges and client ratings/reviews automatically appear on each user’s profile alongside their Job Success Score. Let’s specifically explore the Job Success Score. Stay tuned for future blog posts about other performance metrics displayed on an Upworker’s profile.
What is the Job Success Score?
The Job Success Score (JSS) is a percentage calculated by Upwork that every freelancer receives. This measurement attempts to quantify each user’s average success rate on the platform.
How is it calculated?
The calculation of JSS on Upwork can be boiled down to a straightforward equation:
(successful contract outcomes — negative contract outcomes) / total outcomes
Every two weeks, Upwork calculates 4 Job Success Scores for each freelancer by capturing their 3-, 6-, 12- and 24-month work history on the platform. The best score out of these 4 time windows is the JSS that appears on a user’s profile.
While this may seem a straightforward way to measure a freelancer’s performance, this calculation relies on contract outcomes being either “successful” or “unsuccessful.” The calculation of the JSS does not take extenuating circumstances into account. Freelancers run into many precarious situations when finding and completing work digitally. As they navigate the Upwork platform, they deal with scammers, problematic client communication, and payment issues. These types of extenuating circumstances make it hard to label all contract outcomes as good or bad.
Impact on Freelancers
How does the JSS dictate the user experience?
A freelancer’s JSS and their ability to secure work are highly correlated. Given the lack of social contact between the freelancer and the client on the remote platform, the JSS percentage is one of the only ways a client can gauge a freelancer’s reliability, reassuring or raising concern for a client when hiring a freelancer. When a freelancer has a near-perfect JSS, most of their previous contracts have been completed successfully, and clients heavily consider a freelancer’s track record before hiring them. This is a self-fulfilling prophecy: the higher an Upworker’s JSS, the more reliable they appear to clients, increasing their ability to secure work. Each successful contract continues to “cushion” their JSS. As a freelancer’s number of successful outcomes increases, a negative contract outcome is less damaging to the calculation of their JSS.
Additionally, higher-paying gigs have a more significant impact on the JSS calculation. A low JSS makes it more difficult for freelancers to gain client trust and secure contracts. A freelancer’s JSS can only change once they complete new gigs, and a low JSS makes securing new contracts much more difficult. Adverse contract outcomes impact a freelancer’s JSS until the old activity is phased out of the calculations. Freelancers with a JSS lower than 75% will likely face difficulties finding new contracts and clients.
A freelancer’s JSS and other platform-generated metrics are essential to securing contracts and higher wages on Upwork. To build up these metrics, a freelancer must accumulate a work history to demonstrate their previous success. It’s common for freelancers to take on contracts that lower-than-desired pay in order to build up their work history.
Despite how vital the JSS is in finding gigs, this metric exists only within that freelancer’s Upwork account. This profile-specific capital does not follow the freelancer to a new profile on the platform if they delete their account. The JSS is not only profile-specific; it’s platform-specific. A good (or bad) JSS doesn’t affect a freelancer’s reputation anywhere besides Upwork. Consequently, for some successful freelancers, these metrics cause their continued reliance on the platform, because users with a high JSS tend to have an easier time securing work.
What Our Participants Think
The level of awareness and emotions regarding their JSS score varies among the freelancers we follow. Some freelancers don’t know what it is, and others are ambivalent about its effect on their ability to find contracts. However, most freelancers have an acute awareness of their JSS, and it’s direct effect on their Upwork profile. Quotes from our participants give our research team insight into how freelancers feel about their Upwork JSS.
One freelancer we interviewed expressed worry that a potential decrease in their JSS could affect other aspects of their profile. A freelancer’s JSS can qualify them for one of three Upwork badges (more on these in upcoming posts). Similar to the JSS, it’s a profile feature that dramatically affects a freelancer’s platform identity and subsequent ability to secure contracts.
I: And do you pay attention to your job rating score?
R: I do, because if it goes down, I’m going to lose the badge I have right now. I think you cannot go less than 93% for your job success score. I’m actually worried now that I don’t have hours with Upwork right now, because I’m not sure how that’s going to work now that I’ll be kind of inactive on the platform.
Besides explaining the correlation between badges and JSS, the freelancer also mentioned another factor– the effect of inactivity on their JSS. Many of our participants discuss how inactivity on Upwork affects their platform identity. Since work history informs a freelancer’s platform metrics, a decline in the number of new gigs means that a freelancer’s JSS and other metrics will change at a much slower rate than a user who maintains a consistent number of gigs. If a freelancer secures long-term work, off-platform work, or is simply taking a break from Upwork, their JSS does not reflect these other factors– it is simply an analysis of the quality and quantity of completed gigs.
Freelancers face situations with clients that impact their JSS constantly– sometimes, circumstances are entirely out of their control. Two of our participants outlined how their JSS was impacted by factors outside of their jurisdiction.
“I’ve had a couple of clients ghost me, where I did all of the work that we had established, but they just never closed it out. And I would be like, ‘Hey, can we close this out?’ because somehow — I don’t understand their formula, but somehow, all of that factors into what Upwork calls your job success rating, and it’s a really weird algorithm that they have, so if you close out the contract, it negatively affects you, versus your client closing it out, which I don’t understand. And I learned that the hard way because I closed out contracts after some clients ghosted me when I should have just kept them open.”
To get their payment, these participants described having to close out a contract themselves when a client becomes unresponsive. Despite successfully completing the job, a freelancer’s JSS is penalized when they end the contract themselves to receive the payment. Freelancers then face a choice: do they forgo their payment and allow the client to ghost them to preserve their JSS? Is getting paid more important, despite the decrease in JSS (that could make securing future work more precarious)? This freelancer’s situation demonstrates how the Job Success Score is just that– a score.
While the JSS is only one of the profile metrics that impact freelancers, it’s one of the clearest examples of how Upwork’s algorithms influence a freelancer’s reputation by quantifying their platform experiences. By boiling client-freelancer transactions down to successful and unsuccessful, the JSS does not leave room for the ‘gray areas’. Freelancers independently navigate the platform, securing clients and completing contracts. Interactions with clients are often nuanced– these unique situations fail to be quantified in the JSS, and have the potential to damage a freelancer’s reputation. Platform metrics are the lifeblood the Upwork marketplace, but they don’t always paint a clear picture of a freelancer’s platform experience.
Written by Clea O’Neil. Special thanks to Heba Salman for her edits.
