How fair are social rating systems?

Google has announced at its I/O Developer conference that it’s to change the way its Play Market app score ratings work by weighting them to favor those from more recent releases and will also remove scores from older versions, meaning they will better reflect an app’s current performance.

The idea is to offer developers a way to combat one of the problems of social scoring systems: the impact of a specific episode. In the case of Apple with its App Store, the problem is solved by allowing developers to reset their scores each time they upload a new version, something that the company says requires careful management and should not be done too frequently.

The impact of poor scores obtained long ago is just one of the problems that plague application platforms’ scoring systems. In practice, these ratings have a lot of impact: a bad score can put users off installing an app and in such a highly competitive environment, this can be a death blow. The problem is that many of the scores users give do not reflect how an app really works and can be highly subjective: there are apps, for example, that are slammed because they aren’t available in a given market or only by invitation, on other occasions, users may have been expelled for bad behavior and decide to get their revenge by going to the corresponding application platform and giving a minimum score, which can weigh on the global score and prevent more users hearing about it or appearing in the rankings in a reasonable position.

The problem is social scoring system’s lack of detail and incorrect use by users who tend to see ratings as a complaints form rather than as a way to rate functionality. There’s no easy solution: every app wants feedback, but if this is initially negative, when applications are still in the testing phase, the repercussions can be disastrous. Occasionally, developers are pressured by impatient users who “want it all and want it now.”

The system is imperfect but has become a universal metric, an inflexible judge whose word is final: apps with a higher marketing budget can launch campaigns that dilute the weight of bad evaluations among a greater number of users attracted by the media, but small developers have it harder and must think very carefully about the balance between expectation and functionality at all times.

The same problem occurs with other types of evaluations on social platforms, such as hotels or restaurants: a poor rating may be due to the fact that your restaurant or hotel is bad, but could be because the person making the rating had difficulty getting a table or there was a misunderstanding, or could even have political overtones. There are cases of many coordinating to issue bad recommendations with no authority, given that they have never visited the establishment, but the platform faithfully reflects a low score that will put people off visiting the establishment, even though the original issue that prompted the low score has been resolved. Things may reach such a state that the establishment no longer appears in a ranking, and of course the economic cost can be huge.

Social rating systems have transformed the way we consume. But we shouldn’t lose sight of the fact that people have many sides, and often, we issue scores due to factors that have little to do with what is supposedly being evaluated, be it the functionality of an app or the quality of a restaurant or hotel. Correcting this will not be easy, and developers or owners of establishments will need to be involved in addressing the challenges. At the same time, it is a good idea to introduce a sell-by date on ratings. When we use this types of systems, whether from the supply or the demand side, let’s think a bit more about what our rating will do for people who in many cases have put more than just time or money into a project.


(En español, aquí)