4 Ways Netflix’s All-Thumbs Ratings Leave Users of the 5-Star System Behind

On April 5, Netflix moved from its five-star ratings to a binary system of thumbs up and down.

Netflix released the video explanation below in a blog post.

A survey of 1,103 US-based Netflix subscribers this weekend found that 71% disliked this new system.

Many of those users who dislike the change may not feel strongly about it. But for users who devotedly depended on the star predictions—including me—the change considerably worsened their Netflix user experience.

4 losses to user experience

Reading others’ complaints and reflecting on where my own usage practices broke, I’ve compiled four ways that these users’ experiences have been threatened by Netflix’s changes.

1. Up/down ratings remove useful nuance that could be expressed in 5 stars.

If one title deeply moved me and another was barely good enough to finish, the new system collapses both into a thumbs up. There’s now no apparent way to teach Netflix what types of movies and shows will provide peak viewing experiences, memorable and even transformational ones.

Though I agree with this issue for my own personal usage, Netflix’s Director of Product Innovation, Cameron Johnson— who oversaw the change — points out in this interview that the five-star system had confused many users. They apparently associated online star ratings with averages of others’ ratings, whereas Netflix’s stars were predictions of what each user would rate the title if he or she watched it.

Reading people’s complaints on Reddit and elsewhere, it’s clear that there was and still is much confusion regarding the Netflix star system. Specifically, some people are upset that Netflix is now trying to predict their tastes, which it’s actually been doing for many years (with some major media coverage).

The new “match” language does seem less confusing and likely to achieve the desired effect of making everyone aware that Netflix is providing a personalized prediction.

For those who did understand the system, however, it dials down what we can teach to Netflix’s prediction system. It’s ironic that Netflix was built upon personalization and yet is now leaving behind the dedicated users of its personalization, in a move that treats all of its users the same and that Netflix justifies by pointing at typical behavior.

2. The old system was accurate for many users, making it a powerful tool. Conversely, the new match percentages are often highly out of sync with already rated titles.

Here’s what one Reddit user shared:

This lack of correspondence between known ratings and match percentages has been reported widely. For example, on Twitter:

And I’ve confirmed these discrepancies on my own account. Here’s one of the worst I found, during light digging:

I gave five stars.
And yet, a very low match score. (I’ve found few match scores below 60% on my account, so this score is lower than it might seem.)

Netflix wrote in their announcement of the thumbs system that they will “continue to use [users’ star ratings from the past] to suggest great content.” So we shouldn’t be seeing such inaccuracy. Yet we are.

The inconsistency we’re seeing between our past star ratings and the match percentages seems to indicate that either

  • Netflix is not sufficiently using our past ratings, despite what they’ve said, or worse,
  • there are some added factors affecting the match percentages that don’t align with our tastes. These factors—vaguely specified, if mentioned at all—could even be used to cynically manipulate us towards Netflix Originals or other content Netflix wants us to consume.

3. You can no longer sort titles by predicted rating, finding the movies and shows you are most likely to enjoy.

Before the change, you could go to Netflix’s website on a desktop computer, click to browse a genre, and then sort the entire genre by how much Netflix thinks you’d like each title. This option disappeared with the arrival of thumbs.

Until the switch to the thumbs system, you could sort by “Highest Rated”.
A seemingly minor feature deeply beloved by some.

I derived a lot of value out of this feature, often watching movies and shows that I would never had guessed I would have liked (e.g., Terrace House and Puella Magi Madoka Magica). For a while, I even subscribed to the DVD service for the single reason of being able to sort a larger selection of titles. (I did not order any DVDs, so effectively paid several dollars a month for access to their predictions.)

Nothing about the new rating system is incompatible with this sorting feature. The sort option’s phrasing would change, maybe to “Match percentage”, but it would have the same effect.

Those who sorted by rating were likely a niche set of Netflix’s users, but for those of us who did use it, it was immensely valuable. Since this feature is compatible with the new system, there’s greater hope of it returning.

4. MOST shows and movies don’t show a match percentage at all.

Here’s a sampling in the Documentaries genre in my account.

From browsing around, this seems to be more common with older items.

Perhaps the system is waiting to have enough ratings — from me on other titles and/or from others on this title — to confidently judge the title for me. If that’s the case, then match percentages should become more common after some time, during which more ratings have been accumulated. Nonetheless, part of the value of Netflix’s recommendations was that it would sometimes recommend obscure titles; if it withholds predictions for such titles (which would tend to have less ratings), then we’ll only be able to find recommended titles that are more mainstream.

Netflix’s announced change to thumbs up/down only required the first of these four downgrades. The other three could still be fixed. The typical Netflix user may not notice whether Netflix addresses these issues, but many of its most passionate users will.