What Does A Year Of Dismal Reviews Do To A Yelp Rating? Not As Much As We’d Hope.
Yelp gets a lot of things right, but their ratings algorithm needs to be tweaked so that newer reviews are sufficiently factored into the overall rating. Take a look at the Palo Alto restaurant Caffe Riace. Over the last year, the restaurant has received 45 reviews, and over half of them were 1-star. Despite this horrendous track record, the restaurant still has a 3.5-star rating on Yelp.
How is this possible? Because Yelp’s algorithm puts too much stock in years-old ratings, even when it’s clear that a business is a shadow of its former self. Years ago, Caffe Riace was a lovely restaurant with reviews averaging 4 stars. In 2015, its ratings dropped off a cliff, and it averages just 1.8 stars over the last year.
But Yelp does not present this glaring downward trend to Yelpers. When you load the restaurant’s Yelp page, you see a 3.5-star rating, which is not amazing but also not half bad.
Then you scroll down and start reading the reviews, which by default are presented in “Yelp Sort” order. The first review presented is a 4-star review, which seems to confirm that the restaurant is pretty good. Why does Yelp Sort choose a 4-star as the first one, even though the majority of Caffe Riace’s recent reviews are 1-star? Not based on user feedback on the review—this particular review has only 3 “Usefuls”—whereas several of the 1-star reviews have as many as 13 Usefuls. Why does Yelp prioritize this uniquely-positive review over the dozens of 1-star reviews that dominate the restaurant’s recent ratings? We don’t know.
But we can see that Yelp’s rose-colored glasses extends beyond this ill-chosen review: when you look at the first page of reviews in Yelp Sort mode, the average rating is nearly 20 percent higher rating than when sorted based on recency. On Yelp’s five-point scale, this is basically an entire star.
On desktop, you can change the sort order and see how bad the recent reviews are. But on mobile, it’s impossible to escape Yelp Sort—which means that anyone Yelping on a smartphone had no way of telling that the 3.5-star Caffe Riace had a 1.8-star rating over the last year.
Of course, Caffe Riace is just one restaurant, and we shouldn’t extrapolate too much from one data point. However, the trend that their reviews have followed is not a unique trend (basically, a good business gone south), and it’s likely that other businesses that have followed this trend are similarly treated by Yelp’s algorithm. While no algorithm is perfect, we should expect that after a year of 1.8-star reviews, a business doesn’t still have a cheery 3.5-star rating.
