An App To Turn Product Ratings From Junk Into Gold
Today, Customer Ratings Aren’t Very Useful But They Could Be
by David Grace
WWW.DavidGraceAuthor.com
The Problem With Crowd-Sourced Ratings
Let’s say you’re looking for a great pizza. You pull up Yelp or some other service and you see ten restaurants with all kinds of bar graph ratings and maybe an average rating number. But how do you figure out which is the best one? Comparing the bare ratings is an apples and oranges exercise. One restaurant has 3 five star reviews, 7 three star reviews and 2 one star reviews (12 reviews) for an average of 3.17. Another restaurant has 12 five star reviews, 38 four star reviews, 12 three star reviews, 19 two star reviews and 8 one star reviews (89 reviews) for an average of 3.3. It seems as if they are pretty close: 3.17 vs. 3.3. But one has seven times more reviews than the other. Doesn’t the fact that it is seven times more popular count for something?
Now add in the other seven or eight pizza places on the list and you’re likely to be thoroughly confused. Is there a way to know which merchant is really the best and is there a way for an entrepreneur to make money from that answer?
I would say “Yes” and “Yes.”
Improve The Useability Of Product Ratings
First, you need to apply some statistical thinking to the numbers. You need a better way to manipulate them than a simple average. I’m no math expert but I reasoned that high scores are disproportionately more important than low scores. People will give a low score because they didn’t like the waiter or hated the parking or the table was wobbly or lots of other fringe reasons. On the other hand people usually give a product a high score when it does the primary job very well, e.g. the food tastes great.
Also, while a restaurant with 5 five stars and 5 one stars will have the same average as one with 10 three stars, my reaction is that the restaurant with all threes is completely mediocre and I will never get great food there while the other restaurant has at least a few terrific dishes but maybe one bad aspect like bad waiters or it’s too loud. Between the two, I’m going to want to pick the one that at least has a chance of giving me a great meal.
Massaging The Numbers
So, I built an algorithm that massaged the numbers to give more weight to higher scores and less weight to lower scores then I applied it to some famous restaurants in San Francisco and Napa (The French Laundry).
Here are the restaurants listed by the standard average of their ratings on a review site:
The French Laundry: 4.4
The House Of Prime Rib: 4.4
Restaurant Gary Danko: 4.2
Michael Mina: 4.3
Incanto: 4.2
Swiss Louis (Pier 39): 3.2
Here’s how they rated under my revised algorithm which accentuated the highs and the lows and spread out the scale.
Restaurant Gary Danko: 4.75
The French Laundry 4.4
The House Of Prime Rib: 4.1
Incanto: 3.8
Michael Mina: 3.7
Swiss Louis (Pier 39): 2.1
But these numbers still don’t really tell a meaningful story, at least to me. So I substituted a rating in English for the numbers:
4.5 — 5.0 Best Of The Best
4.0 — 4.5 Excellent
3.5 — 4.0 Very, Very Good
3.0 — 3.5 Pretty Good
2.5 — 3.0 Solid Good
2.0 — 2.5 Just OK
1.5 — 2.0 Barely Tolerable
1.0 — 1.5 Emergency Only
0.0 — 1.0 Skip it
But we still need to consider the number of ratings. The fewer the ratings the higher the possibility that they are skewed one way or the other and the less popular the business is. Maybe the vendor is less popular for a reason that relates to the quality of the product being offered. Let’s call this the Volume Factor. I’ve played with the math and come up with the following Volume Factor Chart
Number of Ratings — Volume Factor
1-5 — 5% — Nil
5-10 — 15% — Very Low
10-20 — 30% — Low
20-50 — 45% — Moderate
50-100 — 65% — Medium
100-150 — 85% — High
150-200 — 95% — Very High
>200 — 100% — Extremely High
Suppose that the ratings for these restaurants were presented in this way:
Name ————————— Rating ————Volume Factor
Restaurant Gary Danko — Best of the Best — Extremely High
The French Laundry ——- Excellent — Extremely High
The House Of Prime Rib — Excellent — Extremely High
Incanto ———————— Very Good —— Extremely High
Michael Mina ————— Very Good — Extremely High
Swiss Louis (Pier 39) — —- Just OK — Extremely High
Now we have an idea of how to more usefully present ratings but we have no chance of convincing Yelp or Amazon or Travel Advisor to change their business model. Besides, that won’t make us any money. We solve both of those problems by developing a free Ratings App.
This need translates into the opportunity to make money.
The Product — The Business Model
Suppose you write an app that leaves a tiny icon in a corner of the user’s cell phone screen. The user calls up Yelp or Trip Advisor or whatever and searches for “Italian Restaurants” in San Francisco. A page comes up showing a ratings chart for some vendor. The user puts his/her finger on the Ratings App icon and drags the icon over to the chart. The app is smart enough to recognize the chart and calculate the relative values of each bar based on the relative sizes of each bar if the actual numbers are not available. Yelp doesn’t show numbers. Trip Advisor does.
The app then runs the algorithm using the relative sizes of the bars, computes a rating, converts that number to a text score (Excellent, Very Good or whatever) and puts up a window on the phone along the following lines:
——————————————————————————————— Search [x] where this vender is located OR [ _ ] where YOU are located?
Look for Other Venders [x] in contiguous 9 digit zip codes?
OR [_ ] more distant 9 digit zip codes?
Our Rating: Pretty Good
Ratings Volume: Low
We’ve found three other Italian restaurants in contiguous zip codes that are rated equal to or higher than this location.
1) Tony’s Palace — Very Good — Ratings Volume: High [Tap Here To See Details]
———————————————————————————————
Visit Tony’s Palace in the next 8 hours and get 10% off any pizza
———————————————————————————————
2) Sorrento — Very Good — Ratings Volume: Moderate [Tap Here To See Details]
———————————————————————————————
Want a great Italian Deli? Click Here for Info On Gino’s Deli
———————————————————————————————
3) Little Napoli — Good — Ratings Volume: Medium [Tap Here To See Details]
———————————————————————————————
Advertising Model
You would tie the type of business being rated to a database of your advertisers. So, when a user checks the ratings on an Italian restaurant you might show him ads and coupons and specials for Italian delis, bakeries, restaurants, grocery stores, pizza parlors and the like.
Each ad could offer a limited time discount, free soft drink, or other bonus to shop there. The discount would be claimed by pressing a button on the app that would display a list of the ads the user has seen in the last 24 hours. The user taps the ad for Tony’s and a bar code appears which the merchant scans. The system would notify your web site when the coupon was redeemed.
Geography Of The Ad Market
For the search parameters the user could select one of four geographical zones:
(1) very near the business being rated
(2) farther from the business being rated
(3) very near where the user is located
(4) farther from where the user is located.
The last four digits in a nine digit zip code designate subsections in that zip code. There may be only two or three subdivisions or more than twenty subdivisions in any given zip code. Surrounding every zip code subdivision are other nine digit zip-codes subsections in the same and/or adjacent zip codes. If you think of the zip code subsection in which you are located as the center of a target then other zip-code subsections surround it on all sides like the next target ring. More zip-code subsections surround those contiguous ones like the next farther ring out on the target.
Locations that are in the closest ring of zip-code subsections to the center or in the nearest two rings around the target center can be plotted depending on whether the user had elected to make his/her location the center of the target or to make the vendor he is checking out the center of the target.
The app would be free. It would provide a useful rating service to the user and it would present the user with information on other similar venders which are nearby, including venders with better ratings or who offer discounts or other incentives.
Amazon?
Of course you would design the app so that it would work on Amazon product rating graphs. Suppose that the Amazon shopper was looking at the rating for a Le Creuset Dutch oven and he could pop up an interpretation of that rating. Our app then pops up an ad for a Lodge Dutch oven just as highly rated as the Le Creuset but at one-fifth the price, and maybe even with a manufacturer’s direct-to-user rebate coupon if they buy one within twenty-four hours? Wouldn’t that be a valuable ad platform? Also, suppose the app linked that Lodge Dutch oven back to the Lodge Dutch “buy me” oven page on Amazon. That would keep the shopper there which would make Amazon happy. If the shopper bought the Lodge, then Lodge, your advertiser, would be happy too.
That kind of an app could have a lot of ad value if it could be made to work. Could that be done? I don’t know.
It’s something to think about.
