Exposing fake reviews with Python

Image on the Thesis website showing their box of supplements

Highly rated brain enhancers?

Out of 7411 reviews, the nootropic starter pack had at least 4.5 stars from what I could tell. For a $119/month supplement, I want to make sure I’m spending my money on something that’s going to work! The reviews section shows handfuls of snippets from people like “Elliot S” and “George L”,. claiming “Love it!” or “So far, super great!” with a 5 star rating.

Where are the reviews coming from?

In the Chrome Inspect menu, you can record network traffic. Every time I clicked “next” on the reviews page, it was making a request to https://api.yotpo.com/ .

Chrome inspect menu displaying the request to Yotpo

Scraping time!

To find out if the reviews have been tampered with, I figured I should first download all of them.

cURL command to download reviews
--data-raw '{"page":10,"domain_key":"starter-kit","sortings":[{"sort_by":"votes_up","ascending":false}],"free_text_search":null}'
JSON results of querying for reviews
Array of reviews returned by the API

Downloading all the reviews

Because the API is paginated, I’ll need to query it repeatedly to get all 7411 reviews.

A Python script to run a cURL command and combine the results into a JSON file.
Script to run the download reviews and merge them into a JSON file
Wrote 7411 reviews to reviews.json.

Analyzing the reviews

To start, I counted how many reviews of each star rating there were.

A Python script to plot the a histogram of reviews
Script which reads reviews from a JSON file and plots a histogram
A histogram displaying the number of reviews for each star rating
Only 0.2% of reviews are 3 stars or less
Script to plot the dates of reviews
Script to plot the dates of reviews
  • Recent reviews had been deleted?
  • After spending all summer generating fake reviews, they took a sabbatical.
Script to plot the most common names of reviewers

In conclusion…

The distribution of scores is enough evidence that Thesis tampered with their reviews. Don’t believe me? Try finding a single Amazon product with comparable ratings. The highest rated products on Amazon all have from 8% to 10% negative (3 or less stars) reviews. Even this perfect product has 9% reviews 3 stars or under. Nothing comes close to 0.2%.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Chromatic Descension

Chromatic Descension

Software engineer with a knack for fun pet projects.