B2B Buyer Beware — It’s not just consumers that are falling prey to fake reviews
It’s long been known that websites such as Amazon and TripAdvisor have had problems with fake reviews. In fact, recent reports suggest that the problem is worse than ever on Amazon, and the story of how one man made his shed the top rated restaurant on TripAdvisor highlights the ease with which it’s possible to play the system, should you want to. Clearly the financial rewards for being listed at the top of the reviews are distorting some people’s moral compass.
Unfortunately, with the consumerisation of IT, we are also seeing many of the same traits coming to the fore in the B2B software sector. Over the past few years there has been an explosion of crowd-sourced review sites where users can report back on their experiences of using enterprise software. Of course, we are now conditioned to treat online reviews with some scepticism, but one review site that stands out from the crowd is G2 Crowd — whose USP is that reviewers must use their LinkedIn accounts to post a review and they must submit a screenshot of the product containing their user ID, in order to verify they are an actual user of the product. So, these should all be real reviews, right? Wrong.
The company I work for has a subscription billing solution and consequently I started following the corresponding section on G2 Crowd 2 or 3 years ago — always useful to keep tabs on competitors. They even send me an email of weekly updates, so I don’t miss the latest reviews. However, it was a recent weekly update that aroused my suspicions — two reviews of competing products by the same person?!
It seemed somewhat unlikely that Elliot R. would be using both products, so I decided to dig a bit deeper.
The two reviews were posted 2 days apart — 7th January and 9th January:
So who is “Elliot R”?
Well apparently he works for a company called Hello! Destination Management:
A quick search on LinkedIn and I found him — Elliot Rounds. https://www.linkedin.com/in/d-elliot/
But then it appears he only has 4 connections — when almost everyone on LinkedIn has at least 200+ connections! However, he seems to have been rather prolific with 6 reviews on G2 Crowd:
Next stop to check if Elliot Rounds is who he says he is. I go to the Hello! Destination Management website where they have a handy “Team” page which includes photos and names of what seems to be everyone who works there. No sign of Elliot. Thinking that this is probably not a complete list, I submit a contact request via their website to ask if Elliot Rounds works for the company. A few hours later the response comes “Good morning, in response to your inquiry we do not have a Elliot on our team”.
Elliot Rounds is clearly not who it is claimed, and it is reasonable to assume that these reviews are not real either.
At this point, it is also worth pointing out that G2 Crowd offers incentives for people completing reviews (hover over the “Review Source” under a reviewer picture). So, whoever Elliot Rounds is, he seems to have received a gift card for his troubles.
It is not unusual for companies to offer something in exchange for completing a survey, usually entry to a prize draw of some kind. However, by offering a financial reward for every review completed (up to a maximum of 7 per user), this really opens up the system for widespread manipulation and there are already reports of the negative impact this is having on some customer relationships.
It also seems to have created a market for fake review writers.
When on the G2 Crowd website looking at these questionable reviews, the handy web chat window popped up and I decided to enquire about the validity of these reviews. “Are these reviews real?” I asked. The response came to say that all reviews are validated and that if I had any concerns I should contact their outreach team.
As noted on the G2 Crowd website “We monitor all <product name> reviews to prevent fraudulent reviews and keep review quality high. We do not post reviews by company employees or direct competitors. Validated reviews require the user to submit a screenshot of the product containing their user ID, in order to verify a user is an actual user of the product.”
However, in the age of photoshop it is really not difficult to manipulate a screenshot to put in a different user id.
So, I contacted the outreach team at G2 Crowd and raised my concerns about the 2 questionable reviews described above:
“Thanks for reaching out. We actually have a team that looks into all review concerns! On the review you are unsure of you can click “Report a Concern” and someone will look into it ASAP.”
I replied to explain that the web chat team had advised me to contact the outreach team, and pointed out that since my first email to them I had spotted two more apparently fake reviews:
…which had been written by a LinkedIn user with 0 connections who seems to only exist for writing reviews: https://www.linkedin.com/in/brandie-davenport/
Once again, the response came:
“The best place to submit any review concerns will be by hitting that “report a concern” button on the bottom of the review you want looked into. Unfortunately this address is not set up to look into those review issues. The fastest and most efficient way for a review to be re-checked is by reporting to the team through the “report a concern” option.”
It may be a crowd-sourced review site, but I have no intention of being a crowd-sourced reviewer of reviews.
In an article on VentureBeat, G2 Crowd co-founder and CEO, Tim Handorf, is quoted as saying that “Every review is checked by a human being for quality and we use a LinkedIn login to verify that the reviewer is real, not a company employee or a competitor”. Well it took me about 30 seconds to find and check the LinkedIn profile of the reviewer and see that these were fake. I have also looked at many other reviews which are undoubtedly real.
In fact, after looking at a few reviews it becomes quite easy to identify suspicious reviews that need further investigation — they are usually a similar length, typically score 4 or 4.5 stars (not 5 stars which may rouse further suspicion), the language / grammar used is very poor, and they say very little about the context of the business that is supposed to be using the software.
I have followed with interest the growth of G2 Crowd over the past 2 or 3 years and have noted their move towards publishing their own industry reports where they collate their crowd-sourced reviews and scores in a “G2 Crowd Grid” in an effort to move in on the traditional industry analyst grounds such as Gartner Magic Quadrants and Forrester Waves. But these Grids perpetuate the problem, as they further incentivise manipulation of the system by vendors in order to achieve a favourable position in the Grid, and hence it now seems to be possible to buy reviews in the same way that you can buy Facebook and Twitter followers.
I think there are several problems here that need to be addressed:
1) G2 Crowd really needs to sharpen its validation process so that reviews on its site can be trusted. At the moment it seems to be an automated process which leaves the door open to widespread abuse.
2) Vendors need to stop buying fake reviews. Do what you do, and if you do it well, customers will be happy to shout about it.
3) And finally, LinkedIn needs to start validating that people actually work for the companies they say they do. As things stand, anyone can claim they work for any company. Interestingly, LinkedIn participated in the latest round of funding for G2 Crowd.
I think what G2 Crowd is trying to do is great; there is definitely a place in the market for independent review sites, but there needs to be a much more rigorous validation before these can be taken seriously. In the meantime, love them or hate them, Gartner, Forrester et al, have expert analysts who specialise in each industry sector and follow rigorous methodologies to complete their research.
So, if you’re a B2B Buyer looking for your next enterprise software purchase, do your own online research and by all means check the review sites, but you would be well-advised to talk to reputable industry analysts and preferably some actual users too, before making your final decision.
Since G2 Crowd inadvertently highlighted this issue with the aforementioned “New reviews” weekly email notification, I now look forward with interest to see what gems the next email will bring! Meanwhile, the fake reviews I reported to the outreach team more than a month ago are still published…