Predicting Business Revenue with Yelp Consumer Metrics
How we tapped engagement data to understand Texas alcohol sales
This post was authored by Peter Weir and originally published on the Yelp Blog. All graphics below were designed to be interactive so please visit the original post for full functionality.
At Yelp, we recognize how hard small business owners work to give their customers great experiences, and how essential it can be to identify which elements lead to success. So we asked ourselves, could our data help business owners, investors, and others predict business outcomes?
We found that a simple model combining information about how consumers engage with the business, where the business is located, what type of business it is, and when it was founded, enables us to predict the most critical business outcome: revenue. And the most telling of elements for businesses of a given type? Consumer engagement with the business.
Discovering the secret sauce
As the saying goes, everything is bigger in Texas, and that is definitely true of the data. The Lone Star State requires all businesses that serve alcohol to report revenue from beverage sales on a monthly basis. Alcohol sales often make up over a quarter of total sales for restaurants*, and that fraction can be much higher for bars and clubs. The Texas data set is thus the gold standard by which we can judge a model that attempts to answer the question “how much money will this business make?”
We first looked at the raw data for a local favorite, the Texican Cafe in Austin, and immediately noticed the growth of beverage sales starting around 2013 that resulted in its present, enviable position. But what caused the growth in revenue? Contributing to the success were phenomenal performance in customer engagement, intent, and (of course) transactions. Given the central role these factors play in business success, Data Science at Yelp formalized them into three metrics** for every individual business location in the US and Canada: the Yelp Consumer Engagement Metric tracks customer engagement with each business, the Yelp Consumer Intent Metric indicates intent to transact with the business, and the Yelp Consumer Transaction Metricmeasures transactions with the business. (More information about Yelp data elements is available on the Yelp Knowledge homepage.) In the case of the Texican Cafe, we can see that growth in the Engagement Metric coincides with the surge in revenue.
The two examples above illustrate that consumer engagement and revenue are closely linked for some businesses, but we’re still left wondering how predictive of success the Yelp Consumer Metrics are for a much larger collection of businesses. To answer this question we collected data on over 6,500 Texas business locations that were open for all of 2017 and sold beverages for more than one month in that year. For these businesses we built a model to predict the revenue for each business individually based on how customers engage with the business, where the business is located, what kind of business it is, and when it opened.
Another round for my friends and me!
To evaluate the utility of Yelp Consumer Metrics in predicting business success, we constructed the simplest model possible: a linear regression. Using only the Consumer Engagement Metric to predict alcohol revenue explains 16% of the variance in the log-transformed revenue data. (Note that the vertical scale is logarithmic in the figure below — revenue data are highly skewed, so throughout the following we will use log-transformed revenue as the target variable in our models.) Using all three Consumer Metrics performs slightly better, explaining 17% of the variance†.
But looking at the figure, we can see that there is considerable variability remaining. For one thing, both of our examples (indicated in black) made substantially more revenue than predicted by the simple model: The Texican Cafe had total sales of almost $600k, whereas the model is only predicting $400k based on user engagement. The disparity for Floyds is similar — we underestimate revenue for both businesses by more than 30%.
The local watering hole
Business location is an important determinant of sales. We have extraordinarily fine-grained location data for every business in the Yelp data set, but for the purposes of this study we aggregated them into fairly coarse regions: congressional districts. Although gerrymandering could be a concern, using political divisions for grouping business locations is convenient because they are designed to encapsulate similar numbers of people.
By adding congressional district to the simple model, we can account for the differences in revenue that would be expected just by the geographic differences between businesses. After this addition the model’s performance jumps to explaining 19% of the variance.
Pick your poison
So far we’ve been neglecting an obviously important feature for determining how much alcohol revenue a business will earn: what type of business are we talking about? A sports bar probably sells more booze than a movie theater. Below you can explore the relationship between customer engagement and revenue for all business categories containing at least 50 businesses††. It’s noteworthy that for every category higher engagement drives higher revenue, but the relationship is often shifted up or down depending on the category.
By adding category information to our simple model, we are able to explain 43% of the variance in revenue. This large increase in prediction power demonstrates how important category information is when evaluating sales potential. We used color in the figure above to indicate the relative adjustment that being in each category entails, after accounting for the other features in the simple model. To get more insight into the role played by category, we have plotted the same data below, showing which categories are most (and least) likely to drive higher beverage revenue:
What’s your vintage?
Finally, we can consider one more indicator of the health of a business — how long it has been open. After adding this feature our model explains 44% of the variance. This iterative approach to constructing a model can be continued almost indefinitely, but at this point we’ve explored a variety of indicators and explained almost half of the variance using an extremely simple model. In the process we’ve identified some of the major predictors of business revenue.
In vino veritas
Despite the simplicity of the model we generated, it can provide several important business insights. For instance, business owners or investors looking to expand to new locations can use Yelp data to forecast business outcomes across geographies. Given the positive correlation between consumer metrics and revenue even when controlling for other elements such as category, location, and age, business owners can compare their own consumer engagement with similar businesses in their locale to see how they rank. This could in turn expose opportunities for enhancing consumer engagement. Investors can identify which business categories deliver the strongest outcomes across geographies and among them, which specific businesses have the strongest engagement and thus the highest likelihood of success.
Nightcap
By combining Yelp Consumer Metrics with other data about local businesses, we have explained almost half of the variance in alcohol revenue using the most basic models. The linear regression using only location, category, age, and Yelp Consumer Metrics is able to predict the annual beverage revenue of Floyds Cajun Seafood and Texas Steakhouse to within 9% of the actual amount. The same model performs even better on the Texican Cafe, achieving an error of 5%, just a fraction of the tip we’ll leave next time we stop by for one of their famous margaritas!
*See for example this article from Uncorkd.
**The Yelp Consumer Metrics are constructed every week by taking a weighted sum of various user interactions (page views, clicks on website URLs, phone calls, reviews posted, etc.) for each business and then ranking these sums to remove the effect of growth of the Yelp platform over time.
†All values reported for explained variance are the means of performance on the validation set using 5-fold cross validation. The two example businesses mentioned throughout the text were excluded from the training set.
††We removed several categories to reduce collinearity.
