What to watch out for when building a recommendation engine
So you’ve decided to build a recommendation system for your product. Cool — maybe you have an e-commerce store, serve media on a subscription model, or just want to bump conversions on your landing page. Personalized recommendations are one of the many tools in the growth toolbox that can help to boost revenue in the long term, but there are some things to keep in mind. In this post, we’re compiling our experiences from building real-time recommendation engines for our customers and collecting the top questions you should ask yourself before jumping into the game.
If all you have is a hammer
Machine learning is a big and powerful hammer. It’s easy to smash your thumbs when applying it blindly. ML models are especially appealing because they promise to take a lot of complexity off your shoulders. But caveat emptor — like the mythical siren luring sailors to their demise, black-box machine learning models are easily outperformed by someone who spends a few days understanding the user and business. Instead of going straight to the data, ask yourself first: What are your users hoping to get out of your product? A fashion e-commerce store will have very different answers than a workout planner app. In the former case, users might be trying to explore new content that complements their existing wardrobe. The latter users are looking to continue their workout journey, perhaps with helpful tips on other areas they should improve.
Secondly, what are you hoping to achieve? Perhaps you aim to build a better product with greater long-term user satisfaction or boost KPIs like fighting churn or driving conversion rates. For example, booking.com — their pushy and aggressive messaging is undoubtedly working for them to increase conversions and revenue, but does it satisfy the customer? These tactics work because customers generally book only infrequently. On the other hand, a grocery delivery app is trying to recruit frequently recurring customers where customer satisfaction is paramount to long-term success. If you’re optimizing for revenue, you also have a vested interest in recommending items that might be slightly less appealing to the user, resulting in higher expected revenue overall.
Thirdly, how frequently are you acquiring new users, and how often are you adding new content? What is the average timespan between recurring visits of a user to your product? Booking.com, for example, is facing a unique set of data science challenges because users visit only so infrequently.
While subtle, these factors become highly significant at scale. Use them to guide the design of your recommender system by defining:
1. How much “exploration” should your recommender have.
2. What kind of metrics should your model be optimizing for.
3. How to best implement and deploy your model at scale.
The whole is greater than the sum of its parts
Personalized recommendations work best when complemented with other tweaks to your product. If users are having a hard time finding relevant content, then perhaps a redesign of your navigation is appropriate, in conjunction with a personalized recommendation engine. This will both serve as a fallback for when recommendations aren’t entirely on point and users are looking for something else and improve the quality of data that your algorithm is working with. The saying goes in machine learning: “garbage in, garbage out.”
Perhaps annotating your content with better tags and other metadata could be handy as part of this process, allowing you to provide better navigation and give your ML engineers more data to work with. Finally, it can be hard to know what users think of the content. Are they enjoying it? How did it make them feel? Integrating a feedback mechanism can go a long way but is not always the right call to make. The optimal decision depends on how easy it is for you to gather implicit feedback (such as a user watching most or all of a 15min video), how much data you have available, and most importantly, what user experience works best for your product. Some apps don’t work well when users are constantly interrupted by feedback dialogues.
ML is more than just models
Think not only about developing a recommendation engine but also about the infrastructure and maintenance required to keep it running. If you’re updating recommendations in real-time or making frequent adjustments to your algorithm, having an excellent infrastructure to support these operations pays big dividends. Depending on your desired uptime, investing in fallback infrastructure and a robust error handling plan might also make sense when things go south. Ingesting real-time user event streams is a challenge. The load can come in sporadic but heavy bursts. Having your product collapse during those critical moments of highest user activity is the last thing you’re looking for.
Ask for help
At Flike, we’re building recommender systems as a service. We’re always happy to chat and help fellow engineers and data scientists deploy recommender systems for their business — feel free to reach out to us via email at hello@goflike.app.
