How Xeno grew marketing conversions upto 100% using Machine learning

Today there’s very little technology at Google that isn’t using Artificial Intelligence(AI) and Machine learning. AI is reinventing existing products, from Maps to YouTube, and it’s powering new experiences.

Take the Google Assistant. Today, the Google Assistant is the first real progress we’ve made in creating a true conversational experience that essentially brings people their own personal Google. It uses speech recognition and natural language understanding to help people get things done in the real world — from managing their calendars to helping them control their lights.

AI is reinventing existing products and powering new experiences like iPhone Siri, Amazon Alexa and Netflix Recommendations.

Customers today expect the level of relevance that Netflix or Spotify delivers to them in every product & the marketing that’s delivered from those products.

Higher the marketing expectations of customers, more personalisation of marketing opportunities using technology.

Putting another way, consumer expectations will be higher than ever. This will pose a challenge for brands — and a great opportunity.

A big part of the opportunity for marketers is how Artificial Intelligence will help us fully realize personalization — and relevance — at scale (reaching out to millions of customers with each getting a unique marketing campaign) With platforms like Google Search and YouTube reaching billions of people everyday, digital ad platforms finally can achieve communication at scale. This scale, combined with customization possible through AI, means we’ll soon be able to tailor campaigns to a consumer’s intent in the moment. It will be like having a million planners in your pocket.

One of the most important questions that we are repeatedly asked at Xeno is how do we use Machine Learning to improve the marketing conversions?.

This is a very important question given the recent hype around Machine learning and AI.

With this blog, We wish to describe how we use statistical models and machine learning to improve the customer conversions from the marketing campaigns that we run for different brands at Xeno.

Most of the brands that we are associated with collect thousands of customer transactions every month where 30–40% of them are their first time customers. Every time a brand has to plan a digital marketing campaign, it requires a lot of time for them to plan. Adding to this is the back and forth that the marketing team has to do with the data science team. It’s also very difficult for the data science team to analyse such tremendous customer databases within short span of time.

Different Aspects of Sending Personalized Campaigns

After sending out millions of campaigns at Xeno, we identified 4 different aspects to personalize a campaign

1.Finding the right audience for your campaign

2.Selecting the right marketing channel for every customer

3.Predicting the right time for every customer and

4.Designing the right content to promote at scale

Finding the right audience for your marketing campaign

Selecting the right audience for your marketing campaign becomes the most difficult and cumbersome part of planning the entire campaign. It requires a lot of data analysis and tuning while simultaneously tracking how every customer is moving across different clusters or what you call a group of customers.

After running so many campaigns at Xeno for different brands, we realised that most of the marketing managers have specific goals in mind when they plan to run a marketing campaign which could be one or more of the following:

If you have more than a million customers, it does not make sense to reach out to all of them together. Therefore, it becomes very important to keep in mind the goal of your campaign before filtering out your customers.

At Xeno, we start by dividing your customers into different clusters along multiple attributes which can be how recently the customer has visited you, the number of visits that he has done till now, how often does he react to different marketing channels and how much on average and total he has spend with your brand till now.

We divide all of these attributes/dimensions along multiple quantiles or ranks and create a group of clusters to tune them with different brand goals.

Effectively, if you wish to plan a campaign to launch a new product, you should reach out to customers who have higher brand affinity, have visited you recently, are very frequent customers and react to your marketing campaigns quite often.

Selecting the right channel for every customer

There can be multiple channels that marketers can use to reach customers like SMS, Facebook, Instagram, Email, Push Notifications,Google Ads etc. To understand how to select the right channel for every customer, we need to understand what kind of data needs to be collected before we actually start predicting it. For every individual customer, it’s very difficult to depend on a few static variables like age or gender to predict their most preferred channel of communication. A working millennial might prefer an Email while a guy in his 30’s would prefer an ad on Instagram. Similarly, the reactivity might keep on changing for different customers on different channels. So the historical data might get irrelevant in a few days.

The only way we can solve this is to use Reinforcement Q based Learning using which we can train an artificial agent to predict the next best channel for every customer.

The artificial agent is rewarded and penalised based on how the customer is reacting on different channels. Also, the artificial agent is penalised if the conversion on a channel gets outdated.

This way, the artificial agent keeps on updating reactivity scores on different channels for every customer.

Predicting the right time for individual customers

This FIFA World Cup final, we saw an uplift of over 50% in orders for most of the restaurant chains that use Xeno. Out of these, 90% were repeat orders who love ordering from the same place time and again out of which 60% of them were customer who only order on weekends or big events like the World Cup final.

Eventually, it became evident that sending them a communication on normal weekdays was actually spamming them, while sending a communication just before a weekend, before a league final or a special occasion like Christmas had a higher chance of conversion.

So, there are multiple data points related to time that we keep on analysing for different customers which become the feature variables for our time prediction system including:

Weekday vs Weekend

Month end/Start/ Mid of the month

Customers who prefer to order on occasions

Using the above variables, we do a regression analysis to predict the probability of them ordering at different days and times

Designing the right content for every customer

Our entire Machine Learning algorithms revolve around building an effective recommendation system. Recommendation Systems are already a huge value addition to large companies like Facebook, Netflix, Google, Amazon etc.

Mckinsey analysts estimated that 35% of AMAZON’s revenue and 75% of what they watch on NETFLIX came from product recommendations.

In today’s world, every retailer has thousand of products to promote but it is very tough to find the next best product out of thousands to promote for every customer. Therefore, a recommendation system solves this major problem at scale.

We at Xeno use a mix of algorithms typically defined as content based methods (based on similarity of item attributes) and collaborative methods which calculates similarity from interactions between item and users.
Skip the next few lines if you don’t wish to go into more detail about the algorithms, but it will give you better context about how recommendation works in real life.

Collaborative methods work with the interaction matrix for which machine learning is used to learn a function that predicts utility of items to each user where the matrix is typically huge, very sparse and most of values are missing.

User-User Collaborative Filtering: Here we find look alike users based on similarity and recommend items which first user’s look-alike has chosen in past. This algorithm is very effective but takes a lot of time and resources because it requires to compute every user pair information.

Item-Item Collaborative Filtering: It is quite similar to previous algorithm, but instead of finding user’s look-alike, we try finding item’s look-alike. Once we have item’s look-alike matrix, we can easily recommend alike items to user who have purchased any item from the dataset. This algorithm is far less resource consuming than user-user collaborative filtering. Hence, for a new merchant, the algorithm takes far lesser time than user-user collaborate as we don’t need all similarity scores between users. And with fixed number of items, item-item look alike matrix is fixed over time.

Generally for a Restaurant, the next purchase can be predicted using item based similarities. As an example, a guy who generally eats Donut will have higher chances of trying a similar or different donut in his next visit while a guy buying a shirt at H&M, would buy a jeans or a watch in his next visit which can be predicted looking at similar users who have bought the similar items at H&M.

There are 3 distance similarity metrics that are used along with collaborative filtering:

1. Jaccard Similarity: Similarity is based on the number of users which have rated item A and B divided by the number of users who have rated either A or B. It is typically used where we don’t have a numeric rating but just a boolean value like a product being bought or an add being clicked.
2. Cosine Similarity: (as in the Content-Based system) Similarity is the cosine of the angle between the 2 vectors of the item vectors of A and B where each vector can be values from different attributes like Popularity of Product, category of product etc of individual item. Closer the vectors, smaller will be the angle and larger the cosine.
3. Pearson Similarity: Similarity is the Pearson coefficient between the two vectors. For the purpose of diversity, we will use Pearson Similarity in this implementation.

Measuring the Conversions from Campaigns

Till now, we discussed about what algorithms we use for running different campaigns but the more important part is to track if these algorithm's are impacting the overall conversions. For that, we start by measuring the uplift

Measuring uplift is important because it tells the brands about the impact that the Xeno way of campaigning is making over and above their organic return rate and there is no better way to do other than using A/B test.

A/B testing (bucket tests or split-run testing) is a randomized experiment with two variants, A and B.It includes application of statistical hypothesis testing or “two-sample hypothesis testing” as used in the field of statistics.

Generally We break a set of customers into 2 equal segments normally called the A/B test campaigns. For set A, we either don’t send the campaign or send it across in the manner the brands are already sending it. For segment B, we send the campaigns using Xeno’s algorithms as mentioned above.

After getting the results, we compare the conversion rates from both campaigns A and B. The increment of B over A is what determines the actual uplift.

The End Note

The most important thing that we aim with our marketing campaigns is to always send a relevant campaign to the customer and the data point we consider to measure is how many customers are getting converted with each campaign.
For all the customers that don’t return from the marketing, we believe that our marketing communication was not relevant to them which keeps us motivated to improve our algorithm every week.

Hope you found this post useful 💯 and it gives you some insight about how Machine Learning and statistics is helping us to improve conversions at Xeno. 🙌

Thanks for reading. Please give it some 👏 if you liked it 🙏 and also share your feedback in comments 🙌.