The Power of You: Launch of NUDGE — Homepage Personalization at AirAsia.com
More than 50 million passengers fly with AirAsia every year. Our website airasia.com receives every day at least 2 million unique active users who interact with us at various touchpoints and leave a trail of data points that tells us the story of who they are, what they want, when they want it and why they want it. And now, we are listening.
Till recently, www.airasia.com appeared the same for everyone — you, me, the globe-trotting consultant, the mother of three who is planning a get-together, and for everyone else. However, everyone is unique and every customer who visits our website has unique needs and preferences. The new and improved Airasia.com now knows what you want and shows you exactly what you are looking for.
AirAsia.com offers much more than flights — as Asean’s fastest growing travel and lifestyle platform, we now also offer hotels, activities, deals, food, groceries and even content.
Using bleeding-edge machine learning technologies like extreme gradient boosting machines implemented on top of data accumulated from the decades of relationships that our customers have with us, we are now able to predict what our users would most likely be interested in. And by marrying this intent with the desired business outcome, we now render them their own unique homepage experience. The smarter AirAsia.com experience means that no two people potentially see the exact same website given that it would be personalised at an individual level.
The machine learning engine builds predictions based on the rich data that we have on user behavior, purchase sequence and context, collected from our database of the 75 million customers whom we have served over decades.
User experience and product marketing
Personalisation engine is used to customise all the elements of the homepage, that means showing those specific deals that we feel are most relevant for you.
This means, for example, that if the machine learning engine believes you are more likely to buy hotels as compared to activities based on your purchase history, the website will then display the hotels carousel before the activities one. Not only that, but also we will show you the specific hotels and activities that you might be most interested in. We will also re-personalize the homepage for you every time you come back to it, even within the same session.
Beyond just the homepage, the same engine is going to be used to power personalised sorting inside specific product lines including hotels and activities — within these pages, each user would have their own personalised feed.
For illustration, we offer more than 550,000 accommodations worldwide, but not all choices are relevant for you. Using the personalisation engine, we would be able to show you the hand-picked options that you might be most interested in.
The same personalisation engine is also the key driver to our customer communications. Imagine being on holiday and conveniently receiving notifications of exciting offers for tourist activities nearby. Using the personalisation engine, we would be able to understand your present situation and predict your interests at the moment and then communicate that to you via the most efficient channels.
Personalisation becomes even more critical in case of small screen devices like mobile phones which limit the amount of information that we can present to the user at one point of time. Showing engaging content to users on a phone is challenging but even more rewarding.
NUDGE: The Personalisation Engine
At the heart of the entire personalisation effort across www.airasia.com is the intent prediction module which takes the user identifier and fetches the historical data of the user along with a bunch of contextual inputs about the user and creates a rich profile for the user that can be parsed in multiple manners for each use case. The personalisation engine is composed of three separate modules that run independently from each other — the user behavior module, user context module and user sequence module.
User Behavior Module
The user behavior module takes the user identifier as an input and retrieves the entire historical data (inc. geographical, demographic, psychographic) of the user. It uses both strong signals (as demonstrated by transactions) and weak signals (as demonstrated by browsing behavior) and builds a unique profile for each user. Using this profile, it estimates the intent of the user. This module is powered by a extreme gradient boosted tree.
User Sequence Module
The user sequence module takes the user identifier as an input and retrieves the recent interactions of the user with our website. We have observed that most users exhibits some sort of a sequence while catering to their travel needs: Flight booking -> Hotel booking -> Activity booking -> Ancillary booking -> Web checkin, though the exact sequence differs in different scenarios. Using these sequence as an input, the next most likely intent of the user is predicted. This module is powered using a sequential model like recurrent neural nets.
User Context Module
The user context module considers a large number of contextual inputs like the user’s geo-location, browser, device, source/channel, etc.to estimate the user’s most likely intent. It also considers the user’s present and previous sessions’ browsing details to predict the user’s intent better. Since this model runs on a live basis and performance is a key, this module is powered by a logistics regression model.
Why Three Modules?
Why do we need three different modules? Why can’t one module do the job?
It’s tempting to reduce the complexity by aggregating the different components, however if each component serves a specific purpose then that might not be ideal.
Look at the figures on the left. A single linear regression model is unable to capture the nuances of the model with sufficient accuracy. Even if we increase the model complexity, it’s likely that the performance would still be limited. Instead, using multiple simple models increases the accuracy significantly, while also keeping the model complexity under check.
Serendipity Score
To increase engagement it’s often a good idea to display different content every time. This not only has the benefit of exhibiting freshness but can also be used to showcase the immense variety that a large platform has to offer. We do this by adding a serendipity score to the outputs of NUDGE. However, balancing exploitation (showing the most relevant results only) and exploration (showing unique content on every session) is a continuous undertaking.
Incorporating Business Inputs
NUDGE, the intent prediction module is responsible for identifying highest relevance for each of our users. However, to be able to generate true business value, it is crucial to match this with current business priorities. This could be either the key focus for the month, profitability, monetization opportunities or even past performance.
Criteria for Success
It’s important to identify and document the key success metrics before embarking on any project. For NUDGE, the click-through rate is the metric that identifies how well we’ve been able to match users with respective content. To be able to identify the business value that this project generates, we also look at the transactions as well as revenue share via the personalized properties compared to via static content through a standardised A/B test with 99% confidence level threshold. A/B testing is also used to validate continuous improvements as any new versions of a model are subject to the same rigorous tests against the baseline to be scaled to higher share of traffic.
Maintenance
Unfortunately, no models as of now have the ability to stand the test of time and they need to be refreshed on a regular basis.
- Data Drift: Users’ preferences change with time and it’s critical to ensure that the models also get updated to be able to catch up with these trends. Our model is thoroughly retrained using the latest data every 3–6 months.
- System Changes: The platform on top of which NUDGE is built is not static, but is dynamically evolving. Tables, fields, variable names that were available yesterday can disappear after a release, wreaking havoc if not mitigated correctly. Hence, it is imperative to liason closely with other teams to ensure inadvertent down-stream impact of any changes is accounted for and corrected in a timely manner.
- Performance Monitoring: It is critical to track performance on a regular basis, of not just the final output but also of critical paths in the middle. To avoid over-dose of information, the best practice is to set alerts when the performance deviates beyond the pre-defined thresholds. Also, it’s important to log not just the final outputs but also the inputs as well as the model parameters for us to ensure full trackability to enable efficient debugging.
Best Practices for Collaboration with Stakeholders
Large projects span across multiple stakeholders and it’s crucial to align with them to be able to get their full support. Here are a few of the best practices that we follow:
- Get buy-in: Before embarking on a new project, it is crucial to get buy-in from the respective stakeholders. The only way a project can generate business value is when it gets implemented and it’s important to align with the relevant parties to get that done in a timely fashion.
- Clearly define success metrics: Success metrics should be defined before the onset of the project. This brings clarity into the mind of project owners about the target, in terms of either the user experience or the financial objectives. This also helps other stakeholders prioritize this project.
- Close collaboration: It’s important to have weekly meetings to discuss progress, seek clarity, resolve issues, and identify next steps. In addition, there should be clearly defined milestones so that the progress can be measured better.
Workflow Management
Agile way of working has revolutionized software industry and we have observed great benefits by adopting the same methodologies to be able to track our regular progress and manage dependencies. This coupled with detailed documentation allows us to pen down our ideas and share with others for validation and approval. Daily stand-ups help review the open action items and align the next set of priorities.
