Popularity Engine: Heuristic-Based Ad Serving Algorithm

Airtel offers cutting-edge B2B and B2C solutions in today’s dynamic digital landscape through multiple digital channels and applications. Airtel provides services through its Wynk Music App, the Airtel Thanks App, and the Xstream video streaming app, with approximately 120 million monthly active users across these platforms.

In addition to their core services, these digital platforms also have profitable segments where they promote other brands and their products to their users. The Airtel Ads team manages all advertising on various digital platforms as a separate vertical, aiming to deliver relevant advertisements that users may choose to view.

Prerequisite

Let’s delve into how ads work in the Ad Tech ecosystem:

Brand/Advertiser: This is the entity that promotes its product or seeks to advertise to raise awareness or achieve specific Key Performance Indicators (KPIs).

Publisher: These are platforms where users encounter ads, such as Wynk.

Ads System: This encompasses various components that facilitate the connection between the Advertiser and the Publisher (e.g., Thanks App, Wynk, Xstream) through targeting.

Within these components, the Ads System comprises a broad array of elements, including the Brand (along with the goals that need to be achieved), Audiences (encompassing in-market segments like affluence, etc.), Creative Management (visual components of ads), Bidding System, and many more.

Problem Statement

In the world of advertising, it is essential to display relevant ads to users in order to enhance the Click-Through Rate (CTR). We employ collaborative recommendation techniques and a fundamental model of the Epsilon-decreasing strategy. This approach utilises dimensions such as geography (cities) and cohorts (audiences/segments) and continuously refines ad delivery based on user feedback.

While operating within this Ads system, the engine typically serves advertisements to users by considering user-specific details and constructing audiences. These audiences are formulated based on factors like geography and demographics to ensure greater relevance to the users.

One of the other expectation was to adhere to a response time of < 100 ms (p99) which was accomplished by the below mentioned architecture.

Solution Proposed

Building on the aforementioned points, we’ve developed a Heuristic-based algorithm, known as the Popularity Engine. In this approach, brands have the chance to showcase their popular creatives (graphics), which are randomly displayed to all relevant users up to a specified threshold limit of impressions served to users. Furthermore, we analyse user feedback and persist in showing ads in alignment with emerging trends. This approach is commonly referred to as collaborative recommendation.

Explanation

Airtel Ads recently ran a campaign called ‘Buddy Loan,’ featuring three creatives. Two of these creatives performed well, while the one with lower-than-expected results was removed from circulation.

Because of the popularity algorithm, which operates based on geography and cohorts, we identified that the first and third creatives are more popular in their respective geography and cohort.
For example, Creative 1 achieved a CTR of 1.7% for South Delhi and crypto users, and a CTR of 3% for Chennai and crypto users. This performance exceeded that of the second creative. As an optimisation strategy, the algorithm served relevant creatives to users based on their geography and cohorts.

Creatives for Buddy Loan campaign

Technical Implementation

• Client/User: These individuals are the ones who view advertisements on their devices.
• AdSystem: The heart of the Ad Serving system, which comprises multiple components for processing and delivering ads.
• Performance Campaigns: These are marketing campaigns that concentrate on achieving specific, measurable goals and key performance indicators (KPIs). The primary aim of these campaigns is to produce concrete results and demonstrate a return on investment (ROI).
• Personalisation: An intelligence/recommendation engine constructed upon user demographic data. It can employ various algorithms, including Popularity, to serve ads.
• Amazon S3: This is an object storage system used to gather user feedback data from previously served ads.
• Amazon EMR: Elastic Map Reduce, built on top of Apache Spark, is employed to process large-scale data with implicit data parallelism and fault tolerance.
• Airflow: Built on top of Apache Airflow, this is a workflow management platform for data engineering pipelines.
• Mongo: A NoSQL database used to store key-value pair data.
• Postgres: A SQL database employed to store aggregated data, which allows for low-latency aggregated queries.
• Redis: An in-memory storage system utilised for distributed data structures.

To serve ads using the popularity-based approach, there are several stages involved, from data collection to delivering popular ads, as outlined below:

HLD for Popularity Engine

Stage-1 (User Feedback)

• While ads are served to users through Airtel Apps (Wynk, ThanksApp & xStream), the metrics related to serving those ads are collected from the Ad Server.
• The data is then stored in the AWS S3 Object Store due to its substantial size. Typically, user feedback is generated on an hourly basis, with a compressed format data size of 2 GB and a consumable format size of 15 GB. This feedback data is produced every hour, with a transaction rate of 5K for ad serving.
• This data includes all relevant details regarding recently served ads to users and how the users have interacted with them, along with all metadata, including demographic information. It encompasses all the dimensions that can later be utilised for probabilistic algorithms.

Stage-2 (Data Transformation Layer)

• The Airtel Ads system processes user-feedback data on an hourly basis to compute the logic required for delivering the Key Performance Indicators (KPIs) for any campaign run by brands on Airtel. To achieve this, Airflow has been configured to trigger jobs on an hourly basis.
• Airflow is utilised to trigger the Spark job (EMR) responsible for processing the substantial amount of data. In the event of data size scaling up, this setup won’t impact processing time since Apache Spark can efficiently handle high-volume data.
• Within Airtel, we’ve identified specific dimensions for aggregating the overall user feedback loop, which currently include Cohorts, City, and City * (cross) Cohorts.
• The processed data is then pushed to MongoDB in a somewhat raw format, making it accessible for multiple use-cases.
• Subsequently, a job is used to trigger another process, which calculates the Click-Through Rate (CTR) against all the required dimensions. This aggregated Key-Value pair data is written to PostgreSQL, ensuring low-latency access.
• For example, consider a user who is a food enthusiast and has interacted with an advertisement. This user will be categorized as someone from Delhi who is interested in food and has interacted with an advertisement falling under campaign categories like entertainment, BFSI, etc.

Stage-3 (Data Serving via Popularity Algorithm)

• The metrics on each defined dimension computed from the feedback logs for any campaign can be visualise as

Campaign’s creative feedback by CTR metric

• Case-A (Cold Start)
  
  In the scenario where all campaigns are newly launched and their impression count served to users is below the threshold of 10,000, the list of campaigns is randomly shuffled and their graphics/visuals (creatives) are delivered to users in a Round-Robin fashion.
  At Airtel, we have set a threshold limit of 10,000 impressions for each campaign, and this threshold is used to distribute the creatives in a Round-Robin fashion. This distribution is facilitated by Redis via the RPOPLPUSH source destination mechanism.

Round Robin across Popularity Service

And once each creative will cross the threshold we compare the effectiveness of each creative by expected KPI.

• Case-B (Some Campaign has crossed threshold)

In the above example, Campaign A consists of two creatives, namely creative1 and creative2. When we evaluate the effective Click-Through Rate (CTR), a metric for ad clicks, we find that creative1 has a CTR of 1.5, while creative2 has a CTR of 1.3.

Suppose the expected Key Performance Indicator (KPI) for each creative is set at 1.4. In this case, creative1 surpasses the threshold and is served to users. However, creative2, performing below the threshold, is removed from the list of creatives.

Regarding the overall campaign list (as there are typically multiple ad slots in any app to display ads to users), ads are served through A/B testing. This means that 50% of users receive popular creatives, while the remaining 50% are exposed to fresh campaigns that are still in their initial stages (part of a cold start).

• Case-C (All campaigns has crossed)

When all the creatives have exceeded the threshold, the underperforming ones will be removed, and the rest will be presented in descending order of CTR value. In other words, the most popular creative will be displayed at the top, with the remaining creatives positioned accordingly in the ad slots within an app.

At Airtel, we employ this algorithm for performance campaigns, allowing each brand to establish its expectations for the campaigns. Based on the visual and performance data, we can assess whether a campaign has the potential to achieve the defined Key Performance Indicators (KPIs) through this mechanism.

Conclusion

In summary, we have delved into the fundamentals of the Heuristic-based algorithm, also known as the Popularity Engine, and have gained insights into using the user-feedback loop to enhance the performance of any system.
Moreover, the Popularity Engine is a versatile technique that can be applied across various industries, such as e-commerce and social media platforms.

Through this exploration, we have achieved the following milestones:

1. The development of a Heuristic-based algorithm to serve each creative for fresh campaigns, facilitating continuous learning.
2. Improved Click-Through Rates (CTR) for each category, serving as a benchmark for setting KPI expectations for brands running campaigns with Airtel.
3. The automated removal and enhancement of low-performing or inefficient creatives (visuals) from any campaign.
4. Realising an approximately 100% increase in CTR compared to campaigns served without any recommendation logic.
5. Future enhancements will involve the implementation of pacing algorithms in conjunction with the popularity algorithm.

Acknowledgement

I hope you’ve found this article insightful and valuable. Your feedback is important to us, so please don’t hesitate to leave your thoughts or questions in the comments below.

Special thanks to Airtel Ads team and all stakeholders who played a crucial role in bringing this feature to life.

Airtel Ads Team:
Ajit Singh, Sanket Arora, Parshant Verma, Rajat Kukrety, Shubhi Garg, Tapesh Gupta, Rajat Kumar, Sanjeev Yadav, Ahana Nath, Ayush Gupta, Rahul Sharma, Shivang Chaturvedi