Machine Intelligence in Ad Tech

Much has been written about the rise of AI in ad tech. Much has also been written debunking the myth of AI. The truth seems to be somewhere in the middle though. It’s important to understand that AI is a very broad term thrown around, similar to big-data. Whenever you have promising technology, the hype-cycles are bound to happen. But, it shall not be prudent to disregard the progress of AI independent of the hype-cycles.

Actually, AI does not have to signify artificial ‘general’ intelligence (AGI) which possess human level intelligence. An ant trying to lay markers from its nest to the food source is still considered to be intelligent. Any machine intelligence that models the underlying functions of a solution to a problem, without human’s having to hand-craft those solutions can actually be termed as AI.

Here is a Layman’s intro to AI if you are interested to know more about the technical aspects of AI.

Of course you do have a scale, measuring how weak or strong the modeled function is before you apply it to the specific domain depending on the utility you gain from such machine autonomy. The current crop of AI being applied to specialized fields of study are more domain or problem specific and narrow in nature. It is still AI though and is categorized as ANI or artificial narrow intelligence. Most of the AI research being conducted by known tech giants like Google, Facebook, Baidu, Apple, Microsoft, IBM falls into the category of ANI. However, notice that few among them have a significant portion of revenue attributed to in-house ad tech (media revenues).

The recent rise of AI can be attributed to two factors. Advancements in the field of deep neural networks (or Deep-Learning) and affordability of accelerated hardware called the GPUs (graphic processing units) that reduces the compute time for learning. Deep neural networks (or DNNs) are quite powerful models which are broadly used for pattern recognition, anomaly detection and prediction.

Why Deep Learning?

It’s important to note that DNNs work very well on unstructured data which has very high dimensionality. Let’s get these terms straight.

Unstructured Data — Any data that is quite raw, is not organized, and does not represent a definitive form is called unstructured data. Any place you have a human-narrative is quite unstructured. For example, emails, spreadsheets, reports, consumer complaints, social media posts, comments etc. are all forms of unstructured data. I can throw audio and video into the mix which is on a significant rise as well. Independent studies by organizations like IDC, Merrill Lynch, EMC, Computer World claims that 80%-90% of organizational data is unstructured and this data can grow in excess of 40 zettabytes by 2020.

High Dimensionality — A dimension of data is a feature that describes one specific aspect of the data. For example, in healthcare, blood pressure which is a ratio of systolic (maximum) over diastolic (minimum) pressure can be considered as one dimension. Likewise, you can measure many dimensions of a patient’s heart condition, immune system, state of internal organs, genetic makeup, nutritional status etc which can each add hundreds of thousands of dimensions making the patient healthcare high dimensional.

AI in Advertising?

Advertising is a field that is ripe with unstructured and high-dimensional data. Advertising edicts, bylines, articles, reports, jingles, ad copy, ad creatives, social sentiments, user generated data, brand guidelines, audio and video content are all forms of unstructured data. From a dimensionality perspective, the number of products and categories in the advertising space and feature of each product and the utility it provides contributes to dimensionality explosion. Added to this, the demographic, behavior, geography, consumption habits, social bias, cultural make and content consumption channels of an individual consumer explodes the dimension of data beyond comprehension.

Also, unlike search where a user has a stated intent by typing in what she is looking for, advertising relies on deciphering the ‘moments’ of the consumer to understand the probable subtle intent of the consumer in order to influence her with relevant propositions. A moment can be made up of a multitude of dimensions such as the time of the day, weather, day of the week, seasonality, region, channel of engagement, device of access, consumer cohort, individual behavior etc.

Given this, hand-crafting human rules to such unstructured and high-dimensional data is not only intractable but also dilutes the relevancy of propositions. AI could be what is required in this time of need for ad tech to break the plateau of irrelevance and annoyance to consumers.

Where do we start?

The current research and applications of AI in ad tech are in the areas of fraud-detection, audience insights, recommendations and auction pricing.

Fraud Detection — One of the biggest pain points for advertisers and Ad-Tech players is publisher fraud, be it on mobile or desktop. Unlike the yesteryears, fraud has become quite sophisticated and is becoming almost undetectable to human-rules. Given the high-dimensional nature of publisher signals which can be in excess of 100+ dimensions under the category of device, carrier, internet service provider, access points, publisher site and consumer engagement patterns, it becomes quite important to distinguish a bot engagement patterns from a real human. Deep learning is quite powerful in anomaly detection on engagement patterns and categorize fraudulent traffic.

Audience Insights — The ability to infer the state of an individual at a given moment is the holy grail of advertising. If you can properly infer the state, then one can choose to either influence the consumer through a relevant custom crafted advertisement or actually leave the consumer alone and reduce ad-spillage. AI comes in handy during inference. Given that ad creatives are highly unstructured, you can use AI to start drawing correlations to the compositional structure of what the creative image contains to the preference of a consumer. We can start doing chromatic analysis (color scheme preferences) to the moment of the consumer.

• Does she prefer ads with certain color schemes during a certain time of the day?
• Does she prefer simplistic ads with less visual clutter with no-text or does she prefer decorated creatives?
• Does she prefer certain artistic style of creatives? This can be decomposed to beyond just a static ad creative and insights can be explored for interactive ads, audio ads and video ads as well.
• Also, note that the brand edicts or ad content also is unstructured and is part of the advertisement and word correlations can be identified through deep learning.

Recommendations — Once you make some progress on identifying the preference of a consumer, recommending the right advertisement which approximately matches the preference of each consumer is another area of investment for AI. At any given moment, a consumer can be classified under hundreds of personas. She can be a pet lover, frequent traveller, soccer mom at the category level, but also someone who prefers a particular type of interactive creative on a particular publisher at particular time of the day. A pet food advertiser and a travel booking advertiser may both want to grab the consumer’s attention at that same moment. The ability to identify the dominant behavior of the consumer at the given moment and the ability to recommend the right ad which is higher in relevancy curve (independent of the highest bidder) is an area where AI can excel.

Auction Pricing — As stated, real-time bidding for the attention of a consumer packaged as ‘media’ on ad-exchanges is the norm of the day. How do you know how much to bid for? The price of a bid should be proportional to the benefit you gain from influencing the consumer at a given moment. The benefit ranges across the lifecycle of the consumer actually. Is the ad proposition for first time consumer acquisition (app download?), consumer engagement for mid-funnel prospecting? cross-sell or up-sell for increasing lifetime value of the consumer? Direct response for feedback loops? Also, you have to consider the premiums or going-rate for a consumer moment which might be high due to other advertisers also bidding for the same moment. If you have to bid to win, you have to consider the win-ratio for the similar moments in your immediate history. If you have to preserve your margins, you should be aware of the rebates that are possible during off-peak times for certain other moments which may have similar utility. All these makes pricing a bid in an auction quite complicated and deep learning is proving to be quite handy in this process. Improving your bid efficiency can save in-excess of 40–50% on your margins actually.

This is only the beginning and implementation across the ecosystem is already happening. AI in ad tech can then move towards operational activities like campaign management, account management, and inside sales. The next step is to automate the generation of creatives in realtime with machine generated images and text. The possibilities are astounding and probably right around the horizon.