Optimizing Bid guardrails by constructing demand curves in real-time bidding

Manoj Kumar Rajendran, Principal data scientist, MiQ

In programmatic advertising, we use real-time bidding to ensure we are always buying advertising inventory at the best price for both the buyer and the seller. Our automated technology relies on pricing analytics models to calculate the optimal bid, but most of the pricing analytical solutions that are designed and deployed in the conventional retail space don’t fit the programmatic world. Getting the models right is critical to ensure advertisers neither waste budget, nor miss opportunities to reach their audience. They need to be tailor-made to optimally suit the enormous complexities of the ever-growing and super competitive real-time bidding (RTB) environment. The key focus area of the work outlined in this article was to construct a customized demand curve for the programmatic setup to smartly bid and optimize the campaign spending.

Introduction to real-time bidding

Advertising on the internet uses targeting to show ads dynamically to users. Simply put, the ads we see when we use the internet are intended for us as individuals. For example, whenever I load a web page, the potential ad slots (whitespaces in the webpage) which we will refer to as inventory, are up for grabs by advertisers. Thousands of advertisers now compete to show their ads to me. Basic anonymous details of that browsing instance are sent to them, most of which are contextual, including browser, ISP, geography, time of the day, and URL. The advertisers will now analyze if they want to target me with their ads. The willing advertisers now make bids and the one with the highest bid gets to show their ad to me. All this happens within 1/10th of a second. Welcome to the fascinating world of real-time bidding (RTB)!

Conventional demand curve (price elasticity)

Price elasticity, in mathematical terms, is the percentage change in demand for a percentage change in price. This relationship is always negative, i.e. when prices rise, demand reduces — unless we talk about the rare cases of “Giffen Goods”. The magnitude of the elasticity indicates “how price-sensitive” the item is. Except for perfectly elastic items (elasticity = -1), the economics literature doesn’t draw clear boundaries for highly elastic and inelastic items. Apple products could be considered to fall under the inelastic banner. Given the dedicated and affluent customer base, a $20 price change isn’t going to drop the demand significantly. Now, imagine a budget smartphone with a price tag of $200 going through a $20 price increase. This could cause a landslide drop in the demand. The low price point is the USP of this segment, so they are extremely price sensitive.

The following graph demonstrates the drop in demand with an increase in the price of the item (elastic Item).

Fig. 1 — Variation in demand with price

The widely adopted methodology to calculate price sensitivity is building a log-log linear model. In simple terms, build a linear model with ‘log(demand)’ as the dependent variable (Y) and ‘log(price)’ as the independent variable (X). For simplicity, I am going to use the definition to calculate the Elasticity value, percentage change in demand for a percentage change in price, using the first two data points (famously referred as point to point elasticity) in the table; initial price (p1) = $100; initial demand (q1) = 366; new price (p2) = 102; new demand(q2) = 350;

Elasticity = ((q2-q1)/q1)/((p2-p1)/p1) = ((350–366)/366)/((102–100)/100)= -2.18

The interpretation is straightforward, for a 1% increase in price, the decrease in demand will be -2.18%. Using a different set of points will result in different elasticity values, so it is recommended to build a log-log linear model with all the data points as suggested earlier.

Challenges in modeling demand curve for RTB

If we are to extend the concept of the price elasticity concept to RTB, it will be the percentage change in the number of impressions/ad inventory units we secure for the percentage change in bid amount. Note, that the elasticity values will be positive here. The higher the bid amount, the more inventory we will be able to secure.

Like physical inventory, ad space in different websites carries different values depending on the daily digital footprint, customer base, brand value, position on the web page, etc. Similarly, a video slot is pricier than a display slot, mobile ad space could be costlier than that of tablets.

When setting up the campaign, we need to decide the audience list (target segments) we want to reach, which might have several different variations, and each of these audience segments could be viewing a wide range of ad space features such as website/App, display/video, ad size, ad position, etc. Every combination of these features can be considered as a unique item with a value/price tagged to them. In short, billions if not trillions of items with varied value propositions are up for grabs when we set up the campaign.

Herein lies the challenge — it is impossible to predict the value of this massive amount of inventory and have those individual bid values incorporated during the campaign setup as the resulting dataset would be too large to process. This gets further complicated by the nature of RTB, meaning the inventory doesn’t have a fixed value. Ad spaces may have a winning bid of 0.2 cents this second and 1 cent in the next second.

What if I told you that when I explored the winning bid distribution for a specific website, I realized the winning bid could be as low as 10 cents CPM to as high as 10 dollars CPM (a 100x difference). The law of elasticity gets tossed out in these cases.

Fig. 2 — Impressions secured at different winning bid price

Monotonic cumulative demand function

Let us break the section header into three parts to understand our novel approach better. A monotonic function is a function that is either entirely non-increasing or non-decreasing. The only way to make the curve singular (directionally), to ensure that there are no price increases that are so big they deter bidding, is if we take the cumulative values. As demand is a non-negative entity, the cumulative value is going to make the above demand curve monotonic, the only permissible relationship in the realms of elasticity. Given that the bid value is a real number, binning them is essential. In short, bin the x-axis bid values and calculate the cumulative y-axis impressions value.

Fig. 3 — Achieving Monotonic relationship between impressions and bid price

The process of taking cumulative impressions is intuitive in a bid environment. You need to bid more if you want to secure more. As the demand curve is now linear, we can calculate elasticity using point to point estimate as previously done, initial price (p1) = $1; initial demand (q1) = 3662644; new price (p2) = 2; new demand(q2) = 5792113;

Elasticity = ((q2-q1)/q1)/((p2-p1)/p1) = ((5792113–3662644)/ 3662644)/((2–1)/1)= 0.58

For a 1% change in the bid price, the impressions we can secure increases by 0.58%, isn’t that neat? To protect our IP, I demonstrated elasticity calculation using simple point-to-point estimation. We will be publishing the details of the actual methodology in a research paper soon. This is critical because a wrong estimation of elasticity results in either overpaying or under-delivering the promised impressions to your business.

Leveraging linearity and discarding saturation

One of the assumptions of linear regression modeling is that the relationship between the dependent and independent variables should be linear. Well, this assumption/requirement is mandatory but is not necessarily true for the entire range of the values. The dependent variable reaches saturation at some point. Assume that I am trying to predict store sales using linear regression. The number of employees will be one of the important independent variables. Their relationship will be linear, but it doesn’t necessarily mean that store sales will be infinite as I hire thousands and millions of employees.

Now drawing parallels to the RTB setup, the number of impressions we secure will increase linearly with the bid values to some extent. After that, we may still win impressions, but it saturates, mainly because of limited inventory — limited number of people in our target audience viewing the pages and channels available to target. It is not worth the money; this is the point of inflection (where linearity ends and saturation kicks in).

In the graph below, we can see the relationship between impressions and bid values across weeks. The red dots indicate the inflection points across weeks and the blue vertical line indicates their harmonic mean. We can incorporate this learning in the DSP and never bid above $4.82 CPM. It is interesting to note that the bid values are as high as $35 CPM for a few impressions across the weeks. Instead of securing very few impressions with an astronomical bid price on this specific website, the budget could be better spent on other/similar websites in the linear range to reach more of your target audience at a good price.

Fig. 4 — Inflection points separating the linear and saturation regions of demand curve

Using this model, we have answered the question, what’s the maximum I can bid for an inventory? But what about the minimum bid? This is critical when we consider securing impressions in a highly competitive but limited inventory category. Let’s look at this example: a primetime slot on leading news websites during elections/political campaigns. In the graph below, we see that the minimum bid price is as high as $10 CPM.

Fig. 5 — Demand curves with multiple linear and saturation regions

The last two plots are quintessential to my point. In the first case, $4.82 is more than enough to secure millions of impressions but falls short massively in the second case (not even one impression!). Setting up the big guard rails helps in bidding in the sweet spot of the demand curve, where for every extra cent, we get significantly extra impressions. There is one key difference in the above plot compared to the previous plots, this demand curve has multiple linear and saturation regions. This indicates that a 10 cents price variation in the first linear region has a different impact than that in the second linear region. Simply put, the price slightly higher than the price point at which the first linear region starts could be used as the minimum bid price.

Conclusion

“Every problem has a solution; it may sometimes just need another perspective.” This quote is the essence of our attempt to create a tailor-made solution to estimate price sensitivity in the RTB setup. As a trader or an analyst in the programmatic domain, have you ever wondered why certain impressions were bought at a very high price? Maybe you targeted the inventory in its saturation region and wasted your precious campaign budget. The in-depth analysis of bid patterns across campaigns revealed that bidders secure approximately 3% of the impressions in the saturation region, indicating that the revenue savings is higher than 3% (as they are costly impressions). Have you wondered why some line items don’t spend at all? Well, the bid values could have been less than the minimum bid (not referring to the clearing/flooring price set by the auctioneers). The suggested minimum bid approach will address this problem. This article is just the first step in MiQ’s long challenging journey to create an in-house smart-bidding solution.

Reference

1. Xiang Li and Devin Guan, Programmatic Buying Bidding Strategies with Win Rate and Winning Price Estimation in Real Time Mobile Advertising http://www0.cs.ucl.ac.uk/staff/w.zhang/rtb-papers/bid-drawbridge.pdf
2. The statistical determination of demand curves https://www.jstor.org/stable/pdf/1883264.pdf
3. Weinan Zhang, Shuai Yuan, Jun Wang, Optimal Real-Time bidding for display advertising https://www.researchgate.net/publication/264397624_Optimal_Real-Time_Bidding_for_Display_Advertising
4. Price elasticity https://en.wikipedia.org/wiki/Price_elasticity_of_demand