How to Build Ads Dynamic Bidding
Amazon Sponsored Products (ads) has an ad bidding product called “Dynamic Bidding.” It works by bidding up to 200% of your cost-per-click bid to increase ad-attributed sales. From Amazon:
“With dynamic bids - up and down' strategy, Amazon will increase your bids in for clicks that may be more likely to convert to a sale. With this strategy, we’ll also reduce your bids for clicks that are less likely to convert to a sale… With the fixed bids [CPC] strategy, Amazon will use your exact bid for all opportunities and won’t adjust your bids. Compared to dynamic bidding strategies, you may get more impressions, but fewer conversions for your ad spend with this strategy
Dynamic bidding works by making a cost-per-click (CPC) ad auction more similar to an ideal ad auction by bidding for sales using machine learning while keeping the CPC click bid in the insertion bid formula. Here’s why dynamic bidding works and how to implement it. This guide is designed for tech leads and product managers creating dynamic bidding in their marketplace or e-commerce app.
Step by Step Dynamic Bid Equation Derivation
Step 1: Start with CPC Bid Equation
Assume you have a calibrated click prediction model to predict the probability of a click given a specific insertion. It may be personalized and contextual. In simple CPC systems, p(click|impression) may be implemented as a global or per advertiser constant rate for all insertions.
Step 2: Show the implied relationship to sales
Expected Sales: Advertisers don’t value clicks. They value sales. Implicitly, advertisers expect some incremental sales per click when bidding for clicks. A lift test would measure this as total incremental ad sales divided by total ad clicks. Note that these are not necessarily “post-click” or “last-click” sales.
Implied CPA Bid: Likewise, implicitly, advertisers are willing to pay some cost per incremental ad sale, or “Implied CPA Bid,” when bidding for clicks. A past willingness to pay for sales for cost-per-click ad delivery estimates an effective lower bound on the value-per-sale to the advertiser on a cost-per-click basis.
Step 3: Re-combine to show the relationship
Multiplying Equations 2 and 3 together, we get back the original cost-per-click Equation 1.
Step 4: Implement “sales per click” numerator with ML model
𝑢(Δsale/click) is assumed to be a constant for every ad click in the CPC bid formula. However, some ad clicks are more likely to generate sales than others. Model this directly with a calibrated machine learning model to replace 𝑢(Δsale/click) in the numerator with a post-click conversion model p(sale|click). A reasonable implementation of p(sale|click) uses de-duplicated clicks and sales per insertion, p(∃sale|∃click|insertion), to better represent a user state transition model.
Step 5: Implement “clicks per sale” denominator to preserve a “stable” mean of 1
We can’t re-implement 𝑢(Δsale/click) in the denominator with the same p(sale|click) model because then the ratio between numerator and denominator would always be 1, and there would be no difference from the CPC bid Equation 1. Going back to Step 2, consider that the value-per-action to the advertiser is stable — it may vary over weeks or seasons, but per day, it’s a constant. Reasonable implementations include:
- OK: The observed average sale per click rate. This estimator is problematic for ads with few recent attributed sales, which may be most ads.
- Better: The average p(sale|click) prediction over a week or some other “reasonable” observation period. This is like borrowing information across ads when data are sparse. When the per-ad p(sale|click) numerator is persistently miscalibrated, the denominator will also be miscalibrated by the same factor, canceling out the bias.
The ratio of p(sale|click) to avg(p(sale|click)) is the raw bid multiplier, see Equation X. It represents how much higher or lower to bid than the average per click as proportionate to the expected sale attributed to a click on the insertion.
Step 6: Add Bid Caps, Transform to “Dynamic Bid”
The raw bid multiplier is unbounded. To provide the guarantee and avoid unreasonable overbidding, cap the bid multiplier to the guaranteed limits. The upper cap is 200% or 150% for “up and down” and 100% for “down only.”
The “Dynamic Bid” is the capped bid multiplier by the click bid. It can be used as a direct replacement for the ClickBid in any CPC auction.
Below are additional discussion topics on building and designing Dynamic Bidding for ads.
Characteristics of a useful p(sale|click) model
For Dynamic Bidding to work (see “The Ideal Ad Auction” below), the p(sale|click) model must correctly distinguish between high and low-value clicks for the same ad. This means the model must include features that are either or both:
- Contextual: certain pages, queries, devices, products, times, or surfaces are higher value than others for the same ad.
- Personalized: clicks by some users are worth more than others, including authenticated versus unauthenticated users.
Many ad systems start with a per-ad average rate to implement terms like p(click|insertion) and p(sale|click). Per the discussion above, a constant per ad simplifies to the CPC bid equation, which defeats the purpose of using Dynamic Bidding.
Further, many ML systems in Search and Discovery are per-item estimates of “quality.” Or, features are predominately per-item historical rates or item features like “price.” These features may help distinguish between overall “better” ads and help explain bias in a personalized and contextual model to improve fit. Still, they will not generate a useful p(sale|click) model for Dynamic Bidding.
To verify that a p(sale|click) model is promising for use in Dynamic Bidding using offline analysis:
- Verify that some feature importance is attributed to user and contextual features (e.g., regression weights, GBDT feature importance measures, NN dropout loss)
- Show that per-item (versus global) normalized entropy and AUC are promising (<100% and >50% respectively).
Refinements to p(sale|click) could account for incrementality or the expected value of the sale, including the quantity or order value, probability of cancelation or return, and delivery costs. However, all these improvements must still be realized as improvements per ad versus modeling across ads to work with Dynamic Bidding.
Incrementality
p(sale|click), or “post click sale,” is a model of “incremental sales” caused by an ad impression. (See how Promoted estimates incrementality for conversion joining). It’s important to optimize towards “incremental” sales because if the ad does not cause the user to buy, for example, then showing that the ad will not increase sales in live production. Not all post-click sales are incremental, not all non-post-click (aka “view-through”) sales are not incremental, and sales can be “fractionally” incremental. However, an increase in post-click ad sales is usually highly correlated with the total lift in true incremental sales as measured in lift tests. Therefore, p(sale|click) is typically a directionally good estimate of incremental sales, but it may be miscalibrated.
Example: Assume a uniform random 50% probability that a “true incremental” conversion has an ad click, and nearly 100% of post-click ad conversions are 100% incremental. Then, a p(click|imp)p(sale|click) model predicts incremental sales with a miscalibration constant of 50%. This miscalibration may not impact the relative priority in value allocation, so p(sale|click) is appropriate for search and ads ranking without adjustments. However, the miscalibration may impact ad pricing and ad value estimation.
The Ideal Ad Auction
The goal of Dynamic Bidding is to move the CPC ad product closer to an ideal ad auction. In the ideal ad auction, each advertiser bids exactly their expected value in dollars from winning each ad insertion to maximize advertiser profits.
Advertisers who most value each ad insertion win that ad insertion. Given no interactions between insertions, this maximizes total platform advertiser value and total advertiser profits, and given a “reasonable” reserve price, it maximizes platform ad revenue.
However, in a fixed bids CPC ad auction, advertisers can only bid a fixed price per click, see Equation 1. This CPC design has three problems in practice:
- Advertisers cannot directly bid for each ad insertion. They can only bid for clicks. However, advertisers don’t value clicks directly. They value incremental sales.
- Some clicks are more valuable than others because some are more likely to convert to a sale. However, a “fixed bid” cost-per-click (CPC) bid values all clicks equally.
- Real auctions are not “dominant strategy” or “bid your true value” auctions. Maximizing the allocation’s (assumed) advertiser value does not necessarily maximize profits because advertisers may be able to win more profit by bidding lower, and therefore, advertiser bids are not reliable signals of true value.
Dynamic Bidding addresses allocation maximization problems (1) and (2), allowing the system to bid for “true value” more correctly per insertion on behalf of advertisers. The more correct the value estimates used in bidding, the more that estimated value will be realized in expectation. Problem (3) relates to pricing and bid incentive theory, which are beyond the scope of Dynamic Bidding (this definition). Assume that any advertiser value measurement using past advertiser bids is a lower bound of true advertiser value in most auction designs.
In summary, bidding closer to the ideal generates more ad value and, therefore, more ad revenue.
Measuring Value
In brief: sum sales. More details below.
The goal of Dynamic Bidding is to increase ad value. Ad value is the total value to advertisers from ads and the maximum sustainable ad revenue that can be billed. Advertisers value sales, so ad value for Dynamic Bidding is the sum of incremental sales. Not all sales are valued equally. They are each valued by, at least, the effective CPA Bid or cost-per-sale, see Equation 3. That is, advertisers are willing to pay for more sales rather than others. This is an important consideration in ads value and potential ads revenue.
If Dynamic Bidding is working correctly, then compared to CPC, Dynamic Bidding will:
- more correctly predict advertiser value (in sales) per insertion better than any per-ad constant multiplier of any click prediction model. Measure this using per-ad distribution of Normalized Entropy (<100% is good) and AUC (>50% is good). If any constant is better, then either Dynamic Bidding will not outperform CPC, or CPC can perform as well as or better Dynamic Bidding with fixed CPC bid adjustments.
- change the allocation of ads. If there is no substantial allocation change of ads, then there can be no difference in ad value generated (prices may differ, but this should be negligible if the allocation is unchanged)
- increase total ad value because ads are more correctly allocated to generate more ad value. Assuming ad volume is constant when compared to CPC ad delivery volume, the total ad system must have become more efficient overall and in expectation per advertiser to generate more total sum ad value if total sum ad value increases.
- does not penalize specific advertisers for realizing value. If a particular advertiser is heavily penalized, and the additional ad value in an A/B test derives from the penalty of this advertiser, then in production, that advertiser will rationally refuse to use Dynamic Bidding, and the increase in ad value will not be realized.
Ad Value Metrics
Ad Value is the sum of incremental ad sales weighted by each sale’s value to the advertiser. The Cost-per-sale bid is a lower bound for each sale’s value. See Equation 3: Implied Sale Bid, or Cost-per-action (CPA) bid.
Generally, ad-attributed conversions that are rarer and have higher click bids are worth more ad value because they have a higher implied conversion bid. An increase in such higher ad value conversions supports higher ad revenues. This is related to an increase in ad CVR but where some conversions are worth more than others in billable advertiser value.
In practice, the effective cost-per-action bid can be difficult to estimate, especially for advertisers with sparse historical information. Therefore, all things being equal, more attributed ad sales (conversion rates (CVR), total sales) and more ad revenue are reasonable proxies for increases in ad value.
An easier heuristic to compute than effective cost-per-action bid is to use the sum click-bid-weighted attributed conversions. This assumes that all conversions have equal conversion rates (false). It is an improvement over CVR and sum sales because it accounts for different click bids, and click bids are not estimates and, therefore, are more appropriate for use in metric definitions.
Ads Value Metrics for A/B testing from simple to more advanced. Note that the dollar value of sales in Ads Value derives from the bidded advertising potential revenue, not the sale price, sale profitability, or cost-of-revenue.
- Total sales and CVR. All things being equal, total sales should increase.
- Post-click sales and CVR. Post-click sales are directionally a better estimate of incrementally.
- Click-bid weighted post-click sales in $. The click bid is one of two terms used in the impliedCPABid, and it’s not an estimate, so it can be computed exactly, making it a more appropriate metric. Generally, sales are worth more in ads value proportional to how much an advertiser is willing to pay per click.
- Implied CPA-weighted post-click sales in $. Sales that require more clicks to realize in expectation are worth more in ad value. However, this rate is an estimate, and so is uncertain and may be unobservable for some low-volume advertisers. Uncertainty is an undesirable property of a launch metric even though this is technically a more correct metric. A reasonable heuristic is to assume a global rate that shrinks to an observed post-click sales rate when data are available. If all advertisers used the global rate, then this metric would be functionally the same as Click-bid weighted post-click sales.
Ad Revenue should also increase because the Ad Value increases. However, it may require advertisers to raise bids or budgets to realize the revenue depending on the ad auction design and billing.
