Flexible Daily Budgeting at Pinterest
Kelvin Jiang, Software Engineer, Ads Intelligence | Keshava Subramanya, Engineering Manager, Ads Intelligence | Perrye Ogunwole, Product Manager, Ads Intelligence
Introduction
The Ads Intelligence team at Pinterest builds products that help advertisers maximize the value they get out of their ad campaigns. As part of that initiative, we have recently launched Flexible Daily Budgets (FDB) to US advertisers in open beta.
FDB is an automated ads product that benefits advertisers by distributing the advertising budget for each campaign across multiple days in an automated manner. The goals of this product are to:
- Enable advertisers to spend across all hours of the day
- Give advertisers the ability to “roll over” unspent budgets while meeting their target daily goal across a period of time
- Help advertisers achieve the lowest cost per result by allowing the campaign to spend more on days with greater opportunities and vice versa
Problem statement & motivations
Advertisers have two options for how they set budgets for their campaigns on Pinterest:
- Daily budgeting: “I want to spend no more than $X per day” either indefinitely or during a specified period of time
- Lifetime budgeting: “I want to spend no more than $Y over a specified period of time”
For both budgeting types, the technical challenge is to figure out how to spend the advertiser’s specified budget amount in the allotted time while still spending the budget in full. This challenge is addressed through our budgeting, pacing, and bidding systems.
Under our old daily budgeting offering, the advertiser’s daily budget became a strict daily cap in our system. If the advertiser is using automatic bidding, our pacing and bidding systems would aim to spend up to this strict daily cap before the end of the day, leaving some buffer in case we need more time to fulfill the amount. If we’re not able to spend the full amount on a given day, the unspent budget is lost and we start over the next day, aiming to spend the same amount again.
This system can potentially create two different problems for advertisers.
Problem 1: Advertiser A can’t reach their target audience. Let’s assume Advertiser A sets a daily budget of $100 for a holiday campaign lasting two days. The following could occur:
Day 1:
- We aim to spend $100 by ~21:00
- At 21:00, actual spend is $99.07
- At 21:22, actual spend reaches $100 and the campaign stops spending for the day
Day 2:
- We aim to spend $100 by ~21:00.
- At 21:00, actual spend is $99.59
- At 21:04, actual spend reaches $100 and the campaign stops spending for the day
The advertiser has spent their entire campaign budget of $200, but on both days their ads stopped showing at around 4:00 PM in their target audience’s time zone, exactly when Advertiser A knows they’re most likely to be on Pinterest.
Problem 2: Advertiser B can’t fulfill their budget. Let’s assume Advertiser B sets a daily budget of $500 for a holiday campaign lasting two days. The following could occur:
Day 1:
- We aim to spend $500 by ~21:00
- At 21:00, actual spend is $399.52
- At 23:59, actual spend is $438.52
- Advertiser tweaks their campaign to improve performance (i.e. broadens their targeting)
Day 2:
- We aim to spend $500 by ~21:00.
- At 21:00, actual spend is $499.73
- At 21:02, actual spend reaches $500 and the campaign stops spending for the day
The advertiser only spent $938.52 of their $1000 total campaign budget, even after making some improvements to their campaign once they observed underdelivery on Day 1. While performance was better on Day 2, the $61.48 in unspent budget on Day 1 can’t be utilized unless they manually increase their daily budget on Day 2.
Our approach
FDB modernizes the daily budget offering at Pinterest by changing the mechanics of daily budgeted campaigns.
First, it removes the strict daily cap on campaign spending by allowing for rollover of unspent budget from each day, while aiming to average out to a daily spend equivalent to the advertiser’s target daily budget across a full weekly period.
Second, it introduces flexibility by allowing for overspend each day, up to a certain limit.
Additionally, FDB introduces a mechanism for fine tuning the pace at which each campaign spends throughout the day, in order to maximize the amount of time it is eligible to spend. This is powered by a prediction system — modeled as a regression task — that forecasts the delivery performance of a campaign for the upcoming day, based on historical data.
The system uses the predicted delivery performance of each campaign to customize the underlying pacing controller to allow each campaign to smoothly and efficiently deliver ads across an entire day.
With the capabilities introduced by FDB, advertisers whose campaigns finish their budgets “early” can continue spending past their daily budget amount so they can reach Pinners during all hours of the day. Also, advertisers whose campaigns don’t finish by the end of the day can automatically have unspent budgets rolled forward to the next day (without needing to manually update their daily budget after they notice some underspend).
Once a campaign gains the ability to spend across the entire day, instead of being somewhat “constrained” to spend within a segment of the day, it is expected to achieve lower costs as its spend is spread out.
Experimentation
Prior to testing FDB with advertisers’ live campaigns, we ran a large-scale experiment to test our hypothesis that rolling over budgets and enabling spend across the entire day can improve the delivery efficiency (lower cost per result) of daily budget campaigns while maintaining neutral overall spend.
Since the rollover mechanism needed for testing did not exist for daily budgets at the timing of this experiment, we developed a novel approach of using lifetime budgeted campaigns (which already have this mechanism) as a testbed to simulate daily budgeted campaign behavior. This was done by allocating slices of each campaign’s budget to different treatment experiences, while leaving enough of a buffer to allow for any overspend or underspend that may unevenly occur across the slices of budget.
We tested multiple FDB candidate treatments that introduced full-day pacing with budget rollover for campaigns utilizing daily budgets, trying a combination of heuristic-based, data-driven, and model-based implementations. Our experimentation showed that all experiment treatments resulted in significantly lower campaign costs with minimal impact on overall spend.
Future work
FDB is the first step in our journey to improve the ads budgeting products on Pinterest. In the future, we plan to explore the use of more sophisticated forecasting techniques to actively seek out opportunities for each campaign on the platform, and to use those forecasts to optimally allocate advertiser budgets across time. The challenges that lie ahead are a refreshing mix of problems that span time series analysis, machine learning, and optimization techniques.
If you’re interested in helping build the next generation of ML powered advertiser solutions and build products like Advertiser Recommendation Systems, Campaign Budgets, and Audience Sizing, do drop us a line!
Acknowledgements
Ads Intelligence: Kelvin Jiang, Yeming Shi, Joey Huang, Keshava Subramanya
Ads Quality: Christina Huang, Robert Gordan, Nitish Ratan Appanasamy, Andrei Curelea, Akanksha Baid
Ads Infra: Mingsi Liu, Sameer Bhide, Aniket Ketkar, Ge Huang, Priyan Agarwal, Marc Yang, Crystiane Meira, Vincent Phan
Ads Reporting: Sanchay Javeria, Insu Lee
Advertiser Automation: Dani Gnibus, Jayanth Mettu, Eric Le
PMM: Kimmie O’Callaghan
PADS: Andrea Burbank, Kandarp Srivastava, Van Wang
PM: Perrye Ogunwole, Dmitry Lubensky
To learn more about engineering at Pinterest, check out the rest of our Engineering Blog and visit our Pinterest Labs site. To explore life at Pinterest, visit our Careers page.
