Improving Efficiency of Thumbtack’s Online Marketplace

Authors: Bharadwaj Ramachandran, Pietro Montanarella, Navneet Rao

Online marketplaces exist to facilitate transactions between buyers and sellers, with buyers representing the demand and sellers representing the supply. Homeowners on Thumbtack, riders on Uber or shoppers on Amazon are all examples of customers who represent the demand side of an online marketplace. On the other hand, the supply side is made up of those offering goods or services through a marketplace — professionals on Thumbtack, drivers on Uber or vendors on Amazon. Online marketplaces typically seek to create a great customer experience and maximize fruitful transactions between buyers and sellers on their platform. Trying to deal with imbalances between demand and supply is an important challenge faced by online marketplaces. In this blog we’ll explore how one can categorize the health of a market based on demand/supply characteristics. We’ll give an overview of demand constrained and supply constrained markets and how online marketplaces approach these constraints. We’ll then dive deeper into how supply can get constrained in Thumbtack’s marketplace, and how we design initiatives that create a more efficient online marketplace that optimizes the customer experience for millions of US households.

Categorizing Market Health

Within a marketplace, there may be distinct “markets”. What constitutes a market depends on the marketplace — at Thumbtack, a market might be defined as a combination of an occupation and a city, for example Lawn care professionals in San Diego.

One of the ways in which we can categorize the health of a marketplace is based on supply and demand characteristics:

1. Healthy markets: High demand, high supply
2. Supply constrained markets: High demand, low supply
3. Demand constrained markets: Low demand, high supply
4. Unhealthy markets: Low demand, low supply

Figure 1. Categorization of Market Health based on supply and demand characteristics

As facilitators of a thriving marketplace, we strive to build one where we are creating great outcomes for customers & service professionals (pros). We might look at multiple metrics (like conversion or supply retention) and have a fairly nuanced way to define which quadrant a market falls in at a particular moment in time, or perhaps have even more subdivisions in each. However, for the purposes of this blog lets assume there exists an easy way to define this, and a market falls cleanly into one of these 4 quadrants.

With or without any initiatives that drive efficiency, over time, markets can move between the four quadrants we’ve defined. At Thumbtack, for example, we might see seasonal variation within markets (e.g. lawn care demand picking up in April in New York), but we might also see broad changes that affect all markets, such as the sharp drop in demand in March 2020 (relevant blog).

Since we want to increase the number of fruitful transactions, as well as longer term retention on both sides of the marketplace, we create initiatives to help optimize this. If a market is healthy and runs efficiently, there’s less of a need to make any changes. Unhealthy markets, despite presenting significant opportunities to improve efficiency, are harder to jumpstart. Demand constrained & supply constrained markets tend to present interesting opportunities to create fruitful optimizations. Before we go into the nuances of a supply-constrained world, let’s first consider demand constrained ones. This will help illustrate supply constraints later on.

Demand constrained: The incremental demand opportunity

Demand is an important currency for a marketplace. When a marketplace’s supply outstrips its demand, both the supply-side and the marketplace have an opportunity and desire to create more business. The excess supply becomes underutilized. On Thumbtack, excess supply might mean professionals have the capacity to do more jobs and have emptier calendars than they’d ideally like. Having some excess supply isn’t always a bad thing; it can create healthy competition between various sellers and deliver more value for customers at lower prices. But marketplaces need to balance that with the desire to drive more demand for their sellers and in effect facilitate more fruitful transactions on their platforms.

Marketplaces can generate more demand either by acquiring new customers or generating incremental demand from their existing customers. Performance marketing is an example of how a marketplace like Thumbtack could acquire new customers via channels like Instagram or Google. But it also requires us to spend more money on marketing. For homeowners, there may be many times in a year where they could use help from pros on Thumbtack. Thus when a customer completes a project on Thumbtack, we could suggest other ways in which we could help and thus generate incremental demand without performance marketing. For example, when a customer books a mover for their home, we might recommend move-out cleaning services to that customer. In more demand constrained markets, we could also create price discounts on the follow up or bundled service and thus help with creating more incremental demand for our pros, and potentially do that more cost-effectively than with marketing.

Supply constrained: The need to improve the customer experience

In online marketplaces, customers tend to benefit from excess supply. This could mean more listings to choose from on AirBnB, an Uber driver just around the corner, or more pros to choose from on Thumbtack. Conversely, when supply is low, the customer experience might suffer and the number of customers creating fruitful transactions might fall, sometimes dramatically. This is intuitive; for example if your ride share from the airport requires a 30 minute wait, you might open up a competitor’s application (hoping to see a lower wait time) or take public transit. More importantly, these less-than-ideal customer experiences lower the likelihood of a customer relying on your marketplace in the future.

To deal with such situations, some marketplaces introduce instantaneous pricing incentives. Uber famously has surge pricing. They dynamically track when and where rider demand spikes, and raise fares accordingly. Drivers are given some information to help them find areas where prices are surging, further incentivizing them to take rides in that area which increases supply. Riders who have the luxury of time can wait out the price surge or switch to an alternative, simultaneously bringing demand down in the short term. Riders who urgently need a ride and cannot find a better alternative will accept the higher prices.

For a marketplace like Thumbtack, pros in different categories (e.g. plumbing) sign up to serve customers in specific neighborhoods (e.g. zip codes in Miami, Florida).

• Since there may be certain kinds of plumbing jobs they have expertise in / want to do, they set up their job preferences.
• Depending on the number of jobs they can take on in a week, they set up a weekly spending budget on Thumbtack, which relates to how many jobs they may ideally want and the maximum amount they are willing to spend on Thumbtack over that period.

When customers search for pros who can help get their job done, we respect the job & budget preferences of our pros and try to find the most relevant pros for our customers. But based on the preferences of pros in that neighborhood, this also means that there may not be enough plumbers who want to accept jobs in that neighborhood in the coming week.

So in a supply constrained market, how does a marketplace like Thumbtack improve customer experience and in effect create a healthier, more efficient marketplace?

Over a longer time horizon, when marketplaces remain constrained on supply, the obvious way to drive efficiency is targeted acquisition of more supply in those markets. Onboarding and activating more pros in specific markets is thus ideal, but often takes time.

On the other hand, when a marketplace is constrained on supply it can launch initiatives that boost supply in the short term. For Thumbtack, supply constraints can be caused by two challenges:

• You might have too few pros willing & able to do certain kinds of jobs (as defined by their job preferences), for example, if more house cleaning services want to target only larger homes in a neighborhood and prefer not to target apartments.
• You might have enough pros in the neighborhood willing to do specific jobs (e.g. apartment house cleaning), but they only want to spend a limited amount of money on acquiring customers from online marketplaces.

In the following sections, we will share examples of initiatives that target these two supply constraints over shorter time horizons.

Determining relevant pros for a customer’s job

In order to understand where in the product these initiatives come into play, it’s useful to first understand how we determine which pros are most relevant for a customer’s job preferences.

We first ask for information about the customer’s job like the type of job (e.g. house cleaning), the zip code, some job preferences (e.g. 2 bedroom home, monthly recurrence), and then use these job preferences to find the most relevant pros who operate in that neighborhood.

Figure 2. Customer searching for house cleaners in their neighborhood

At a high level, this involves the following 3 steps:

1. Candidate selection: Fetch a list of pros whose job preferences match the job details that the customer provided.
2. Ranking: Rank the pros based on relevance to the customer job preferences, the quality of the pro and their max lead prices.
3. Policy-specific filtering & re-selection: Use supply utilization policies to determine if any filtering or re-selection needs to apply based on any supply boosting initiatives.

Note: This high level overview is specific to the topic in the blog, if you’d like to know more about how we rank pros based on relevance you can read about it in a previous blog related to ranking.

Now that you have some context on how we show relevant pros to customers, let’s jump into the initiatives that target these two supply constraints.

Initiatives to expand job preferences

How pros set job preferences: As mentioned previously, pros control the job preferences they set on Thumbtack. For example, they might only be available to work Tuesday-Sunday with Monday being a weekly holiday for their business. They might also prefer house cleaning jobs within a 25 mile radius of their business. Some preferences, like the days they are available, might be non-negotiables for their business. But some, like distance or whether they accept bookings for smaller homes, might relate more to factors like convenience, price margins, etc.

How job preferences create supply constraints: When we perform candidate selection we select pros that match both the customer’s job details & the pro’s job preferences. For example in a supply constrained market, there could be a lot of customers living in small apartments, and not enough pros willing to accept smaller house cleaning jobs. We thus design initiatives that seek to expand job preferences in those markets. Here’s an example of one such mechanism we introduced.

Initiative 1: Personalized Mismatch Insights

Problem: Generally we try to get as much detail as we can about a customer’s job preferences before we show them pros on the platform. But sometimes customers don’t share all of their job preferences until they go down the path of messaging or booking a pro and that’s when we might learn that the pro’s job preferences don’t quite match what the customer wanted. This results in what we call a “mismatch”, and the pro misses out on potential customers due to this discrepancy. In supply constrained markets, this problem can be more acute due to fewer overall pros in that market.

How we addressed this: We know that some pro preferences tend to be non-negotiable, but some tend to be more flexible. So here are the steps we took to incentivize pros to expand their job preferences and get more jobs via Thumbtack:

1. We identified preferences that we deemed potentially negotiable for a business, like the neighborhoods (zip codes) that pros were willing to travel to.
2. We aggregated data around how often pros were missing out on customer bookings due to those potentially negotiable preferences.
3. We created and shared actionable insights personalized to each pro on the jobs they were missing out on every week.

Result: As measured through our A/B tests, in just a few weeks we were able to improve the customer experience by incentivizing pros to expand some of their preferences and get more jobs via Thumbtack.

Initiatives to expand pro budgets

How pros set budgets: Thumbtack charges pros a fee on bookings created on the platform, so that we earn a small proportion of the revenue created through our platform. Based on the size of their business and the number of bookings they might want from Thumbtack, pros either set unlimited weekly spending budgets or limit their budgets to a certain dollar amount per week. Let’s say, for example, a small business can handle 12 bookings per week. Let’s say Thumbtack charges them a $20 fee on average for each booking made by customers on the platform. If they assume they can acquire half the customers through word of mouth and other half through Thumbtack, they might set a weekly spend budget of approximately 150 dollars (approximately 6 bookings * $20 + $30 buffer), even though their true capacity for the number of jobs they can take on is higher.

How budgets create supply constraints: In a previous section we mentioned how we select relevant pros for a customer. In general we only select pros who have the budget to cover Thumbtack’s booking charges when we run our candidate selection step. In demand constrained markets, this isn’t an issue as we might have more than enough pros to select for our customers. But in supply constrained markets, we may not have enough relevant pros to surface to our customers which can affect the customer experience on Thumbtack and thus we need to introduce some ways to optimize efficiency. Pricing incentives can work quite well in such scenarios where we might incentivize our pros to expand their budgets and offer them one time discounts for taking up our budget increase incentives. But we can also create intelligent discounts like the one described below, which can operate more responsively to drive efficiency and improve the customer experience.

Initiative 2: Price discounts based on conversion modeling

Problem: As we mentioned above, in general we only select pros who have the budget to cover Thumbtack’s booking fees. Supply levels tend to influence the customer experience and thus the expected levels of conversion in those markets. Sometimes even if we have a few pros who have the budget to cover booking fees, the lack of customer choice might prompt customers to not book a pro for a job.

How we addressed this: We select pros who are relevant to a customer’s job preferences and rank them based on relevance. We thus already have a way to determine how relevant a pro is to the customer’s job preferences. In supply constrained markets, we use this information to:

1. Model the expected conversion rates for the aggregate list of relevant pros.
2. For pros who fully match all job preferences, but who can only partially cover the booking fees, we model the expected conversion rate based on their addition to the list.
3. We use this to determine whether we should add this pro to the list. If we add this pro to the list and the customer chooses to book them, we end up creating a discounted booking for this pro, since we respect their weekly spend budgets. This means we don’t earn our usual share for these jobs.

Result: This allowed us to selectively boost our supply on an as-needed basis in order to improve the customer experience and help more customers get their job done via Thumbtack.

Conclusion

Hopefully, these scenarios we shared give an overview of demand/supply challenges in online marketplaces and how we create initiatives in supply constrained markets that improve the customer experience and get more jobs done on our platform. If these types of problems seem interesting to you, check out Thumbtack’s career site!

Acknowledgement

A huge shout out to the many teams across the Marketplace & Growth pillars at Thumbtack that contribute towards creating healthy marketplace dynamics, including some of the initiatives mentioned above. A special thanks to Richard Demsyn-Jones and Cassandra Abernathy for suggestions refining this blog post. Also thanks to Nadia Stuart, Richard Domurat, Eric Ortiz, Dhananjay Sathe, Shishir Dash and Brandon Sislow for their diligent reviews of this post.

Additional Reading

1. Rysman, M., 2009. The economics of two-sided markets. Journal of economic perspectives, 23(3), pp.125–143.
2. Han, B., Lee, H. and Martin, S., 2022, August. Real-time rideshare driver supply values using online reinforcement learning. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (pp. 2968–2976).
3. Mehrotra, R., McInerney, J., Bouchard, H., Lalmas, M. and Diaz, F., 2018, October. Towards a fair marketplace: Counterfactual evaluation of the trade-off between relevance, fairness & satisfaction in recommendation systems. In Proceedings of the 27th acm international conference on information and knowledge management (pp. 2243–2251).