How Opendoor Manages Buyboxes and Product Eligibilities at Scale

Introducing Merchandising — An extensible, configurable, and scalable eligibility decision engine

Introduction

As of writing this post, Opendoor operates in 44 markets. And we have the ability to make offers on the majority of homes in the markets that we’re in. But, there are a number of reasons why we might not want to make an offer on a given home. In fact, for each market we have a strictly defined set of parameterized criteria that homes must meet in order for us to provide an instant, all-cash offer. We refer to this set of homes as our buybox.

When a customer enters their address on our site, looking to find out if we will make an offer on their home, we need to be able to give them an immediate and accurate answer. We also need to be able to determine up front what other products, if any, we can offer them in order to help them sell their home with confidence. To handle this, we needed a reliable and efficient end-to-end solution for answering what might at first seem like a simple question: is this particular customer eligible for this specific product?

In this post, we cover how we solved this problem by building Merchandising, an extensible, configurable, and scalable eligibility decision engine that is used throughout Opendoor as the single source of truth for buyboxes and product eligibility. In particular, we will provide some additional context on why we built it, how it works, how it’s used, and how it has helped us operate at scale.

A Brief History of Buyboxes at Opendoor

Back in the old days at Opendoor, we only offered one product to customers: an all cash offer for your home. We were also only in a handful of markets. We had a service to define the buybox parameters of this single cash offer product and execute a basic rule engine for determining if a home was in our buybox, but it had a lot of missing pieces and limitations. A few key ones:

1. Limited flexibility. There was a lack of flexibility around building rules for buybox criteria. Our more complex rules could not be modeled fully with the implemented rule engine. There was no support for building rules that needed more contextual information, such as feature flags, experiments, customer request details, etc. There was also no way to support new products.
2. No configuration. Any change in criteria for a given rule required code changes, which created a lot of engineering overhead, especially when the change in criteria had to do with more contextual information.
3. Inefficient. Any client calling this service had to reach out and fetch all the data needed to run through the buybox rules. This created a ton of overhead for clients and led to an extreme amount of duplicated and inefficient code across a variety of applications and surface areas.

The limitations of this became especially apparent as we needed to start supporting more tricky use cases such as using buyboxes for creating more optimized marketing and exposing eligibility checks via API to build new integrations with some of our partner companies. We built some ad hoc solutions, but the reality was that, at a foundational level, we were not capable of providing support for these use cases nor could we support our rapidly increasing scale.

Enter Merchandising

We needed to put ourselves in a position to support a much more diverse set of use cases at a considerably higher scale, so we decided to go back to the drawing board. We began by collecting a detailed set of needs and requirements from teams across Opendoor that had either existing or future use cases for checking product eligibility. We then boiled all of it down into a set of goals and primary feature requirements for what would become Merchandising:

Goals

• As a client of Merchandising, I know if a customer is eligible for an experience
• As a client of Merchandising, I can know what experiences this customer was historically eligible for
• As an experience provider, I can register eligibility criteria for my experience
• As an experience provider, I can register new evaluation functions
• As an experience provider, I can register a data source for my relevant eligibility data
• As either a client or experience provider, I know that this is the source of truth for all experience eligibilities in Opendoor

Immediate Feature Requirements

• Must work for all existing Opendoor products
• Must be easy to add a new product (experience)
• Must be able to model arbitrarily complex eligibility rules
• Must automatically retrieve and utilize relevant info from internal Opendoor services
• Must be able to support overriding internal data with externally provided data when calculating eligibility
• Must return all denied eligibilities with reasons for the denials
• Must be easy to gradually roll out
• Must be easily configured
• Must be reliable. Available >= 99.99% of the time and accurate.

With these goals and requirements, we went to the drawing board to design what would become Merchandising.

Concepts and Terminology

High level Merchandising service architecture

Experience

As you might have noticed, we began this post by talking about product eligibility, yet in the goals we use the term experience. During the design phase, we found that a “product” is an overly specific use case for what we were building and an “experience” would provide us with much more semantic flexibility. Products are typically limited to service fulfillments like “Sell your home directly to us,” “Buy your next home with our cash,” “List your home with us,” etc. However, there was no reason that we couldn’t support building eligibility criteria for a more diverse array of experiences, such as product add-ons, new application features, experimental versions of an existing product, etc. That being the case, we eventually settled on the term “experience”, which can represent all of the above and more.

Client

A service or application that needs to know if an experience is available for a given customer context.

Data Source

A service that provides data for data field(s) used to calculate eligibility for one or more experiences.

Data Fields

Data fields are the keys used when constructing rules for a given experience. Each data field is filled in by one or more data sources. Data sources have a priority and the highest-ranked data source for a given field is used when more than one data source is available. A data field can represent a specific data attribute about some model, such as a home’s `dwelling_type`, or a data field can represent data that is more interpretive such as whether a customer `has_legacy_offer`. Data fields can also represent feature flags, with the data provided by an external data source such as Optimizely. We will get into how data fields are used to construct rules in a later section.

Rule Sets

A rule set is a collection of rules defined for a unique experience and market combination. Each rule can have one or more versions, but only one can be active at any time. Rule set versions can inherit rules from one or more parent rule set versions. Each rule on a given version’s set has a unique name and a boolean expression.

Rule Expression Evaluation

Rule expressions are modeled as trees where each node has an expression type and child nodes that are sub-expressions or operands of the expression. Expression trees are evaluated recursively by executing the evaluation logic for each expression type. The following are some examples of supported expression types:

1. Operations

• AND, OR, NOT
• IN
• IF
• IS NULL
• <=, >=, =, >, <, !=
• CONTAINS

2. Data fields (described in the terminology section)

3. Values

Visualizing a rule expression tree

Rule expressions also have the following useful properties:

1. Highly composable. Expressions can arbitrarily nest other expressions.
2. SQL-like semantics. Expressions can be drafted like sql `where` conditions.
3. NULL aware. Expressions are NULL aware and operations involving NULLs usually return NULLs (IS NULL and IS NOT NULL are exceptions).
4. Strongly typed. Expressions are well typed and are strictly validated when a new version is saved. For example: comparison operations requires operands to have the same type (INT < INT is okay, but not FLOAT < INT). AND and OR requires all operands to be boolean typed (AND(BOOL, BOOL, …) is okay, but AND(STRING) is not), etc.

Handling CheckEligibility Requests

If a service or application wants to interact with Merchandising, it will generally call its primary endpoint: CheckEligibility. This endpoint allows eligibility consuming services to define all the identifiers relevant to the request (i.e. customer info, address, etc) and the set of experiences they want to determine eligibility for. Optionally, the client can also define any data fields that are known by the client as well as a list of data sources that should be excluded as part of the criteria evaluation in order to avoid round trips.

Taken together, the shape of the request looks a bit like this:

After receiving a CheckEligibility request, Merchandising uses its RuleSetProvider to obtain all rule sets needed for the set of experiences requested. From the rule set(s), Merchandising then extracts all data fields needed to compute eligibility and in parallel fans out to load all fields from their respective data sources. Next, Merchandising evaluates all rules using the loaded data fields and aggregates each result.

Each result will either pass, fail, or resolve to `missing`. In the latter case, this is analogous to a null value and lets clients know that some data field(s) were not found and the corresponding rule(s) could not be fully evaluated. Each requested experience will also receive a top-level result that is derived from the aggregated results. In the case of failed results, a reason is determined based on the data field that caused an expression to return false during rule evaluation. Altogether, Merchandising will respond with a top-level result for each requested experience, each individual rule result, and reason(s) in the case of failed eligibility. The shape of that response object looks something like this:

There are a few key benefits we get from this response shape:

• Clients can opt to use the top-level result or rely on specific rule results depending on the use case.
• Clients can use the result reasons to provide context in-product for customers.
• Clients can use missing results as a trigger to request additional data inputs from customers, then re-check.

Overall, the CheckEligibility endpoint has served as a highly performant and effective API for eligibility consuming services across Opendoor!

Configuration

Another key requirement for Merchandising was ease of configuration. Historically any changes to buybox configuration would require engineering work that could be quite cumbersome. Our business and operations counterparts should be empowered to tweak the buybox on their own terms and without having to involve engineers. To support this, we built an application UI for interacting with Merchandising. From this app, you can view all existing rule sets and versions, edit existing rules, add new ones, remove old ones, etc.

In order for the application to be easy to use for non-engineers, it was important that viewing/editing rules was intuitive for folks without them having to understand how exactly Merchandising works under the hood. That being the case, our rule presentation translates rules (which are stored as JSON trees) into a much more readable SQL-like format:

And when making edits, we have a UI that supports dropdown/select based configuration for drafting rule expressions:

Though the UI system is not perfect, it has worked well for us so far, and our ops team has been making changes without engineering support for quite some time!

In general, we have found that investing time up front to build a system and process that scales well from an operational perspective leads to massive time saving dividends in the long run.

Efficiency Gains

Improvements in accuracy and performance are not the only benefits we have seen from building and migrating to Merchandising. Some of the greatest impact has been realized in the form of operational efficiency gains resulting from the flexibility and centralized configuration that we now have in Merchandising.

Buybox Expansion and New Market Launches

Despite going through a difficult period during 2020 due to Covid-19, 2021 proved to be a monumental year for us in terms of expanding our buybox and market footprint. Over the course of the past 9 months, we launched our products in 23 new markets across the country. In addition to new markets, we have also dramatically increased the coverage of our buybox across a number of dimensions in existing markets. This means everything from acquiring more expensive homes, older homes, new home types (e.g., townhomes), etc.

When it comes to launching new markets and expanding our buybox, there is an enormous amount of work teams across Opendoor need to do. Our pricing team for instance needs to run extensive testing on our models to have conviction that we can accurately price homes in the new market or expanded buybox. In fact, we probably need an entirely separate blog post to cover what it means to launch a new market at Opendoor.

Once everything is ready, though, the actual “launch” can be done with a few simple button clicks to edit rules in Merchandising and unlock the floodgates for the new set of homes. We for the most part take that operational efficiency for granted now, but the business impact of that efficiency and launch control can only be understated!

What’s Next

In this post we covered what Merchandising is, why we needed it, a brief summary of how it works, and it’s impact. The impact in terms of both reliability and efficiency has been huge, but since launching Merchandising we have layered some projects on top of it to make it even more useful and reliable. Keep an eye out for future posts that deep dive into how we built some of these features:

1. Automated backtesting on rule changes to make it easier, more streamlined, and less error prone to implement changes to the buybox, e.g. increasing buybox price cap.
2. Using merchandising rules to auto-generate pre-computed and parameterized analytics tables in our data warehouse to help teams understand our market coverage for a given buybox. We refer to this as our Merch-based Buybox ($MBBB), and it is now a critical part of how our data science and pricing teams consider buybox adjustments and addressable market size as well as how our marketing teams optimizes audiences.

If there are any other parts of this post that were interesting and you’d like to hear more about them, then let us know in the comments! And if you want to work on building out systems that solve complex and large scale problems like these, Opendoor is hiring! Check out some of our open engineering roles :).

Acknowledgments

Building Merchandising was a large and difficult effort made possible only by all the people who worked on it. In particular, we would like to thank Martin Liu, Travis Hairfield, Jerry Yang, Vicky Zhang, Luke Groesbeck, and Rohan Dang, all of whom were instrumental in designing, building, and launching Merchandising.