Introduction

As of writing this post, Opendoor operates in 44 markets. And we have the ability to make offers on the majority of homes in the markets that we’re in. But, there are a number of reasons why we might not want to make an offer on a given home. In fact, for each market we have a strictly defined set of parameterized criteria that homes must meet in order for us to provide an instant, all-cash offer. We refer to this set of homes as our buybox.

When a customer enters their address on our site, looking to find out if we will make an offer on their home, we need to be able to give them an immediate and accurate answer. We also need to be able to determine up front what other products, if any, we can offer them in order to help them sell their home with confidence. To handle this, we needed a reliable and efficient end-to-end solution for answering what might at first seem like a simple question: is this particular customer eligible for this specific product?

In this post, we cover how we solved this problem by building Merchandising, an extensible, configurable, and scalable eligibility decision engine that is used throughout Opendoor as the single source of truth for buyboxes and product eligibility. In particular, we will provide some additional context on why we built it, how it works, how it’s used, and how it has helped us operate at scale.

A Brief History of Buyboxes at Opendoor

Back in the old days at Opendoor, we only offered one product to customers: an all cash offer for your home. We were also only in a handful of markets. We had a service to define the buybox parameters of this single cash offer product and execute a basic rule engine for determining if a home was in our buybox, but it had a lot of missing pieces and limitations. A few key ones:

1. Limited flexibility. There was a lack of flexibility around building rules for buybox criteria. Our more complex rules could not be modeled fully with the implemented rule engine. There was no support for building rules that needed more contextual information, such as feature flags, experiments, customer request details, etc. There was also no way to support new products.
2. No configuration. Any change in criteria for a given rule required code changes, which created a lot of engineering overhead, especially when the change in criteria had to do with more contextual information.
3. Inefficient. Any client calling this service had to reach out and fetch all the data needed to run through the buybox rules. This created a ton of overhead for clients and led to an extreme amount of duplicated and inefficient code across a variety of applications and surface areas.

The limitations of this became especially apparent as we needed to start supporting more tricky use cases such as using buyboxes for creating more optimized marketing and exposing eligibility checks via API to build new integrations with some of our partner companies. We built some ad hoc solutions, but the reality was that, at a foundational level, we were not capable of providing support for these use cases nor could we support our rapidly increasing scale.

Enter Merchandising

We needed to put ourselves in a position to support a much more diverse set of use cases at a considerably higher scale, so we decided to go back to the drawing board. We began by collecting a detailed set of needs and requirements from teams across Opendoor that had either existing or future use cases for checking product eligibility. We then boiled all of it down into a set of goals and primary feature requirements for what would become Merchandising:

Goals

• As a client of Merchandising, I know if a customer is eligible for an experience
• As a client of Merchandising, I can know what experiences this customer was historically eligible for
• As an experience provider, I can register eligibility criteria for my experience
• As an experience provider, I can register new evaluation functions
• As an experience provider, I can register a data source for my relevant eligibility data
• As either a client or experience provider, I know that this is the source of truth for all experience eligibilities in Opendoor

Immediate Feature Requirements

• Must work for all existing Opendoor products
• Must be easy to add a new product (experience)
• Must be able to model arbitrarily complex eligibility rules
• Must automatically retrieve and utilize relevant info from internal Opendoor services
• Must be able to support overriding internal data with externally provided data when calculating eligibility
• Must return all denied eligibilities with reasons for the denials
• Must be easy to gradually roll out
• Must be easily configured
• Must be reliable. Available >= 99.99% of the time and accurate.

With these goals and requirements, we went to the drawing board to design what would become Merchandising.

Concepts and Terminology

Experience

As you might have noticed, we began this post by talking about product eligibility, yet in the goals we use the term experience. During the design phase, we found that a “product” is an overly specific use case for what we were building and an “experience” would provide us with much more semantic flexibility. Products are typically limited to service fulfillments like “Sell your home directly to us,” “Buy your next home with our cash,” “List your home with us,” etc. However, there was no reason that we couldn’t support building eligibility criteria for a more diverse array of experiences, such as product add-ons, new application features, experimental versions of an existing product, etc. That being the case, we eventually settled on the term “experience”, which can represent all of the above and more.

Client

A service or application that needs to know if an experience is available for a given customer context.

Data Source

A service that provides data for data field(s) used to calculate eligibility for one or more experiences.

Data Fields

Data fields are the keys used when constructing rules for a given experience. Each data field is filled in by one or more data sources. Data sources have a priority and the highest-ranked data source for a given field is used when more than one data source is available. A data field can represent a specific data attribute about some model, such as a home’s `dwelling_type`, or a data field can represent data that is more interpretive such as whether a customer `has_legacy_offer`. Data fields can also represent feature flags, with the data provided by an external data source such as Optimizely. We will get into how data fields are used to construct rules in a later section.

Rule Sets

A rule set is a collection of rules defined for a unique experience and market combination. Each rule can have one or more versions, but only one can be active at any time. Rule set versions can inherit rules from one or more parent rule set versions. Each rule on a given version’s set has a unique name and a boolean expression.

Rule Expression Evaluation

Rule expressions are modeled as trees where each node has an expression type and child nodes that are sub-expressions or operands of the expression. Expression trees are evaluated recursively by executing the evaluation logic for each expression type. The following are some examples of supported expression types:

1. Operations

• AND, OR, NOT
• IN
• IF
• IS NULL
• <=, >=, =, >, <, !=
• CONTAINS

2. Data fields (described in the terminology section)

3. Values

Rule expressions also have the following useful properties:

1. Highly composable. Expressions can arbitrarily nest other expressions.
2. SQL-like semantics. Expressions can be drafted like sql `where` conditions.
3. NULL aware. Expressions are NULL aware and operations involving NULLs usually return NULLs (IS NULL and IS NOT NULL are exceptions).
4. Strongly typed. Expressions are well typed and are strictly validated when a new version is saved. For example: comparison operations requires operands to have the same type (INT < INT is okay, but not FLOAT < INT). AND and OR requires all operands to be boolean typed (AND(BOOL, BOOL, …) is okay, but AND(STRING) is not), etc.

Handling CheckEligibility Requests

If a service or application wants to interact with Merchandising, it will generally call its primary endpoint: CheckEligibility. This endpoint allows eligibility consuming services to define all the identifiers relevant to the request (i.e. customer info, address, etc) and the set of experiences they want to determine eligibility for. Optionally, the client can also define any data fields that are known by the client as well as a list of data sources that should be excluded as part of the criteria evaluation in order to avoid round trips.

Taken together, the shape of the request looks a bit like this:

After receiving a CheckEligibility request, Merchandising uses its RuleSetProvider to obtain all rule sets needed for the set of experiences requested. From the rule set(s), Merchandising then extracts all data fields needed to compute eligibility and in parallel fans out to load all fields from their respective data sources. Next, Merchandising evaluates all rules using the loaded data fields and aggregates each result.

Each result will either pass, fail, or resolve to `missing`. In the latter case, this is analogous to a null value and lets clients know that some data field(s) were not found and the corresponding rule(s) could not be fully evaluated. Each requested experience will also receive a top-level result that is derived from the aggregated results. In the case of failed results, a reason is determined based on the data field that caused an expression to return false during rule evaluation. Altogether, Merchandising will respond with a top-level result for each requested experience, each individual rule result, and reason(s) in the case of failed eligibility. The shape of that response object looks something like this:

There are a few key benefits we get from this response shape:

• Clients can opt to use the top-level result or rely on specific rule results depending on the use case.
• Clients can use the result reasons to provide context in-product for customers.
• Clients can use missing results as a trigger to request additional data inputs from customers, then re-check.

Overall, the CheckEligibility endpoint has served as a highly performant and effective API for eligibility consuming services across Opendoor!

Configuration

Another key requirement for Merchandising was ease of configuration. Historically any changes to buybox configuration would require engineering work that could be quite cumbersome. Our business and operations counterparts should be empowered to tweak the buybox on their own terms and without having to involve engineers. To support this, we built an application UI for interacting with Merchandising. From this app, you can view all existing rule sets and versions, edit existing rules, add new ones, remove old ones, etc.

In order for the application to be easy to use for non-engineers, it was important that viewing/editing rules was intuitive for folks without them having to understand how exactly Merchandising works under the hood. That being the case, our rule presentation translates rules (which are stored as JSON trees) into a much more readable SQL-like format:

And when making edits, we have a UI that supports dropdown/select based configuration for drafting rule expressions:

Though the UI system is not perfect, it has worked well for us so far, and our ops team has been making changes without engineering support for quite some time!

In general, we have found that investing time up front to build a system and process that scales well from an operational perspective leads to massive time saving dividends in the long run.

Efficiency Gains

Improvements in accuracy and performance are not the only benefits we have seen from building and migrating to Merchandising. Some of the greatest impact has been realized in the form of operational efficiency gains resulting from the flexibility and centralized configuration that we now have in Merchandising.

Buybox Expansion and New Market Launches

Despite going through a difficult period during 2020 due to Covid-19, 2021 proved to be a monumental year for us in terms of expanding our buybox and market footprint. Over the course of the past 9 months, we launched our products in 23 new markets across the country. In addition to new markets, we have also dramatically increased the coverage of our buybox across a number of dimensions in existing markets. This means everything from acquiring more expensive homes, older homes, new home types (e.g., townhomes), etc.

When it comes to launching new markets and expanding our buybox, there is an enormous amount of work teams across Opendoor need to do. Our pricing team for instance needs to run extensive testing on our models to have conviction that we can accurately price homes in the new market or expanded buybox. In fact, we probably need an entirely separate blog post to cover what it means to launch a new market at Opendoor.

Once everything is ready, though, the actual “launch” can be done with a few simple button clicks to edit rules in Merchandising and unlock the floodgates for the new set of homes. We for the most part take that operational efficiency for granted now, but the business impact of that efficiency and launch control can only be understated!

What’s Next

In this post we covered what Merchandising is, why we needed it, a brief summary of how it works, and it’s impact. The impact in terms of both reliability and efficiency has been huge, but since launching Merchandising we have layered some projects on top of it to make it even more useful and reliable. Keep an eye out for future posts that deep dive into how we built some of these features:

1. Automated backtesting on rule changes to make it easier, more streamlined, and less error prone to implement changes to the buybox, e.g. increasing buybox price cap.
2. Using merchandising rules to auto-generate pre-computed and parameterized analytics tables in our data warehouse to help teams understand our market coverage for a given buybox. We refer to this as our Merch-based Buybox ($MBBB), and it is now a critical part of how our data science and pricing teams consider buybox adjustments and addressable market size as well as how our marketing teams optimizes audiences.

If there are any other parts of this post that were interesting and you’d like to hear more about them, then let us know in the comments! And if you want to work on building out systems that solve complex and large scale problems like these, Opendoor is hiring! Check out some of our open engineering roles :).

Acknowledgments

Building Merchandising was a large and difficult effort made possible only by all the people who worked on it. In particular, we would like to thank Martin Liu, Travis Hairfield, Jerry Yang, Vicky Zhang, Luke Groesbeck, and Rohan Dang, all of whom were instrumental in designing, building, and launching Merchandising.

How Opendoor Manages Buyboxes and Product Eligibilities at Scale was originally published in Open House on Medium, where people are continuing the conversation by highlighting and responding to this story.

At Opendoor, we think getting an offer on your home should be as simple as the click of a button. We are building an API platform so that other online real estate companies can build on top of our automated home valuation services and add a button like ours to their sites.

Internally at Opendoor Engineering, we use GraphQL to develop nearly all of our application APIs. Although REST is still the dominant player for building external facing APIs, we decided to double down on GraphQL and use it to build out our external API platform. In this article, we cover the things we learned while designing and launching this API.

Best practices for schema design

In a traditional REST API, data fetching is structured around the set of endpoints that your API makes available to the client. In GraphQL, however, it is structured around your API’s GraphQL Schema.

Design your schema with client use cases top of mind

A core principle of GraphQL is giving power to the client. GraphQL makes this possible by having a declarative syntax that allows clients to describe exactly what data they want from the server then ask for and receive it in a single request. When designing your schema, it is important to think about the use cases of your clients so that these benefits that GraphQL can provide are maximized.

Currently, the primary use case of our API is to provide an external application answers to the following questions for an arbitrary home address:

Would Opendoor make an offer on this home, and if so what is the estimated value of that offer? Furthermore, if the home is eligible and the value estimate is attractive to the customer, how can I refer said customer to Opendoor so they can begin to request an actual offer while also tracking our application as the origin of this transaction?

Since our API is using GraphQL, our mindset needs to be that client applications can ask and receive an answer to these questions with only one request to our API. If they cannot do that, then this is a big red flag that we did something wrong when designing our schema. Finding a type/schema structure that allows for answering these questions all in one response body in an intuitive manner is not a trivial task. It is, however, an achievable goal when this mindset is maintained throughout the entire design process. Along the way we found several actionable ways for how you can keep your client use cases as the north star signal while designing your API:

1. Hold design reviews. Hold one of these for each new proposed addition/change to your schema, and constantly ask each other whether or not this makes things the implementation easier or more intuitive for the client use cases.
2. Get external feedback. Show your schema to an engineer who does not already have context, and ask what does and doesn’t make sense to them about it. This will help you understand what the people developing client applications would think.
3. Build an integration. Create a very simple client application and attempt to integrate your API with it. The pain points you may or may not face while implementing will greatly improve your understanding of how usable your API is. This task in particular has helped us a lot with building more usable APIs.

Keep your schema flexible

Another very important consideration to make while designing your schema is maintaining flexibility. This was particularly important for our use case. For the first launch of our API, we only needed to support the idea of customers wanting to sell their home directly to Opendoor. Moving forward, though, we will need to support a variety of products that we can offer for a given home. This created a very important requirement for how our schema needed to be designed. We now needed to design an API that intuitively answers the questions from above, but also is flexible enough that we can easily add new products for new questions in addition to the simple direct sell one.

Luckily, GraphQL is designed to make handling these types of requirements easier, and it is important that you take advantage of this fact. There are many ways to develop a GraphQL API that achieves both our goal of use case oriented while maintaining significant flexibility. In the end we arrived at the following schema:

Our schema and response types are designed to handle our responses in terms of Products from the outset, but in the initial launch there is only a single product entity type that we call homeSell. Each product will have product-specific data about it that we want to provide back in the response to clients. So, for each product we will create a custom product type object, with SellProduct being our first

The SellProduct object in turn includes eligibility and our value estimate of that home. Furthermore we generate a referral url that can send a client application customer directly into our direct home sell user flow for the provided address and accurately track what client application they originally came from. Also, since we capture these data fields as distinct GraphQL types, we can reuse ValueEstimate and Referral for other products that we add to our schema in the future.

The case for having good documentation

We have all very likely heard the common claim that all of us developers repeat to one another at some point in our careers: good code is self documenting. There is definitely a lot of truth to this statement. Well written code that uses intuitive naming conventions and function structure is self documenting in the sense that someone who reads this code will probably be able to discern what it does just from the code itself.

When we consider the case of an API, it is still somewhat true that a well designed API should, to a certain degree, be self documenting. With a GraphQL API, if the schema design and type naming is intuitive, then in theory an external developer working on a client application should be able to derive a sufficient understanding just from reading the schema. But here is the issue with that statement: it is only true in theory.

In practice, however, that is rarely the case and it is very important that external developers are provided with robust, well written documentation. We have learned this fact the hard way. Already client applications that have integrated our API since launch just a couple months ago have made key mistakes in their implementation that have led to less than ideal user experiences for their customers. We could simply chalk this up to being a result of lack of care on the part of the external developer, but in doing so we would be making a categorical error.

Fundamentally, there is no real difference between an external developer making a mistake while implementing a use case of your API, and a random user improperly interacting with your website or application.

In the case of the latter, we would always treat this as a UX failure of our application and attempt to remedy the situation by improving usability and user guidance within the application. The former should be handled in a similar manner.

These sort of implementation errors that external developers might make can and should be remedied by creating robust, clear, and well maintained documentation. This is something we failed to do when first launching our API, but we have learned from our mistakes and have since been trying to improve our documentation in every way we can think of. We have already seen less confusion among developers working on recent integrations, and we believe this is a direct result of our vastly improved documentation, but there is still a lot work to be done to eliminate the confusion and mistakes entirely.

Create a developer experience that scales

When building any kind of API, whether it be GraphQL or REST, it is important to create a good developer experience, in particular one that scales with an increasing number of developers using your API. At launch, and during the first couple of months following, we would work directly with developers at other companies to help them to get started and launch an integration of our API. This included us semi-manually creating and entering details regarding their application that our system needs to know about, including everything from authentication keys to webhook endpoints for us to hit with notifications regarding the referrals that we are tracking from their application. As you can imagine, that leads to a lot of back and forth, which creates a poor developer experience that definitely does not scale to a large number of integrations.

To fix this, we decided to create a Developer Dashboard that all external developers can use to interact with data regarding their application, such as create/deleting API keys, adding/editing webhook endpoints, and viewing logs of requests from their application. Furthermore, our dashboard allows them to turn on Test Mode, and do all of that using just test data. We very recently launched this dashboard and although we have not yet gathered feedback to see whether this had led to an improvement in experience, we have already reaped the benefits it provides to us from a scalability standpoint. One client application recently needed to rotate the API keys for their application, which previously would have meant us needing to coordinate with them and do it manually, but now they were able to successfully do everything from the dashboard without any help from us. We view this as a huge win!

Some final thoughts

It is difficult to try and build something from scratch without having a ton of experience in the area, and especially without having many examples to go off of. We hope that what we learned along the way can help make things easier for you, should you ever find yourself building an external facing GraphQL API, or any API for that matter. If you have any questions or general thoughts, please share them in the comments below.

We’re hiring! If building an API platform to help people move rapidly and freely is something that interests you, consider joining our team.

What we learned while designing and launching an external GraphQL API was originally published in Open House on Medium, where people are continuing the conversation by highlighting and responding to this story.