Aligning Product with Strategy by Unifying Models

How unifying pricing models moves from tactical to strategic, to better power life’s progress one move at a time

By Sam Stone, Director of Product Management, and Amit Misra, Staff Research Scientist

Intro

At Opendoor, our mission is to power life’s progress one move at a time. Selling your home and buying a new one is complex; a successful move necessitates literally hundreds of decisions and tasks. Advancing our mission requires Opendoor product teams to understand all these decisions and tasks, identify those that generate the most frustration or inefficiency, and relentlessly chip away at those pain points to improve the experience for our customers and reimagine the real estate process.

In our early days, product teams identified important decisions and tasks where models offered to improve quality and/or efficiency. The bottom-up, task-oriented nature of early model development led to models which were valuable and effective — but highly tactical.

Strategy without tactics is the slowest route to victory. Tactics without strategy is the noise before defeat.

— Sun Tzu

As Opendoor has grown, unifying models across related tasks has expanded their benefits from tactical to strategic. These benefits flow through to our customers, both sellers and buyers — the more accurate and efficient Opendoor can be in navigating the lifecycle of home buying and selling, the less financial risk we take on. And the less risk we have, the higher prices we can offer sellers, the lower prices we can charge buyers, and the more homeowners we can serve. In other words, model unification feeds directly into powering life’s progress one move at a time.

While model unification might sound like the pure purview of data science, product managers are heavily involved in it — success here also depends on engineering, operations, and many other stakeholders. In short, a classic product management challenge! In fact, this (largely non-technical) blog post is written primarily for product managers interested in machine learning products‚ although we believe data scientists, machine learning (ML) engineers, and anyone else who works on ML is likely to find it interesting and, hopefully, useful for their own work.

We’ll explore model unification through a real journey that Opendoor went through recently, related to how we sell homes that we’ve acquired (a process we call “resale”). There are three major decision types in the resale lifecycle for each home:

• When the home is first listed, set the home’s initial list price
• Each week that the home is on the market, decide whether to drop the list price, and if so, by how much
• Each time a buyer offer is received, decide whether to accept it

Overview of the Resale process for Opendoor-owned homes

Initially, all of these tasks were done manually by our in-house team of portfolio managers, with little support from customized software (and no model support). First, we developed a model for recommending price drops. We then developed a second model for setting initial list prices. And then we developed a third model for deciding on buyer offers.

Deciding to unify our models

Identifying the above-mentioned structural issues with stacked and parallel models was only the first step towards the decision to unify our resale models. To put model unification into our roadmap, we had to move from identification of a theoretical problem to sizing the benefits and costs of solving that problem in real-world terms.

Estimating the benefits of model unification had two parts. First, we began to estimate the direct business benefit of joint optimization of resale decisions. In other words, by removing the “conditional assumptions” inherent in the stacking and parallel setup, how much more accurate would a unified model be than the “sum of the parts” from the multiple model status quo. And then how would that increase in accuracy translate into business outcomes like customer volume and customer-facing prices? There are lots of models that are theoretically “wrong” but where “correcting” the theoretical problem yields negligible customer benefits; we wanted to make sure we were not headed down this path. Product management started with a back-of-the-envelope estimate (done in a spreadsheet) of the accuracy benefit from model unification; when that looked promising, data science built a simple, backtestable prototype of a unified model and produced a more refined estimate with it.

Product management also explored more qualitative benefits of model unification. Some of these were technical, like the fact that building a new (unified) model would allow us to migrate to newer infrastructure. Some were more business-facing, like the fact that a new model could natively incorporate unusual home types that the status quo models did not support.

With the prototype model in hand, we then estimated the costs. In the short term, there would be a large cost in both engineering and data science team time to take the prototype model to production — time estimates that our lead engineers and data scientists were responsible for making and detailing. This was partially mitigated by estimates that the long-term maintenance of the unified model would be significantly lower than the status quo.

Finally, with the cost/benefit analysis in hand, product management (in conjunction with data science and engineering) compared this project to others on our roadmap, slotting it into the cross-functional group’s backlog (the placement was quite high, e.g., “Let’s get going ASAP!”)

Implementing a unified model

Three decisions were key to implementing the unified resale model. First, we chose to keep the model deployed in the initial release quite simple, but with strong support for extensibility and future development. In fact, the initial version was so simple it did not include some signals that we knew mattered and had been incorporated in the precursor models. The decision to exclude these signals was hotly debated! The root of the issue was that different information (or “signals”) is available on a home at different events in the resale lifecycle — as a home sits on the market, not only does more information accumulate, but entirely new types of information become available.

Availability of information by home lifecycle event

Previously, since there had been a separate model for each event, each model used all the information that was available for that event and nothing more. But with a unified model, there was new complexity in deciding what information should be processed for each event. We were able to show that with just the minimum set of information — information relevant to all events — a unified model would still outperform the status quo. That led us to a “sparse” v0 unified model, but it also led us to build it in such a way that adding in additional signals would be straightforward, as we knew that would be follow-up work.

Second, we built the unified model in conjunction with heuristics and operator guidance focused on the areas we knew the unified model would be weakest initially. For example, while we didn’t include certain signals in the v0 unified model, that didn’t mean we ignored them. Instead, we developed simple ways of surfacing these to our operations team, flagging situations where they check the model results due to the presence of those signals, and guidelines for consistent and principled model overrides or adjustments.

Finally, we invested in educating our operations team on the “why” behind the new unified model. This was critical to building an effective feedback system, in which operators flagged truly problematic model recommendations to our product teams, and did not raise a high volume of false alarms. The new unified model generated recommendations that often looked quite different from the precursor models, which could have elicited lots of false alarm feedback. For example, the unified model allowed us to implement new types of strategy shifts through resale decisions, and to implement them very quickly. When a macroeconomic event occurred, we could now respond the next day with new jointly-optimized recommendations for all homes in our inventory. This meant that significant day-over-day variations in model recommendations were often an expected behavior, something that our operators did not need to flag. Conversely, significant variation in model recommendations on the same day between similar homes was generally unexpected, and something we wanted operators to flag.

Where to next?

Model unification has proven to be a powerful way to align our products with our strategy. And this is not just limited to resale; we’ve observed similar benefits across many different product areas. As we look ahead, we’re excited about two things that build on (and contribute back to) model unification. First, integrating models across entirely different task groups (as opposed to unifying models within a task group). An example of this is home acquisition versus home resale; they’re different task groups, but they both are key to managing inventory and portfolio risk. Model integration offers benefits that go beyond simple manual attempts at reconciliation. We’re also excited about new venues for experimentation, to generate counterfactual data that helps us understand what would happen if we deviated from our status quo systems. This information is critical to understand how far our systems are from optimal, and it shapes our roadmap around where and how to improve model-driven systems.

Unifying models is not easy. It requires collaboration across product, data science, engineering, operations, and many other disciplines. At Opendoor, product management plays a key part not just in coordinating this type of product development, but also in laying out the strategic vision that motivates. We believe in working backwards, starting with our mission, and identifying the largest opportunities to advance it. And we’ve found unifying models, despite its challenges, to be a consistent winner in that regard.

Interested in joining our team? Opendoor is hiring across all teams, including Product Management and Data Science! Check out our current open positions!

Want to learn more about working at Opendoor? Check out our Product Management and Engineering and Data Science blogs.