How Data Science Drives Innovation at Wheelhouse with A/B Testing

One of the biggest differentiators between Wheelhouse and other pricing tools out there is our investment in and dedication to Data Science, and our ability to run pricing experiments on our own rental units to improve the overall algorithm.

Our Data Science team helps us navigate through the uncharted waters of short-term rentals. They turn hard data into useful insights and compelling stories. From optimizing our pricing algorithm to guiding revenue management best practices, they challenge conventional wisdom, leverage data to build thoughtful visualizations, and develop effective solutions for the team.

In the beginning, we relied on observing external rentals to guide how we built our pricing system. And while this took us far, helping to develop our initial algorithms, we hit some hurdles. Most notably, was our inability to experiment with pricing strategies. We didn’t really have a way to proactively test any of our ideas.

“Observation is a passive science, experimentation an active science.” — Claude Bernard

And so, after a few years building Wheelhouse we began managing our own rentals. Doing this allowed us to ‘dogfood’ our pricing. Our Data Science team can now experiment and iterate on different hypotheses. We review experiments that work well and build them into Wheelhouse Pricing. Similarly, we refine experiments that don’t work well to get better for the next one.

This experimentation is key. We see it as the only way to provide the world’s best pricing algorithm. And to do it, we break it into 3 stages: build, measure, learn.

Build

We first generate a hypothesis. These hypotheses come from everywhere — from a failed experiment we tried to something one of our team member notices. Our Data Science team meets and discusses each hypothesis and how to test it. Once we green light it, we build it.

Measure

Soon after building, we begin running A/B tests. We’ve designed our system to compare apples-to-apples to limit the edge cases in testing. Depending on how successful the tests are, we will refine the code or scrap it and move on. Total revenue optimization is the main goal in our testing. We also measure the impact on occupancy, RevPAR (revenue per available room), average booking lead times, and cancelation rates.

Learn

Validated hypotheses are further analyzed post-testing. We then build the key learnings into Wheelhouse Pricing. For Wheelhouse customers like you, this is a win-win. You get the best pricing to maximize revenue without taking on the risk of testing strategies that don’t ultimately succeed.

Read more about the first A/B test that we conducted around temporality, and how we built those learnings into the Wheelhouse Pricing Engine.

Have an idea for a pricing experiment you want to see us run in the future? Drop us a line at hello@usewheelhouse.com or via live chat on our website at www.usewheelhouse.com.