How Wheelhouse Used A/B Testing to Improve Overall Revenue
As we’ve mentioned in a previous blog post, one of the biggest differentiators between Wheelhouse and other pricing tools out there is our investment in and dedication to Data Science, and our ability to run pricing experiments on our own rental units to improve the overall algorithm.
In the beginning, we relied on observing external rentals to guide how we built our pricing system. And while this took us far, helping to develop our initial algorithms, we hit some hurdles. Most notably, was our inability to experiment with pricing strategies. We didn’t really have a way to proactively test any of our ideas.
And so, after a few years building Wheelhouse, we began managing our own rentals. Doing this allowed us to ‘dogfood’ our pricing. Our Data Science team can now experiment and iterate on different hypotheses. We review experiments that work well and build them into Wheelhouse Pricing. Similarly, we refine experiments that don’t work well to get better for the next one.
Temporality A/B Test
One of those recent experiments was around temporality, which refers to how pricing recommendations change as the potential stay date approaches. Conventional wisdom says to increase prices for far-future stays to capture higher revenue from ‘advanced planners’ and to decrease prices as the date of potential stay approaches to capture revenue from ‘last-minute’ travelers.
The chart below shows how we’ve translated this conventional wisdom around temporality (the red line) into our pricing model. Stay dates within 14 days are discounted up to 15%, and any stay dates more than 45 days out are increased by about 10%.
As a data-informed company, we decided to pressure test this assumption. How much of a difference did temporality really make on booking performance? We have spent the past few months (July-October) trying to figure this out.
The results were a bit surprising. The units in the A group used our existing temporality curve. The units in the B group used our experimental, dampened temporality curve. Our hypothesis was: by not raising prices as significantly far into the future as we did previously, we’d see an increase in booking lead time and occupancy. That’s exactly what happened.
What we didn’t expect, though, was that total revenue would increase by as much as it did. Our original hypothesis was: As the stay date gets closer, prices from A (the group with far future temporality) would eventually be lowered to be more like prices in the B group, and total revenue would be roughly the same. Instead, the test group (B) outperformed the control group in total revenue. In fact, units in the experimental B group had 37% higher RevPAR than units in the A group. So, our units performed better across the board without increasing prices for far-future dates on the calendar.
The Wheelhouse Data Science team concluded this improvement was due to two major factors:
- Guests that book far into the future are more price sensitive than originally thought. Showing high far-future prices did not mean potential guests would come back and book later, but more likely to book a cheaper place at that time.
- Capturing earlier guests does not diminish bookings in the weeks before the reservation day. This is when Wheelhouse’s last minute discounts kick in, incentivizing last-minute travelers.
As a result, Wheelhouse will be modifying our temporality curve to reflect these findings. We will dampen the price increases for far-future stay dates and maintain last-minute discounts for near-term stay dates.
We’re excited to start rolling out these changes for Wheelhouse customers the week of Monday, November 26th, 2018. We expect these changes to not only have a potentially significant positive impact on your revenue but to also increase booking lead times, helping to give you more certainty that your short-term rentals are performing at their highest potential.
Any questions or comments? Let us know at hello@usewheelhouse.com or via live chat on our website at www.usewheelhouse.com.
