Learnings from a failed product launch
(In)validating a new product offering using design thinking and what we could do differently next time.
RateIntelligence was a free rate shopping tool by Booking.com for partners (property owners) to help them predict their occupancy and maintain competitive prices. The tool enabled partners to gain insights into their local market, compare rates with their competitors, and set competitive rates by using market data to forecast trends.
In 2018, we adjusted our strategy and set out to validate if a paid offering was desirable, viable, and feasible. My role was to lead the UX strategy and design for RateIntelligence and the new product, RateIntelligence PRO.
Understanding the partner
For the free version of RateIntelligence, we needed to better understand the mental model of partners and how they set their rates as part of their revenue strategy. We developed personas and journey maps based on partner research to empathize with our partners’ needs and challenges.
Personas
We put ourselves into the shoes of our partners and created empathy maps. Using the Jobs to Be Done framework (JTBD), we analyzed existing research to identify the key requirements that partners “hired” our product for. By the end of the session, we had identified 3 primary users: Owner Olivia, Reservation Manager Rebecca, and Revenue Manager Rick.
Journey mapping
I organized a 2-day workshop with the team to better understand the partner journey. The intended outcome was to have journey maps that visualized the primary use cases, actions, thoughts, feelings, and opportunities during each stage of their journey.
UX competitor analysis
To understand the current market and primary competitors, I led a UX audit of the top 5 rate shopping tools. We researched the competitor profile (e.g. mission, competitive advantage), marketing and brand (e.g. traffic, target segment, social media, pricing model, reviews), UX (e.g. content, information architecture, usability), and feature set. This data allowed us to establish a baseline and identify where our product fell short, where we held a competitive edge, and where there were potential opportunities.
Ideation and testing
Brainstorming opportunities
We used the personas and journey maps to determine what pain points to prioritize. To generate ideas that would make it easier for partners to set their room rates, we sketched ideas using the Crazy 8’s framework.
Storyboarding the ideal scenario
After generating a lot of ideas, sharing back to the group, and dot-voting on the best ideas, we switched to storyboarding. This helped create an end-to-end flow from the partners’ perspective. We used the best parts of the individual storyboards and added steps that were missing. We played these back to the group and discussed.
Quick validation
Using the storyboard that each team agreed upon, we spent an hour creating paper prototypes. It wasn’t necessary to have high-fidelity designs since the goal was to quickly test and validate with proxy users. Later on, we would iterate upon the prototypes and test them with real partners.
Partner interviews
To quickly test and gather regular feedback, I established a bi-weekly partner research framework for the team. This was a step-by-step guide on defining the audience, managing communication, tools and logistics, best practices for preparing designs, creating the discussion guide, capturing feedback, analyzing, and sharing insights with the team. This framework was continually refined and enabled our team to quickly iterate on concepts and get direct feedback.
RateIntelligence PRO pilot
The objective of the pilot program was to validate product-market fit. Success would be achieved if we could convert 10% of partners from free to paid during the trial period. In addition, we wanted to test the value proposition, acquisition channels, and gather qualitative feedback.
Challenge
After the product strategy pivoted towards monetization, the current RateIntelligence product was lacking in 5 key areas:
- Difficult to monetize based on the existing target audience
- Insufficient data to make informed pricing decisions
- Difficult to customize the data to fit their needs
- Didn’t integrate easily with property management software (PMS)
- Poor navigation and user experience
Target audience
We analyzed data (e.g. property type, # rooms, region, star rating) from over 44,000 active partners so we knew what type of partner to prioritize. From in-depth interviews, there was little indication that small to medium size properties had a desire for a paid rate management solution, therefore we needed to focus on larger properties that had dedicated Revenue Managers. Using qualitative and quantitative data, we were able to create a 360° view of our new target audience.
We conducted additional research and revised the persona and journey map of Revenue Manager Rick, to ensure it accurately depicted the needs of our user.
Meet Revenue Manager Rick
The main difference between Revenue Manager Rick and the earlier personas was that Rick generally had a business degree or background in revenue management and was hired by large properties specifically for that reason. Owner Olivia’s and Reservation Manager Rebecca’s worked for smaller properties, were less experienced, and juggled a lot of different responsibilities.
Rick hired RateIntelligence for 3 main jobs
- Automate the process of transforming data into pricing decisions
- Optimize hotel performance to increase revenue
- Validate the rate strategy
Rick’s 3 main motivations
- Meet revenue targets that were set by management
- Be viewed as the expert on revenue and pricing strategy
- Correctly predict occupancy and market demand to maximize revenue
Rick‘s 3 main challenges
- Lack of clear market insights and too much time spent interpreting data
- Didn’t trust one tool so would cross-check data against multiple resources
- Required more data to make confident decisions
Understanding Rick’s journey
We generated a journey map, in order to put ourselves in Rick’s shoes and better understand his goals and challenges.
Journey map components
- Stages in the journey: onboard → plan and budget → maintain → analyze data → optimize.
- Sub-stages: For example, in the analysis phase, there were 3 main steps Rick focused on — competitor analysis, market demand & event analysis, and data analysis.
- Frequency of stages: Some stages occurred yearly or more frequently, which gave insight into how we should best solve a specific pain point.
- User actions: Specific activities that Rick had to perform were captured, along with the goal he wanted to achieve.
- Pain points: Challenges that Rick faced so we knew what to prioritize.
- Legal constraints: Legal oversight was required to ensure we were aligned with regional requirements and best practices, including GDPR.
Opportunities
- Planning: Comprehensive reporting to make forecasting, budgeting, and review of historical data easier.
- Performance: Improved tooling, analytics, and alerts to actively manage current performance.
- Competitors: More control in selecting competitors and increased visibility into their data.
- Market demand and events: Increased clarity and data into variables impacting market demand and insights into guests booking preferences.
- Optimization: Product improvements that would give users more control when viewing and customizing the data.
Opportunities
I worked closely with the Product Manager and Designer to prioritize the opportunities based on effort and value to the partner. A few that we explored are shown below: alternative views, ranking, and guest insights.
Alternative views
Revenue Managers always want additional data — the more the better. It’s beneficial to have access to different types of data, quantities, and time-frames depending on their goal.
The existing homepage was a 30-day calendar view that showed the property’s ADR (average daily rate) and the average ADR of their competitors. There was also a graph that showed the ADR rate compared to the market demand, so you could easily compare the trends. This was sufficient for smaller property owners that wanted basic functionality to guide them.
For RateIntelligencePro, we focused on providing additional analytics, controls, and actionable insights. For example, a miniature calendar view showed upcoming dates that needed attention — there was either a significant discrepancy between their rates vs their competitors, or they were missing out on an opportunity to capitalize on a shift in market demand.
We also added additional data such as property restrictions (for example, if a property sets a minimum number of nights required to book a room) to give visibility on how competitors were setting rates and why. Promotions (e.g. last minute deal) and local events can drastically impact market demand, meaning there’s a good chance more people will want to book on those dates. These additional data points help Revenue Managers better understand the current trends in the market, so they can price their rooms accordingly and maximize revenue.
Ranking
Ranking shows how popular properties are on Booking.com search results and is based on a number of factors. The challenge is that it’s not clear how ranking is calculated, not having actionable advice on what to improve, and insufficient data.
To solve this, we improved upon the copy to clarify what ranking was and added additional data points (e.g. conversion rate, room nights sold, cancellation rate) to paint a holistic view of how and why they’re performing in a certain way on Booking.com search results.
Guest insights
Revenue Managers use the guest review score to monitor their competitors. They don’t respond to guest reviews or requests, but they need to know the overall sentiment so they can instruct the reservation or operational teams on what changes the property should make in order to improve their score. Challenges they face are not knowing how to compare competitor data, insufficient segmentation data, and reviews from all channels aren’t aggregated.
In order to solve this, we created a guest reviews dashboard that would provide the most relevant data in one spot (e.g. segmentation data, change in guest sentiment, comparison to competitors).
Learnings
These were just a few of the opportunities we investigated and released as part of the pilot program. Unfortunately, by the end of the 6 months the trial run wasn’t successful and a year later we had to sunset the product.
Reflecting back on the UX process that we followed, you’d think we did everything right. We empathized, we talked to users, we ideated, we understood the problem and opportunities, we did all the design thinking stuff, right? So what went wrong? If you recall from the beginning, there were 5 challenges we set out to solve…
- Difficult to monetize based on the existing target audience
- Insufficient data to make informed pricing decisions
- Difficult to customize the data to fit their needs
- Didn’t integrate easily with property management software (PMS)
- Poor navigation and user experience
Some of these challenges were big risks that we should have validated first. For example, #4: Didn’t integrate easily with other property management software — this is a deal-breaker for larger properties. Booking.com is one of many online travel agencies (OTA) that large properties work with. Some partners churned because our new solution didn’t integrate easily with their existing tools.
#2: Insufficient data to make informed pricing decisions — relates to point #4. Since our product didn’t integrate well with other types of PMS, we were missing a lot of data that properties relied on to have a holistic view. To a property owner, it doesn’t matter what channel the data was coming from.
#1: Difficult to monetize based on the existing target audience — We launched the pilot in one country and 1/3 of the partners ended up churning due to the price point. Trying a few different countries and better understanding of the market, could have helped us to price the product more accurately.
Hindsight is 20/20. In the end, technical and legal challenges were big enough hurdles that we couldn’t overcome. There’s a lot of learnings I’ll take away from this and apply to new products I work on. Next time, as a team we’ll focus more on (in)validating the riskiest assumptions first.
Thank you for reading! 😊 Kudos to the team and especially Daniela Guerrato, my design comrade, who was an amazing collaborator and designer.
