Optimizing the Hostelworld Web experience — a User Research case study
--
Uncovering opportunities by following a User Centered approach
About Hostelworld
Hostelworld is the leading online booking platform of hostels in the world with over 36,000 properties across 170 countries globally. The platform is mostly used by the younger generation to discover destinations and meet new people while staying in budget friendly hostels.
Research objective
The challenge that we decided to tackle was to analyze and suggest changes to optimize the hostel listings page on the website. The user is presented with this page after he/she has filled in the necessary details i.e. Location, Check in and Check out dates and Number of guests.
My Role and Timeline
We were a four person team. My role in the project was of a user researcher — specific tasks included designing questionnaires, conducting contextual interviews, usability testing and qualitative data analysis.
The project was completed within a duration of 2 weeks.
The Design Process
Usually, any product solution needs to cater to both the user needs and the business goals.
But it’s also true that for any business, happy customers lead to success and growth. Hence in order to solve our challenge, we followed a user centered approach to understand the pain points and motivations of the users to get a deeper understanding of their needs and behaviors.
We also assumed a common business objective of increasing the booking conversions of hostels on the website.
Our 2 weeks
We followed the double diamond model as our design process for the two weeks.
In this case study, I will focus on my role in the project which was the Discover and Define phase i.e. Designing for the right thing.
Discovery Phase
Understanding our Users — Developing Context
As we began our journey, we knew nothing about hostels. None of us had lived in hostels. The discovery phase required us to understand the context about hostels and gather insights about the same.
Hence, the first step was getting to know our users. A quick search on Hostelworld, and we landed on two hostels in Porto — Nice Way and Pilot Design.
We conducted contextual interviews with 4 hostlers and the managers in both hostels. We grouped the questions for the hostlers in the following themes:
1. Traveling Choices
Example: How often do you travel? With how many people?
2. Reasons behind choosing a hostel
Example : Why did you choose to stay in a hostel and not Airbnb?
3. Ways of booking a hostel
Example : How did you book this hostel? Is this usually how you do it?
4. Expectations from the hostel
Example : What are your expectations from this hostel keeping in mind the ratings and reviews?
The people we interviewed were from different nationalities, cultures with varying traveling preferences which helped us in gathering diverse perspectives.
As an example, one of the hostlers was a 23 year old from Netherlands who travels roughly around thrice in a year. An interesting thing he pointed out to us, was that he always books a hostel from the hostel’s personal website. He uses booking platforms like Booking and Hostelworld to search and find hostels but he never makes a booking using them. He believed that by making a reservation for the hostel from the hostel’s personal website, they would know for sure that he is coming. It was interesting to know that he trusted the hostel’s website more than the booking platforms.
How much can we achieve in 2 weeks?
We identified a lot of opportunities from our own unique viewpoints in our initial research. But we realized the need to define the scope of what we wanted to achieve together as a team, given the time constraints.
Building on top of our context and assumptions, we decided to use a variation of user story mapping which we internally called “Applied User Story Mapping” to define the scope of our research.
Applied user story mapping
Aim : The goal of this exercise was to identify themes that would capture the most value for the end user given the context and assumptions built from the initial research.
The following are the steps for the exercise:
1. Each team member creates a user story that defines a user’s journey using the product.
2. Different parts of the user story are allocated relevant themes.
3. We identify and label parts of the user story that can be solved by features in the current website
4. Finally, we vote on the bits of the user story which show the most value and impact.
Following the exercise, we came up with the following themes:
- Events : While attending a concert in a city, people might want to stay in a hostel.
- Social influences: Staying in a hostel based on suggestion from friends or through social media platforms.
- Trip planning flexibility: Choosing a hostel because the duration of their stay could be flexible.
- Personal Passion : Living in hostels as a way of fulfilling their personal passions like traveling or adventure sports.
Designing experiments to deliver learning
We wanted to dive deeper about the core themes identified previously. The way to do that was to design experiments that deliver the intended learning and also validate our own assumptions. But before designing the experiments, we had to identify the opportunities to uncover through the experiments.
Our intended learnings were as follows :
- What makes a user choose one property over another?
- Where does the user face pains/gains during the buying process within the dynamic/static pages?
- How the hostel location played a role when a user is choosing hostels?
- How does a user discover new hostels?
We crafted a combination of Usability Testing and User Interviews to capture our intended learnings.
2 Hostels, 10 Users from 5 different countries
Back in the field, we visited 2 hostels — Pilot Design and Nice way. We conducted sessions with about 10 users from 5 different countries.
In terms of usability issues, we knew we could discover 85% of the issues with 5 users, based on this research by Jakob Nielsen.
Usability Testing task :
We created scenarios of two different types : Group and Solo Travel.
Each task had constraints regarding the location, check in and check out dates, the budget and any necessary amenities.
Given these constraints, the users were asked to book the hostel of their choice.
Some of the User interview questions:
- Have you ever made a trip where you stayed in more than one hostel? If many, why? (Trip Planning Flexibility)
- Do you know about theme based hostels? If yes, What are they? How did you find about these hostels? (Social Influence, Personal Passion)
- Do you follow hostels on social media? Why? (Social influence)
Define Phase
Having gathered data from 10 users, the objective was to analyse it from different perspectives and reach a problem definition. We wanted to identify the most valuable and feasible opportunities to solve for, given the time constraint.
Analysing the data
We recorded the usability testing and the interviews. We went through all recordings and each and everyone observed and identified behaviors we thought were worth examining(shown above). Utilizing all of our different perspectives helped us to cross examine each other and not bias the analysis by a single opinion.
We organized the various observations with respect to the corresponding section of the website as shown below.
Prioritizing Usability Issues
Having organized our data, we wanted to find a way to prioritize usability issues. We came across this article by Steve Bromley, lead user researcher at PDS, who explained why usability issues should not be prioritized based on their frequency when testing with small sample sizes.
The result is depicted in the graph below which shows even if an issue has occurred once in testing, it might occur 65% of the times in the real world while if it has occurred five times in testing, it might still occur just 60% of the times in the real world.
Keeping the above result in mind, we decided to use this decision tree(shown below), created by userfocus to prioritize usability problems.
The above decision tree shows that to compute the severity of a usability problem, it has three key attributes:
- Does the problem lie on the Red Route? : The red route is a route that is either frequently visited by the user or is a critical task. An example could be a user booking a hostel on the website.
- Is the problem difficult to overcome? : Understand whether the problem affects task completion? An example could be a functionality hidden under a right click which is hard for the user to identify and hence becomes difficult to overcome.
- Is the problem persistent? : Identify whether the problem exists on multiple pages? An example could be all hyperlinks on a website are not underlined, as a result the user faces the same problem in multiple pages.
Based on this matrix, the problems could therefore be categorized into critical, serious, medium or low issues.
The following is the screenshot of the prioritized usability issues based on the decision tree.
We’re almost there. So far we have seen the entire process of how we went from understanding the challenge to talking to users to analysing and synthesising the user data. But before we jump into defining the problem statement, let’s probe into the most interesting finding we had during the usability test.
Looking into the Serious Problem
Users preferred using Google maps instead of Hostelworld map view
If you didn’t see the video above, here’s what happened: the user wanted to gain better context of a hostel and instead of using Hostelworld’s map view, he decided to proceed to Google Maps. Although the process seemed to help the user we found something which could adversely affect Hostelworld and it’s business.
As you can see, once the user abandons Hostelworld’s website to visit Google Maps, he can find other booking platforms offering the same property for a cheaper rate.
At the beginning of the project, we had assumed the business objective of increasing the booking conversions on the website. This particular issue could adversely affect booking conversions and might also cause a problem of retaining users.
Why were users abandoning the Hostelworld Map View?
Problem Definition
The issues highlighted above gave us clarity in defining our problem definition. We decided to frame it as follows :
How might we improve the Hostelworld Map Experience?
With that we finished first, of the two diamonds in the double diamond process.
We won’t dive into detail but I would like to give you a peek into our final solution that was done by my teammates : João Araújo and Harita Sahadeva.
Conclusion
My personal take away from the project was the significance of designing for the right things and eventually building solutions on concrete research. I have always loved user research and this was quite a fulfilling experience. Lastly, I would like to mention my team members without whom this could not have been possible : João Araújo, Harita Sahadeva and Damon E. Redding.
