Usability Test Case Study
Project Info
Design challenge to run usability testing on the VRBO mobile App.
Duration
2 Days
Background
VRBO is part of the HomeAway family and is a leading community-driven hospitality company where people can list, discover, and book unique accommodations around the world. Whether travelers want an apartment for a night, a castle for a week, or a villa for a month, VRBO.com connects people to unique travel experiences in more than 34,000 cities in 190 countries.
Methodology
Research questions. The study collected qualitative and quantitative data to answer several research questions, including:
· Task completion — How well does the site support customers’ ability to accomplish key goals and tasks?
· Navigation and information architecture — How does the site structure support customers’ ability to accomplish their tasks? Can they navigate to where they want to go and accomplish their tasks quickly and efficiently? What pathways do they take?
· Content and terminology — Do the customers understand the content and does it help them accomplish their tasks?
· Layout and visual design — What are the customers’ impression of the visual design?
· Communication and site impressions — What are the customers’ overall impressions of the site? Does it adequately communicate what users can/are required to do with the site?
Study design
I conducted an in-person usability study in order to gather insights into user performance and unmet needs. The study collected information such as task completion rates, time on task, navigation and content insights, overall satisfaction, areas of concern, and unmet needs.
Using a tripod and camera, I recorded several users to study their physical behaviors while using the app in order to asses subconscious behaviors. Each participant was asked to sign a release form before beginning any testing.
Tasks
1. Create an account or sign into VRBO.com on your mobile device (tablet or phone).
2. Find a listing of places to stay in Tokyo that are under $60 and are either a private room with private entrance or an entire house.
3. The place you choose to stay at has to have been rated 4 stars or more.
4. You’ve decided to invite a friend/spouse/significant-other to go with you. Change the number of guests that will be staying to 2.
5. You would also like to add a place that is pet friendly.
6. Add to your profile by adding your photo.
Participants
I used a public library as my testing center and set up in high traffic area to gain access to as many people as possible. I found the best way to talk participants into helping me test the product was to tease their curiosity and entice their stomach. A simple sign and a basket full of goodies goes along way.
“If you feed them, they will come…”
User Responses to VRBO Mobile Product
Price Slider
- Takes to long
- Hard to use
- Confusing
Pet Filter
- Can’t find the “pet” filter option
- Can’t find “pet friendly” filter option
Filter
- Does not give specific enough criteria
- Love the live filter
Calendar
- Confusing at first on how to use
Other
- Slow to load
- First pop up is a deterrent
Findings and Recommendations
1. Filter options are not specific enough for the criteria given in the study.
- Change option compliment to reflect more popular criteria EX: Add “Private Entrance” to “Features” category.
2. No way to sort by star rating.
- Add sort option.
- Label listing without star rating as “Not Yet Rated”.
3. “Property Type” has possible bug.
- Some options are not available until “House” is selected.
- Submit to QA.
4. Calendar is confusing at first, even with text explanation.
- Add secondary directional cue on selected days.
5. Filter sliders are hard to use and not intuitive.
- Change to box option.
6. Filter close “X” was repeatedly confused with “Clear Filters”.
- Change “X” to “Close”.
Hi-Fi Mock Up
Below is the previous search filter (left) next to the recommended changes (right). Based on usability testing, this is what the user actually wants.
Outcomes
The study provided me with:
· Metrics. Objective and behavioral performance data that provided a usability baseline to measure future improvements.
· Audience insights. Actionable insights on how to optimize the user experience for customers.
· Actionable improvements. Concrete recommendations for improvements based on research findings.
Contact Info
Want to know more about my process? Feel free to contact me here.
Disclaimer: I am not affiliation with VRBO or its partners. All opinions and and finding are my own and do not reflect those of the VRBO brand.
