Case Study — Library Search Tool
UX Research: Needs assessment and usability evaluation of University of Michigan’s Library Search tool
Background
The University of Michigan Library (U-M Lib) has rolled out an improved version of their resources search feature in the last year. We have been given the task of measuring the usability of the new U-M Library search application. We are looking at understanding how we can improve the library experience for seasoned library users. Our target audience are sampled from a broad range of academic disciplines and roles (Adjunct, Lecturer, GSI, Assistant/Associate/Full Faculty). Our main goal is to identify usability barriers related to the design, performance, and information architecture of the U-M Library Search application, with a focus on the “Catalog” search option.
Research Questions and Goals
- Are users able to successfully locate and access search and advanced search options?
- Are users able to find specific content (known search) using the search bar?
- Are users able to search for an unknown item when they are unaware of a topic?
- Are users able to discover and understand the usage of various filters?
- Are users able to leverage various features like favorites, exporting citation, sharing books and booking a resource online?
- Do users get useful feedback for errors and how do users seek help?
Process
Step 1: Interaction Mapping
We first analysed the user flow with interactions and associated screens of the library search website. Find the details in the presentation below.
Step 2: Stakeholder Interviews
Through four interviews with both advanced users and low-use users, we found multiple areas in which users are having issues as well as where some features are working well. Overall, users are satisfied with the tool’s performance and overall look (specifically in comparison to the old tool). Users find that searching for resources and finding relevant results is a simple task, although there are caveats to whether users find relevant resources or choose to use the library tool versus others.
- Interviewee 1: A male PhD, MFA professor in the English Department Writing Program at the University of Michigan
- Interviewee 2: A female staff member in Communication Studies and Information Science at the University of Michigan Library
- Interviewee 3: a female Lecturer at the School of Information
- Interviewee 4: a female Graduate student GSI at the School of Information
The detailed report can be found here.
On the basis of these interviews, we developed a set of relevant personas and use case scenarios.
Step 3: Comparative Analysis
For this comparative analysis, we identified and compared a total of Eleven different competitors and weighed their strengths and weaknesses. Each competitor’s features were studied in detail and grouped into categories based on the range of functionalities they offered.
Find the detailed report here.
Step 4: Surveys
In this particular study, our aim was to understand the behaviors of our audience in terms of the usage of certain features. We designed a survey in order to understand — our audience demographics, usage of library, usage of library features, inclination of certain demographics to certain features, and usage of other platforms.
The survey had 22 questions, comprised of multiple choice, text entry and rank order. The survey was sent to the audience comprised of lecturers, professors, GSI, and doctoral candidates.
The survey received a total of 26 responses of a sample size of 662 individuals. Our audience included Faculty, GSIs, and PhD students of various colleges within the University of Michigan (including the School of Information, Pharmacy, Social Work, Rackham, Engineering, LSA, etc).
Find the detailed report here.
Step 5: Heuristic Evaluation
For this heuristic evaluation, our primary goals were to identify barriers, pain points, obvious usability and/or interface issues, and review the site’s overall flow — all through a user’s typical scenario when using the site. This scenario includes logging in, searching, finding materials, and requesting to access materials. Our key questions come directly from our goals, including: what about the interface could be improved? Are there barriers to access or clear usability issues? Are alerts used properly? Are there issues with the flow? And more.
The goals of this study were a clear fit for heuristic evaluation because heuristic evaluation allows designers/researchers to review the site with their own knowledge and a clear set of conditions with which to evaluate the site. Having a clear set of conditions allows the team members to be more objective/quantitative with their evaluation. The goals of the study fit well with delivering value to the U-M library’s team by offering recommendations to some clear issues without having to take time and resources to conduct in-depth usability tests. Heuristic evaluation allows us to identify functionality or flow issues that users in the field also wouldn’t identify due to their lack of knowledge of the platform.
Find the detailed report here.
Step 6: Designing an Analytics Dashboard
Based on our research thus far, we decided it would be helpful for the designers and developers of the U of M library to understand how users are currently using the library system. A quantitative analysis of the usage of the U of M website would help discover the strengths and possible areas of improvement in the website. Providing a metric to understand user search flows, and measure the use of search features could lead designers to understand the effectiveness of the website. We believe an analytics dashboard along with a monthly report would provide better insights as to how users are using the current library search features and how they could be improved. The primary goal of our report is to help -
- Find patterns within user flows
- Measure search relevance
Our dashboard consists of 11 visualizations, each displaying the nature in which users browse the ‘Catalog’ section.
Find the report with details here.
Step 7: Usability tests
This study aimed to create a deeper understanding of the usability issues that our target audience — seasoned users of the library, face when using the catalogs search feature. Having users perform specific tasks allowed us to capture how they interact with the website, how they feel and what the problems were. For this study, we had participants perform a set of predefined tasks. Each test included a moderator script, with a preamble explaining our goals, roles and what the purpose of the test was. We also provided pre-questionnaires to help understand the demographics and background of each participant along with questions pertaining to the usage of the library website.
Find the detailed report here.
Step 8: A brief video presentation of consolidated findings and recommendations for the client
Conclusion
This project gave me a good understanding of the user experience research process including various methods, activities and planning timeline. I also learnt presenting the findings and recommendations in concise (presentations) as well as elaborate (reports) formats for the client. This was a great first time experience with a huge amount of learning and I hope to incorporate all the knowledge gained in future projects.
