Using UX Research to design and test a website for educators
Anuj Gupta, Katelyn Southard, Ryan Winet
University of Arizona
Context
The Office of General Education at the University of Arizona (UA) designs, implements, and supports changes to the General Education (GenEd) curriculum. The University of Arizona is refreshing its GenEd Program for the first time since 1998 to update its curriculum based on the latest research in education and in order to equip the students with 21st century skills. Specifically the curriculum emphasizes perspective-taking, interdisciplinary thinking, and reflection. Importantly, the curriculum also represents a shift in the culture of teaching at UA, paying particular attention to engaged, dynamic learning environments.
This is a large-scale curricular transition for both students and teachers. General Education is intended to help students explore multiple disciplines in order to become well-rounded individuals. The vast majority of students at the University of Arizona will engage with the transformed GenEd curriculum throughout their time at the University.
To learn more about these program changes:
📖 or explore this webpage
The Problem
The curriculum changes involves providing massive instructional support to GenEd instructors as they transform their courses for the new curriculum. This instructional support at UArizona has come in the form of several web-based resources, workshops, mini-courses etc. These resources were housed on the GenEd website and grew organically as and when the resources became available. Over the course of 2020 and 2021, given the pace at which the team was working, it was not possible initially to think of the design and organizational structure of these resources.
In Fall 2021, we started to become more intentional about applying UX principles to make these resources more usable. This is when the Directors of Communications and Instructional Support in the Office of General Education decided to take a critical look at the information architecture and design of the instructional resources spread across the website.
Following this initial review, we decided to centralize resources for our educators on a focused instructor hub page within the website. Our goal was to make these resources easily accessible for our target audience, GenEd instructors.
Even though we had not received negative user feedback on the accessibility of these resources, we felt these resources could be more logically organized. We decided to collapse these resources into one landing page: the “Instructor Hub” on the GenEd website.
The Goals
🎯 To improve the usability of instructor resources on the GenEd website by improving their information architecture.
🎯 To make our design processes more user-input driven.
The Team
🤝🏿 Katelyn Southard, Director of Instructional Support, Office of GenEd
🤝🏿 Ryan Winet, Director of Communications, Office of GenEd
🤝🏿 Anuj Gupta, Graduate Associate, Instructional Support, Office of GenEd
The Process
✍🏽 Step 1: Goal setting and storyboarding
The team met to take stock of the design of the instructional support resources already existing on the website, consolidate these resources, and identify gaps and future projects.
It was important for us in this process to identify redundancies and outdated information. The product output of this meeting was a short list of elements to include in the new “Instructor Hub” page.
🃏 Step 2: Card sorting to improve information architecture
We realized that the main problem was one of information architecture for which card sorting has been proven to be very effective. Card sorting is a UX research method where researchers “ask people to group cards into categories, and use the results to guide your information architecture design” (Curl, 2022). Based on this, Anuj designed a card sorting study by using a free tool called KardSort.
🔨 Tools
We used KardSort do conduct the card sorting study then Google Sheets to collect and analyze the results.
🧪 Methods
In the study, we created cards based on the items from the initial list we created in Step 1. The participants were asked to sort the cards into categories that made most sense to them. They were also allowed to suggest new categories.
🧮 Participants
The GenEd instructional support team, which consists of 5 designers and content creators for the GenEd instructional support resources.
🏆 Results
Fig 1 above shows a portion of these results from our card sorting study. See more detailed results.
Participants organized each card into one or more categories. For example, you can see in the image that the card “Assessment” got classified into categories like “Assessment,” “Curriculum,” or “Pitching a Course.” Or, our participants let us know that they were unsure what this card really meant or who it was for.
👀 Insights from the results
First, the results told us which cards were getting sorted under a similar category by all our users (for example, users sorted the “Our Team” card quite consistently under the “About Us” category) and which cards were getting classified under a wider range of labels (for example, users sorted “Deadlines” under multiple categories, including “Updates,” “General Information,” and “Pitching a Course.”) We made a list of the cards that were sorted into multiple categories to discuss more deeply in our design team meetings.
Second, card sorting also gave us a range of choices for what we could label our main menu categories on the website as. For example, we saw several possible categories like “Get Support,” “Resources,” and “Help for You.”
💡 Inferences drawn from the results
Based on the data generated from card sorting, we had several team meetings where we discussed the pros and cons of different information architecture choices based on the range of results. Through these discussions, we synthesized the following inferences that we interpreted from the card sorting data:
- Inference 1: Information should be presented to users in easily skimmable and findable way since for our target user population (faculty and instructors), time is the top priority. We wanted to use menu labels that contain action words that tell them whether their desired action matches with the page content or not.
- Inference 2: Make sure each card’s distinct purpose is clear. Some ideas presented by the cards were overlapping and needed further conversation to pull apart under distinct headers.
- Inference 3: By synthesizing the card sorting results, we came up with a new draft of the information architecture.
- Inference 4: We decided we needed to do usability testing with a focus group on the prototypes to get real time feedback on the new information architecture.
Step 3: Prototypes
Based on the results of the card sort study, Anuj created two prototypes:
🎨 Prototype 1: See our first prototype in full resolution or in the screenshot below:
🎨 Prototype 2: See our second prototype in full resolution, or in the screenshot below:
✅ Step 4: Deciding between prototypes 1 and 2
As a team, we had a thorough discussion on the pros and cons of each of these prototypes. While prototype 1 is more concise, prototype 2 allows users to see everything in a bird’s eye view.
Prototype 1’s pull down structure requires additional clicks and users have to make decisions or guesses for opening the pulldown, adding to the cognitive load.
Prototype 2 on the other hand, would allow users to see all options in one go and reduce the amount of time they needed to spend to find their desired resources. This is why, we decided to continue with prototype 2 for further testing as we believed that this wireframe would help us increase the usability and efficiency of our website.
🔬Step 5: Focus group style usability testing
Next, to get feedback on our selected design, we decided to conduct a focus group style usability study. While conventionally, a usability study is one that evaluates “the usability of a product by asking participants to complete tasks and then observing their behavior” (Blakiston, 2022), we decided to improvise this method. Instead of observing participants as they completed tasks, we asked them to document their process and then we engaged in a focus group style conversation with them.
🧮 Participants
We chose to conduct this study with two groups of stakeholders:
GenEd team: These members are involved in various aspects of GenEd content development and communication but had not helped with the the Instructor Hub designs. This is why we thought that running our design by them would give us a good first check about our work given their expertise.
GenEd faculty experts: We also ran our design with our faculty experts who coordinate with our end-users, GenEd instructors. Apart from teaching GenEd classes themselves, they also act as expert liaisons between GenEd educators and the GenEd Office. Thus, while the earlier focus group would give us an internal team perspective, this focus group would give us feedback from the GenEd instructor perspective, which is our target user group for our design.
🧪 Methods
Usability testing: During the focus groups, we asked individuals to perform tasks on the website and collected their feedback through an open discussion. In each session, we first asked participants to list out tasks that instructors perform on the Instructor Hub. For example, how to register for the Quick Start Live-Online mini-courses. Then we asked them to choose one among this list of tasks and use our prototype to complete that task. We encouraged them to use multiple devices (phones, laptops, tablets etc.) to complete these tasks.
Discussion questions: Once they completed their chosen task, we then asked them to answer questions about their experience. Finally, we had an open discussion to gather deeper insights into their experience using these questions:
- What device did you use to access the Instructor Hub page?
- Which task did you try to accomplish?
- How easy/difficult was it to complete your task?
- How fast/slow was it navigating the Hub to complete your task?
- How clear/unclear was the language of the information as well as its organization in the Hub?
- How welcoming-friendly/ unwelcoming-unfriendly was the Hub’s layout, colors, images and other design features?
- Did the Instructor Hub have all the functions and capabilities you expected it to have?
- If you were designing this page, what changes would you like to make to it?
🔨 Tools
- Google Docs for the usability study and Zoom for the focus group discussions
🏆 Results
We found common themes that emerged in the focus groups:
Focus group 1: In terms of navigation, we got consistent feedback that users preferred links to open out to new tabs to facilitate keeping track of all the pages they were accessing. Several of the members also felt that re-arranging the order in which we presented our content categories in order of priority to users would be useful. They recommended that we should have Announcements (top-left), Course Proposal Support (top-right), Class & Workshops (bottom left), and About the Gen Ed Team (bottom right).
They also suggested that we add a Course Proposal FAQs page as the team recurrently received similar questions about that aspect of the Gen Ed workflow.
Finally, we also got some feedback on the need to change our language to make it more inclusive. For example, our earlier design had said “Instructors, you are the backbone of transformational education and we are here to support you”. Our Diversity, Equity, and Inclusion (DEI) expert alerted us to how that language de-values the labor that administrative staff puts in to make GenEd work. Keeping this insight in mind, we changed the banner to say “Instructors, you are a key part of transformational education and we are here to support you.”
Focus group 2: Our faculty experts highlighted the need to link out to specific GenEd curricular information (like explanation of course learning outcomes, signature assignments etc.) more prominently rather than having users find that information by navigating to secondary pages.
They also recommended that we change names of many of our links to be comprehensible to a non-GenEd audience. For example, while earlier we had an link called “Quick Start GE Course”, they recommended that we revise it to something like “Training for course proposal preparation (Quick Start GE Course).”
💡 Impact of these results
Recommendations from these focus group studies became our actionable items for redesign described in Step 6.
🗺️ Step 6: Affinity mapping
Through affinity mapping which “refers to organizing related facts into distinct clusters” (Pernice, 2018), we then collected all the feedback we got through our various studies and created a to-do list for the changes we wanted to make to the design and information architecture of the webpage:
🎯 Action Step 1: Rename webpage sections to be more clear. Rewrite the content (including links) to include less GenEd jargon and be more intuitively accessible to the public.
🎯 Action Step 2: Reach out to appropriate curricular stakeholders to coordinate content for their specific pages in each menu section.
🎯 Action Step 3: Ensure consistency in color, font, and formatting. Ensure that all hyperlinks open out to an external tab in the browser to ease navigation.
🎯 Action Step 4: Ensure all webpages adhere to web accessibility guidelines by using the automated Spotlight tool in Google Chrome.
🎯 Action Step 5: Create an Instructor Hub tab on the GenEd website navigation toolbar.
Summary of Design Changes Informed by User Research
🃏 Design decisions impacted by our card sorting study
First, it helped us make the titles of each card in our design distinct to avoid users getting confused about which card to go into for which category. For example, our study had told us that some resource titles like “Deadlines” were clubbed under various categories like “General Information” and “Pitching a Course.” Our study led us to infer that we should create cards based on the tasks that our users want to achieve to allow for more distinct categorization. We now have a “Learning about GenEd” as a card which contains all the resources that instructors need to understand the different features of the GenEd curriculum, and a “Course Proposal Support” card which contains the specific resources that users need to start developing their own course materials and pitch it.
Second, it also helped us create content that is accessible both internally to the team as well as externally to our users. For example, one our links in the “Course Proposal Support” card is called “Training for course proposal support (GE Quick Start Course)”
🔬 Design decisions impacted by our focus group style usability testing
First, in terms of navigation, we edited all links to open out to new tabs. Additionally, we also got very useful feedback on arranging the cards in order of priority: Announcements (top-left), Course Proposal Support (top-right), Class & Workshops (bottom left), and About the GenEd Team (bottom right).
Second, we had also gotten feedback that some of our language was not inclusive. For example, we had a banner in our original design that said “Instructors, you are the backbone of transformational education and we are here to support you”. Our Diversity, Equity, and Inclusion (DEI) experts alerted us to how that language de-values the labor that administrative staff puts in to make GenEd work. To address this, we changed the language to: “Instructors, you are a key part of transformational education and we are here to support you.”
Third, our faculty experts had highlighted the need to link out to specific GenEd curricular information, like course learning outcomes, signature assignments, etc.
The Final Design
We launched our webpage on 10/25/22. We plan to collect website analytics data and do more usability studies with instructors in the future to continue to improve our design as the program evolves.
What we’ve learned
🤔 Lesson 1: What feels intuitive and natural as a design choice, even the most simple ones, varies from person to person which is why it is really important to test out different design options with a sample of your target users.
🤔 Lesson 2: Just asking users what they feel is never enough. Triangulating this data with observational data where you ask them to complete a task on your prototype really helps get a better insight into what kind of changes are needed.
🤔 Lesson 3: UX writing is just as important as UX design. Usually we tend to prioritize the more visual aspects of design, but we learned that paying attention to making our language more inclusive and accessible is just as important.
