Testing our Content & Design with Users
Part of the process of developing our Career Management Module has been to test key interactions and designs with WGU students. Our first round of user testing involved walking students through a low-fidelity prototype and having them do a think aloud as they were going through the module. Our 2nd round of user testing involved students learning content and taking assessments of the module and having them do a goal setting activity with us. Below we describe our findings from each round of user testings.
1st Round of User Testing Findings and Implications
We completed 3 user testing on the content of our career management module. We quickly put our design ideas into a powerpoint presentation to illustrate the pacing and content of the course. From our user testing we found 5 key findings and design implications.
Users may already think they have a career goal and plan.
Design Implications:
One user already had a system he used for his job to plan and track his career goals. He didn’t see the value in the course and would be reluctant to take it. This made us think about how we could effectively motivate students to take the course before entering the module and give students a variety of examples and use cases for career goal setting and planning.
Users think something like career planning should be done in-person.
Design Implications:
One user strongly felt that something as serious as career planning should be done in-person, however after going through the content, he found some value in the module and process. In our next design, we want to make it clear to users that this should not necessarily replace the in-person coaching that career services provides.
Career experts coach students and lead them to the proper resources
Design Implications:
After talking with our subject matter expert (SME) on career management, we found that sessions with students usually focus on one particular aspect of the career exploration and planning process. In our initial design, we had students briefly walk through the whole process of career management, however, using the feedback we received from our SME she suggested we only focus on the planning and goal setting part of the process, because realistically in one session you would not cover so much material. We will include more of a coaching tone throughout the design and providing them with interactions and examples to practice with feedback along the way.
The module as a conversation starter with alumni or career services expert
Design Implications:
Although the module will be designed to help students think about the career management process we want to make it clear to the students that they can go and validate their goals and plans with other external resources such as alumni or WGU’s career services.
Users found it difficult to answer such open ended questions
Design Implications:
In the first iteration of our design, users were asked to write their career goal. in an open ended way with little support or structure. After consulting with our SME, we decided on an appropriate way to ask students to formulate their career goal. This new way of asking about career goals helped students more easily come up with answers to the question.
2nd Round of User Testing Findings and Implications
We completed 3 user testing on the content of our career management module. We wanted to test the key components of our design including the learning objectives, videos, assessment, and goal setting activity.
Users liked how to video showed an example of someone going through the whole career management process.
Design Implications:
Two of the users enjoyed the video and liked how the example showed someone going through the whole career management process. Our design implication from this insight is to incorporate more examples throughout the other videos because they are easy for users to relate to.
Users were able to answer the assessment questions correctly after watching the worked example video.
Design Implications:
All of the users were able to answer the assessment questions and videos correctly. This signaled to us that our instruction and assessment questions were aligned.
Users had a difficult time distinguishing between skills, experience, and education.
Design Implications:
When asked to state the skills, experience, and education they needed to achieve their career goal, students frequently confused skills for experience. For example, one user said she needed experience using Quickbooks software, however, this is commonly seen as a skill on resumes and on job listings. Using this insight we have changed some of the instructional content for the module to include a graphic and an explanation of skills, experience, and education.
Once users articulate their long-term goal, they want to reference it all the time.
Design Implications:
Once students were able to articulate their long-term goal, they wanted to be able to see it throughout the process of setting their short-term goal. One way we want to change our design to reflect this insight is to have the long-term goal visible to them as they are brainstorming other goals, skills, experience, and education they need to reach their goal.
Users wanted examples for when they were writing their SMART goals.
Design Implications:
Students were asked to write SMART goals for their long-term goals. They had a difficult time coming up with a SMART goal even after hearing what the definition of all the elements of the goals were. In our future design, we will include examples for students to look at while they are writing their SMART goals and include a checklist of questions to self-check their goals.
Next Steps
Our final round of user testing will involve asking students to take a pre-assessment before the module, have them complete the entire module, then have them take a post assessment on the content.
