Encouraging engagement and interactivity during online classes
by Team Perogies
As a part of our CS 449 Human-Computer Interaction course, we worked as a small team to design a mobile solution to solve a real-world problem. This was a remote group project completed by Alexandra Girard, Sarah Ma, Samantha Rempel, Moeyyad Qureshi, and Heather Musson. Our group focused on encouraging attendance, participation, and group discussion in online learning. This topic was particularly relevant with courses shifting completely online this year.
Recent years have witnessed increases in online learning around the world without a matching increase in the development of tools to support it. The cumbersome troubles of balancing courses — each seemingly requiring a unique suite of tools — are especially highlighted by the current global pandemic. There is an absence of a single comprehensive platform actively engaging students in their online learning experience.
Being post-secondary students ourselves, we are keenly aware of points of friction when communicating remotely with course staff, campus resources, and other students. Additionally, students across the country are currently taking online courses, making the impact of this project even greater.
Currently, instructors feel like they are “talking into a black-void” or missing out on important body language that would be present during in-person classes. Students, however, are challenged to keep focused and engaged during lectures.
Our solution is a mobile application that allows students to participate in discussions, take short quizzes, provide feedback to instructors, and earn rewards all while attending online lectures.
Throughout the term, we learned about all aspects of the design process and implemented what we learned in order to create an application that solves our problem. We started by conducting user research and creating user personas before beginning to ideate about how best to solve our problem. This report outlines all of the work we completed this term and our final product.
Value Proposition
The project started with creating a value proposition answering “why” our product will be useful to users. We outlined customer needs, wants, and fears, along with the benefits, features, and experience of our product.
Students’ key needs included an “all-in-one” platform for lectures and real-time discussion of questions and answers. Our platform aims to complement lectures with interactive content to keep students engaged, provide instructors with feedback, and consolidate resources.
User Personas and Empathy Maps
Achieving this value proposition required identifying our target users. Two groups of users emerged; those providing content and those consuming content. The first group consists of course instructors, usually Professors and Teaching Assistants (TAs). The second group consists of students.
In order to represent the slight difference in use cases, we chose to create four different personas: a professor, a TA, and two types of students. Empathy maps were then created for each persona to create a shared understanding of our users and their needs. These serve as a reference for a typical day, the priorities, the influences, the goals, and much more for each user.
Meet Professor Peters
Professor Peters is a married, 38-year-old, English, and Linguistics professor. Being very passionate and knowledgeable about what he teaches, he loved his job before COVID-19. Being able to lead classroom discussions was a favourite teaching tool. He felt a sense of accomplishment from seeing his students enthusiastic about the content. Online learning causes concern about whether he successfully engages students because teaching well is what gives him his sense of purpose.
Meet Teagan the TA
Teagan is a single, 24-year-old female, pursuing her Masters of Science in Thermodynamics. She spends a lot of time at the university holding office hours, tutorials, and working on her thesis. She stays very organized to not forget important deadlines or events. Lacking face-to-face student interaction makes online learning a struggle for Teagan. Without in-person office hours or tutorials, she worries that some student questions have gone unanswered or even unasked. Teagan wants to become a professor so good feedback from students and professors is crucial.
Meet Studious Sam
Sam is a single, 21-year-old female, studying biomedical sciences in university. Sam is very career-focused. Good grades are very important when applying to medical school following her Bachelor’s degree. She is a hard worker and values growth and achievement. Most of her time is spent balancing her full course load. Although it was not an issue in the past, with COVID-19 Sam worries about losing focus during remote classes. Additionally, she is fearful of not being able to interact with her instructors one-on-one.
Meet Textbook Thomas
Thomas is a 21-year-old male studying computer science at university. Thomas has already completed five internships for his co-op program. He focuses on the final internship placement, followed by graduating to a F.A.A.N.G. full-time position. As a result, he tends to prioritize achievement and power overgrowth. This includes choosing Leetcode practice instead of regularly attending lectures. Thomas fears failure, especially given his high-achieving classmates. He would pursue a “good job,” even at the expense of his grades.
Questionnaire and User Interviews
To gain more information from our users, we conducted a short questionnaire and user interviews. Although we did not receive many responses, each persona type (professor, TA, and student) was represented. The questionnaire inquired about current online tools and the interviewee’s opinions of them, along with experiences comparing online and in-person learning.
Course staff found that human-to-human interactions more rewarding. Doing so allowed them to easily explain concepts without “clunky” screen sharing through online substitutes for blackboards or pen and paper.
Instructors felt pressure to improve the organization of course delivery in online formats. However, insufficient time was available to do so. One interviewee referenced studies demonstrating eight months are required to develop an online course, but the pandemic demanded a sixth-month transition.
Students appreciate the change in online content, with materials being constantly available for review rather than only by word of mouth during lectures. Friction in attendance arose without formal schedules.
Current online solutions hindered all interviewees. There are still audio/video errors and insufficient collaboration. Further, most interviewees expressed displeasure at the superficial conversation over messaging tools like Slack and MS Teams.
Overall, instructors feel overwhelmed by the timeframe to perfect their online course delivery. The creeping sense of disconnection is a detriment to their job satisfaction. Students are also feeling isolated and unmotivated with online learning.
Affinity Diagrams
Following our exploration of user needs, we started to define our problem space by using an affinity diagram. We focused on information and insights gained during user interviews.
Some of the key insights displayed in our affinity diagram are inadequate or challenging tooling, a strong feeling of disconnection, lacking student motivation, and the sorely lacking organization of online courses.
User Tasks
The affinity diagram assisted us in defining the needs and wants of our users and allowed us to create an application to fulfill them. For this, we identified some key tasks that users want to accomplish with our application:
- Asking and responding to questions
- Providing and receiving feedback
- Discussing content with peers and instructors
- Viewing a course content release schedule
Storyboards
After identifying four key features as user tasks, we outlined five design arguments:
- Instructors viewing lecture data
- Students reacting to lectures
- Gamification of viewing lectures
- Instructors creating interactive activities
- Professors and students engaging in discussion
These five design arguments are what we would base the remainder of our work on this term. We then created storyboards to represent user stories associated with these design arguments.
User Flows
With our storyboard created, and our use cases further defined we began to focus on preliminary layouts for each design argument. We started with a group Crazy 8’s session to help us brainstorm as many ideas as possible for each feature. Based on this activity, we decided on a layout that we thought was best for each feature and moved on to create basic sketches and user flows.
Low-Fidelity Prototypes
From the five feature user flows completed previously, we chose to build two as low-fidelity prototypes. As a group, the choice was made to prototype the in-class quiz activities and the question and discussion feature because they provided more complex interactions. As an example, the discussions required interaction to move between current and past lectures, as well as changing the visibility of someone asking the question. Therefore, these features were of the highest priority of gathering feedback from test participants.
Due to the remote nature of this term, we used Figma instead of paper to create our “paper” prototypes. This choice ensured that we could easily share the prototype amongst team members and with our test participants.
Lecture Discussions
This first paper prototype showcases how a student or instructor can participate in live discussions during lectures. Check out this link to try out the prototype for yourself:
In-Class Quizzes
The next paper prototype was more complex since it needed to cover how an instructor would create a quiz, and how a student would participate in the quiz. Below you can view a working prototype that showcases how an instructor can create a quiz:
The second flow is for students to complete quizzes during a lecture. Students will be notified of quizzes through a popup in the application. This will only be shown to students who have selected to enter the live lecture.
Low-Fidelity Prototype Evaluations
To evaluate our paper prototypes we asked participants to complete a series of tasks. The chosen tasks centered around asking questions, having discussions during lectures, creating quizzes, and participating in quizzes.
We conducted these evaluations with an instructor and a student. These occurred over video calls with shared screens. This facilitated observation of users’ facial expressions, body language, and cursor movement while completing tasks. These notes allowed deeper understandings of any hesitation or confusion users experienced.
Collectively, users enjoyed the chosen feature set and the cohesive prototype layouts. No difficulty was encountered while completing tasks. Throughout, participants provided great insights on how to improve certain design choices.
Participants felt that the messenger style discussions would become confusing to search for questions and responses. The team felt that having one discussion thread per lecture caused this risk to be outweighed by the benefit of the familiarity. Additionally, it was suggested that past and future lectures be explicitly separated into different pages. The reminders were well received. One participant suggested adding more customization to lecture reminders — including setting reminders for all lectures.
Quiz creation was very intuitive to users, with similarities to other tool interfaces. Unfortunately, some customizability was lacking. Though excellent, these remarks were not addressed due to time constraints. Users also felt similarities to Kahoot! in answering quizzes. One participant commented on how the application removed the need to purchase iClickers with batteries.
The prototype evaluation gathered valuable feedback for future iterations. Many of the above recommendations were later implemented, including the organization of lecture cards and quiz customization.
High-Fidelity Prototypes
The name we chose for our application was Participation Pierogies. The primary color is blue-green, reflecting trust and dependability (blue), and growth (green). The goal is to convey to users that the application will be a dependable addition to their learning.
Similar to the in-class quiz paper prototype, we created two different high-fidelity prototypes — one for an instructor, and one for a student. The instructor prototype has all of the features available to a student plus the ability to create new courses, schedule lectures, set up quizzes, and view analytics.
We created this short demo video to showcase our application:
To view our click around our high-fidelity prototype as a student, check out this Figma prototype:
To view our application as an instructor check out this Figma prototype:
If you are interested you can also check out our Figma board:
High-Fidelity Prototype Evaluation
As we were wrapping up our project we wanted to make sure that we were producing the best possible application. Therefore we connect with our peers to conduct two heuristic evaluations and one cognitive walkthrough to evaluate our high-fidelity prototype. The participants gave us insights into further improvements for our application’s usability.
Our heuristic evaluations focused on: user control and freedom, consistency and standards, aesthetic and minimalist design, the match between system and the real world, and visibility of system status. Both evaluators were able to complete the high-level tasks without issue.
Evaluators suggested adding confirmation buttons throughout the application to prevent accidentally sending incorrect student input. Another adjustment to improve user control and freedom is reordering the quiz list so that the most recent question is first, along with clearer control to navigate out of quizzes.
Both evaluators felt like the consistency and aesthetic, as well as a match between the system and the real world, met expectations. The application gave a polished professional air which helped them intuitively navigate. However, the symbols used in the reaction feature were not the most intuitive; an example being a turtle rather than a rewind button for the slow down.
Most suggestions centered around the visibility of system status. Amongst these was scrolling to the end of discussions after posing questions. Another suggested only temporarily highlighting a submitted reaction. One evaluator requested a digital countdown to replace the progress bar during quizzes, though Figma limits creating this transition. Another remarked that the reaction recipient is vague, and could be remedied with an in-feature explanation.
Overall the evaluators enjoyed our prototype and felt three of the five heuristics were strong. User control and freedom and visibility of system status needed improvements.
For the cognitive walkthrough, our evaluator completed all three high-level tasks without issue. The evaluation, nonetheless, offered helpful insights. The choice of an exit button rather than automatically exiting a completed quiz confused the evaluator. Similar to the heuristic evaluation, the intuition of the reaction icons was questioned. The bottom two reassembled fast-forward and rewind controls, which semantically differ from slowing/accelerating the lecture. Furthermore, a raised hand emoji was suggested as beneficial.
These evaluations were extremely beneficial to us. We had been iterating on our designs throughout the entire process whenever we received feedback from our peers, and these evaluations were no different. Based on this feedback we changed our reactions to be more intuitive (i.e., a turtle for slow down), added more indicators for system status, and outlined how we would onboard a user to our application.
Reflection on the Design Process
We all found this project, and CS 449 extremely valuable. It was one of the most applicable courses we have taken during our undergraduate degrees at the University of Waterloo. We learned about how to empathize with our users, and how to solve real-world problems with technology. Instead of creating the first thing that came to mind, we connected with users to truly understand their points, and how our application could improve their workflow.
When learning more about our users, one of the first things we found was that instructors want to connect more with their students and vice versa. Since moving to remote classes, daily interaction between students and professors has decreased dramatically, and this has affected students and instructors alike. By providing features such as live reactions and discussions we hope to increase the connection between students and instructors. Furthermore, we hope that by providing reaction data to instructors we hope to provide instructors with the tools they need to understand their students’ needs and make adjustments based on this.
In addition to wanting interaction in their classes, students also want to be more engaged. From our user survey, we found that the majority of students said that they are less engaged in online classes compared to in-person classes.
Aside from learning more about our target users, we also learned a lot about the design process. As students studying Computer Science, or Software Engineering we were used to be told what to build and had little experience talking to users. Following a full design cycle, we were able to speak with users and determine what to build based on our discussions.
Current Limitations
While we are very happy with what we created over the past four months, we were limited by the constraint that we had to create a mobile app. We found that users were unlikely to watch lectures on their mobile devices. Instead, we had to focus on creating a tool that could be a companion to watching lectures on another device. Given the option to create a web application we could have expanded our project into an all-in-one platform that provided all of a user’s lecture viewing and interaction needs.
Another one of our major limitations was time. With each member of our group taking 4–5 courses in addition to CS 449 we only had a couple of hours a week to work on this project. Therefore, we were only able to participate in one design cycle. Given more time we would have loved to participate in more iterations of the design process.
Future Plans
In an ideal world, our app would enhance the online learning experience and support both students and professors in getting the most from online learning.
We also hope that when in-person lectures begin again our application could supplement in-person lectures, engaging students, and acting as an improved iClicker that allows for participation during large lectures.
We believe that this application could also integrate with school sites like LEARN to incorporate participation grades for quizzes and discussion.
The possibilities for this project are immense and we are excited to learn more about how an application like this could improve online education in the future.
What are your thoughts about our design process and final product? We’d love to hear about how you think we could improve our application.
