Optimizing Learning Management Systems for Remote Classes
In the spring of 2020, our world was suddenly thrust into the midst of a pandemic, the likes of which many had never experienced before. Students, teachers, and workers had to quickly adapt to virtual instruction, meetings, and extracurricular social interaction. Now, over two years later, our society has shown that we aren’t easily letting go of the privileges afforded to us as a result of the pandemic response. Gen Z, or the 18-to-24 demographic, encompasses individuals that are in school or just entering the workforce in entry-level positions. In a poll from November 2021, a survey indicated that 71% of this demographic would consider looking for a new job if their employer required them to return to the office full-time. (Hoff). In the Forbes article, author Peter Georgescu argues that an educational revolution is waiting to happen, something that will democratize online remote learning which typically comes with a much lower price tag. It is apparent that remote schooling and work will stick around as options for those who prefer it long after the COVID-19 pandemic. In this research study proposal, our team seeks to examine the effect of optimizing remote lectures for greater engagement, interactivity, and information retention. For the purposes of this study, we will look at the Blackboard platform.
Blackboard is a learning management system (LMS) that is used at many universities across the US including Drexel University. Blackboard Collaborate, a sub product, is a relatively newer platform service that is powered by the Amazon Chime SDK for messaging, audio, video, and screen share capabilities. (Kelly). It is frequently used for online instructor-led lectures. This research study proposes a proof-of-concept field study format, which examines how novel technology functions in the real world. This form of study tends to be shorter than other formats like studies of current behavior or experience of using a prototype. It seeks to primarily understand if the novel technology functions effectively. (Krumm).
The proposed proof-of-concept study consists of a separate web browser window that appears next to the lecture content being shared and/or the instructor’s webcam video. The window consists of a constantly updating set of 10 questions based on the previous and current lecture slide. These questions are not directly addressed by the lecturer but are instead are historical selections based on what previous students have asked in the class via the text chat feature. Students can “upvote” and “downvote” questions to be addressed by the instructor live. The study conjures an image of a “pick your own adventure” tale a la Black Mirror: Bandersnatch but applied to an education setting in a fun and interactive way. At the end of the class, the system will collect and tally student responses to support learning analytics (LA) objectives and help the professor tailor content better towards students or help struggling students. The system is seen as having the potential to boost student interaction and information retention.
Our primary research questions for the study will be:
· How do students interact with the live poll system?
· Do students find the system useful?
· Are students participating more in lectures than otherwise, like having to type in the chat or unmute to speak?
· How are instructors using the system?
· Do instructors find the system useful?
Answering these research questions will require a field-study approach. For this study, our team recommends the choice of a between-subjects design to account for the individuality that is a classroom lecture. A within-subjects approach would not work as effectively here as the student would be experiencing the same lecture twice, and therefore would already have the prior background about the material from the instructor. Taking a between-subjects approach of control and test groups would advance this research study the most effectively. Control groups would consist of a traditional lecture to a group of students without the poll/response component, and the test group would be using the poll app along with the instructor. All our research questions besides Do students find the system useful? and Do instructors find the system useful? can be answered using quantitative analytics tracking (logging) built into the platform. For example, in response to How do students interact with the live poll system?, the browser would track whether it is an active window on the computer (not closed or minimized) and report this data back to the application. Additionally, the system would track mouse clicks and question selections. The study would make extensive use of all technical logging capabilities for student engagement. Additionally, the data could take into account quiz grades based on information from the lecture and statistically analyze this.
However, not every component of this research study can be studied by logging and quantitative technical measurements. The research team has decided to incorporate the survey method to collect presurvey and postsurvey opinions of students and instructors to answer the qualitative research questions. The presurvey will feature many Likert questions on a scale of “Strongly Disagree” to “Strongly Agree” to assess the student and instructor respectively of how they felt they were learning from the instructor and how much they felt the students were learning from them. The presurvey would be brief in length, approximately 5 questions. To keep the initial research study brief, the team does not deem it necessary to use using other methods such as experience sampling methodology (ESM) or diaries or interviews.
This study will be conducted over the course of a 16-week semester of class (or 10-week term at quarter schools). The longer timeframe is to avoid bias from novelty of the system. In fact, the research term will prioritize conducting the study at semester-based schools to have a longer timeline. The effort required from participants will only consist of opening an additional browser window during each lecture, so the team does not anticipate this being a large hurdle. Opening the browser window will factor significantly into the attendance grade for the class.
As discussed previously, the questions appearing in the live poll will consist of questions that have been asked in the Collaborate chat from previous sections of the class and from questions asked over microphone by students in these past sessions. The researchers will parse through the logs and recordings of previous lectures to assemble these questions. Additionally, professors will also populate questions based on what they have understood to anticipate from some students from historically teaching the class. Unfortunately, brand new instructors and courses will be excluded from this research study. After the quantitative data has been collected, the research team will examine it using different statistical analysis methods such as ANOVAs, paired t tests, and other significance tests.
With the results of the data analysis, the team will begin to assemble themes from the research study. For example, the team will identify outliers and state the reasonings for these data points.
Finally, the team should be able to answer the questions outlined in the original research study proposal to see if it was effective and if there were any interesting results. A possible outcome could be that students largely ignored the live poll during each class and instead preferred to get the lecture material delivered in a non-interactive format. Another possible outcome from this could be that some students found it helpful, but many did not feel as if the questions were relevant enough. A third outcome could be that the system was incredibly beneficial in improving student engagement and information retention. Students could reflect positively on the opportunity to engage their instructors without as much effort as typing in the chat, raising a hand, or unmuting themselves on the stream to talk.
Knowing the answers to these questions and the quantitative data collected could prove incredibly useful to Blackboard and the education community at large. For Blackboard in particular, this software could be integrated directly into their Collaborate product and content could be pulled from the LMS. Blackboard could present the results of this study to education customers and offer a higher tier subscription plan that allowed instructors to customize the live polls promising higher engagement and knowledge retention.
To benefit the broader education community, the results from this survey could be used to develop other open-source or low-cost EdTech (education technology) tools. For example, Quizlet could offer the product at a free tier to educators.
In conclusion, this research study has the potential to derive significant insights about the potential for remote education. Individual’s habits and preferences have changed to pursue remote learning and work more, and so institutions and other platforms must similarly adapt to serve this demographic. By understanding the possibilities of enhancing a traditional remote lecture, researchers can advise and shape the world for more equitable remote education for all.
Works Cited
Krumm, John. Ubiquitous Computing Fundamentals. CRC Press, 2010.
Kelly, Rhea. “Blackboard Partners with AWS to Enhance Virtual Classroom.” Campus Technology, 19 July 2021, https://campustechnology.com/articles/2021/07/19/blackboard-partners-with-aws-to-enhance-virtual-classroom.aspx.
Georgescu, Peter. “An Educational Revolution Waiting to Happen.” Forbes, Forbes Magazine, 29 Apr. 2022, https://www.forbes.com/sites/petergeorgescu/2022/04/28/an-educational-revolution-waiting-to-happen/?sh=58df6d1d695b.
Hoff, Madison. “Return-to-Office Is Driving Gen Z to Quit.” Business Insider, Business Insider, 25 Apr. 2022, https://www.businessinsider.com/return-to-office-great-resignation-gen-z-thinking-about-quitting-2022-4.
