Computer Vision in Education — What, How, and Why?

“Just like to hear is not the same as to listen, to take pictures is not the same as to see and by seeing, we really mean understanding” says Standford AI Lab’s Dr. Fei Fei Li in her TED talk. She talks about the exponential advances in the field of computer vision — computers being able to understand and even briefly describe the objects in the images they see, after having been “fed” extremely large data sets of pictures (millions of representations) which were processed through the means of neural networks. Such a technological achievement has led to massive progress opportunities in diverse industries and fields, ranging from transportation (e.g. autonomous cars) to healthcare (e.g. medical imaging) and security (e.g. surveillance cameras).

While further exploring the untapped potential of computer vision, I cannot help but ask myself: how could we use it in education? How could computer vision enhance the learning process by offering educational content creators relevant feedback which leads to the production of more engaging, effective classes (online/offline)?

In order to come up with a response, I have started doing research on how computer vision is currently being used in education. One main goal that I have identified among initiatives in this field has been the improvement of the user (learner)’s experience through the use of computer vision. For instance, a team of researchers from The Laboratory of Technologies for Interaction (LaTIn) of the University of Sao Paulo (USP) has developed a low-cost eye-gaze tracker, “a device that measures the position and orientation of the eye”. When the EGT is integrated in an Intelligent Tutoring System that involves students in different learning processes which aim to improve knowledge in specific areas, it leads to an enhanced learning process by offering the teacher/mentor relevant data feedback on the learner’s reactions. The system does this by identifying the user’s level of attention and involvement through the recognition of facial features. This technological breakthrough has the potential to dramatically enhance learning experiences, making them more adaptive and thus effective.

Such computer vision applications may play a significant role in improving the effectiveness of online courses (MOOCs). In his Foundation’s 2015 Annual Letter, Bill Gates discusses the major role online courses will play in the future of education. Software will democratize and dramatically decrease the price of education. Moreover, as Gates states, “software will be able to see when she [the learner]’s having trouble with the material and adjust for her pace. She will collaborate with teachers and other students in a much richer way”. As online courses are becoming more and more popular, the potential applications and impact of computer vision in online education increase substantially.

However, computer vision also has applications in traditional class settings. For instance, one of the awarded projects in the Harvard Initiative for Learning and Teaching has used computer vision analysis to explore students’ behavior and interaction during diverse group tasks, trying to better understand how students teach each other, in order to optimize peer instruction. Software has offered valuable insights into students’ posture, orientation of face, or gesticulation during the team activities, by recording all students’ interactions. This is only one example which proves that computer vision can play a beneficial role in a traditional class environment.

While the convergence of computer vision and education may still be in its incipient phase, it’s worth imagining and exploring ways in which such technology can mutually benefit the parties involved in the education process by making learning more effective, fulfilling, and, most importantly, fun.

Works Cited

Li, Fei Fei. “How We Teach Computers to Understand Pictures | Fei Fei Li.” YouTube. TED Talks, 23 Mar. 2015. Web. 21 Mar. 2017. <https://www.youtube.com/watch?v=40riCqvRoMs>.

Coutinho, Fl´avio L., Thiago T. Santos, and Carlos H. Morimoto. Computer Vision Tools for ELearning. N.p., n.d. Web. <http://www.vision.jhu.edu/iccv2007-cvdc/CVDR-coutinho.pdf>.

Gates, Bill, and Melinda Gates. “2015 Gates Annual Letter Our Big Betfor the Future Bill and Melinda Gates.” 2015 Gates Annual Letter. N.p., n.d. Web. 21 Mar. 2017. <https://www.gatesnotes.com/2015-annual-letter?page=4&lang=en&WT.mc_id=01_21_2015_AL2015-GF_GFO_domain_Top_21>.

Mazur, Eric, Todd Zickler, and Rachel Scherr. “Transforming Education through Computer Vision Analysis and Automated Assessment.” Harvard Initiative for Learning and Teaching. N.p., n.d. Web. 21 Mar. 2017. <http://hilt.harvard.edu/pages/transforming-education-through-computer-vision-analysis-and-automated-assessment>.