Computer Vision for Classroom “Engagement Detection”: Is it Ethical?

When you’re teaching in a traditional classroom, there is one thing that is usually obvious: When students are bored, lost, or distracted, they let you know. While most educators don’t love teaching to a sea of slouched bodies and yawning faces, these bodily signs offer important cues to help educators increase student engagement. But how do educators gauge engagement online? In the past, it was assumed that engagement detection was something beyond the scope of online education. Now advancements in computer vision have reached a point where it is possible to detect emotions in photographs and video footage. These advancements could transform engagement detection and online learning, but some people continue to wonder if the use of computer vision in educational settings is even ethical.

Why Instructional Designers and Educators Are Concerned About Online Engagement

Engagement continues to be raised as a concern by educators, especially in online learning contexts. In fact, research shows that many online courses suffer from low levels of student engagement. The engagement problem is especially apparent in large online courses or MOOCs where completion rates are often as low as 7%.

In addition to findings indicating that online courses often have lower levels of student engagement than on-campus courses, some researchers have found that engagement may be more important online than it is in face-to-face settings. In a 2018 study on student engagement published in the Online Learning Journal, Florence Martin and Doris U. Bolliger emphasize, “Student engagement in online learning is very important because online learners seem to have fewer opportunities to be engaged with the institution.” Unfortunately, creating the conditions for engagement in online courses tends to be challenging both due to course size (online courses are often larger than in-person courses) and delivery mode (some online courses are asynchronous, which means students rarely engage with course materials, assignments or discussion forums simultaneously).

What is “Engagement Detection”?

“Engagement detection” refers to any method used to measure engagement. In the past, most research on engagement, whether being carried out online or in a traditional classroom setting, has relied heavily on self-reporting. While self-reporting (e.g., student surveys issued at the end of a course) is useful, it rarely tells the whole story and is not always effective at measuring how students engaged with specific content or a specific activity. As new technologies enable us to record and collect data on everything from postures to eye movement and even facial expressions (as an example, see Microsoft’s Emotion Detector software), self-reporting is no longer the only or even the best way to measure engagement.

In an article published in Smart Learning Environments in January 2019, M. Ali Akber Dewan, Mahbub Murshed and Fuhua Lin explain:

The main advantage of computer vision based methods is the unobtrusiveness of the assessment process and easy to use, similar to the classroom situation where a teacher observes whether a learner is motivated without interrupting his/her activities. Affective computing techniques and [the] low cost of cameras and wide-spread of its [sic] availability in cell phones, tablets, computers, and even automobiles, are allowing [researchers] to detect learners’ engagement using computer vision.

In theory, using computer vision to detect student engagement could transform online learning. With instructors’ eyes no longer in the classroom picking up on students’ subtle and often not-so-subtle cues, there is hope that computer vision may not only replace but even enhance the ability to detect, measure, and respond to students’ levels of engagement. But there are a few concerns about how this new form of engagement detection may also erode privacy rates and even unfairly target some students.

The Ethics of Using Computer Vision to Measure Engagement in Educational Settings

While the use of computer vision to track student engagement is not yet in widespread use in Europe or North America, in China, the experiment is already well underway, and it’s not simply happening in online courses.

As reported on Reuters last year, many Chinese schools are already using a “smart classroom behavior management system.” The technology not only captures students’ expressions and movements but also analyzes these expressions and movements to make sure students are paying attention. Many Chinese educators and government officials believe the technology can improve interactions between teachers and students, but the ethics of placing students under such surveillance in physical classrooms and online is debatable. There are also concerns that the technology itself may not yet be ready to be widely implemented.

As William Michael Carter, a professor at Ryerson University in Toronto, wrote in a recent article published in The Conversation, to date, facial recognition tools are not yet accurate and consistently report higher error rates for both women and people of color. Carter speculates that this may be due to the biases possessed by coders. Whatever the cause, however, Carter fears these new tools often still suffer from programmed bias.

While computer vision may not yet be advanced enough to measure engagement in online or on-site classrooms, it seems likely that as accuracy levels increase, computer vision will transform engagement detection in educational settings. Between then and now, one thing is clear: Educators, instructional designers, students, parents, and policymakers need to start exploring both the benefits and ethical concerns of this new form of engagement detection.

Photo by oldskool photography on Unsplash.