Researchers Found Positive Effects from a Personalized Virtual Reading Coach. But Teachers Don’t Want to Use It.

As more and more adaptive personalized learning solutions enter the market, educators and schools are increasingly looking to use them to augment their instruction. The subject of reading is seen by many as an area that could benefit substantially from this technology. In a recently published study, researchers from the University of Cincinnati and North Carolina State University, Raleigh looked into the MindPlay Virtual Reading Coach (referred to in the study as MVRC). They investigated its use in three separate Midwestern districts where students, like most American learners, were often one or more grade levels behind in literacy and reading skills. While the researchers found MindPlay significantly outperformed traditional instruction and other reading tools to improve literacy, none of the teachers using it wanted to continue to do so once the trial was over. The findings indicate that there’s more to edtech than academic outcomes.

Professor Heidi Kloos at the University of Cincinnati has a background in psychology. “I study how the mind works and how children learn on a more basic level,” Kloos said over the phone.

Her initial foray into investigating education technology was in teaching math.

“I had assumed, wrongly, that surely we had solved the reading problem,” Kloos said. “We all know we need practice, we have to be exposed to reading. That’s half the battle. But when you dig a little deeper, you realize that proficiency rates on a national level are just as bad [as those in math], especially with low-income schools.”

The Average American 4th Grader Reads Below Grade Level

According to the latest National Assessment of Educational Progress (NAEP), commonly known as the Nation’s Report Card, 37% of American 4th graders read at grade level in 2017. That’s better than 1992, when it was 29%, but it’s obviously not ideal. The amount of below-proficient learners, furthermore, is far more pronounced among learners of color. Just one in five (20%) black 4th graders read at grade level, while the figure is 23% among Hispanic students. 

“It’s interesting to me that kids struggle with reading,” Kloos said. “They’re not reading at grade level when it’s not that hard (comparatively) for the mind to learn. Where does that fall apart?”

Recognizing this prompted Kloos to investigate the issue further. Colleagues who work in K-12 districts had told her they’d had success with MindPlay. So Kloos decided to study it.

Experts currently understand reading as an amalgamation of numerous different skills. The MindPlay courseware is intended to develop three elementary areas: phonetics, vocabulary, and fluency. The last one is defined as the pace at which readers read. It’s a crucial component of literacy, because to string together meaning, a reader needs to recognize and understand a sentence or string of words in quick succession. 

To develop this skill, the courseware displays sentences on the screen. At a regular speed, words begin to disappear, thereby pushing readers to get through the sentence quicker, while still understanding the meaning.

This function is set within a wider AI-powered adaptive personalized learning supplement. It begins by assessing students’ abilities, and then targets lessons according to where they need the most work.

Investigating the MindPlay Virtual Reading Coach

The researchers broke their study down into three parts. The first used existing data from a district containing roughly 20,000 learners. Students used MindPlay for one school year, and the researchers assessed their abilities before and after. They were also able to determine how much time each student had spent using the reading coach. They broke each grade cohort down into quartiles based on the amount of time each learner spent using MindPlay over the year.

On average, the 3rd grade cohort of students was one grade level behind in reading skills. Every other cohort was, on average, two grade levels behind.

In most grades, learners who spent more time using the virtual reading coach recorded better post-trial academic outcomes. But there were a few interesting exceptions. 4th graders who fell into the second lowest quartile actually scored the highest in their grade, while all other quartiles were roughly equivalent. In 5th and 6th grade, meanwhile. the second highest quartile recorded the best achievement (although the top quartiles were close behind in each).

In two other districts, the researchers conducted randomized control trials. In these, students were divided into three groups: a control who learned via traditional instruction, a test group who used MindPlay, and a third that used a different reading supplement chosen by the district.

This trial lasted nine weeks and, again, the researchers assessed learners before and after. Here, it became evident just how effective MindPlay’s Virtual Reading Coach could be. Certain cohorts improved their fluency skills during the nine week period by over three-quarters of a grade level. The 4th graders in the third district studied showed gains in fluency of 88% of one grade level, while their peers in the control and other groups improved by 23% and 43%, respectively.

Objectively, the MindPlay Virtual Reading Coach Works Really Well. But Teachers Didn’t Want to Continue Using It.

This would appear to constitute a highly positive report on the efficacy of MindPlay’s Virtual Reading Coach. Kloos and her team were interested in exploring the technology further. But then something unexpected happened.

Just a small percentage of teachers participating in this study agreed to continue using MindPlay. The vast majority didn’t want to use it anymore.

“We talked to the teachers before we had our data, and their responses made us think that maybe this particular technology didn’t work,” Kloos said. “And then we find that it actually did work, and yet teachers didn’t like it. That was surprising, because either teachers didn’t track that it worked or it didn’t matter that it worked. But then even after we told all the teachers that it worked and asked if they could try it again so we could get to the bottom of why they didn’t like it, they declined.”

The teachers gave various reasons for not wanting to continue using MindPlay. Some said their students were frustrated by it. Others said it didn’t work well with their teaching style.

There’s More to Edtech than Academic Outcomes

“After those results, I explained MindPlay to parents and teachers as follows: ‘We have this program that works surprisingly well to improve a child’s reading fluency. But at the same time, teachers had some concerns about the program. It’s like going in to a tunnel. You need to keep walking and, once you get out, you will read faster. But it’s not necessarily a pleasant experience.’”

The conclusions drawn from this study are relevant for research in adaptive personalized reading courseware. But they also reverberate more broadly. Many education researchers consider efficacy research a main bellwether in edtech.

“Some people assume that if I can show you that something works, all you need to do is push the button and turn it on,” Kloos said. “I have to say, that idea is naive. Using education technology is not like swallowing a pill. You need that social-cultural component for technology to work.

“Every problem in education has a million sources. It’s not just one reason. Some people might read this research and think, ‘Oh, it’s just those teachers who don’t like using the technology.’ That’s not it. These teachers were highly competent. We were impressed with how they ran their classrooms. And yet, there’s challenges.”

While the results of this study provide more questions than answers, Kloos remains convinced that education technology will be a primary driver in improving the American education system.

“If something doesn’t work, it’s not because we don’t need technology. We need technology, there’s no doubt in my mind.”

Read the full study here.

Featured Image: Johnny McClung, Unsplash.