Reboot Report Compares International Use of Devices in Class with Educational Outcomes

Many K-12 edtech initiatives have a very specific stated purpose: put a device in every learners’ hands. Whether its personal computers or mobile tablets, researchers remain divided on whether or not they improve learning or make it worse. A recent report out by the Reboot Foundation, a think tank headed by Helen Lee Bouygues, analyzes large data sets provided by the 2015 Program for International Student Assessment (PISA) and the U.S.’s 2017 National Assessment of Educational Progress (NAEP), which is commonly known as the nation’s report card. The report was compiled and authored by Bouygues and her team. It primarily compares the amount of time students report using devices in class with their academic achievement, while controlling for a variety of factors.

The Relationship Between Devices in Class and Standardized Test Scores

The researchers found highly nuanced results. They found that, internationally, there was a “weak link” between technology use and outcomes. In the U.S., meanwhile, results were highly mixed.

In her introduction and discussion of previous research, Bouygues wades through the various perspectives on the roles and efficacy of devices in class. A huge body of evidence suggests both positive and negative outcomes.

“The debate over educational technology isn’t black and white, either,” Bouygues writes. “Context makes a tremendous difference, and students can use technologies such as a tablet or the Internet in so many different ways that it can be hard to say that technology will — or will not — improve learning. Are students using devices to perform research? Take notes? Play games? Engage in a virtual reality-based simulation? In this sense, learning technologies are tools; they can be used effectively or ineffectively.”

The 2015 PISA data tested 15-year-old students from 90 countries in math, reading, and science. Insufficient data from some countries, including the U.S., made proper analysis of those regions impossible. The NAEP, meanwhile, tests 4th and 8th grade U.S. students in reading and math.

For evidence of computer use, Bouygues and her team looked at the use of personal computers and tablets used for math, reading, and language arts learning. They then compared that data to standardized test results, controlling for a wide variety of factors, including GDP, teacher experience, student demographics, and more.

The PISA Results

One narrative appears to stand out from analysis of the PISA scores: a little technology use goes a long way. Learners who reported a small or medium amount of device use tended to score higher than those who reported no use at all. But those with heavy technology use scored worse than learners who had used no devices in class. In Australia, for example, learners who reported 2-4 hours of device use scored a 524 on average on the PISA. Australians who had no device use scored a 468, while those who logged over 6 hours scored a 453.

In the U.K., meanwhile, the ‘sweet spot’ was 1-30 minutes of technology use (average score of 525), but fell off a cliff after 2-4 hours.

NAEP Results

In the U.S., results did not fit so straightforward a narrative. For 8th graders, there appeared to be no relationship at all between device use and academic performance in math.

This was not the case with 4th graders. “Fourth-grade students in Arizona who reported using tablets in class “all or almost all” of the time scored 26 points lower than their peers who reported “never” using tablets,” Bouygues writes. “Students in New Jersey who reported “never” using tablets in class scored 29 points higher than their peers who reported using tablets “all or almost in all” classes. This score difference is the equivalent of nearly three grade levels.”

In general, more device use tended to lead to lower academic achievement.

Bouygues concludes, writing, “Our results underscore the complexity of the relationship between classroom technology and student outcomes. A variety of factors influence the degree to which computers can have a positive, negative, or negligible relationship with student performance. Access to particular devices, the frequency of exposure to these devices at school, and the length of exposure during the school day all affect the direction and strength of the relationship between technology and achievement.”

Read the full report here.

Featured Image: Niamet Ullah, Unsplash.