Columbia Teachers College Study Casts Doubt On Personalized Learning

From tech leaders to educators to government officials, many people now believe that the future of education will be personalized. But a new study by a team of researchers at Columbia University’s Teachers College cast doubt on claims about the potential of personalized learning. The study questions whether personalized learning holds the potential to serve students in the long term.

Scope of the Study

The study, Final Impact Results from the i3 Implementation of Teach to One, was carried out by Douglas D. Ready, Catherine Conn, Shani S. Bretas, and Iris Daruwala with a grant from the U.S. Department of Education. The evaluation had two key strands: a quantitative assessment of the personalized learning initiative Teach to One’s impact on student mathematics performance and a broader qualitative analysis of the personalized learning system. The research team’s most recent report describes results from the quantitative half of the study, which includes data from 36,158 student-level measurements across 209 school-by-year cohorts located within five Teach to One schools in Elizabeth, New Jersey.

Key Findings on Personalized Learning

This study sought to describe the impact of Teach to One: Math (TtO) on student mathematics outcomes over a three-year implementation period in a group of schools in Elizabeth, NJ. While their findings were not entirely negative, the researchers did find troubling inconsistencies. As they explain in the study’s conclusion:

“We found that performance patterns across the five TtO treatment schools were quite inconsistent, and the TtO estimates for each of the three implementation years were statistically non-significant. We found similar results using a measure indicating the combined (averaged) three-year impact and models that excluded a number of relatively higher-achieving control schools.”

The researchers raise specific concerns about whether or not personalized learning is aligned with the expectations found on state tests and whether or not personalized learning does, in fact, serve learners in the long term. “One result of this differentiation,” they write, “Is that TtO and non-TtO students with the same initial test scores likely receive quite different mathematics content during the academic year. Because TtO students may receive content that is below grade-level, their performance on state-mandated, grade-level standardized assessments may lag behind their non-TtO peers.”

The Study’s Noted Limitations

The study’s authors admit that their findings do have several limitations. First, the results come from a single school district staffed by a particular group of teachers. Moreover, while the study was adequate from an analytical perspective, the researchers note, “Our ability to identify statistically significant treatment effects that were modest in size—again, either positive or negative—was somewhat limited by the school sample size.” Finally, they note that the findings of their current study should be taken with a grain of salt given that the TtO model is constantly evolving.

Despite the study’s limitations, it does raise legitimate concerns about whether or not some claims on personalized learning have been exaggerated.

Photo by Joshua Eckstein on Unsplash.