Quizzing is a fundamental aspect of any course, unit, or learning module. Numerous studies confirm that quizzes help in the learning process and improve long-term retention, especially if the assessment includes feedback. But not every online quiz aid is created equally. One main difference is testing for recognition vs. recall, or in other words, multiple choice vs. fill-in-the-blank, respectively. While a good deal of studies show that recall requires more work and leads to more effective learning outcomes, these studies typically produce results in a lab setting, and not in a classroom. For this reason, researchers Piers Howe (University of Melbourne, Jason Lodge (University of Queensland), and Meredith McKague (University of Melbourne) recently carried out a study on online quiz aids to see which would actually improve student performance. The paper was published this month in Frontiers in Education and draws some surprising conclusions.

Cram vs. PeerWise

For subjects, the team tested a second year biological psychology course at the University of Melbourne. Out of nearly 700 learners, 262 participated in the survey. Participants could use two different study tools throughout the semester. The first was Cram, a flash card-based aid that represented recall quizzing. The second used PeerWise System, a multiple choice, or recognition-based tool. Learners also had the choice of using neither. For ethical reasons, the researchers could not use a randomized assignment method and were forced to make each tool available to any student who wished.

Based on previous research, the authors hypothesized that students who used an online quiz aid more would do better on the course. Furthermore, those who chose Cram over PeerWise would do better still.

Still, they emphasize that, due to several complicating factors, “It is unclear to what extent these laboratory-based findings” which they used to form their hypotheses “apply to actual higher education courses.”

Which Online Quiz Aid Is Better?

In summary, their first hypothesis held water, while their second did not stand up.

“Both participation with Cram and participation with PeerWise uniquely predicted final exam scores,” the authors write. “The degree of participation with PeerWise was still significantly correlated with final exam score, even when both student ability and the degree of participation with Cram was controlled for.”

The same was true regarding participation with Cram when Peerwise was controlled for.

Neither online quiz aid, furthermore, helped students particularly more than the other. “Running a bootstrap analysis with replacement revealed no significant difference between these two correlations,” the authors write.

In general, they found that the more hours spent using either of the study tools, the better students did.

“We ran a linear regression to estimate the degree to which participation with Cram and PeerWise affected final exam scores, accounting for performance on the first two assignments. To increase their final exam score by 5% (i.e., by one grade), on average students would have needed to have interacted with PeerWise on 10.1 days and with Cram on 24.3 days.”

More interesting, perhaps, is the subjective data that came from student reviews. 63% of learners thought that Cram prepared them more effectively than PeerWise, and learners generally spent more time with the former online quiz aid.

Yet this made no difference on their performance. More interesting still, Cram tended to make students more confident in their abilities. And “the more confident Cram made them feel, the worse they did on the final exam. A similar trend was observed for PeerWise, but this did not reach significance.”

Much more evidence is needed in a classroom setting. But for instructors looking to design an effective online study tool or to choose one for classroom use, these Australian researchers conclude, “we believe that university instructors should not favor recall-based learning aids.”

Cover Image: Juliette Leufke, Unsplash.