Regardless of where you stand on the debate over open educational resources, you’re probably wondering: Does OER actually improve learning outcomes?
This question came up in a handful of discussions this week at the OpenEd conference in Niagara Falls, NY. And the short answer is, most experts still aren’t sure. But it’s probably not making things more difficult for students. At least, that was one of the main takeaways from a short session led by Phillip Grimaldi, director of research at OpenStax, a nonprofit OER initiative out of Rice University.
Grimaldi started things off by asking the audience—many of whom are advocates for open educational resources and use them in their practice—to interrogate their assumptions about why open resources could improve learning outcomes.
The so-called access hypothesis goes that students who have access to the textbook learn better than students who don’t. And in cases in which traditional textbooks are prohibitively expensive, OER can provide students with an opportunity to gain access to course materials they may not have otherwise. If all students in a course have access, then “students in a course with an OER book will learn better than students in a course with a commercial textbook,” Grimaldi explained.
What makes it hard to measure those outcomes, Grimaldi stressed, is that adopting OER in a course largely affects students who would not have had access to the materials in the first place. “The problem is, the research today treats OER like any other intervention and we expect it to impact all students,” he said. “But it won’t affect a student who could have easily put out $300 for a text before.”
That may sound familiar to those who follow the open education movement. But Grimaldi’s team at OpenStax wanted to better understand those research gaps. So they identified nine existing OER studies and examined the 42 comparisons in those papers between students who used OER in a course versus those who did not.
The majority of those studies, it turns out, has found little to no evidence on whether OER improves learning or not.
The team dug further, and decided to create their own study in which they could try to eliminate external variables that might affect a study in a real classroom. The researchers generated a simulation sample of fake students and assigned various subsets that did have access to a text and a subset that did not. Then they randomly assigned the entire class to OER or non-OER materials.
They ran the study again and again, tried sample sizes from 100 to 5,000, and varied the size of the group with access. But even with potential room for error minimized in the simulation, the overwhelming outcome of the simulations were also null.
So where does that leave us? According to Grimaldi, it means that “access [to course materials] has a devastating effect on statistical power. If the population has high levels of access, the likelihood of detecting an effect of OER is very low.”
He continued: “These results are for perfect experiments… real world data is probably worse. What do these studies have a chance of even observing.”
Issues of Access
Earlier in the day, students and instructors shared ideas around how OER can make learning more affordable and accessible to all students—and how the technology and open pedagogy needs to go further to support students who need it most.
Not all openly-licensed learning materials are free. But even those with lower price points can prohibit students from obtaining materials. Sam Shzu, a student at Rice University, said that digital textbooks, when they are not free, are often shared by students who have friends or connections in a course. “The people who are among the ‘in-groups’ have access to the PDF where someone else paid,” he said. “Other students who aren’t in that in-group can suffer.”
Later, one audience member shared how open resources aren’t an easy alternative for low-income students at her campus because many do not have access to devices or internet at home. “OER is not a solution that fits all the issues,” she said. “Where I work, most people are poor and don’t have internet at home. Moving away from a print book is terrifying for faculty.”
Even if OER is implemented, these ongoing challenges can prevent the kinds of learning outcomes that open education advocates hope to see. Some in the group shared ways their institution has fought to get around those barriers.
One attendee said their school negotiated with their printshop to try to lower costs and keep print texts, while others have partnered with technology lending programs. Others are leasing Wi-Fi hotspots to students who rent devices for a period of time.
“If you’re a librarian and you haven’t considered wifi lending spots,” a librarian from Arkansas said, “it’s not as scary as you think it is.”
