There is no doubt technology has revolutionized most aspects of our lives, including what’s happening in our classrooms. But what are the indicators of quality when it comes to the technology we put in front of students? Educators face a daunting task of keeping up with rapidly evolving edtech products, identifying the best available applications and effectively implementing them in their classrooms. A common barrier for educators trying to incorporate technology is a lack of knowledge about the products and their alignment with course objectives, not to mention a heavy reliance on self-taught product use and integration.
The ISTE Seal of Alignment product certification program has developed a reputation for identifying excellent edtech products that align with the ISTE Standards. Recently, EdSurge talked with ISTE researchers who managed Teacher Ready, a research-driven project that (1) provides edtech decision-makers with tools to validly and reliably evaluate edtech, and (2) helped expand the ISTE Seal of Alignment certification by integrating user experience and product usability components.
What is the work the Teacher Ready team set out to do?
Brandon Olszewski, the director of research at ISTE, underscores the need to determine the best way to evaluate edtech products. “It’s really about edtech product quality, particularly usability. Did the tool help you accomplish your task effectively and efficiently? Is it easy to use? Will this technology help my students achieve their learning goals? Those are indicators of success.”
Nicole Langford, a research associate at ISTE, describes the purpose of the Teacher Ready project as developing a “framework for teachers to use as they score and evaluate products that they’re bringing into their systems, to help them choose better products from an oversaturated market.”
How did the team embark on this project? Caitlin McLemore, a senior research associate at ISTE, explains that early research activities encompassed a literature review, input from an expert panel, teacher focus groups and “think-aloud” interviews. After developing the framework, McLemore notes that “user testing allowed us to hone in on what specific dimensions and indicators were most important and ensure reliability and validity within the framework.”
Why is evaluating edtech product quality so important?
Billions— somewhere between $26 and $50 billion—are spent annually on K-12 edtech. Yet decision-making for K-12 districts purchasing edtech has historically lacked research-based practices, with many district leaders identifying cost and convenience as top reasons for purchases. Further compounding the situation is the fact that most edtech products lack efficacy data to support claims of value and effectiveness. The Teacher Ready project team prioritized the need for evidence-based guidelines for decision-makers to make more informed purchases.
“This framework is a way to standardize the product evaluation process,” McLemore says. Whether making decisions for a large district or one classroom, this framework provides educators with foundational criteria from which to assess edtech products. The standardization allows for adaptability to a variety of contexts and needs.
Given the billions of dollars spent on edtech, Olszewski sees this evaluative framework as a necessary step toward establishing quality-assurance standards. “With the proliferation of edtech today, we are talking about a standardized process that has never been more imperative. Teachers and students use edtech products for dozens of hours each week. Wouldn’t you want to know that these products align with everything we know about how humans learn?”
How can this edtech usability framework serve as a change agent?
The Teacher Ready team recognizes two primary challenges with this project. First, they must ensure that the criteria for product evaluation are both human-centered and evidence-based. This means employing an iterative process of design and revision based on research and user feedback. “We intentionally keep the educators’ voices at the center of our process,” says Olszewski. “We created and repeatedly revised the framework with the guidance of teacher feedback and user experience experts.”
The second challenge the team faces is the distribution of the framework in a method that is most effective and impactful. McLemore notes that while the Teacher Ready framework is valuable for teachers and district leaders who want to be more critical consumers, it is also informative and actionable for product developers. Langford sees this mission as educating vendors on market signals while arming district decision-makers with specific indicators of quality. “Having research-based criteria helps school leaders speak a shared language with concrete indicators regarding product quality with edtech vendors about what is truly needed to achieve learning outcomes.”
“I could see this framework being applied by an individual teacher without evaluation training and then also incorporated into a district’s broader procurement plan,” says McLemore. Langford agrees that the focus of the team was to provide guidance for all scenarios: from technology experts through untrained evaluators. Olszewski adds, “Educators and even parents should be able to use this framework and make some kind of reliable assessment of the product.” But at the same time, he sees this framework as changing how practitioners evaluate edtech quality, which could ultimately change product development. “If you want to revolutionize learning with technology, you have to use quality technology to do it.”
Ready to learn more about the indicators of quality edtech products? In the next EdSurge interview, the Teacher Ready team will dive into the five dimensions that address product usability and the ISTE Standards for Students—and how these dimensions inform an expanded ISTE Seal of Alignment.
