Judge education products & services by their impact on student outcomes? Why not?

Can you imagine a world in which education decision makers had access only to education products and services that were known to have a positive, measurable impact on learners?

Education economist Tom Kane has recently advocated for two separate, but complementary, systemic structures regarding the effectiveness of education products and services. The first structure is a formal research function within the education sector that would review and vet the efficacy of education research, and ensure education decision makers ready access to such information — like an FDA for education. The second structure calls for the education sector to begin systematically measuring the efficacy of textbooks, curricula and other educational materials by their true purpose: to generate learning.

Education research in the US has never been systematically validated by an external agency, and the availability of education products or services on the market remains without oversight.

In medicine, by contrast, physicians and patients regularly rely on the rigorously vetted research overseen by the Food & Drug Administration to inform their approach to medical treatment. The FDA not only vets pharmaceutical research, but it also uses that research to inform its decisions about which medicines, for example, will be brought to market. Moreover, and at the risk of oversimplification, pharmaceutical companies would never attempt to bring a new medicine to market without rigorous evidence as to whether that medicine did, indeed, deliver its intended effects.

Although a slightly different kind of oversight agency, the US Consumer Product Safety Commission works on behalf of consumers to ensure product safety. In both cases, such oversight results in enhanced transparency to both products and industry about “what works” in a given product area. As Kane notes, although the What Works Clearinghouse exists in the education sector, its purpose is to store, not vet, education research.

While neither analogy is perfect to the education sector, the reality is that such infrastructure simply doesn’t exist in education and, therefore, rigorous evaluation of education products and services is the exception rather than the norm. The consequences of this reality include stymied innovation and a lack of progress on the sector’s most intractable challenges, including the achievement gap. At the classroom level, educators continue to make daily decisions about education products and services with little evidence about whether learners will actually benefit.

In an era of widespread high-stakes accountability, it is fair to ask ‘why are those who provide education products and services exempt?’

At Pearson, where I serve as Chief Education Advisor, we’ve embarked on our own journey to measure the impact of our products on learners; we call it our efficacy work. Our goal is to provide our customers with ready access to robust research that demonstrates a products’ measureable impact on learners. We’re still in the early stages of this work, but already we have:

● developed an efficacy framework which guides our work;

● collaborated with our customers to define learner outcomes at the product level;

● reviewed over 250 products against their intended outcomes; and

● leveraged that learning back into product development to ensure continuous improvement and ever-improving learner outcomes.

Such work is captured in our preliminary efficacy reports, which by 2018 will be externally audited. In the meantime we’re documenting and sharing our progress publicly and regularly, and seeking feedback. (See: On the Road…to Delivering Learner Outcomes.)

If Kane’s suggestions become reality, decision making in education could shift from evidence-based to evidence-informed. Evidence would inform the products and services available to educators, ensuring that educators had efficacious tools available, giving learners a greater chance of increased learning. Product development and innovation would be pushed to new heights as true evidence of what increases learning becomes the new standard by which all education products and services are measured, forever altering the marketplace.

Let’s ask ourselves, what will it take to deliver efficacy in education at the systems level? Let’s not imagine a world in which education decision makers had access only to education products and services that were known to have a positive, measurable impact on learners — let’s make it a reality.