Building Evidence to Guide EdTech Adoption in Schools
This is the first blog in a series highlighting the Office of Educational Technology’s (OET) EdTech Evidence Toolkit resources. To learn more about how OET is supporting evidence-building practices for impactful EdTech adoption in schools, visit https://tech.ed.gov/evidence.
OET has developed this EdTech Evidence Toolkit as a resource to support educational leaders tasked with using evidence to inform the selection of educational technologies (edtech) in schools. In the first of a series of EdTech evidence blogs, we offer “how-to” guidance and suggestions for ways educational leaders can use OET’s EdTech Evidence one-pagers to build their school district’s evidence base and support professional development.
This blog discusses the use of OET’s EdTech Evidence Toolkit as introductory resources for navigating the edtech evidence landscape, as well as considerations for supporting evidence use in schools.
What is the EdTech Evidence Toolkit?
The Elementary and Secondary Education Act (ESEA) encourages state and local educational agencies to prioritize evidence-based decisions on the use of edtech in schools. Educational leaders charged with making edtech decisions on behalf of school districts have expressed the need for OET’s support in using evidence to inform edtech adoption decisions in schools.
The EdTech Evidence Toolkit was developed in consultation with federal, state, and local partners with expertise in use of evidence and edtech to ensure that they met the expressed needs of state and local education leaders seeking guidance on the use of evidence to support edtech adoption in schools.
The purpose of the EdTech Evidence Toolkit is threefold: 1) to be an introductory resource for understanding the four tiers of evidence, as outlined by ESEA, in relation to edtech adoption, 2) to present a practical school district case study across the four tiers of evidence for using evidence-building activities incrementally to inform edtech adoption, and 3) as a resource to inform professional development efforts supporting use of evidence in schools.
How can schools use the EdTech Evidence Toolkit?
The EdTech Evidence Toolkit introduces evidence-building activities for four tiers of evidence using a one-pager format. Each one-pager provides schools with background knowledge for each specific tier and offers a case study example that schools can use as a thought partner resource.
There are four evidence one-pagers in the toolkit, each one focusing on a different tier of evidence, as defined under ESEA: (1) Strong Evidence, (2) Moderate Evidence, (3) Promising Evidence, and (4) Demonstrating a Rationale.
The first half of each evidence one-pager introduces readers to key background information and relevant terms for that specific tier (e.g., the first one-pager is Tier 4: Demonstrating a Rationale).
The second half of each evidence one-pager uses a case study, starting at Tier 4 and continuing through Tier 1, to introduce the reader to evidence-building activities that incrementally build an evidence-base for edtech adoption.
The case study assumes little to no evidence-base for a proposed edtech intervention — necessitating building evidence “from scratch” as follows:
- Identify the educational need. A school district has determined that they would like to support students’ reading engagement. Through anecdotal evidence they have identified an edtech intervention that teachers said helped improve their students’ reading engagement.
- Determine what evidence exists. The school district partnered with an evaluation organization to conduct a landscape analysis — a review of previously established evidence on a topic. Upon completing the landscape analysis, the school district found no existing evidence-base for the use of the proposed edtech intervention.
- Develop an evidence-building strategy. When the evidence-base is minimal, the school district identified steps for building an initial evidence-base (e.g., Tier 4). When an evidence-base existed, the school district developed next steps for continued evidence-building (e.g., Tiers 3, 2, and 1).
The chart below illustrates this process, including sample evidence-building activities for each tier:
The chart presents evidence-building in a consecutive manner (i.e., Tier 4 to Tier 3 to Tier 2 to Tier 1) to illustrate the cumulative, incremental, nature of building evidence. However, evidence-building does not necessarily need to move consecutively across each individual tier. For example, evidence-building activities in Tier 4 or Tier 3 could be used to develop a plan for evidence-building in Tier 2 or Tier 1.
School districts will need to base their decisions about what level of evidence is “sufficient” on a variety of factors. These factors include access to partnerships and resources, as well as state and federal funding requirements. For many school districts, building evidence towards Tiers 4 and 3 suffice, whereas some school districts may seek to build toward additional tiers of evidence depending on their funding and resource circumstances.
Finally, it is important to note that evidence-building activities are not always successful in producing positive evidence supporting the use of an edtech intervention. Negative or inconclusive results may require revision and re-evaluation of the intervention. As a result, some edtech interventions may not be designed or implemented in ways that can be supported by evidence. It can also be the case that evidence-building activities designed to meet Tier 1, but having some flaws, could qualify for Tier 2 — or that evidence-building activities designed for Tier 1 or Tier 2, but not meeting those requirements, could qualify for Tier 3 or Tier 4.
How can Educational Leaders Support Evidence-Building in Schools?
Across government, education, and industry, leaders tasked with making decisions about edtech use in schools recognize that improving student outcomes necessitates engaging in evidence-building activities. We conclude this blog with considerations for what educational leaders can do to support evidence-building in schools.
Government and Research Organizations.
- Develop evidence-building resources for state and local education agencies.
- Support open-access mandates for government funded research.
- Provide guidance to state and local education agencies on access and use of evidence.
- Expand existing repositories to include education science and modernize interfaces to be user-friendly and accessible (e.g., National Institutes of Health (NIH) and Education Resources Information Center (ERIC)).
Developers and Industry.
- Create tools to document evidence building in schools and to support school efforts to incorporate evidence into edtech adoption decisions.
- Expand product development to consider the evidence needs of schools.
- Create just-in-time processes that facilitate schools gathering, building, and interpreting evidence to inform day-to-day instruction and use of resources and tools.
- Support the development of Education R & D infrastructure across sectors that amplify evidence-based practices, resources, and tools.
State Education Agencies and School Districts.
- Partner with education research organizations to support evidence building in schools.
- Collaborate to identify and use teacher- and student-centered evidence-building practices.
- Facilitate and streamline the process for schools to access, use, and build evidence.
- Provide professional development and technical assistance to schools in accessing and using evidence to inform educational decisions.
Educators and community leaders.
- Explore available evidence on educational topics of interest using search tools such as ERIC and What Works Clearinghouse.
- Ask for access to evidence (many education researchers will share their work upon request).
- Collaborate with educators to formalize how you already use evidence to inform teaching.
- Request professional development and technical assistance from your school district or state education agency to support use of evidence in the classroom.
To conclude, incorporating evidence-building into education evaluation empowers schools to iterate and improve their educational practices, including decisions regarding edtech adoption. When schools have access to resources for building and using evidence, they can better position themselves to do what schools already do best: leverage existing resources for maximum impact.
