A Math EdTech Mixed Methods Study: Trust is Key (Part 4 of 6)

Silicon Valley Education Foundation (SVEF) is a nonprofit resource and advocate for students and educators. We are dedicated to supporting all students to be on track for college and careers, focusing on the critical areas of science, technology, engineering, and math (STEM). SVEF’s Learning Innovation Hub (iHub) leverages research, practice, partnerships, and policy to raise student achievement through technology equity.

In our second and third posts, we shared the context for our research study of EdTech products used in middle school math classrooms. We discussed how we prepared and trained to conduct classroom observations with WestEd by challenging our assumptions and biases and by writing descriptive and focused observation narratives.

Image credit: MediaShift

Trust is the mainstay of a research study. We started with ourselves. To achieve our two-fold goal to build both a study and enhance SVEF team capacity, our WestEd researcher became embedded into the iHub team for this project. She has consistently emphasized the need for researchers to be close to the field, in this case, both the iHub team and the students in the classroom. At a systems level, demonstrations of trust include establishing an MOU or data contract with a district partner and earning the approval of WestEd’s Institutional Review Board. Trust must then extend all the way into the classroom, and our next step was to build relationships with the educators and students sharing their EdTech experience with us.

Recommendation One: Be a resource and start early

Teachers are often observed by other teachers, school administrators, district leaders, and/or even parents. That can mean many people moving in and out of the classroom, which increases the risk of disruption and lost instructional time. We had to acknowledge our role in that environment. When we arrived, we wanted the students and teachers to understand that the study was for their voice to be heard and that the information would be shared to benefit their district and other educators. If those two messages did not resonate, then trust would be difficult to attain. We followed a philosophy that gaining trust is not gained easily; rather, it was earned through slow, steady, and consistent communication that started on the first day of school.

At the same time that we were reaching out to districts to participate in our study, we were also seeing if any schools could use some help on the first day of classes. From our school experience, we knew that the first day can be hectic, and extra sets of hands are usually welcome. We were invited to help at two school sites. Our team members made friends with a cheerful school nurse who passed out health forms and consoled a student who had to get another pair of shoes from her house because the ones she wore didn’t meet the dress code. Since we had established a presence on the campus from day one and had small but meaningful interactions with students and staff, when we returned to conduct the research for our study, we were met with a sense of trust and familiarity.

While working through an MOU with the district and WestEd’s Institutional Review Board to secure approval of our research plan, we used the time to anticipate the needs of the school-based participants and plan for how to address them. We suggest to:

1) Assess each school’s breaks, shortened days, and other school events that could impact a research schedule and arrange to limit disruption of teachers’ weekly plans.

2) Research the major languages spoken in the districts to provide translated documents and make the study inclusive and welcoming.

3) Start a dialogue with the teachers involved in the study as soon as possible to develop relationships and better understand the culture of the classroom.

4) Visit every school to formally meet the principal and the teachers in order to have a window into classroom structure and school environment.

Recommendation Two: Acknowledge where students are

Even though we were proactive to address classroom needs, we continued to navigate issues, including peer-to-peer dynamics, teacher needs, scheduling, changing lesson plans, technology troubles, new students to the class, or students who left the school. While teachers were generally trusting of us, some students were skeptical.

Prior to our first observation, we asked for class time to describe the study and solicit the students’ participation; however, each teacher gave us a different amount of time-based on their lesson plans. One teacher allowed us to speak with the class for 45 minutes, and students were able to ask us what we did, where we came from, and any other questions that they wanted to ask. Another teacher gave us the floor for five minutes to explain the nuts and bolts of the study. Students in each class were given an assent form to sign or opt-out, in addition to parental permission forms. In the class that provided the most time for the students to get to know us, 84% of the students agreed to participate. In the class that provided us with a short amount of time to present, 68% of the students agreed to participate. In the class that had less trust, some students followed the leaders of their group. In two instances, a male student who had chosen not to participate appeared, through his visible disapproval, to influence his table mates. We quickly realized that our study would not be immune to middle school peer relationships, and we moved forward with the students who had agreed to participate.

Recommendation Three: Lead with support (and coffee!)

Despite a smooth start to the study, one of the educators we worked with began to feel the pressures of home and school. The teacher was hesitant to continue with the study because they worried that the classroom was not up to the expected standard and that there was not enough time to plan for using the technology. We had to decide how to respond. Should we give up on including that class in the study? Or could we change something that would make the teacher feel more comfortable? We opted to give the teacher space and find a way to re-approach. We reached out to the principal, who was instrumental in helping the teacher feel less overwhelmed by offering check-ins and substitute days. With that additional support in place, we re-approached the educator with a cup of coffee and a conversation reaffirming our intent to observe the technology and not the teaching practice. We expressed empathy and assured the teacher that we would respect a decision to remain or exit from the study. The teacher decided to continue, and we were able to finish our observations and conduct focus groups with those students.

Every research project will challenge its researchers in unique ways. Qualitative research in particular directly engages involved human behavior, thoughts, and feelings. We learned to adjust to each day in the classroom. Because we planned well, we were able to avoid some of the bigger challenges and overcome the smaller ones. Ultimately, we would suggest that the key factors to building trust for research with students and teachers are:

1) Communicate regularly

2) Be involved

3) Be empathetic

4) Offer coffee

This has been a year of learning for our team, for which we are especially grateful to our anonymous student, teacher, and district staff partners. Thank you also to WestEd, our Learning Assembly peers, and data partner LearnPlatform.

This is the 4th of six posts we will be releasing on our research. We will release our post on Khan Academy next week, so please follow us and check back. If you are interested in hearing more, email ihub@svefoundation.org for additional information.