It’s Checklist Time! Diving into Data Collection

If research is a puzzle, data collection is a process of dumping the pieces out of the box and diving in to sort through it. For some, it may be a satisfying commencement to see the data begin to roll in, but for me, I find it comes with anxiety. Will it be easy to piece everything together or will I be forever wading through looking for connections?

Part of my worry comes from the nature of the data, which is largely open-ended and qualitative. Since graduate school I have been particularly fond of qualitative research. Following a bachelor’s degree and student research which was heavily quantitative, I find the nuance of qualitative and oral research fascinating. It is, by its nature, much more subjective, so I am seeking to triangulate my research with some quantitative elements to anchor the qualitative analysis. You can read more about my research methods here.

It has helped me to ground me to remind myself of my research question [What are the potential barriers for students with learning differences to access primary source documents in the history classroom?] and the existing research as I begin to collect data.

Student Checklists #1: Reading sections of the Constitution

My first point of data collection through student checklists was for a lesson giving students access to the Constitution as a primary source.

I primed students by telling them they would only have a small section to read, that they could use Newsela to change the reading level as needed, and that they would have several days to work with their levels to become experts on their sections. Students also picked 10 vocabulary words that they or others found confusing, usually in the original (unmodified) text. This gave them an opportunity to interact with the original text, but also use easier levels, as needed.

Student Checklist #1 was a short survey of likert items and short-answer open ended questions. After introducing the assignment, I had students complete the first half of the checklist questions about their level of interest and confidence related to reading this primary source. I asked students to keep the form open during the lesson, and complete the remaining questions at the end of the lesson as an exit ticket. Here, I asked students to assess their prior confidence level for accuracy, indicate whether or not they learned from the lesson, would feel confident on similar readings, and whether they still had questions.

So, How Did It Go?

When I collected my first student checklists after students spent one class period working with the Constitution, my first instinct was to seek out trends, to find the insights within.

Following some digging and discussion with my SET Lab team, I’ve come to the conclusion that this first checklist was only one point of data. Once I collect the same information on a new lesson, this first set will be a more meaningful point of comparison. While the checklists served as helpful feedback on how students were feeling after the lesson (and I was glad to have this formative data), there is no major picture emerging just yet. The only trends of interest thus far are the interaction between confidence and interest.

Despite all my enthusiasm, there was not intense interest from students about reading the Constitution, but this did not seem to stand in the way of their confidence about reading it (note that in the chat, confidence 1 was pegged at “not confident” and 4 at “really confident).

While the pre-assignment questions were not dramatically positive, the post-assignment data was much clearer (but needs comparison). To my surprise, all but one student felt they understood the lesson and learned from reading it. We’ll see if the trend continues with the next primary source lesson (coming soon)! Follow my journey here on Medium and follow me on Twitter and follow through #setlab #educationscientist and #changebydesign