A research-based approach to coding education
One of the things we’ve seen lacking in the technical education space is an intentional focus on pedagogy and school design. It isn’t enough for us to ask “What makes an effective software engineer?” We also need to ask: “What makes an effective school?” and “What makes an effective teacher?”
As it turns out, being a good software engineer doesn’t necessarily make someone good at teaching it. In fact, it can actually make someone worse.
According to the National Research Council in How People Learn, “expertise in a particular domain… can sometimes hurt teaching because many experts forget what is easy and what is difficult for students.”
Research shows that being a great teacher requires more than just content knowledge; it requires a fundamental understanding of pedagogy– the theory and practice of how to teach.
Recently, our team used research by the world-renowned educational academic John Hattie to analyze how we could improve our school structure and teaching practices to better support student outcomes.
Hattie is best known for undertaking the largest-ever analysis of quantitative education research data. He synthesized the data from over 800 research studies that synthesized other studies and produced a ranking of the factors that actually impact educational outcomes.
Hattie ranked each factor by “effect sizes”. An effect size of 0.4 means a year’s worth of progress for a year’s worth of effort. According to Hattie, everything with an effect size above 0.4 is worthwhile, and anything below 0.4 can be discarded.
There are over 250 factors in Hattie’s most recent work. We’ve included a chart of some of the factors we find most interesting and relevant.
The top factor, collective teacher efficacy (a teacher’s personal belief that they can influence a student’s achievement), can give students a boost over 3.5 times higher than average with an effect size of 1.5.
Factors below 0.4 (and thus, negligible) include things like the teacher’s subject matter knowledge or lack of sleep.
Our team has a spreadsheet of many of these factors. For each factor, we’ve asked ourselves the following questions:
- What does this mean?
- How can we assess this?
- How are we doing this well?
- How can we do this better?
As we ruminated on this research, we realized that there were a lot of things we were already doing exceptionally well.
From day one, we have used deliberate practice and targeted feedback to push students towards mastery. They build their first full stack app by themselves from scratch in week six and practice those skills in repetition throughout the second half of the course.
From day one, we have used conceptual change programs to help students shift their mindsets about learning. As Kani, our CEO, likes to say: “We make you comfortable with being uncomfortable.”
We have also helped students develop their soft skills and critical analysis skills through technical discussions, student-led meetups, and improvisation techniques.
Analyzing this research also made us realize there were some things we could put a heavier focus on.
Below, we’ve included a list of changes we have made since our first cohort, and how it is related to Hattie’s research.
Response to intervention (1.29) :We added ongoing formative assessments and created a scaffolded system of support for struggling students.
Jigsaw method (1.20): We changed our solution lectures from teacher-led to student-led after each activity.
Collective teacher efficacy (1.57): We implemented a research-backed teacher effectiveness rubric to improve both our collective abilities and confidence in our impact.
Cognitive task analysis (1.29): We shifted the focus of lectures from delivery of content to task decomposition strategies.
Lack of sleep (-0.05): We reduced the time of our classes from 6 days per week to 5 days per week.
By using research as part of the design process for our school, we’ve been able to refine our school model and improve student learning outcomes at an accelerated rate.
At Code Chrysalis, we’ve been intentional about creating a team of highly qualified software engineers AND highly qualified educators. And we have also been intentional about using educational research to inform our teaching practices.
We don’t just want to be the best school for software engineers.
We want to be the best school.