Using statistics to focus our efforts
A quick peek into how we at I Hate Statistics use data and statistics to expand our online practice exercises
Practice makes perfect
I’m confident to state there is one essential ingredient to mastering statistics: practice. Everyone can master statistics, as long you keep practicing and testing yourself.
At I Hate Statistics, our mission is to make statistics accessible and understandable for everyone. In particular to those who feel they hate it.
One of the ways in which we try to achieve that, is by helping students in university courses to practice statistics. Online, with direct feedback.
Do students practice? Oh yes, they do.
One example of a course we supported is a first year statistics course at the psychology program at the Vrije Universiteit Amsterdam. In seven weeks, the 350 students completed over 137.000 exercises!
Although I’d love to claim that’s our work, it is not: their teacher Ilja Cornelisz did a terrific job in motivating students to practice a lot. (Which he did so well that he even received his university’s Teacher Talent Award. Congrats Ilja!)
What we hoped for is that students would practice and that our online environment might make it easier for them to do so. What we and Ilja did not expect though was the high amount of exercises they completed (137.000 exercises divided by 350 students, thus hundreds of exercises per student)
So we were happily surprised. But it also introduced a serious problem: at some points, we ran out of available exercises. This meant that after a dozen exercises on one subject, students were working on problems they had already encountered before. They remembered the answer and could not challenge themselves anymore.
“I Hate Statistics really helped me practice and understand statistics. The only problem I had was that there were not enough questions.” — Kyah (student at Vrije Universiteit Amsterdam)
students were practicing so much that we were simply not able to keep up.
Using data and statistics to prioritise our work
The quote from the students above was voiced by many more students. We decided that adding new exercises needed to become our highest priority. With our team of educational content creators, we jumped in. As we cover over 90 distinct subjects, our main challenge was: where do we start creating new exercises?
We first asked students, but that did not lead to a coherent list. So we decided that we needed more data. Together with technical co-founder Thijs, we spent an afternoon creating dashboards.
Finally we came up with the following approach: new questions are most needed at subjects where students most often encounter exercises they have already seen before. We averaged this per student, to correct for the fact that some subjects are seen by more students than others (some subjects are covered more often than others as we create tailored curricula for all the teachers we support). We started calling this the Average Double Question Rate.
By creating a list of subject with the highest average double question rate, we were able to prioritise our work. Our educational content creation team started off by creating exercises for these subjects.
Did it have an effect?
Interested in evidence as we naturally are, we soon started wondering: does adding new exercises really help to lower the double question rate?
Again, we turned to data, statistics and in this case some graphs :)
Our take-away: The double question rate per student though makes a sharp drop, right after we added the new questions. And only for the subjects where we added new questions, not for the other ones.
(What you don’t see in this graph, but that we did check: although the number of exercises done per subjects varies over time, it stayed roughly equal).
Constantly adding new material where it’s most needed
Although is not a perfect metric, we’ll keep on using the Average Double Question Rate to indicate where new exercises are most needed. This way we can ensure that students always have plenty of opportunities to practice on new problem sets.
Data is not everything
One thing we always try to teach our own students: data and statistics are not a holy grail. Always make sure you really understand the data and check and support it with qualitative insights.
So let’s practice what we preach: are there subjects that you feel could really use online practice exercises? Which type of exercises work well and which ones don’t?
Add your own
Do you have your exercises that you would like to bring online so they can be automatically graded and your students receive direct feedback? You can now add your own exercises as well. Please send me an email to learn more about it!
Let’s make sure that students have enough opportunities to practice and test themselves. It will surely help to ensure it will be the last time some people think: I Hate Statistics! ;-)
PS: Are you a statistics teacher and interested how the online exercises and direct feedback work? You can request a free teacher demo here.
