12 months at the Raspberry Pi Foundation
It’s been over 12 months now since I joined the Raspberry Pi Foundation in a new role leading their research work. It’s been a busy year, and I had meant to blog more as I went on what I was working on, but as happens the actual work took over a little. It’s good to look back though, and easy to forget how much has happened in a year. I’ll likely blog in more detail about some of these elements, but for now an overview.
Annual reviews
The first project I took on when I started was the 2015 annual review, and I’ve done two of them now (available at raspberrypi.org/about). The first was a good opportunity for me to talk to everyone across the foundation about what they did and get an in depth understanding of all the programmes. This helped in the early days to begin conversations about research and evaluation and figure out my first steps.
Many people know the Raspberry Pi Foundation for the computers created by the trading company. There is also a wide range of educational programmes that the foundation runs. The annual reviews give a good overview of all of the work we do, and I hope are a useful document for people to find out more.
These were really interesting projects to work on, both in terms of the project management and the writing. They involved working with almost everyone across the organisation to really understand the work they are doing and the impact they are having. It was then a case of communicating this in a way that was informative and accessible for a range of audiences.
Creating the finished publication involved working with our illustrators, designers and publishing team to create something that looked the part and communicated the fun ethos of the organisation as well as the substance of the serious work we do. I think we reached that balance well and I’m pleased with the final result.
Research strategy
Having produced the first annual review I had a good overview of how the various programmes fitted into the foundation strategy. This put me in a good position to devise the strategy for how research and evaluation should support and inform this work. I spent some time developing this and getting feedback and input from others, resulting in presenting it to the board of trustees. It’s now shaping everything we do in terms of research and evaluation.
Evaluations
A priority was to develop the approach to understanding the impact our programmes are having. I started this by undertaking some analysis and evaluation of programmes that had recently ended, or were coming to an end. The Creative Technologists mentoring programmed and the Education Grant Fund were at ideal stages to evaluate in very different ways. The grant fund involved delving into reporting and documentation to provide an analysis of impact to trustees. The mentoring programme involved running some qualitative research such as interviews, focus groups and observations to analyse impact. Both of these went on to influence future decisions and future work.
Data
End of programme evaluations are useful, but even more so is evaluating as you go and tweaking delivery accordingly. Doing this requires the right information being collected at the right time. After the evaluations I moved on to working with various programmes to develop this approach.
The most comprehensive work on this has related to the training we run for educators. I’ve worked with the team to develop and improve their collection of information and data. This includes running an annual survey and developing the thinking around the journey people take as they go on training, join the community of educators, and develop their practice with young people. I’m pleased with how so far we have managed to integrate data collection with activities that are genuinely useful to educators and also help them progress and increase their impact in the process. One such initiative are the project logs we are asking people to complete. These are a data collection device, but more importantly they help people plan and execute on their plans for translating the skills they gain from training into their educational practice.
Theory of Change
It’s all very well collecting data and evaluation, but you need to have a strong sense of what you are trying to achieve to make any of it meaningful and useful. Businesses have their financial bottom line as a strong reminder of this, but social organisations can find it harder for everyone to understand the focus. This deserves it’s own blog post entirely, but I’ve spent about six months working with everyone across the foundation to develop our theory of change, explaining what we are trying to achieve and how we believe our activities contribute towards this. It’s been a significant strategic exercise doing this work, and I’ve really enjoyed digging into every aspect of what we do, challenging assumptions and hopefully bringing some clarity to the complex set of work the team does. This will be published soon, and I’ll go into more detail on the process and what I’ve learned then.
Publications
It’s so important to share and publish what you are doing if you want to be a learning organisation. For a charity like Raspberry Pi we can contribute to education not just in terms of the practical activity we do but also in sharing the thinking behind it. That’s why I’ve been keen to set up a mechanism for publishing the research work we do, and we now have it as a ‘Research and Insights’ area on the website. This currently features the independent revelation that NfER conducted of Code Clubs. It’s a big deal to publish something like this, especially as it contains feedback both on success and areas that are more challenging. I think it’s testament to the ambition here to share and learn with and from other people. Setting all of this up and managing the publication of this report has been a lot of work, but well worth it. Plans are afoot for more publications regularly now.
Culture
I’ve mentioned being a learning organisation, and a lot of my work has involved working with people to develop this culture even more. I’ve started a research section on our internal Slack for discussing relevant (and tangential) research projects and papers. We have a series of informal get togethers (on hangout for remote and office based staff) digging into research papers and publications that I select for us to learn from and be challenges by. I’ve also done lots of supporting and consulting with programme managers and teams who want some research influence on their work or want to get better at understanding what’s out there they can learn from or how to get ongoing feedback.
Learning from others
As well as working with NfER on their evaluation, we’ve been working with the Behavioural Insights Team on various experiments into growing the number of Code Clubs across the UK. We’ve learned a lot from their experiments, especially on engaging with people online and making it as easy as possible for them to turn good intentions into action. I’ve also been a member and supported of the Computing At School Research Group, hosting a meeting and getting to know all the researchers involved and the interesting projects they are working on. Engaging with researchers and keeping up to date with what people are discovering in computing education and digital making has become an important part of my role.
New research projects
I’ve also been working on developing some empirical research of our own, testing some of the assumptions that have been made clear, particularly from the theory of change project. I’ll share more soon, but I’ve got some interesting projects up my sleeve that are just coming together with some partners.
On reflection it feels like a year of developing, sense making, planning and building on the existing work of others. I came to this job as a new role, with no predecessor or legacy, although lots of ambition from myself, Philip the CEO, the trustees and the team. I think we’ve achieved a lot both strategically and practically, and the next twelve months looks a lot more certain than the last twelve have been. Research now feels like an established area at the Raspberry Pi Foundation, although I have many plans to continue developing it now that we have got the area moving. A year well spent, and now on to the next…
Originally published at Oliver Quinlan.
