How we rejected 15,000 users to find 50,000.
How choosing users improved adoption of technology in classrooms.
Australia has 2.4 million primary school aged children and 1 million attend primary schools below the national socioeconomic average; these children are the 40 per cent that lack opportunity not capacity, and form a central part of what we do at One Education, a One Laptop per Child spin-off.
In May 2012, the Australian government provided us with funding to deliver a low cost, low power, rugged computer, the XO, to 50,000 underprivileged children; joining our early funders Commonwealth Bank of Australia(CBA) and Telstra to increase our reach 10 fold.
Convinced that the current approach of distributing technology in schools was failing, we decided to do something extra-ordinary — we decided to filter classrooms that would receive XOs.
I like to call it ‘Organic Adoption’.
In early 2009, with funding from CBA and Telstra, we visited a few remote Aussie communities and delivered 500 XOs as part of a pilot programme.
A few years later, these early projects where we hand delivered XOs to schools were a resounding flop!
But why? We did the teacher training, we stuck around for a week to embed the technology and made sure it was supported. We had just replicated the OLPC formula, and voilà… fail!
It wasn’t until two years and around 5,000 XOs later that we developed a methodology to increase the likelihood of success (measured by how much teachers used the XO to deliver the curriculum).
What we found was given the ease of distributing technology there is a natural tendency to ‘force adoption’; this notion that everything is ‘plug-n-play’ — including acceptance of technology. Unfortunately — to believe that is to deny the essence of what makes us human, our fears, our insecurities and our varying life experiences.
After visiting at least 40 remote communities, the challenging situations these teachers found themselves in exposed a crucial insight; early adopters actually didn’t need us and that early adopters exist in every community and in unlikely places.
However, to ensure success you needed to have both early adopting principals and teachers, as without one in each camp, you would be doomed to failure!
We found early adopters by creating an artificial gate: $100 per XO (well below the market price, including unlimited support) and we asked teachers to complete online training before receiving XOs for their classrooms.
If a teacher wasn’t prepared to complete the training, we didn’t ship the XOs for her students; but invariably there was always one passionate teacher who unlocked a set of XOs for her classroom first, and that was often enough to spread the programme throughout the school.
After delivering 50,000 XOs; we have at least 93 schools that now have full saturation, accounting for 25,000 machines; the programme grew organically within these schools — in many cases the children pushed late adopting teachers to complete training.
We also have 14,800 XOs in withdrawals; where schools have decided the XO wasn’t for them or individual teachers were not ready for the project. Without our selection criteria, nearly 15,000 XOs would be collecting dust in cupboards.
What we didn’t expect was that we haven’t had to visit communities in order to distribute XOs — in fact now we flatly refuse to do this, seeing this as an indication that the school/community isn’t ready for the project and the school lacks the stickiness to keep it going long term.
School staff turnover is disproportionately high in low socioeconomic areas, and acts as a reset button on the adoption curve in schools. We successfully navigate this by offering straightforward online training.
What this approach has also meant, is with a small team of fifteen staff, we have reached over 300 schools across this massive continent without leaving our offices — allowing us to be really selective in who enters the programme and receives sets of classroom XOs.
Selection bias — the only way to guarantee success
Not convinced this was enough, I wanted to see if this was short lived or whether this resulted in the XOs being used more often; after all if the XOs weren’t being used regularly — why bother?
To validate the efficacy of our programme we collected anonymous usage statistics and we ran a survey across participating teachers.
The results of the survey (320 respondents across 138 participating schools):
- 97 per cent teachers agreed their students really enjoy using the XOs — this is really important as it links to engagement
- 87 per cent of teachers use the XOs at least once a week, 55 per cent daily
- 79 per cent agree the XOs have provided new learning opportunities
- 65 per cent of teachers noted that students have asked to use the XOs outside of class (e.g. before/after school, during recess/lunch)
- 80 per cent of teachers would recommend the programme to their colleagues
A year ago we also started to collect anonymised usage statistics to learn how the XOs were being used in classrooms.
After 14,000 XOs had reported back to our servers, we found that:
- 44 terabytes of data had been downloaded to the XOs over a 4 month period — which tells us that the children were using them and using them regularly!
- 95 per cent of the time was spent using the XO for browsing, word processing and using the camera/speak tools
- The gender split was 50:50, a no brainer in a school environment; but a huge leap for providing equal exposure at a young age to technology
Combined with the anecdotal stories, it is clear the XO is being used regularly and as a tool to deliver class instruction and individual exploration.
Shortly after reviewing the results I was gloating to friends and anyone who cared to listen. I remember spilling the beans to Prashan Paramanathan, CEO of Chuffed.org and his response was sobering “mate, you have selection bias in your survey results” — he was right… but the truth is our whole programme is predicated on selection bias (some people might call this audience targeting? ;) ) — and success would have been a fairy tale without it.
