50,000 Open Badges in 18 Months: What have we learned?
18 months ago, the team at RWA implemented an Open Badge reward system for our online learning and assessment platform, The OBELISK, which has now become the Aviva Development Zone. The OBELISK has been running for over 9 years as a means to assess knowledge of UK insurance brokers and provide evidence of understanding and to highlight knowledge gaps as a means to improve competency. Since implementing Open Badges, we have just passed the 50,000 badges issued milestone and think we should feedback on our experiences and share our thoughts and learner feedback.
Why did we move to Open Badges?
Up until 18 months ago, we used to issue system generated certificates which learners could activate and download as a simple means of evidencing they had done something. This could have been a pass or fail but showed that time had been spent on a learning and assessment task and not many learners activated the certificates, so they were not seen as engaging for our community.
The RWA team were focusing on a big development project to build a new version of the OBELISK (version 3) that would enhance and make a huge difference to the online learning experience of our learning community that would both educate our learners and engage them to want to learn more on a regular basis. This to us meant that a large cultural change had to be started to shift the perception of online learning for professional people and to create a low cost solution that was both feature rich and that could be scalable to any size firm.
We knew that evidencing knowledge and rewarding learners was key to that development and we also wanted to make sure that we steered clear of the gamification craze which we saw as a distraction and not really that conducive to our learning model.
Open Badges seemed the right way to go and ticked all of the boxes we needed for our development project:
- Digital verification of task achievement (baseline assessment score)
- Expiration dates (allow assessments to be completed annually for ongoing measurement)
- Full customisation to allow RWA to maintain our own brand recognition
- Social network integration for learners to share achievements
- Displayable (every learner on the Development Zone has an achievement wall to display badges)
- Full support for our formative assessment learning model
- A growing and very supportive online badges community to share ideas and progress
An instant success?
At first, there was no denying that we had to educate our own people to see the benefits of the new rewards based system and then work to educate our own learners on the benefits. Some of our clients saw them as a gimmick and didn’t want to include them in their own systems and individual learners struggled to understand the relevance of a badge reward.
This meant that not only did we just have to roll out the technology, but we had to work consultatively with our clients and in our own marketing to manage the understanding and to highlight the benefits of badge rewards.
This meant that our experience with Open Badges was a slow burner. It is difficult to show growth in terms of number of badges awarded each month as our user numbers have doubled in the last 12 months and we have added over 100 new courses where more badges can be earned, so the opportunities to issue badges has increased. What our analysis does show is that we are now issuing over 4,500 open badges per month on the Development Zone and 12 months ago, that was more like 700 per month:
The results show that growth was fast and over the last 6 months this has evened out and we are now averaging about 3 badges issued per visitor for that month (compared to 1 per month when we started).
How does a learner get one of our badges?
From the start, the team at RWA believed that it was important not to replicate the certificate model where a learner would get a verification they had simply completed a task or ‘done something’. The key to our approach to using Open Badges was to ensure that learners had to achieve a baseline score on each assessment to unlock the badge. This was set at 70% across all of our assessments. This means that in order to achieve a badge for any of our 250 online courses, the learner must achieve a minimum baseline score of 70% on the gap analysis assessment for that course.
The badges then expire every 12 months, which means that learners must re-assess their knowledge on an annual basis to evidence maintenance of knowledge and to highlight any knowledge gaps or risks to the business.
Badge rewards are issued automatically by the Development Zone learning platform. The learner gets a notification email, a notification on the platform, and the badge gets added to their Achievement Wall, their Badges store and also their annual training record for use in company appraisals.
We have over 250 courses on the Development Zone split between technical learning and business skills learning. This number is growing all of the time as we continue to add more courseware, assessments and badges.
If we consider the Top 5 Badge Collectors on the system, then we can see how the most active users are engaging:
Learner feedback is also important to us and we regularly communicate with our learning community and get feedback from our learners and their supervisors:
“Keiran has confirmed that receiving badges does motivate him to learn more, he sees badges as recognition that his efforts have been worthwhile.”
“I found the badges a great tool to help motivate me to complete more and more courses. The satisfaction from seeing my development plan full of recognition badges is far more positive than just a long list of courses, and gives instant feedback and reward on the courses done.”
“It’s simpler and clearer than just a list of passes and certainly makes to a more rewarding experience. The ability to download badges to my computer means I have a folder full of learning, without the need for multiple certificates of achievement and these are instantly recognisable as to which course earned the badge.”
We have also seen a significant amount of growth in terms of usage of our own Development Zone platform:
- Usage grew 489% in 2015 (measuring the number of interactions on the platform).
- On average, our active learners access the site 5 times a month and visit 110 pages per visit
- User numbers doubled in 2016 and continues to grow every month
And whilst we do not attribute this growth wholly to Open Badges, what it does show is that our approach to online learning and assessment which focuses on delivering a learner centric user experience is keeping our learners engaged, educated and coming back for more on a regular basis. The team at RWA believe that Open Badges play a fundamental role in adding to the great user experience and high standards that we expect.
What we have shown over the last 18 months is that our formative learning model has shown that where learners take an initial gap analysis assessment, the average score has been 67.6%, but having completed learning on the platform and by using the tools we provide, we have taken that average score up to 84.4%.
What about the future?
Over the last 18 months, we have focused on embedding open badges into our e-learning culture and believe we have been successful. We are now looking at a number of strands for future development in a number of areas:
In The Workplace: our next challenge for our existing community is to continue the consultative work and educational piece around using Open Badges as part of an appraisal system within the workplace. Our system provides, downloadable learning records which include activity reports, CPD logs and badge rewards, and we are working with our supervisor clients to look at how we can use the evidence based badge rewards to support appraisals and employee development offline.
Our clients are beginning to look at using badges within this process:
“We believe that badges are a good thing for staff to receive as it provides them with something for their achievement and a lot of them like to try and get as many of the badges as they can. Using their badges within their monthly appraisals is something which has been discussed and is being considered.” — Firm Supervisor
In Education: We are working with a secondary school in Newport, South Wales, to investigate whether badge rewards and our Development Zone platform can be used to engage school children through providing business skills learning and marrying up the rewards process with a clearly defined educational framework that will enable us to also work with local employers to develop evidence based criteria for future job roles.
In Community Development Scenarios : Our work has also led us to start looking at opportunities for skills development in local communities where badge rewards can be used to engage individuals and highlight development opportunities for those who want an evidenced based approach to employability and skills development.
Our learning points
In order to achieve the growth that we have had, we needed to work at embedding badge rewards into our learning process and the learning design within the system. Badges are now an important part of the Development Zone culture, and we reward learners with unique User of the Month badges if they have excelled at learning activity for that month and we share their success stories with the wider Development Zone community.
There are still those who do not see value in the badge rewards and find it hard to engage with them, but we hope to change that over time and feel that we are making progress as we regularly hear feedback that badge rewards in the business are creating a competitive learning environment where employees want to earn a badge that someone else has which they haven’t.
Our focus up until now, has been providing the Development Zone within the UK insurance broking industry and we have shown how our model and system has been extremely effective for gap analysis, risk management and skills development. We are now launching a business skills development version for any business, community group or individual to access and benefit from the wide variety of tools and the growing learning database that we are building.
What will be interesting to see as this grows is how quickly the number of badges that we issue will increase and also how more and more individuals and businesses feedback their experiences of open badges so that we can continue developing and improving how reward based learning engages the learner.
We would be very interested to hear from anyone who has similar or different experiences or would be interested in looking at how our solutions might benefit their own development or businesses.
For more information about RWA or My Development Zone, please visit