Quality considerations in Open Badge initiatives
The Open Badges standard is over five years old and there are many examples of its application, with badge issuers ranging from those in industry to education, government to the 3rd sector. But in Digitalme’s experience of working with organisations to create badge schemes, we still hear questions about the value and quality of badges.
Digitalme has approached the question of the value of a badge via our Design Canvas, which has helped many organisations to consider the role-based value proposition for the individual badges they are developing. Downloaded more than 9000 times, translated into 5 languages, and used effectively in many design-led workshops, we know that the Canvas approach is helpful in charting the key elements of a badge that will make it of value to badge earners, issuers and consumers.
But we have recognised that there is a stage before designing a badge, where it would be useful to provide some help for prospective badge issuers to consider quality in terms of their badge initiative. Before designing a badge of value we need to consider if using badges is even the right approach to take. As founding members of the Open Badge Network, we have been working on the concept of Open Badges and Quality Management through the development of a Discussion Paper and Quality Canvas informed by the findings from a Quality Survey.
The Quality Survey was structured to encompass quality considerations from the perspective of various stakeholders, with survey respondents being asked to consider quality through the lens of a badge earner, badge issuer and badge consumer. The Discussion Paper presents the survey findings in relation to stages of a badge initiative: initial design; implementation; and delivery, and provides recommendations for quality considerations at each stage. The Quality Canvas is designed to help initiatives consider quality before commencing the creation of their badge scheme and like the Design Canvas, helps users to map out the key elements of their badge initiative. It synthesises the findings from the Quality Survey and recommendations from the Discussion Paper into an easy-to-use tool that can be completed by individuals or preferably, a group of stakeholders that have an interest in the badge initiative.
We tested a prototype of the Quality Canvas at MozFest and were pleased that feedback was positive overall. We will be making some tweaks to it for the next version and will also work with Open Badge Network partners Artes, who are experts in quality management, to further refine the Canvas.
Here is an example of the prototype with a completed example:
Quality Survey findings
The Quality Survey was released in February 2016, with data gathered in August 2016. In total there were 39 responses, with 25 complete responses.
The main sector represented by respondents was education.
If you are interested in the survey findings and our recommendations for quality in terms of the initial design, implementation and delivery of an Open Badge initiative, you can find them here. (The summary on p28 gathers the recommendations from each section together.)
The following summary provides the top quality considerations for badges as specified by respondents assuming the roles of issuers, earners and viewers.
Key desired strategic outcomes for badges
All respondents were asked what they saw as valid uses of badges, so that we could gain an insight into the overarching strategic outcomes they would like to see from the use of badges. The joint top choices were:
* Badges as a means of discovering intrinsic motivators, e.g. as a way of recognising what someone already does, is good at and therefore is motivated to do
* Badges to empower people to forge their own skill paths
Issuers
61.3% of respondents assumed the issuer role. Issuers were most interested in issuing badges to give recognition for non-formal learning. In terms of the opportunities provided by the badge ecosystem, they wanted to create badge pathways.
Most of the respondents stated they think about quality when they create and issue a badge, while 11.8% stated they did not know if they did. To support quality, issuers want:
* To be able to choose an evidence type that is relevant to the context, such as asking earners to submit a reflective piece or demonstrate their applied skills via a video etc
* To have quality assurance processes in place
* To develop badges as part of a wider badge scheme
* To use a badge issuing system that enables earners to upload and store their evidence in the system (thereby reducing the chance of broken links with evidence held in 3rd party systems)
Earners
25.8% of respondents took on the earner role, the second choice out of the three role options. They were most interested in earning and displaying badges to make them stand out to employers or in an admissions process for a course or apprenticeship, and wanted to unlock opportunities with badges.
All think about quality when they take a badge. To support quality, earners want:
* The badge issuer to be a well established organisation. (See final paragraph on how developing networks around badge schemes might help in this regard)
* To be able to benchmark their work against others’ work in order to improve their badge evidence
* A system for earning badges that allows them to search for and find badges
* A system for displaying badges that allows them to set the visibility of their badges, e.g. hide or show them
* A system for displaying badges that allows them to share their badges to social media and professional profiles such as LinkedIn
Viewers
12.9% of respondents assumed the viewer role. Viewers were most interested in viewing badges of prospective students as part of a course admissions process and building capability through badges. Please note that the viewer only section was not completed by all who started and submitted responses. The data presented below represents only one respondent’s views.
The respondent confirmed they consider quality when they view badges and how quality is built into the badge ecosystem. They stated they would trust a badge if:
* They were aware of the badge quality assurance procedures implemented by the issuer and believed them to be sound
* The badge earner had received endorsements on their evidence from people working or teaching in that field
* They could quickly ascertain the level of competency the badge indicated
They believed:
* Badges should be aligned to competency frameworks
* Badges should map to a particular framework. (When asked to state which competency framework, they didn’t specify a particular framework but responded “It would help me as a viewer to understand the value of a badge.”)
Finally, some of the responses from the quality survey highlighted where other Open Badge Network resources could provide support. For example, badge earners stated they are more likely to trust a badge if the issuer is a well-established organisation or well known to them; while badge consumers responded they are more likely to trust a badge if they are aware of the quality assurance processes that have been employed by the badge issuer. An Open Badge Network resource that may be helpful in this context, is the Guidelines for Open Badges in Territories, which draws on case studies of initiatives that have supported wide-scale engagement with Open Badges across a geographical area through the formation of networks. Digitalme are in the process of developing a Roll-out Canvas that badge initiatives can use to map networks to support uptake of their badges. A prototype was tested at MozFest, which received some great feedback, so we hope to share details of that soon too.
Keep in touch
Interested in finding out more about our quality and Open Badges, or developing your own open badges? Get in touch!
Download a copy of the prototype Quality Canvas.
