How can we ensure quality in OERs?
In 2015 Cable Green presented the Key note presentation at the OER conference highlighting how far the Open Educational Resource (OER) movement and infrastructure has come, specifically through the use of Creative Commons (CC)licensing.
Creative Commons
The presentation showed significant growth in the usage of CC licenses. With 50 million in use in 2006 and sites like Vimeo and Flickr now supporting these licenses it is estimated there are 1.4 billion licenses in use today (2019).
The development of CC licenses was a vital step for the growth of OER, which wouldn’t have been possible under traditional copyright licensing law.
Repositories
Another key part of the OER framework is the development of repositories to share resources and content at no cost to the developer.
In 2016 Jisc retired its Jorum repository, launching the App and Resource store in 2017. The project’s big goal was to ensure it hosted high quality resources. To do this, Jisc took the step to review every single resource, deciding which deserved a place in the new repository. Although a significant undertaking, it was an important investment.
Despite redesigns like this, OER repositories still have a way to go to improve. With the growth in OER numbers, it is important to improve the discoverability of high quality OERs.
To measure quality, OERs could take inspiration from traditional academic publishing and use peer reviews. However, this would require investment and buy-in from OER developers.
In an area where educators are aiming to share knowledge and practice ‘for free’ it may be difficult to provide an incentive for busy developers to participate in peer review practices.
When looking at the development of Jisc’s repository it seems this was a key part of what they wanted to create. In the project’s purpose statement they describe this:
“Quality assurance… will be community-driven and result in the sharing of useful resources and good practice. .. recommendations… will help members promote and discover relevant resources.” JISC
Barriers to the reuse of existing OERs
In 2016 Olga Maria Belikov & Robert Bodily undertook a qualitative analysis at US universities revealing the perceptions of reusing OERs in the academic community. The results of the study reported 10 themes, coded as either a barrier or incentive to using OERs.
Recurring themes related to the quality and discoverability of OERs. 16.5% of respondents reported issues discovering resources — commenting that:
“…a peer reviewed repository of materials needs to be created…” (Belikov & Bodily)
With 9.2% of respondents commenting that the quality of OERs is below that of traditional resources and 10.6% reporting a lack of time to evaluate resources as a barrier.
These findings are also echoed in a 2006 trial by Nature to make peer review of journal articles ‘open’ by reviews being posted openly on their site with researchers commenting that they:
“…are too busy, and lack sufficient career incentive, to venture onto … Nature’s website and post public, critical assessments of their peers’ work” — Nature
How can these issues be addressed?
Reviewing these results alongside Jisc’s repository I believe there are some clear solutions that would benefit site users.
Discoverability: Further search improvements would be particularly beneficial, such as being able to take a ‘filter out’ approach, where results are excluded or included based on preference. A good example of a filter I’d like to see would be license filters. Being able to exclude CC BY-NC-ND 4.0 for example. Creative commons describes this license as:
“… the most restrictive of our six main licenses, only allowing others to download your works and share them with others as long as they credit you, but they can’t change them in any way or use them commercially” Creative commons
What I find most disappointing is that the only filter option available is to limit your search to ‘Only free resources’. Aren’t OERs meant to be free?
Improving search algorithms also needs to be a priority for Jisc. Currently a search for “Essay writing” will only return resources with the two words in this exact order. So our resource titled “Never a wasted word: writing your essay” doesn’t appear in the results. At University of Manchester Library we develop almost all our online content as OERs, sharing them via Jisc App Store under a CC BY-NC 3.0 license, so it’s frustrating to know that our resources probably aren’t being discovered.
Peer review: The study by Belikov & Bodily showed that respondents want OERs to have peer review to ensure quality. Jisc has taken the approach of including review/commenting functionality but it is quite passive in its approach, and does not incentivise receiving or sharing reviews. A better approach may be to share new resources with contributors and request peer review, creating a more reciprocal relationship between contributors and organisations maintaining these repositories. This method would reassure users that the resources they will find in the repository will be of a higher quality and in turn, increase repository usage. However with a lack of other incentives to take part in peer review it may simply discourage resources from being shared at all. Another challenge for implementing peer reviews will be to avoid the nonsensical model of traditional academic publishing. Where the repository (journal) is in charge of the process and charges for the resources (article) to undertake peer review gaining some rights to the resource in the process such as only being able to host (publish) in their repository (journal).
Incentivising the quality of OERs
To encourage OER developers to meet quality standards, institutions may want to consider developing easy to identify digital badges to show which quality standards a resource has achieved. This would motivate developers to earn badges, whilst also providing further options when deciding which resource(s) to adopt or remix.
What could some of these badges be?
- Remixable- Resources must use a license that doesn’t include a no derivatives clause. Achieving this badge would be a simple automated check of the details when the resource is uploaded.
- Editable — Resources developed using software not widely available must provide an editable version. This may be checked simply by a second attachment being uploaded in the format of .doc, .txt, or other file format which indicates it is a text file.
- Accessible — An example would be including captions and transcripts for video and audio materials. Tools like Blackboards Ally have been developed to do checks and rate accessibility so automation is possible. However the cost of developing a tool for this is likely to be very expensive. A list of accessibility features would either need to be checked by a person or done on a trust and declaration basis.
What next?
The OER movement has come a long way, but that doesn’t mean it’s complete. I think it’s now is time for those of us producing OERS to look at the opportunities to increase quality whilst continuing to advocate for openness in education.
