Open Data, Open Collections
by Adam Moriarty
The Auckland Museum’s collection contains literally millions of amazing items that express connections with the world. They encompass natural science, cultural heritage, art, archives, a research library and the Cenotaph database. They have evolved over the museum’s 163-year history, and bear the print of the people who have collected, cared for and contributed to them. As such, their equivalent can be found nowhere else in the world. We take them as our starting point, they are the very heart of what we do.
Understandably, this diversity is expressed through an assortment of data standards that captures and records the collection in a number of different ways. Internally the Museum uses three source systems (Vernon, Presto and Genie) to database collection information. Within these systems, and even within each collecting department, there is a range of standards which reflects the requirements for describing different types of objects with different characteristics for different disciplines and all of which also reflect the people and preoccupations of our rich institutional history. It’s a fantastic, messy, wealth of information.
In order to exploit this accumulation of knowledge and to provide external users with meaningful online access we needed to find a way of enabling the exploration and exposition of this data in an integrated way, as a single collection.
Future Museum is guiding the Museum through a process of institutional renewal. It offers a set of values which recognises that museum culture is changing — that, although the Museum cherishes its unique institutional history of content curation, it must now embrace a different kind of access to the museum collection; one that allows people to self-navigate, engage with and discover the wondrous landscape of our natural, social and cultural collections on their own terms.
This might not be considered revolutionary for the sector, but for our 163 year old institution, this was the first step towards becoming a truly digital museum. It has demanded at times controversial changes in the way we do our business. We are relinquishing control and are moving our closed world systems into an open world, one where we will encourage sharing and collaborative use of our data. One that recognises that we have a responsibility to a truly global audience — onsite, offsite and online.
We have two guiding principles to help us navigate towards this transformed horizon. That: “we are open as a rule — closed by exception” and that “we are one collection, not many”.
Open as a rule — the filter system
While working towards this vision of an open collection, we recognised there was a requirement to build in methodologies that allowed us to be both respectful and responsive to cultural and personal sensitivities. Because of the fundamental change in process and deliverables, we also had to ensure that a level of trust and good faith was maintained with internal stakeholders by ensuring they were able to retain a responsible level of control around the information we published. For instance, we have hundreds of fields in our collections dataset which pertain not only to objects descriptions, but also to the personal and administrative information of our stakeholders. These are necessary for proper management of the Collection but were never designed to be seen in an un-curated, un-moderated way. This data was not complete, not systematically audited and there was a very real fear that by releasing this raw data, the reputation of the museum and its people would be put at risk.
To manage the release of this data we implemented a series of filters that were customisable to each department. Firstly we created a filter for four grades of openness, we then allowed the collection teams to define which fields would be available to the public for objects in each grade, for each department.
We then asked the collection team to provide the rules that would determine which objects will fit into the various grades. This allowed us to bulk apply the filter. These rules ranged from time based exclusions, geographic selections or by classifications. Although they were designed to be applied at a bulk level, we ensured that they could also be manually applied to individual records when required.
The records are extracted from the three source systems every 5 minutes. This provides the collection staff the ability to quickly update a record’s grading in the event that inappropriate data is inadvertently published. It also empowers them to publish new data quickly if required in line with exhibition, public enquiry, current events or simple data correction.
Although the filter allows us to restrict data that may be of concern, this is not the end goal. We monitor the number of records in each grading and work to solve data integrity problems to continually move more of the collections into the open filter setting.
For an object record to be completely blocked online, there must be a clear cultural, ethical or legal reason that would detrimentally affect the reputation of the museum or the cultural integrity of the object. To date only 5% of the collections are inaccessible to the public. The flexibility of the system has ensured the success in the publishing of our data. It has meant that we can navigate the difficult terrain of the road to openness with confidence. That, we can find a route to offering the full extent of our heritage to a global audience, without compromising the values of the institution and the cultures of people it and its collection represent.
We are one collection — Linked Data
As information was gradually prepared to be published online, we acknowledged that our users should not have to understand our internal, institutional structure (such as collection department names or data field terminologies) to start exploring the collections. We worked to find a way to pull together departmental peculiarities by finding the fundamental links between them which could create a broader context between departments. This was the principle behind choosing to use linked open data (LOD). Our collections are made up of a network of these connections that is only evident when you break the information down into the sum of its parts. Using LOD triples we created a network that gave a permeable collection data overview, comprehensive for a single collection approach but which did not interfere with the day-to-day running of unique collection disciplines. We achieved this by publishing our collection through an API (application programming interface) that followed the principles of LOD to overlay the arrangement of the data, rather than to integrate with it.
This process allowed us to store our information as a giant network in which any piece of data is be connected to any number of others. To create the links we used the CIDOC ontology, which is a semantically rich system that delivers data harmonisation based on analysed contextual relationships. It allows for our diverse and variable data.
The continuing focus of Collections Online is the “linked” part of LOD, to provide the bricks that allow people to build their own pathways through the collection, to make their own connections, to discover the stories that are relevant to them. By opening the data and connecting to external sources, people and resources around the globe, we embed them in other research ecosystems and encourage an active digital engagement with our objects. This not only benefits the user but it also enriches our collection data and so the cycle may begin again, ever constant and ever changing.
As soon as the Collections Online approach was comparatively mature, the Museum began an ambitious, large scale programme of work to make the decades of backlogged collection objects discoverable and usable, anywhere in the world.
