Open in app

Sign in

Write

Sign in

The Royal Academy Summer Exhibition: chronicle250.com

Matt McGrattan
digirati-ch
Published in
9 min readSep 13, 2018

Matt McGrattan, Head of Digital Library Solutions, Digirati.

The Paul Mellon Centre for Studies in British Art digitised 250 volumes of the Exhibition Catalogue for the Royal Academy Summer Exhibition, from 1769 to 2018, and commissioned in-depth scholarly articles for each year of the exhibition to coincide with the 250th anniversary of the Summer Exhibition. Digirati were commissioned by the Paul Mellon Centre to build the online version of the project.

The resulting website can be found at: https://chronicle250.com.

Digirati were asked to develop the website from designs by Strick and Williams and to provide the supporting infrastructure for the site using the Digital Library Cloud Service (DLCS).

Requirements:

  • Each catalogue should be available online using the IIIF Image and Presentation APIs. See https://iiif.io for details.
  • Each catalogue should have searchable full text.
  • Exhibitors should be identified in the catalogue text and linked back, via hot-links on the images, to a searchable Index on the main Chronicle250 site.
  • Index entries for a given Exhibitor should link to all occurrences of that artist in the corpus of Exhibition catalogues.
  • Pages for each year with rich scholarly articles.
  • Index entries for authors and artworks.
  • Thematic indexes across the curated per-year articles.

In building the site Digirati:

  • Provided performant versions of the digitised catalogues and illustrations with deep zoom functionality and support for open APIs (https://iiif.io) using the DLCS.
  • Created OCR for these images, including 18th century catalogues with historic typefaces.
  • Identified exhibitors within the catalogue text and associated exhibitors with regions of images to create hotlinks between the catalogue and the index.
  • Provided a usable search experience both within an individual catalogue and across catalogues.
  • Created a usable index of Exhibitors.
  • Brought the content — catalogues, indexes, scholarly articles — together following Strick and Williams’ design brief to create the Chronicle250 site.

The Solution

A more detailed technical version of this information can be found here.

DLCS

If we had been started from scratch with no existing infrastructure, and no existing code base, the Chronicle250 project could, potentially, have been very costly in terms of both time and budget.

However, Digirati provide a hosted cloud based service, the DLCS, designed to be run as a multi-tenant service shared by users who may be unable to, or may not wish to, run their own image hosting infrastructure. The DLCS uses the IIIF APIs, and is based around open standards, so new projects can be built easily on top of the DLCS. The DLCS can also be optionally enhanced with additional services that can enrich content with tags, transcriptions, and search.

The use of the DLCS was a key requirement for this project, as the existence of the DLCS made many of the core functions required for the site do-able without a large amount of infrastructure work or basic software development. Development time, and thus the budget, for this project could concentrate on front end development and enhancements to existing DLCS services around annotation and natural language processing, and not on core image hosting or text processing and indexing functionality.

The DLCS provides services which:

For the Chronicle250 we were able to use the shared multi-tenant services as-is and then customise the project specific services for Chronicle250 to provide the enhancements we needed to identify, link, and index exhibitors in the digitised versions of the exhibition catalogues.

OCR Services

The catalogues for Chronicle250 span 250 years of Royal Academy exhibitions, which introduces particular demands around OCR quality, as the historic typefaces used are not, typically, OCR’d well by off-the-shelf open source OCR engines like Tesseract or Ocropy. In addition, segmentation of images into blocks, paragraphs, and lines is also difficult because the text is often quite heavily skewed with bleed-through from verso pages, and uneven kerning introduces erroneous whitespace throughout.

We evaluated a number of OCR engines, including:

The DLCS already had integrations for Google Vision and Tesseract, and we found that Google Vision scored well compared to other cloud-based services from Microsoft and Abby, and scored significantly higher than Tesseract. A range of typefaces is used throughout the 250 years of catalogues, so doing specific training of Tesseract with glyphs from particular catalogue years would not have scaled well across the entire project, and would have introduced significant additional demands on staff time for results that would not exceed the cloud-services which could be used immediately.

We were able to use the existing DLCS OCR services as-is to do text extraction and normalisation of OCR text without significant customisation for this project.

Natural Language Processing and Named Entity Recognition

The DLCS has a named entity recognition service which uses IIIF, Spacy.io, and W3C Web Annotations to tag regions of images with people, places, dates, organisations, and other classes of entity.

We evaluated the use of this service using off-the-shelf neural net models untrained on the Royal Academy corpus, and found that the overall quality of tags produced was not acceptable in terms of the number of artists correctly identified, and in terms of the number of falsely identified non-artists.

A typical catalogue page might contain entries that look like:

And also, other pages within the same volume that look like:

We had to identify the artist names on each page, but also identify when different occurrences of a name within the catalogue were references to the same artist. Note the different forms in which an artist’s name might appear.

To improve the results, we:

  • Wrote code that parsed known sources of artist data, from: Getty Union List of Artist Names (ULAN) ; Lists of Royal Academy Academicians provided by the Paul Mellon Centre; Lists of Exhibitors (comprehensive until 1990) also provided by the Paul Mellon Centre.
  • Generated variant forms of these artist names so that the system correctly identified that J. Northcote, R.A. and Northcote, James, R.A. were the same person, and identified that this James Northcote was the painter who lived from 1746-1831.
  • Wrote code to handle (by normalising and/or ignoring whitespace) the kerning and segmentation issues with historic text.
  • Wrote code to filter artsts by date, to ensure that only the relevant artists for a given catalogue year were in the “pool” for tagging.
  • Used the Aho-Corasick algorithm to do fast pattern matching of the OCR text with the known list of artist names.

This code was implemented as an enhanced version of an existing Digital Library Cloud Service (DLCS) service, so we did not have to write an entirely new software stack from scratch, and were able to take advantage of existing integrations with OCR services, and annotation servers (for storing the output as annotations on IIIF content).

IIIF Viewing components: Canvas Panel and the ‘PMC’ Viewer

In order to provide the results of the tagging process alongside the IIIF Image API images, Digirati built a bespoke IIIF Presentation API viewer for the Chronicle250 site.

Prior to the Chronicle250 project, Digirati had built a lightweight IIIF Presentation API Canvas viewing component, which supports annotation display called CanvasPanel, and which has been used on projects for the Victoria and Albert Museum, such as their Ocean Liner’s exhibit.

For the Chronicle250 project, we took CanvasPanel and added additional support for:

The PMC viewer can be found on Github at: https://github.com/digirati-co-uk/pmc-viewer

Search and Indexing

The full DLCS (Digital Library Cloud Service) provides a IIIF Content Search Service Mathmos which integrates with the DLCS message bus, and indexes both full text (provided by OCR) and annotations (provided by machine generated tags).

However, for the Chronicle250 project, the vision was not to rely on the DLCS for on-going delivery of textual content or services to the viewer. The DLCS text pipeline could be shut down after processing, leaving just the Chronicle250 website/application, and the DLCS IIIF Image API and IIIF Presentation API services running as active services. In addition, the IIIF Content Search service on the DLCS provides basic/generic search services which would not fulfil the full requirements of the Chronicle250 site.

Instead, for Chronicle250 we built a bespoke Elasticsearch based index which contained:

And which provided both the IIIF Content Search to the PMC Viewer, and also general search services and indexing on the main Chronicle250 site.

Overall Results

The machine identification of exhibitors across the corpus was extremely successful given the relatively short time spent on bespoke software development and R&D.

We were able to successfully identify 318,690 exhibitors across the catalogues. An upper bound for the possible maximum number of exhibitors, assuming each exhibitor only exhibited once in each catalogue, would be 513,068, however, given how commonly exhibitors exhibit more than once in any given year, we can assume the actual total is certainly lower. There was a very small number of tags for the post 1990 catalogues because we lacked any Exhibitor data for those years.

Using the techniques described in this article offered a very good return on time invested, versus the time it would have have taken to manually tag 400,000–500,000 names in the corpus. Combining these techniques with the services provided by the DLCS made a resource and data heavy project something that was possible to do in a relatively short timescale.

To measure the output, the Paul Mellon Centre produced statistics in Google Data Studio, to show the accuracy and distribution across the entire corpus. Click here to view the data.

Credits

Digirati

Adam Meszaros, Senior Frontend Consultant. Full stack developer. Chronicle250.com; Site indexes and IIIF Content search services; PMC Viewer; Integration.

Stephen Fraser, Front End Technical Lead. AnnotationStudio; CanvasPanel; PMC Viewer.

Matt McGrattan, Head of Digital Library Services. DLCS text pipeline; Natural language processing and tagging; Digirati Product Owner.

Adam Christie, Senior Engineer. DLCS Infrastructure; DevOps;

Ville Vartiainen, Senior UX Consultant. Digirati User Experience.

Ian Farquhar, Head of Project Delivery. Project Management.

Paul Mellon Centre

Tom Scutt, PMC Product Owner and Digital Editor.

Mark Hallett, Sarah Victoria Turner, Jessica Feather, Scholarly Editors.

Baillie Card, Publishing Editor.

Maisoon Rehani, Picture Editor.

Tom Powell, Sean Ketteringham and James Finch, Researchers.

Thérèse Saba, Copyeditor.

Jan Worrall, Indexer.

Strick and Williams

Charlotte Strick, Design.

Claire Williams Martinez, Design.

User Experience

http://www.unaffiliatedworks.com/

Machine Learning
Culture
Cloud Services
Iiif
Cultural Heritage

--

--

Matt McGrattan
digirati-ch

Written by Matt McGrattan

14 Followers
Editor for

Help

Status

About

Careers

Press

Blog

Privacy

Terms

Text to speech

Teams