An access service for a new archival era
The National Archives is opening a new door to our nation’s records. Keeping pace with a digitally-driven world, our nation’s archives are facing both the challenge and opportunity to develop an access service for born-digital records.
Born-digital records are becoming the new standard across government; however, currently there is no tailored solution for government teams to access these records after transfer to The National Archives. The National Archives has teamed up with Viapontica AI, a fast-paced and innovative engineering team, to uncover the opportunities and challenges in developing a sustainable, secure and effective solution for government access to born-digital records.
This blog discusses key takeaways from the discovery phase of this journey and presents highlights from the development roadmap.
Discovery — The Journey
Spanning eight weeks and 36 hours of video interviews, Viapontica AI led a discovery between October and December 2021. Our team spoke to 13 government departments — such as the Department for Education, Welsh Government, Foreign, Commonwealth & Development Office, Ministry of Defence, and Cabinet Office, to name a few — which have vastly different needs, use cases, and which present different challenges for remote access to born-digital records.
Many services at The National Archives were not designed to face the challenges and the scale of working with born-digital records. We kicked off the project by asking ourselves how we can best complement existing services which so far have not been tailored to present-day challenges.
We talked to a range of stakeholders, such as Departmental Record Officers, who shared with us when they would need to access their records and how they would like to do so. The discovery process made clear the need to develop a service with specialised government access, to facilitate the secure requisitioning, sharing, and searching of born-digital records.
Born-digital records provide an opportunity to discover as yet unrealised features and tools within a service which can aid government departments and The National Archives. An access service can provide faster access to more records, allowing departments to use transferred records more regularly in the development of policies and the delivery of services both locally and nationwide.
Discovery — The Findings
After making the decision to develop a service tailored to government departments, the user research team gathered information to validate the need for the service and highlight the challenges born-digital records will present when creating an access service. These are:
1. Departments are embracing remote working
In light of COVID-19, government departments had to adjust and adapt to agile ways of working and were required to fully access records remotely, whether paper or born-digital.
During the pandemic government departments used Egress to remotely view their paper records. Government departments saw benefits from remote viewing, however, manual scanning and sharing of records means this is not a scalable solution in an increasingly digital age. Born-digital records present an opportunity to scale digital remote viewing for government departments.
2. Departments require tailored solutions
Twenty-three ministerial departments, 20 non-ministerial departments and 415 agencies all operate within government but in unique ways. Some departments require access to The National Archives only a few times per year, whereas others require their team to travel to the archives on a weekly basis. A fine balance is required to accommodate these different needs.
3. Some departments are years away from digital transfer
While some departments have begun digital transformation, others are years away. No two departmental digital storage systems are the same and record teams are dealing with vastly different scales of digital records.
The Viapontica AI team documented common needs and use cases from departments who are close to or have begun digital transfer. Departments that haven’t begun transfer were able to draw from experiences transferring paper records and share their potential access needs. Initialising this project now allows The National Archives to create a fine-tuned service with a deeper understanding of future needs.
4. Born-digital record sensitivity requires care
The new access service would provide more access to records than ever before, but wider online access brings digital risks into the picture. Departments cannot predict the future, meaning unforeseen circumstances can alter the sensitivity of a record in an instant. Widespread online access to records could enable bulk analysis techniques, such as data-mining, potentially revealing further sensitivities. The service we are developing seeks to initialise protocols and features to mitigate the changing and increasing risks.
Gradation of access to records, which is being proposed by The National Archives, is a natural step in reducing mistakes and addressing risks. Gradation is by no means limiting access to information, rather establishing secure and sustainable practises that will keep the nation’s information safe. The extent and presentation of gradation is undergoing further research and testing, and is a focus of the team in the alpha stage.
Discovery — The Outcome
The scope and aims of this project will continue to evolve moving to alpha. As we continue to build on our initial discovery, two defining features have become clear.
Access for government departments
To help departments’ business continuity and adapt in alignment with their transformation strategy, time and focus must be placed on the government’s unique needs.
Records are a tool which can be used by departments to provide evidence to past decisions and inform future ones. The new access service allows the potential to enable wider access to records so that they are used to their full potential — balancing the genuine need of departments to protect sensitivities for a limited period of time, with the right of the public to know how their government works.
Access for born digital
Born-digital records set this project apart from others within The National Archives, as the creation of an access service for government departments offers the ability to test and develop solutions for records which have not been the focus of other projects. In addition to helping departments, the access service is laying a foundation and establishing precedents for future projects that deal with born-digital records.
A lot about the future of the access service is still being decided; the service could be standalone or a part of another service within The National Archives. Our aim, therefore, is to build from current knowledge within The National Archives and develop a solution that can be used in many forms for decades to come.
Next Steps
We are continuing to ask questions and look for answers, such as:
· What level of gradation is appropriate?
· What is the role of machine learning in areas such as managing record sensitivities and protecting personally identifiable information?
· How do we best compliment other services within The National Archives and work collaboratively across government?
The research team is talking at the moment to more government users and staff at The National Archives and government departments to join efforts and tackle the big questions. Moving into alpha, the user research team is adopting iterative testing with government users to fine-tune features and build a sustainable, secure service.
The access service allows The National Archives to set a precedent for the management of born-digital records. As the project is progressing at the moment, our team is looking forward to sharing more and keeping you up to date.
