Is Long-Term Digital Storage Really Possible?
Looking for reasonable suggestions to this conundrum…
I was talking to a friend yesterday who was struggling with the requirement to store certain volunteer and activity records “indefinitely” for their non-profit organization. This is required to retain evidence to defend any false allegations in the future against any of their volunteers. It is an insurance requirement.
This raises two obvious questions:
- How long is “indefinitely”?
- How do you store records that will be findable, retrievable and usable over decades of decay and/or technological change?
For the duration question, since volunteers can start at age 15, they have settled upon the need to retain these selective long-term records for 100 years (ages 15–115). Okay, fine.
However, the method of storing this information can be very problematic.
If the organization wants to store paper records, then they have to consider that paper, ink, etc. decays and becomes less readable over decades. The contents also should probably be secured in a fire and water resistant mini-safe, possibly two copies, one safely on-site and a second copy safely off-site (if they wish to avoid a single point of failure in only having one copy). But will such very old, brittle, severely faded paper and ink documents still be readable after many decades or almost a century?
If the organization wants to store digital records, but they want to store a physical copy of the digital data on an external disk drive, USB stick, or other media in a mini-safe, then this has its own issues as follows:
- Will there be technology around in 50+ years that will still be able to connect to the physical drive’s connector, to the USB stick’s connector, or the other media’s connector? Will a physical plug or jack connection still be possible? Does anyone still have the ability to read 8", 5 1/4" or 3" floppy disks? If yes, will this organization of volunteers have such by-then-ancient technology available in working condition if/when it’s needed for evidence retrieval in the future?
- If connection is possible, then will the file system structure still be compatible with future technology from half-a-century in the future? Will such future tech be compatible with the media, its file system and its data storage formats?
- If so, what about the individual files stored on the media. Are they in document format, spreadsheet format, or other application formats? Will Word, Excel or other specific file formats still be readable in 50–100 years in the future when much more modern and sophisticated technology will be available that will dwarf all of our current technological wizardry and data formats that we currently think are so wonderful?
- What about decay of the storage mechanism used in the physical media? For example, if magnetic technology is used to store the data on a physical disk drive, such magnetic storage is not long-term and does decay (become unreadable) over time, sooner if it ever comes in contact with another magnetic field (e.g. a magnet) even once over the decades. What about optical or other storage mechanisms over time? In general, will the mechanism used to do the actual physical storage still have its efficacy after almost a century of sitting in a damp basement in a room which is not temperature or humidity controlled for so long?
So, a few challenges to overcome…
But you say, “Just store it online in the cloud. Don’t retain a copy of the digital data on physical media. Might that work?”
I don’t know…
What cloud storage service will still be available in 100 years?
Online services have been plagued over the past few years with vulnerability exploits by hackers that expose usually millions of people’s login credentials and their data, per exploit. Would such planned confidential information be safe online?
What if the online copy is the only copy and then an accidental deletion takes place? What about automated backups of these online cloud services? If there aren’t backups and the only copy is lost or deleted, then what? If you do have backups but they only retain previous backup data for say three months, but you don’t notice the accidental deletion until your annual update of the archive data and the lost data no longer exists on any available previous backup cycle?
Call me paranoid, but I’ve worked in the computer technology, systems integration, and software development industry too long to believe that the best plans or optimal scenarios ever really take place in real life. It’s usually some combination of various “oops scenarios” that take place when the planned period of time is measured in decades and the workers are a combination of busy staff and volunteers that change over time.
So what’s the solution for my friend?
She prefers to only use digital records, even scanning any paper source documents to get them into digital format. So she doesn’t need storage of paper documents.
How does she put together a strategy that will be compliant with the long-term records retention requirement of the insurer and such that the retained digital records will actually be findable, accessible, readable and usable many decades in the future?
What would you suggest?
Thanks in advance for considering it or providing feedback — And give it a like if you found the issue interesting :)