Thoughts on Building a Lifetime Personal Data Collection App

Creating an application designed to be a central repository to store and display our life’s collection of personal data is a monumental task. Having to factor in the ability to integrate external and cloud based data sources, that will most likely change over time, in addition to our local data makes it even that more complex. Ensuring that the application and our data will persist beyond our lifetime and into the foreseeable future is yet even harder to imagine.

I’ve seen several digital legacy based companies emerge that provide ways to preserve some aspects of the memories of people who have died. These companies are limited in scope usually to photos and a few other data types but I’m glad to see them bring attention to and provide this need. Entrusting our personal data to a single company to manage it requires immense trust and hope that the company will be around for a long time. Eventually future custodians of our data will need to maintain our data for us and in turn move it to other locations as necessary. I believe that the best way we will be able to achieve creating such a monumental application is for it to begin as an open source project.

How Do We Do It?

An application built for the purpose of housing our life’s personal data both stored locally and in the cloud needs to be extremely flexible and built in a way that can be customized with individual components to accommodate the multitude of data types and the ways a person would want to explore it. I believe that building a comprehensive software package for creating a personal data legacy could be quite difficult and everyone’s needs will be different. This type of software should have a very similar architecture to an open source content management system such as WordPress or Drupal in that it could have some core functionality that would then be supplemented by plug-ins or modules for specific features.

One example would be the integration of photographs. This would require one set of custom plugins designed to provide the ability to integrate both local data as well as cloud based services such as Google, Flickr, Instagram, etc). Then a separate set of plugins could provide the display layer of that data (such as galleries) as well as added functionality such as search or location based features. Plugins would be available in a similar way for other data types such as documents, emails, video, audio files etc.

Then in much the same way that Automattic is a professional services company for the WordPress open source project, and Acquia is the professional services company for the Drupal open source project, an ecosystem of for profit companies could be built around this application to mitigate any concerns around a single company owning such a critical application and ensuring its future support and evolution.

This Software Was Almost Realized

A few years ago it appeared that we were going to have such an application developed with the arrival of The Locker Project. I was monitoring this open source project in the early days and it looked very promising. One of the first areas they tackled was creating data connectors both to import and publish data to and from the application. This proved to be a fairly time consuming process having to develop these connectors that communicated differently with so many web services and their API’s. This part of the project was being developed by a for profit company called Singly which also was the corporate support partner for Locker Project. The Locker Project seemed to be making some great strides helped greatly by Singly’s services. It was a great solution for other companies by offering them a great reduction to development time and costs allowing to rely on Singly for API connections to so many 3rd party services for them.

Unfortunately with the success of Singly it seemed like much of the momentum of The Locker Project began to languish and eventually Singly was acquired by another company and The Locker Project development eventually stopped. In fact in writing this post I discovered that apparently the domain wasn’t renewed and has now been taken over by a squatter.

I had interacted a few times with developers from the Locker Project with hopes if it being revived but nothing materialized. Here’s the last interaction I had with them

more conversations are available here

Homebrew Method

I’ve spent a considerable amount of time coming up with processes and cobbling together various pieces of software to try and create a comprehensive way to collect, backup and display all of my personal digital data. I’ve created a guide that provides three crucial steps to provide a roadmap for others as well. I keep searching for ways to improve this process along with the ways I catalog and review my data. My hope is to eventually have a single centralized application to manage most of this for me and eventually provide a simple way to both allow me to reminisce my own memories and provide an easy way for family and future generations to view it long after I’m gone.

Moving Forward

I’d love to see a stronger focus and more of a discussion around the topic of preserving and referencing personal data storage, analytics. I feel that most people aren’t thinking about how they will pass along their personal data that continues to grow in detail and size as so many aspects of our life go digital. I came up with a base plan that is useful to get started and this is something we should all be thinking about. If you have written or are aware of anything related to this topic in the way of software or methods to achieve some of what I’ve discussed please share them in the comments.

please consider sharing this story with others that might find it of value

Originally published at