Unifying our fragmented health data requires a common storage repository
As a data engineer, my job is to make data flow as smoothly and efficiently as possible from the places that have it to the places that need it. I love when all the pipes work and the data they transport generates real value. Like water flowing through the pipes in our walls and out our faucets, data is becoming a lifeblood of our existence. We interact personally with some of it, but most of it flows unseen. Of course, that hidden flow of data is no less critical than the stuff we can see every day — in fact, it is often the most crucial supporting structure of our modern lives.
Nowhere is this data more important than in our healthcare. It informs our doctors of our recent medical history, explains to insurance companies why we need that MRI, and underpins the research that gives us better treatments, both routine and extraordinary. And yet, thanks to the fragmentation of our healthcare system, our own health data is also fragmented. Ilana Yurkiewicz recently wrote a moving piece about how costly this fragmentation can be, to an individual and to our society.
Almost two years ago, I embarked on a side project combining my love of data and healthcare problems. My goal was to build a place where our full medical record could live, securely and privately, so that everyone would always have an answer to questions like, “What was that prescription I got five years ago?”, “What immunizations did I get for that international trip?”, “Where is that MRI I got?”, or even “What was the name of that doctor I saw last year?”
I wanted this repository to remain out of the control of any one entity, be it a large tech company, EMR vendor, or health system. Are you willing to entrust your past and future record to a single organization? I’m not, and I don’t think the rest of us should either. But while they should not control our access to our own data, these organizations need to transfer our data efficiently and securely across state lines, specialties, and institutions. I believed decentralized storage, a very new and emerging technology, would be appropriate for this repository. My medical record would be broken up into chunks, encrypted, and stored on many servers all over the country. Only I and those I give permission to would be able to decrypt those chunks.
After almost two years of working nights and weekends, an initial prototype is ready: Libri. Libri is a simple, performant, decentralized storage network focused on ensuring the data is always there and easily but securely sharable. It enforces end-to-end encryption on everything. It avoids a blockchain or any other distributed consensus protocol, making it easier to understand and maintain.
The individual peers in the decentralized system are called Librarians. The vision is that major healthcare organizations each run a set of these Librarian peers, giving them instant access to all the data stored in Libri that they have permission to see.
Libri is not the first attempt at decentralizing part of our healthcare system, but it is the simplest and most technically mature I know of. While its design focus is healthcare data storage and sharing, it actually has nothing healthcare-specific baked into it. It’s just a simple, performant decentralized storage network, occupying a spot next to other nascent (blockchain-based) decentralized storage efforts like Sia, Storj, Filecoin, and Ethereum Swarm.
For those that are interested in more of the technical details and performance tests, I’ve written a white paper. For those interested in the code, check out the Github repo. For those interested in helping test it, check out our public testnet.
And for those who just want to have visibility and control over their medical records, we’re working on it. My business partner and I are in the very early research stages of what we should build on top of Libri that would meaningfully improve the lives and medical care of everyday people.
Feel free to share your own medical data experiences in the comments if you like.
