Open Data-Driven Tools for Consumers: A Conversation with Charles Ornstein & Thomas Goetz
by Susannah Fox, Chief Technology Officer, HHS
Prescriber Checkup is a collaboration between ProPublica, a nonprofit journalism site, and Iodine, a start-up company focused on drug information. Susannah Fox, Chief Technology Officer (CTO) of the Department of Health and Human Services (HHS), recently talked with Charles Ornstein of ProPublica and Thomas Goetz of Iodine about how and why they created this data-driven tool for consumers — and where they hope the field of health data will go.
Susannah Fox: I wanted to open by saying thank you, because, as you know, a lot of what we do you in the HHS IDEA Lab is try to prove that there is a market for open data. And you both are doing a great job of showing how useful that data can be, showing also that the more data we open up to the public, the more interesting and unique uses that people find for it. So just to start, could one of you tell me where this idea came from? How did you decide that this would be a good partnership?
Charles Ornstein: Back in 2013, ProPublica had started a website called Prescriber Checkup, and the goal was to let people look up their own personal physician, then look to see how they were prescribing drugs in the Medicare Part D program, and how those prescribing patterns compared to others in the same specialty and state. The thought about this was that we really have no idea — as consumers — whether or not our doctors are prescribing the right drugs for us, and if they are in line with other people.
We turned the feature on in May of 2013, and we’ve been updating it for the past few years. Last year, Thomas and I got together and I said, “Wouldn’t it be great if we were able to include the amazing information that Iodine was pulling together on the drugs themselves, to give people sort of a second layer of background information. They can both get information about their doctor, but also get information about the drug.”
SF: And Thomas, can you talk a little bit about Iodine’s tools, and where you get the data?
Thomas Goetz: Our website is a combination of the data that we’ve collected from more than a hundred thousand consumers on their experience of medication and then the open data that we’ve extracted, mainly out of the FDA. Iodine has been the primary contractor for the openFDA project so we’re intimately familiar with FDA data and we saw a lot of consumer value to it. But we also recognized the need for people to understand not only what happens in the highly controlled world of a randomized clinical trial, but also what happens in the real world among real people. What is that experience like, and how do we help people make better decisions for themselves based on what has happened to others. In other words, as somebody has said, giving people access to information, but access to experience — if that rings familiar to you, Susannah.
SF: That sounds familiar, yes.
TG: We created this data set in a collaboration that we did but with Google back in 2014, and we have been able to get a lot of value out of it at our website. We are huge admirers of what ProPublica had been doing, especially in terms of bringing relevant, actionable data to consumers, and so we thought that it would be a great place to go into a partnership, and see what our data can do in the context of other data sets. For us it was kind of a no-brainer to give ProPublica our first license to an API.
One of the cool things about ProPublica is that it’s thinking about journalism not as a static product with a kind of single, pure truth to it, but as something that is always evolving and always flowing.
Rooting that type of journalism in good, real data is essential. Good data can translate into action-oriented, decision-oriented service journalism.
That is what is so powerful about the internet: it’s driven by a much more visceral connection between a need and a fulfillment of that need.
I think the future of journalism is tools. It’s something that ProPublica is doing, and that’s — to some extent — something that we’re trying to do in a different way at Iodine. Some of the stuff that we’re creating is tantamount to tool-based journalism, but it’s approaching it from the point of view of a question: what can the data tell us?
CO: But we also used Iodine’s data to write a story about how the brand-name medications that cost more money were not the ones rated highest in the Iodine data. I think some people would be surprised to know that just because you pay more for a drug doesn’t mean that consumers rate their experience to be better in terms of using it.
SF: That’s music to my ears because what we hope to do is expose as much data as possible to drive the information to consumers so they can make the best decisions possible. We do need to keep costs down in the American healthcare system, and yet we also want people to make the best choices for themselves. Knowing which drug gets the job done with the side effects that you can live with at a cost you can afford is part of that decision-making. The consumer voice is essential to understanding a woman’s experience with a drug versus a man’s, someone in their twenties versus someone in their sixties.
TG: One of the big things that we’ve learned in seeing people who use Iodine is that often times they’re already taking a medication, and they don’t know what to expect. The question becomes: What is happening to me? What should I expect? You are able to put your own experience in the context of other people’s experience and make an assessment: “Am I doing better or worse than people? Am I seeing the progress that other people like me have reported?” That, for me, is the next dimension. Again, we are not recording a static experience, but thinking about it as an active ongoing experience. Medicine is not an exact science. It is a hit or miss game. It is a game of trial and error. It is a game where doctors and patients are working really hard to try to figure out the best course of action, but there’s a lot of messiness in that. And helping people through that course I think is what so powerful about what ProPublica has created. It’s also the core mission of Iodine.
SF: We’re a few years into the open data revolution, and you have both been pioneers, living on the frontier. When I look at the Prescriber Checkup tool, I see data sets that have come from the federal government and now a private sector source of data. What is the current ratio of government versus private sector data for just this tool, and then what do you each predict for the future?
Where do you think the market is going in five or ten years? Do you think that as the government continually opens data that it will continue to be the field that you harvest from? Or do you see more possibility in bringing in the consumer voice and mining data sets that come from the private sector?
CO: I think that we are at stage two of evolution of the open data movement — the integration stage.
We are no longer building tools based on one single data source. We are looking to integrate a variety of data sources to make the information even richer.
What I think is discouraging is that the private sector has not really — at least from my experience — shown an interest in opening up data at the provider level. I think that there’s certainly willingness to open up national data or state data, but when you actually get to information about the way an individual doctor practices, this is really valuable information. It is information that companies like IMS Health and others can sell for hundreds of millions of dollars each year.
It is a business model in itself, and so freeing it up to make it publicly available, or available for an amount of money that a media organization could pay is just not something that we’ve seen in the cards yet — at least a willingness to work with us on this yet. Is that going to change? I don’t know. There’s a lot of money at stake — both for them and the health care system overall — but also in just the exchange of information. And so, unless other parties see a willingness to cooperate and to work together as Thomas and I have figured out a way to do, it’s tough to crack that open.
At the same time, I think if there were a lot more government data sets beyond those that just HHS holds that were available, there would be many more opportunities for unique collaborations like ours. So, right now we’ve largely been dealing with Medicare data. I think that there’s also Medicaid data that’s not been publicly made available, but which could and hopefully will be in the near future. There’s a lot of data that the VA (Veterans Affairs) system keeps, and hopefully we’ll see opened up in the not too distant future; there’s a lot of data that the Tri-care System — the military health care system — keeps, which hopefully we’ll see open up in the not too distant future; there’s data that the Indian health care system keeps; there’s data that the prison systems keep. So if you start adding up all of the government data sets, it’s far bigger than Medicare alone, and you’re starting to look at more than half of the US health care system as a whole.
I’m pretty optimistic that we’ll be able to find ways to open up other government data sets, but I’m less optimistic when it comes to the private sector.
TG: Yes. I’ll answer this question a little differently. So my experience — I don’t know what the ratio is at ProPublica, but at Iodine, the ratio is really about 50/50 between data that we’ve created that comes out of a consumer experience versus the government level data. The data we’re getting from the FDA or clinical trials is not unlike the way that government has created a safe layer for mapping technologies.
In many ways, the government health data that we use is the map. It’s an assessment of what the landscape looks like on a population level. And then what we’re trying to fill in is the personal experience, in some sense we provide the tracking from point A to point B, like the navigation app, Waze. We help people answer questions like: What is the most efficient route? What is the path that most people take? Where are the dead ends?
You can take the metaphor wherever you will, but I really think that is what is powerful about the combination of the government data with our data. The government provides this very broad, vast, rich set of data that is beyond the reach of a startup company to create de novo. But for us to build upon that and layer human experience on top of that map is where we start to get some of that richness and the potential for power here. I definitely encourage the government to open up data.
SF: Fabulous. Well, you know that we here at HHS believe in open data, and believe that the best thing we can do is open as many data sets as we can so that people can understand the healthcare system. The only way that we can improve this system is if we can measure it. And that is certainly true when it comes to people’s decisions about drugs. Thank you so much for talking with me.
To explore our open data work further, visit www.healthdata.gov