Walking, Talking Hardened Interfaces (i.e. How Amazon can fix health care): February 18, 2018 Snippets
As always, thanks for reading. Want Snippets delivered to your inbox, a whole day earlier? Subscribe here.
This week’s theme: How one of the greatest blog posts in tech history helps us understand what’s wrong with health care. Also, Glooko’s Mobile Insulin Dosing System is cleared by the FDA.
Welcome back once again to our Snippets series on Cost Disease, where we’ve been learning about why certain industries like health care get stuck in escalating cost spirals. In the past several weeks, we’ve learned about several potential causes, like Baumol’s Cost Disease and Bowen’s Syndrome, that might explain how we get stuck paying more every year for a college degree, an emergency room visit or a mile of subway track. Health care is an especially difficult industry to understand, given the opacity and complexity of the many intertwining layers and relationships that go into providing medical care. Whatever’s afflicting health care, there’s probably no easy way to fix it: without some major shock to the system or outside force, the current status quo seems destined to proceed on its current trajectory.
Amazon’s latest ambition: to be a major hospital supplier | Melanie Evans & Laura Stevens, WSJ
Meanwhile, during that same time period we’ve now seen two health care-related announcements from everyone’s favourite Disruptor-in-chief and Incumbent-scarer: Amazon. At first blush, Amazon entering the health care market makes sense at the level of, “well, if their ambition is to sell us everything that we consume, health care certainly is something we consume a lot”. And as we think about it more, there may be a deeper reason why Amazon actually has a “Magic Bullet” — or, if you prefer, a Bitter Medicine — that could be just the thing our health care system needs.
Remember two weeks ago in our issue where we talked about the phenomenon we called “Post-it Note Syndrome”, where we identified a particular kind of interaction problem that drives the complexity escalation we find in hospital systems? Here’s the problem: in complex environments where something is urgently at stake (in our case, the life of a patient), system participants fall back on less formal, more “primitive” interfaces and forms of communication in order to get the job done by any means necessary. (Recall an example from David Chapman’s post: “She needs lymphedema treatment, and the only lymphedema clinic around here is booked months in advance, but I know someone there, and I think I can get her in next week.” If you’ve ever heard something like that before, that’s the kind of “falling back on less formal methods of communication” we’re talking about.) These kinds of working around the system for short term gain, but creating long-term pain, compound as you add more parts and stakeholders: medicine, with its huge complexity and also its urgent short-term needs (“take care of the patient, now”) is rife with this problem everywhere.
Now, Amazon’s Silver Bullet for fixing health care isn’t scale; it isn’t logistics; it isn’t IT management. It’s because they’re the best company at the world at building something called Service Oriented Architectures, both in AWS and all across the company, and they could be the bitter but necessary medicine that our health care systems need to escape cost disease. In order to understand what we mean by a “Service Oriented Architecture”, why health care needs that approach so badly, and why it’s so difficult to pull off, we need to do two things. First, hold in our minds the particular nature of the “Post-it-Note” problem of interaction in complex environments. Second, we’re going to call on one of the great blog posts of recent tech history: the affectionately dubbed, “Stevey’s Google Platforms Rant” by former Amazon and Google engineer Steve Yegge.
The point of this post, when Steve wrote it in 2011, was to explain to his Google colleagues why even though Google did many little things better than Amazon, Amazon had one really important core competency that would come to define the their next decade and beyond. (The foresight in this post, especially around AWS, turned out to be completely accurate.) Although he’s talking about the internal IT systems that allowed Amazon to sell books, as you read this excerpt, think about what he’s really saying from the lens of those complex health care systems we talked about. Here’s Steve:
“One day Jeff Bezos issued a mandate. He’s doing that all the time, of course, and people scramble like ants being pounded with a rubber mallet whenever it happens. But on one occasion — back around 2002 I think, plus or minus a year — he issued a mandate that was so out there, so huge and eye-bulgingly ponderous, that it made all of his other mandates look like unsolicited peer bonuses.
His big mandate went something along these lines:
- All teams will henceforth expose their data and functionality through service interfaces.
- Teams must communicate with each other through these interfaces.
- There will be no other form of interprocess communication allowed: no direct linking, no direct reads of another team’s data store, no shared-memory model, no back-doors whatsoever. The only communication allowed is via service interface calls over the network.
- It doesn’t matter what technology they use. HTTP, Corba, Pubsub, custom protocols — doesn’t matter. Bezos doesn’t care.
- All service interfaces, without exception, must be designed from the ground up to be externalizable. That is to say, the team must plan and design to be able to expose the interface to developers in the outside world. No exceptions.
- Anyone who doesn’t do this will be fired.
- Thank you, have a nice day!
Ha, ha! You 150-odd ex-Amazon folks here will of course realize immediately that #7 was a little joke I threw in, because Bezos most definitely does not give a shit about your day. #6, however, was quite real, so people went to work. Bezos assigned a couple of Chief Bulldogs to oversee the effort and ensure forward progress, headed by Chief Bear Bulldog Rick Dalzell. Rick is an ex-Army Ranger, West Point academy graduate, ex-boxer, ex-Chief Torturer slash CIO at Walmart, and is a big genial scary man who used the word “hardened interface” a lot. Rick was a walking, talking hardened interface himself, so needless to say, everyone made lots of forward progress and made sure Rick knew about it.
Over the next couple of years, Amazon transformed internally into a service-oriented architecture. They learned a tremendous amount while effecting this transformation. There was lots of existing documentation and lore about SOAs, but at Amazon’s vast scale it was about as useful as telling Indiana Jones to look both ways before crossing the street. Amazon’s dev staff made a lot of discoveries along the way. A teeny tiny sampling of these discoveries included:
-Pager escalation gets way harder, because a ticket might bounce through 20 service calls before the real owner is identified. If each bounce goes through a team with a 15-minute response time, it can be hours before the right team finally finds out, unless you build a lot of scaffolding and metrics and reporting.
-Every single one of your peer teams suddenly becomes a potential DOS attacker. Nobody can make any real forward progress until very serious quotas and throttling are put in place in every single service.
-Monitoring and QA are the same thing. You’d never think so until you try doing a big SOA. But when your service says “Oh yes, I’m fine”, it may well be the case that the only thing still functioning in the server is the little component that knows how to say “I’m fine, Roger Roger, over and out” in a cheery droid voice. In order to tell whether the service is actually responding, you have to make individual calls. The problem continues recursively until your monitoring is doing comprehensive semantics checking of your entire range of services and data, at which point it’s indistinguishable from automated QA. So they’re a continuum.
…
This effort was still underway when I left to join Google mid-2005, but it was pretty far advanced. From the time Bezos issued his edict through the time I left, Amazon had transformed culturally into a company that thinks about everything in a services-first fashion. It is now fundamental to how they approach all designs, including internal designs for stuff that might never see the light of day externally.”
Reading Steve’s story about Amazon’s abrupt and painful transition from being an online bookstore into an infinitely extendible service platform, we get a sense of appreciation and awe for how hard it was for Amazon to rebuild itself around this new architectural philosophy, starting with a seemingly boring foothold like selling books. But ever since they came out the other side of that transition, they have a solid platform to stand on as they then enter industry after industry, rearranging them around this new service-oriented approach. These type of standardized, service-oriented interfaces between different functioning parts of the health industry will be hard-pressed to find an initial foothold: it’s simply not how the industry has ever been organized, having grown out of the individually crafted practice of medicine into the sprawling networks they’ve become. David Chapman even pointed this out in his piece we read the other week — even any kind of standardization of the way patients and their loved ones could interact and communicate with the medical system would be a win. To get it working at Amazon’s scale, not just between patients and doctors but between every part of the system and every other? Well, if anyone can pull it off, Jeff Bezos and his team can — and their natural starting point just might be something boring, like selling medical supplies.
Next week, we’ll hear from Social Capital venture partner and guest Snippets author Kristen Baker Spohn, who’ll share some of her thoughts on Amazon entering the medical playing field and what that means for the health care world going forward.
The world of synthetic biology, solving problems and laying new ground:
Alzheimer’s Disease reversed in mouse model | Gen Eng News
CRISPR hack transforms cells into data recorders | Heidi Ledford, Nature
In the future, will we be growing fruit in home bioreactors? | Kat Eschner, Smithsonian Magazine
Cryptocurrency protocols wade cautiously into multi-layered scaling solutions:
Dogecoin is helping Ethereum solve its biggest issue | Alyssa Hertig, Coindesk
Resurgence for a new century:
Forward thinking:
Five More Years: some predictions | Scott Alexander, Slate Star Codex
Why competitive advantages die | Morgan Housel
Train PhD students to be thinkers, not just specialists | Gundula Bosch, Nature World View
Other reading from around the Internet:
The changing face of urban air pollution | Alastair C Lewis, Science
Instead of filling cavities, dentists may soon regenerate teeth | Ferris Jabr, Scientific American
Burn Out: inside Faraday Future’s financial house of cards | Sean O’Kane, The Verge
Confessions of a startup mole in a dumb big company | Saumil Mehta
The Homepod, and the next step in home audio | Steve Crandall
The unsung songwriters who helped make Appetite for Destruction a classic | Matt Wake, LA Weekly
And just for fun:
In this week’s news and notes from the Social Capital family, we have some exciting news to share from Glooko. After 18 months of research studies, feedback and clinical scrutiny, Glooko’s Mobile Insulin Dosing System application has been cleared by the FDA:
Glooko’s Mobile Insulin Dosing System — MIDS — is FDA cleared | Michelle Dehaaff, Glooko
Why is this a big deal for patients? Well, some background information might be helpful for anyone interested in diabetes management who doesn’t have direct experience with the day-to-day struggle that it can be. People with Type 2 diabetes take Long-Acting Insulin (LAI), which helps manage your metabolic rate. Since our metabolisms vary on a day-to-day basis, that means the dose of LAI taken by patients isn’t always the same each time. It requires not only taking accurate readings of your blood glucose on a regular basis (which, as a patient, you should be doing anyway) but then taking those readings and doing a series of calculations to determine your LAI dose. That’s a lot of work, and it’s not just an unnecessary annoyance — i’s also rife with potential for errors, lapses, and unnecessary risk for the patient and their treatment plan.
Glooko’s MIDS fixes this problem entirely. With MIDS, Glooko’s diabetes management platform can automatically integrate a patient’s daily glucose readings with their doctor’s measured Fasting Glucose Level (and other measurements that you get at the doctor’s office), monitor ongoing levels, and then automatically compute patients’ LAI doses for them. One might think: “that’s great, but isn’t that a quick feature that could be added in a few weeks?” Well, in the software world maybe, but absolutely not in health care. Any time that software is going to replace something done by a human, you need to go through a rigorous process with the FDA to make sure of two things: that the tool does what we say it does, and that there are no secondary consequences that we didn’t anticipate ahead of time.
Glooko Outcomes: mobile health tool improves Glycemia in people with Type 2 Diabetes
During this process, the Glooko team and the clinicians they worked with obtained some very encouraging results. In a group of patients who were given Glooko’s mobile health app as a daily dosing solution, patients’ mean blood glucose readings, which had been on the high side of safe prior to the study, were brought fully under control when MIDS was available as a tool:
And second, the rate of hyperglycemic readings — or instances where patients took an individual reading that showed a blood glucose reading that was too high — came down from over 35% of all total readings to less than 10%, on average. That’s a huge difference, which when introduced to the broader population will meaningfully save lives and improve health outcomes.
So what’s next? We can expect MIDS to get rolled out to Glooko and their partners’ 1.5 million patients and 7,000 provider sites, which will have an immediate impact on people’s lives and health. As Glooko’s suite of products becomes more integrated and embedded into medical systems and service providers, we’ll really get to see something special happen as the best parts of the software world begin to really change the way that chronic conditions like diabetes are managed, controlled and prevented. If you’re interested in career opportunities at Glooko, they’re currently hiring for product and software engineering roles in Mountain View, as well as Sales in Chicago — please forward along to anyone you know who might be interested. Congratulations to the Glooko team on this milestone, and for all their work helping to solve one of the world’s hardest problems.
Have a great week,
Alex & the team from Social Capital
