Why I Believe My Theranos Blood Test Was Inaccurate And Useless
That’s a picture of my blood in two small vials — which Theranos calls “nanotainer’s.” I had ordered the blood test at a local Theranos clinic shortly after the company announced their “FDA approval” this last July. The FDA announcement coincided with a legislative one here in Arizona that now allows consumers to buy blood tests directly without a doctor's order. The two announcements this year were an interesting development and the high-profile startup had elected to make Phoenix their launchpad through a partnership with Walgreens. The map of Theranos locations inside Walgreens here in Phoenix is impressive — and included one very near to my home.
It was a long road for Theranos to get to those announcements in July. The company was founded by Stanford drop-out Elizabeth Holmes in 2003, but it was a long arduous sequence of events that opened a much larger door into the realm of consumer blood testing. The new Theranos experience was low-cost, almost pain-free and retail convenient. The only remaining question — was it accurate?
First up on that trajectory was the closely guarded technology that allowed for a micro sample of blood to be collected in “nanotainer’s” with a simple finger-stick. The equipment to run the actual blood test was called “Edison” and was invented by Theranos. Next was a business decision to make the tests low-cost and affordable. Another major coup was the retail partnership with Walgreens that put the availability inside a major retail healthcare footprint with pharmacy hours. Finally, there was the work with Arizona legislators to remove any legal barriers that prevented consumers from ordering tests at will without a doctor’s order.
My own experience was generally positive, pain-free and low-cost. In fact, the cost was so nominal I forgot to use my HSA card (which I could have) to pay for the single compound test. I wrote a fairly positive and detailed review of my experience for Forbes.
Given the FDA announcement earlier that month (July), I assumed the test had been reviewed and approved by the FDA. I like to think of myself as a cautious and informed healthcare consumer, so the FDA imprimatur was an important factor in my decision to try the revolutionary service. I’m not an attorney, but the wording the company used to announce their FDA approval appeared fairly broad.
Theranos, Inc. (Theranos) today announced that it has received the U.S. Food and Drug Administration’s (FDA) clearance of its test system and test for herpes simplex 1 virus IgG. The FDA’s decision provides independent validation of Theranos’ patented finger stick and venous blood testing technology and the ground breaking Theranos System upon which the HSV-1 IgG test is run, as well as the approach that the company has supported for FDA review of Laboratory Developed Tests (LDTs). Theranos press release — July 2, 2015
In the end, however, it didn’t matter all that much to me because I wasn’t really in need of clinical accuracy. I was simply a curious shopper who wanted to try the experience of ordering a simple blood test without a doctor’s order. Theranos made it easy and cheap for me to test the service, but I did conclude my favorable review with this disclaimer.
A great consumer experience doesn’t resolve the clinical controversy surrounding the company, of course, but it absolutely does have to behave like a retail, consumer experience.
The first rule of any consumer retail experience, however, is trust — and based largely on that vaunted FDA approval, I trusted Theranos to deliver accurate results for my one, fairly common blood test. I now have reason to believe my results were totally inaccurate. I’ll never know with certainty, of course, because it’s not possible to recreate a blood sample taken in July for comparison through another service.
Beyond the scientific challenges are the new revelations about the company that openly allege falsifying a critical process for every lab that handles blood tests. It’s called proficiency testing and are designed to independently verify results 3 times every year. Turns out there’s a pretty likely reason why the company emphasized the FDA approval side — and completely ignored the proficiency testing. They may have been trying to manipulate the verification testing by using different equipment for the proficiency testing process.
By now, many have already read the allegations by John Carreyrou — an investigative reporter with The Wall Street Journal published last week.
The Wall Street Journal is pay-walled, so I’ll summarize the relevant and troubling allegations.
Some employees were leery about the machine’s accuracy, according to the former employees and emails reviewed by The Wall Street Journal.
Other doctors said they stopped steering patients to Theranos because of results they didn’t trust. “I don’t want my patients going there until more information and a better protocol are in place,” says Gary Betz, an internist in Phoenix.
In early 2014, Theranos split some of the proficiency-testing samples it got into two pieces, according to internal emails reviewed by the Journal. One was tested with Edison machines and the other with instruments from other companies.
The two types of equipment gave different results when testing for vitamin D, two thyroid hormones and prostate cancer. The gap suggested to some employees that the Edison results were off, according to the internal emails and people familiar with the findings.
In March 2014, a Theranos employee using the alias Colin Ramirez alleged to New York state’s public-health lab that the company might have manipulated the proficiency-testing process.
Manipulating proficiency‒testing by a lab is a very serious allegation. If those tests (done at regular intervals throughout the year) were manipulated, Theranos certainly won’t be alone. In 2013, an Ohio State University run lab was fined $268,000 as part of a settlement with CMS around proficiency testing allegations.
The only defense I’ve seen to all the allegations was a hastily orchestrated one with Chairman, CEO and Founder Elizabeth Holmes herself on CNBC’s Mad Money with Jim Cramer — and she made no reference to proficiency testing at all.
That aired Thursday evening and a key quote for me was this one:
We’re not even using our nanotainer’s except for FDA cleared assays. Elizabeth Holmes
Ok — except that as consumers, I believe we have a legal right to know exactly when that decision was made. My test in July — which was not for the one “FDA cleared assay “— used the Theranos developed nanotainer technology. This is critical because the FDA considers that nanotainer a medical device and while they have approved it for use by Theranos for one test — a Herpes test — that was not the test I took. So my test was performed using a medical device that was NOT FDA approved.
The WSJ followed up Friday with an update that was equally troubling and damaging.
Food and Drug Administration inspectors recently showed up unannounced at Theranos, the person familiar with the matter said. The inspection was triggered by concerns the agency had about data Theranos had voluntarily submitted to the FDA in an effort to win approval for its proprietary testing methods, this person said.
During the inspection, FDA officials indicated to Theranos that the agency considers the “nanotainers” made and used by the company to collect finger-pricked blood an unapproved medical device, the person familiar with the matter said. Hot Startup Theranos Dials Back Lab Tests At FDA’s Behest
The questions cascade quickly.
- Did Theranos try to manipulate the proficiency tests in any way?
- Aren’t the proficiency tests — designed to verify accuracy and conducted throughout the year — actually more important than a one-time FDA approval (which has never been required in the blood testing industry)?
- When did Theranos stop using their trademarked nanotainers?
- Does Theranos have an obligation to notify consumers about how it’s using their revolutionary technology — and when — and for which test?
- Should tests that are FDA approved be indicated in some way on the Theranos website? Today it’s just a long list of all the tests that Theranos offers (with retail pricing).
- Since the nanotainer’s are no longer in use except for the one FDA approved test — isn’t there an obligation to scale back the full blown marketing around finger-stick samples using nanotainers? Other than low-cost (which may be heavily subsidized), there appears to be no difference or advantage to a Theranos test — and there’s appears to be a significant risk of accuracy.
- Which test equipment (of several available) is Theranos using at which facility and given that “Edison” is only approved for one test, shouldn’t that be indicated at the time of the consumer purchase?
- The science behind microfluidics — which Theranos is trying to leverage here — is really (really) hard. Shouldn’t there be a peer review process for this breakthrough technology before being made commercially available?
In fact, the whole Theranos process isn’t just opaque ‒ it’s intentionally opaque and contradicts the very “transparency” it so openly markets as foundational to its success. Different processes using different machines for different tests — only one of which is FDA approved — all of which is decided by the company at their discretion isn’t remotely transparent. Nor is it what I thought I was buying.
The company may well be in legal compliance with the FDA for the one test, but that doesn’t extend to all available tests. Maybe the FTC can help consumers sort out what’s actually being sold, what disclaimers are appropriate for consumers and when. The whole process can be frictionless and consumer friendly, but if it’s not accurate — it’s consumer fraud.
Appearing on CNBC’s Mad Money with Jim Cramer Thursday evening, Ms. Holmes’ opened with an odd defense against the allegations by the WSJ. It wasn’t remotely what I was expecting and sounded like it was straight out of an Aaron Sorkin script.
This is what happens when you work to change things ‒ and first they think you’re crazy, then they fight you and then all of a sudden you change the world.
That may well be a satisfying argument for many, but it sure wasn’t for me. I’m not a guinea pig for some healthcare startup to “change the world.” First and foremost — as patients — we all expect and need integrity that’s beyond reproach. Healthcare has enough criminal activity without adding to the pile.
As the lead into her much longer defense, her opening remark was the kind of framing you expect to hear from a very typical, overhyped Silicon Valley unicorn appealing to investors and the startup community. It’s certainly not the kind of framing for a company whose mission is to deliver accurate clinical data around life threatening illnesses to anxious consumers. It’s also not the kind of “transparency” that’s a key requirement for consumer trust around personal and sensitive healthcare issues.
I do have a very nicely designed PDF — that was delivered securely — with some numbers from the test I took in July. It may be suitable for legal discovery in a court action, but I have zero confidence in any of the numbers relative to the blood test I paid for. It’s easy to see why consumers feel betrayed by this company. I’m one of them.