Part 3: Ethics and Experiments
This is the third article of a series called Lessons From 1 Year Of Experimentation.
How many Americans can afford to pay for an unexpected bill? The answer will be shocking to many. 40% of Americans would struggle to pay an unexpected $400 bill (Report on the Economic Well-Being of U.S. Households in 2017) and are living paycheck-to-paycheck, leaving them unable to cope with unanticipated expenses. The pandemic has almost certainly made this desperate situation far worse.
Many unexpected bills are medical in nature and it’s not hard to see why: a simple visit to the ER can easily leave someone with a $400+ bill, whether or not they have insurance. These sky-high healthcare prices just don’t align with the financial realities facing many Americans. The unpredictable nature of healthcare expenses coupled with high prices naturally leads to low payment rates, compared to other industries.
At Cedar, one or our core beliefs is that making healthcare prices more transparent and affordable will lead to better outcomes for providers and patients. In the third part of this series, we discuss our efforts to lower patients’ bills through discounting, and some of the ethical questions that arise related to these experiments.
A win-win
We always want to make decisions based on metrics and data. So when we developed the hypothesis that discounting could be a win-win for both patients and providers, we wanted to immediately ground our objectives in data.
How would we define success? Lowering the financial burden for patients can have multiple positive effects. Of course, patients end up with more money in their pockets. It also decreases the chances that they are sent to debt collections, a pretty bad experience that sticks with you in the form of a decreased credit score. Perhaps less obviously, we believe that it will also encourage patients to find care when they need it going forward. As a result you would end up with healthier patients with better financials.
What about the provider who offered a service that needs to be paid for? Our early results demonstrated that offering discounts can actually improve overall collections, leading to both the provider and patient being better off. Yet there is an important trade off to balance — we cannot discount bills more than a certain percentage without starting to hurt collections. Through experimentation, we are trying to find the proper balance.
A first experiment
A few months ago we started offering discounts to uninsured patients who had recently been to the ER and received an unexpected bill. We randomly offered discounts of either 15%, 30% or 45% off to a portion of this segment, while a control group was not offered any discounts.
Simultaneously, we also began testing the impact of timing on discounts. Some discounts are limited in time (30 or 60 days), while others last for the duration of the billing cycle.
In order to assess the effectiveness of discounts we ran a randomized experiment, similar to the method used for drug trials. Some patients receive a discount and form the “treatment” group. Others will not receive any, they are called the “control” group. At this stage there is no potential bias since we are using a computer system to allocate patients into the different groups. We are not running this process manually, which would open up allocation to unconscious bias.
We also carefully considered the ethics of how the discounting is being applied. We have not used any patient demographic data to determine the discount level, with the discount allocated entirely randomly for eligible patients. We do not believe there is any potential overall downside to patients- patients either pay their existing bill, or a discounted version of it. During drug trials, participants go through an informed-consent process- understanding the risks and the fact that they might receive a placebo. Of course, in this case, there is no downside in participating in the experiment. Still, patients might want to know they are part of an experiment that could change how much they owe on a medical bill. We do feel that warning them could impact patients’ behavior and skew results one way or another, defeating the purpose of the experiment itself.
While we would like to give discounts to the highest number of patients we first need to demonstrate the evidence of a win-win situation. If we do then a very large number of patients would benefit and see their bill lowered.
Optimization
So how large of a discount should a patient receive? Can we use the patient balance as an input in our decision? Intuitively, we might need to give bigger discounts to patients with larger balances. To be able to apply the optimal discount at scale we could build a model predicting what discount size would lead to the highest expected collection rate. A linear model would be a good fit if there was a simple relationship such as “for every $250, we should increase the discount by 5% on average”. Such a model would probably be too simplistic to lead to optimal results. We would need to use a more complex model that can find non-linear patterns between the balance of the patient and the optimal discount size.
One thing that we have noticed though is that health care prices should probably not be treated as a classic quantitative variable. For example, more than half of the patients in our experiment have a bill for exactly $985 or $522; those amounts are related to a procedure or set of procedures that they patient had. What if our model learns that for a specific amount, the discount should be lower than for similar amounts because patients who go through the procedure in question do not react the same way to discounts? This could be the case if, for instance, the procedure could be more common among a certain patient population or associated with a particular predisposition. Then all of a sudden your model has a bias towards a particular population despite only using the patient’s balance as input.
Checking for this potential source of bias in a one-input model is easy but it quickly becomes more complex as the size of the model grows. Let’s assume we had at our disposal the perfect estimation of wealth and income for each patient, I would feel fine using it in our model to identify the populations that need the most help — one would hope that the model would learn to give them larger discounts.
But based on our recent experiments, that’s not necessarily the case. Using the patient’s postal code and census data we grouped patients into buckets using the median household income associated with the zip code. As we can see below the bucket with the lowest income has had slightly higher collections with a 15% discount. This is not a “statistically significant” result but let’s assume that it was the case, Cedar would need to make the difficult but right decision to exclude wealth and income data from the model.
We can ask ourselves if a “perfect estimation of wealth and income” even exists. We used ZIP-code aggregated Census data to see if there was any correlation with how people would react to discounts. ZIP-codes have well known issues and are far from a perfect proxy for wealth and income. (See weaponsofmathdestructionsbook.com)
While discounts could be a win-win for both providers and patients, if we only try to optimize the providers’ collections we might end up in unethical situations. One way to prevent this from happening is to be mindful on how our discounting model is built and with what data. Another way should be to change the function we are trying to maximize in order to advocate for patients as well. We could maximize the amount collected + alpha * the number of patients using a discount. We could fine tune alpha until we are satisfied with the trade off of giving up some collections in order to help a higher number of patients.
While 30% discounts outperformed other discount sizes in the experiment we carried, it’s not the optimal answer for all bill sizes. We have a large population of patients without insurance who owe several thousands of dollars. Building models that improve and train regularly on new data is a way to personalize our help to patients while improving providers’ collections. To build ethical models who can recommend the best policies like the size of the discount, we will not use:
- Demographic data from patient like age, gender.
- Proxies like zip code information, credit scores.
- Information like the medical procedure from which demographic data could be implied.
This list will be regularly updated and expanded if necessary.
We would love to hear from ethical experts, healthcare professionals and data scientists on this topic. We are welcoming feedback and suggestions. You can reach us at: data_science@cedar.com
I want to thank Elliott Fong, Rena Yang and Charles Copley for their help writing this article.
Cedar is a healthcare financial engagement platform for hospitals, health systems and medical groups that clarifies and simplifies the financial experience for patients. If you’d like to learn more about Cedar, join our mailing list at pages.cedar.com/join-the-list. You can view our open roles at www.cedar.com/careers.
