Education Expert: DeVos’ New Borrower Defense Scheme is Nonsensical
The methodology proposed by ED is riddled with significant problems.
Rep. Mark Takano is publishing a guest blog post by Douglas Webber, Ph.D., an Associate Professor at the Temple University Department of Economics & Institute for Labor Economics (IZA), on the Department of Education’s revised formula that provides partial loan relief to defrauded borrowers under the Borrower Defense to Repayment rule.
The Education Department (ED) very recently released a proposed methodology to help adjudicate Borrower Defense (BD) applications from students who claim to have been defrauded by the college they attended. The formula ED proposed would allow students to receive a full discharge of their loans if the program they attended has median earnings among graduates which are 2 or more standard deviations below the median earnings of “comparison” schools, and partial relief if program earnings are less than 2 standard deviations below the median.
There is a lot of statistical jargon to unpack here, but let me start out by saying the methodology proposed by ED is wholly inappropriate and confuses many statistical concepts that aren’t meant to be used together.
What results from this methodology is, predictably, nonsensical and harmful to students.
As one of my econometrics professors used to say: “Garbage in, garbage out”. What follows might sound like in-the-weeds technical details to some, but they are both critically important and also fairly basic.
First and foremost, ED is making the assumption that program earnings follow the normal distribution. This is very clear from the graphic (reproduced below) distributed with the announcement of the BD formula. This figure implies that they expect 95% of programs to have earnings within 2 standard deviations (in both directions) of the typical program. This also implies that students at 2.5% of programs would be eligible for full relief.
The problem is that earnings are not normally distributed, almost by definition. For one thing, earnings are truncated at zero, you can’t make negative money from an employer (more on this shortly). But more generally, there are more likely to be people (or programs) with very high earnings, dramatically pulling up the mean, while effectively leaving the median alone. When distributions are “skewed” like this, the mean can be much higher than the median. This brings us to another problem, the standard deviation, the main tool used by ED, is defined based on the mean, and has no direct relationship to the median.
Putting all of this together, using this formula in this setting will mechanically reduce the number of cases which fall below any standard deviation-based threshold. In other words, ED has proposed a formula which misunderstands and misapplies fairly basic statistical techniques in a way that makes it materially harder for defrauded students to find relief.
The above graphic is thus not only wrong and inappropriate, it is deceptive in that it implies far more students would be eligible for debt relief than would be possible given the design of their formula. How serious is this problem? Let’s look at some example earnings thresholds that defrauded borrowers must fall below in order to receive debt relief.
Based on numbers provided directly from ED, borrowers who attended dental assistant AA/BA programs must have attended a program whose median graduate earns less than -$5,468.27 in order to have their entire loans discharged. You read that right, the median graduate must make negative five thousand dollars. This is not the only example of such an impossible bar to clear, and is a direct result of what I would call statistical malpractice. To be sure, there are many other problems illustrated in the graphic put out by ED (including a fundamental misunderstanding of statistical significance) and the design of the formula. But the most disheartening aspect is that this fundamentally flawed methodology leads directly to students (who have already proven they were the victims of fraud) being shortchanged yet again, this time by the federal government.
Let’s tackle one other glaring problem with ED’s formula: the calculation of earnings only among those who graduated from a program.
The formula is supposed to identify people who have been made materially worse off because they attended a given program. So, imagine a borrower who was duped into attending a very low quality program, but recognizes the lack of value and drops out after a year. In other words, the fact that the program was low quality caused the student to drop out. The outcomes of this, and any similar students, would not count against the program. The program could have a 20% graduation rate, but only the graduates’ earnings would matter.
To understand how problematic this type of error can be, imagine instead that we are investigating whether a chemical company has harmed the health of local residents. Would it be reasonable to not consider those residents who died from cancer on the grounds that they did not make it to the end of the sample period? Of course not! Just as dying from cancer is an outcome of a company poisoning the local environment with carcinogens, a massive number of students dropping out is an outcome of a program which defrauds it’s students. This is a well-known problem in statistics, and is perhaps most closely associated with the literature on the gender wage gap.
To sum up, the methodology proposed by ED is riddled with significant problems. Perhaps most concerning is the lack of rigor or thought put into a formula that will help shape the financial well-being of students who have already been materially wronged.
