Part 2: Predicting the Future is Hard, and a 90% Success Rate Isn’t Good Enough

This is part 2 of a 6 part series. Please refer to the introduction to this series Lending: don’t hate the players, hate the game for more context. Or you can read part 1 here.

This is the meatiest post of the bunch!

Lenders create a model that tries to predict the future. In particular, they are interested in whether or not a person applying for credit will pay back, and also that the loan will be paid on time. If yes, then the lender will offer the customer a loan.

In medicine, doctors do something similar. They develop a model and use various signals — such as a patient’s age, weight, physical appearance, dietary habits, drinking and smoking habits, how often or how hard the patient exercises, the health of parents or relatives, etc. — to anticipate a patient’s likelihood of developing certain health issues.

In financial services, lenders do the same but to predict repayment. Unfortunately lenders don’t get to meet every applicant in person. They aren’t able to ask personal questions and gather the appropriate data to make a prediction in the way that doctors do when predicting disease.

While patients are incentivized to provide doctors with accurate responses of their health and habits to get proper treatment, loan applicants are not always incentivized to provide accurate information about themselves. If it is possible to get a bigger loan with better interest rates, people may exaggerate incomes or hide bad repayment histories. Verifying the information that an applicant provides is not easy.

A comprehensive, up-to-date list of everyone who lives in the US and how trustworthy they are doesn’t exist. It would also be hard to maintain. My ability to repay a loan today may be different than three months from now if I’ve received a huge raise or lost my job in the meantime.

Most lenders are forced to rely on one signal — FICO or credit scores from Experian, TransUnion, and Equifax. Sadly, the credit score system in the US is not always an accurate reflection of an individual’s ability to repay a loan. A Federal Trade Commission study found that 5% of consumers had errors on their credit report that would lead to paying more for credit products.

There are various ways a credit score may not be an accurate reflection of an ability to repay a loan. For example, if someone’s identity was stolen and several loans were taken out by a fraudster, a credit score might be affected negatively in the long term because of the inability to remove those fraudulent, unpaid loans from the credit history.

Another example is when someone opens a store credit card for a discount, never receives any notifications, forgets the credit card exists, and doesn’t pay the $11 bill for several years. Ultimately this long term unpaid loan tanks a credit score, and it can take several years for it to recover.

The U.S. credit score system is also an inaccurate reflection of an individual’s ability to repay a loan when the individual is foreign. Immigrants from other countries, whether France or Brazil, were or still are not approved for credit because their international credit histories aren’t recognized here.

In addition, credit bureaus often make identity mistakes, which damage the wrong credit scores. For example, over several years, Julie Miller’s identity was mistaken for another less credit-worthy Julie Miller. The former Julie Miller discovered 38 collection accounts on her report that did not belong to her, along with an inaccurate Social Security number (SSN) and birth date. Even after sending more than 13 letters to Equifax over the course of two years, initiating the company’s dispute process about seven times, and sending in pay stubs, phone bills, W-2 forms, and her driver’s license to verify her identity, Miller was not able to get Equifax to rectify all of the errors on her credit report. In 2013, she was awarded $18M from Equifax in punitive damages. According to credit experts and lawyers who specialize in lending, the credit bureaus are willing to tolerate these errors — and settle with consumers out of court — as a cost of doing business.

Of course these mistakes are not intentional and are not usually from extreme negligence. It’s a difficult task to differentiate people who share the same name, and especially in cases when father and son have the same address and share the same name e.g. John Smith II and John Smith III.

But for a moment let’s assume we live in an almost-perfect world: credit scores are fairly accurate, no one steals identities or accidentally mixes names up, no one opens a Gap credit card by mistake and forgets about it, and immigrants are able to easily transfer their country’s credit scores to the US. Even in this almost-perfect world, when lenders are 90% right about their repayment predictions, it’s not good enough.

Let me explain:

Let’s say you have $1,000 and you want to make some money by lending $100 each to 10 friends who will pay you back in one year. Time is money, so you charge $5 of interest for each year-long $100 loan. This means you are charging them around 9–10% APR (Annualized Percentage Rate), no compounding interest. Seems like a reasonable rate and interest charge.

The $50 — from the $5 fees from your 10 friends — will be your profit from lending out $1,000 over the year.

After one year, you check results. Your “friend intuition” is quite good — 9 out of 10 of the friends you chose pay you back every month and in full. You make $45 profit! But one friend did not pay you back at all a.k.a. the friend who always loses his wallet and brings his passport to the bar every weekend. He’s a good guy, but he’s a mess. We all have that friend.

Although you make $45 profit from the 9 others, you lose $100 from that one friend who didn’t pay back. Ultimately you are $55 underwater and can’t continue this lending business with those 90% odds of success.

Now let’s tweak this a bit to try to break even with the 90% repayment success rate:

If you increase the APR and charge higher interest rates, you can still account for when that one friend doesn’t pay you back. In order to break even with a 90% repayment success rate, you have to charge all of your friends around $11 of interest per year-long $100 loan, or 20% APR. That still seems like a reasonable interest rate. And $11 on a $100 year-long loan isn’t too bad.

But you’re not in the business of lending for free. To make a profit with the 90% repayment success rate:

If you increase the APR and charge even higher interest rates, you can still make a profit with a 90% repayment success rate. In this scenario you can charge around $17 of interest for each loan, or 30% APR. So even when that one friend does not pay back the loan, you still make ~$50 in profit from all the others to account for the lost amount.

This total profit of $50 equals to 5% of the total $1,000 you had from the start. Not too bad. As an investment, that’s better than putting that money in the average savings account where you’d probably get around 1.5% after one year.

What we’ve learned through this example is that lenders must charge everyone else higher interest rates in order to account for a few bad apples who don’t pay back. To cover for the one friend who didn’t pay back the loan, all your other friends paid 30% APR, or the equivalent of around $17 for their $100 loan. That’s a sizable amount of interest charged, and your friends probably weren’t too happy. In fact, many of them probably would not want to borrow from you in the first place because it’s so costly.

In my experience, customers tend to be comfortable with APRs under 20%. That’s an equivalent to around $11 on a $100 loan over a year. With anything above that percent, people become unhappy and begin to resent the lender. It’s always a bad experience to be charging customers more than what they think is appropriate.

In order to maintain low interest rates, lenders must ensure their future predictions and repayment success rates are close to perfect. So-better than 90%. Better than an A- in school. And that means that lenders reject a lot of good potential customers.

Here’s an example:

You’re a lender who wants to only offer loans with interest rates at 20% APR. Jane is a potential new customer. Based on your model, 90% of other customers who looked exactly like Jane paid their loans back. But that means you must reject her, because you would have to charge everyone else higher APRs in order to take on the risk of lending to her. She’s only 90% likely to pay back the loan.

In order to maintain $50 profit with 20% APR loans, you have to make sure you stick to a 94.5% repayment success rate. Therefore you can only approve customers when you are 94.5% sure that they will pay back the loan. You can’t afford to take on the risk. 90% isn’t good enough.

Lenders can increase their appetite for risk and run the business with lower repayment success rates only when they charge higher APRs and interest rates to everyone else. But as mentioned previously, charging high interest rates make customers unhappy.

However if lenders want to offer only low interest rates, they must maintain high repayment success rates and reject lots of good customers. Of course, rejections are also terrible customer experiences.

Either way, both outcomes are bad customer experiences.

Two more things to remember:

1. There are costs from running your lending business that have not yet been calculated. In the example above, these costs might include the time and cost of gas required to drive and deliver the loans to each of your 10 friends, remind them to pay you back every month, and collect each monthly payment. In the online lending world, these costs include renting an office, employing engineers, customer service representatives, and other employees, and everything else required to build and maintain the product.
2. This example operates in an almost-perfect world in which identities and credit scores are never compromised and you know all your friends personally. When re-introducing the real-world complications of fraud, there are hordes of people that intentionally don’t pay back loans because they aren’t who they say they are, and there are no repercussions to their credit score. This make it that much more difficult to maintain high repayment success rates.

A few bad apples really spoil the bunch.

Next up is Part 3: No one likes rejection, not even lenders — coming soon!

This is part 2 of a 6 part series. Please refer to the introduction to this series Lending: don’t hate the players, hate the game for more context.