Annie Delgado, Chief Risk Officer at Upstart — Expanding access to affordable credit
In today’s episode, Kailee Costello sits down with Annie Delgado, the Chief Risk Officer of Upstart. Upstart is a leading AI lending marketplace, partnering with banks and credit unions to expand access to affordable credit. Upstart was founded in 2012 and went public via an IPO in 2020.
“There’s a lot of discussion in the industry and in regulatory circles about the potential for AI to lock in systematic bias … but I actually am excited about the opposite, which is that AI has the potential to solve systematic bias if it’s done the right way”
In this episode, Kailee and Annie discuss:
- Upstart’s AI lending marketplace
Annie: Upstart has been around for a little bit over a decade. The problem that we saw when we started on this journey is that less than half of Americans have access to prime credit, but about 80% have never defaulted on a loan. We learned this by doing a study through traditional credit bureau data.
So the problem to solve is there’s this big gap between people who can afford to repay a loan and people who actually can get the loan that they need, when they need it. This is a really important problem to solve because most, if not all of us, are going to need a loan for something at some point in our life, whether it’s to go to school or to buy a house or a car. Access to credit is just really a critical component of people being able to run their lives and achieve their dreams. So, we set out on this venture to use alternative data and alternative credit modeling techniques in order to close that gap of who has access to a loan and who can repay the loan. Our core mission is expanding access to affordable credit.
- Annie’s role as Chief Risk Officer
Annie: You can think of a Chief Risk Officer’s role as being a person who’s in charge of operationalizing whatever requirements exist for the company. Some of those requirements are regulatory requirements that come from various laws and regulations. Some of them are contractual requirements because of agreements you’ve made with banks or lending partners or vendors.
Whatever the genesis of the requirement is, the compliance team is going help operationalize that by working with the business unit that’s responsible for implementing that requirement and making sure there are controls in place so that happens effectively, assessing the risk of what happens if that thing goes wrong, and then monitoring and testing to make sure that it doesn’t go wrong.
My role has changed a lot over the last eight years that I’ve been at Upstart, as our business has become increasingly complex. When I started at Upstart, we offered one lending product, an unsecured personal loan product, through one partner bank. Since that time, we’ve really expanded quite a bit. We now offer several different lending products. We do auto refinance, we do auto purchase, we do small dollar lending, and we also now serve over 100 banks. You can imagine the requirements of all those different contracts and all those different regulatory regimes — there’s a lot of complexity to streamline in the business.
- How Upstart’s use of Machine Learning has evolved over time
Annie: It’s evolved a lot; the beauty of using machine learning is that it can evolve quite quickly. 10 years ago, when we were trying to launch the platform, we used machine learning to make a binary decision — “Can this person repay a loan or not?” Over the years, we have really invested in our machine learning models in a few key ways. First of all, we have added increasing amounts of data to the models so that they have more information to leverage in terms of decision-making processes. We started out with a more limited subset of variables, but now have something like 1,600 data inputs that go into the models as they’re making assessments in the underwriting process.
Another way that they have evolved is in the mathematical techniques that are used within the model. There’s been a lot of research and advances technically and mathematically in the last decade, so we’re always making sure that we are employing experts in this field that can continue to invest in the cutting edge technology for the models themselves.
Then, the last piece is the application of the models. At our outset, it was really a binary yes/no decision that a model was responsible for making. But, now we use models in all aspects of the credit process. We now also use it for things like how likely an application is to be fraudulent, which will decide whether a person should go through a fully automated loan process, or they should go through a more manual documentary review process. We also use it for loan pricing — if somebody is approved for the loan, what is the right price to offer them based on their risk? We also use it for things like servicing and collections, to make decisions about what time to call people and who to call if they need outreach from us. So we use our models in just a lot more applications than we did at our onset 10 years ago.
- Insight’s from Upstart’s models
Annie: One of the biggest struggles that people have with loans is the process itself. A lot of times what happens if you go to traditional bank is you have to bring along with you a lot of your financial records, your pay stubs and your W-2s and your tax returns and all kinds of things, and then a human looks through those things and it takes days or weeks or sometimes even months, depending on how much of a backlog the bank might have in application processing.
For us, using our models, 84% of our borrowers are able to get fully approved with no human intervention in the loan process and no document upload. So 84% of people don’t actually have to upload a document or have a person review it, which is pretty staggering when you think about how helpful AI and ML can be in terms of automating the loan process for people to make it a lot faster relative to people who have to wait months to get access to the money that they need.
Another thing that is a really cool thing that our model has done is not only can it make the process easier, but it can actually make the pricing better as well. I think the next thing you would hear from people is that sometimes loans are just too expensive and they don’t like the price of a loan. Using our model, banks are able to approve more people and at lower rates than what you would get under a traditional model using only a credit score. And so you’re really making the borrower’s life better both from the pricing component but also from the experience component when you’re leveraging these models.
- Annie’s perspective on how lending processes and credit scores will evolve into the future.
Annie: Obviously this is a biased point of view, but I think everybody should be using machine learning models in their credit decisioning processes. The reason for this is when you think about the invention of credit scores more than 30 years ago, it was a huge innovation because prior to that the only process you had for loan approval was a human underwriter, which was rife with potential for bias and problematic outcomes. So the credit score was a huge innovation at the time, but it has sort of stagnated. It hasn’t changed much since that time, but the world has changed significantly — to think that one three digit number can be used for all flavors of credit and for all types of borrowers is really just something that our world has grown past.
What ML and AI models have the opportunity to do for the industry is it gives the ability for us as a banking industry to evaluate people individually based on their individual merits and their individual needs. How much credit are they looking for? What is the purpose? Who is this person? Can they repay their loan? That individualized approach I think is really important, so I can’t imagine that the world is not going to eventually get there. How fast it gets there is a question, but I do think there is a point in the future where there will be no lender in the country that’s not using some flavor of ML in their underwriting processes.
The second part of your question about if we see traditional credit scores evolve is a really interesting question. I think we’ve already started to see this. Vantage Score, for example, has started to integrate cashflow data into their score. Just like with any company, they’re going to evolve their processes as well to meet the needs of the market. So if we start moving towards a direction where banks and other lenders more and more need ML and alternative data in their processes, the credit scoring providers will also hopefully evolve over time so they don’t get left behind. It’s going to be an interesting decade to see how this all unfolds.
- How Upstart has focused on making their ML models fair and unbiased
Annie: It’s such an important question. The question to ask when you’re talking about any underwriting system, but especially a new system is “ is it fair?” and “how do you prevent it from locking in any sort of historical biases that exist in other systems?”
The way that I typically think about this is any process that you’re going to use to underwrite alone is a three-part process. (1) There are the inputs that go into the decision making system. (2) There is the decision making system itself, the tool. (3) There is the outcome of the decision making system. If you imagine the pre-credit score days where these were humans. A person walked into a bank branch, they gave a set of documents and paperwork to a human. The human reads it and that’s going into their decision making system, which is their own brain, and their brain is making all kinds of assessments and connecting all sorts of dots about what they see in that application. Then the brain is rendering a decision based on that — an outcome — either an approved or a denied decision and a pricing decision.
So as I’ve thought through how to measure fairness in machine learning models, really all three of those components are things that you want to be measuring. What is going into the system? Is it data that is a potential proxy for something that you don’t want to be involved in the decision-making? How do you test for that? How do you statistically measure it to make sure that what’s going into the system is data that is needed to assess somebody’s credit worthiness and not extra data that doesn’t serve a purpose of assessing their credit worthiness.
The second part is the system itself. The system is making correlation types of inferences. The beauty about an ML model and what’s different from a human underwriter is that you can understand that and see it in an ML model. You can see the correlations and the sort of things that are happening. So, that way you can adjust it if there’s things that you don’t like about those assessments, because it’s a supervised model. It’s much harder to adjust a human brain.
The last piece is the outputs. What are the actual decisions that are coming out of the model? What is the ratio of approvals from one group relative to another group, or the pricing differences from one group relative to another group? You have a continuous feedback loop. If you see a disparity in your outcomes, you can look back at what’s going in and what’s happening in the middle that you want to change or adjust accordingly. So, I think that there’s a lot of discussion in the industry and in regulatory circles about the potential for AI to lock in systematic bias, but I actually am excited about the opposite, which is that AI has the potential to solve systematic bias if it’s done the right way and if people are really thoughtful about doing it. That’s one of the things that I think is most exciting to me and most exciting to a lot of Upstarters, and why we work on this problem, because we see AI as the solution to some of these issues.
- Upstart’s rationale for remaining as a lending platform instead of becoming a chartered bank
Annie: At several points in our history, we had this discussion of, “some of our competitors are pursuing bank charters — is this something that we should consider?” Each time, philosophically, what we’ve come back to as a leadership team is that we are a technology provider to banks. We want every lender in the country to be able to use our technology because we believe our technology is helpful to consumers and it’s helpful to the banks. If you become a bank, your technology is only useful to the customers that you are serving. If you partner with banks, you can have a much broader application of your technology. So each time that we’ve sort of had this question, we’ve come to the same conclusion, which is that as a platform, we’re a partner to banks, not a competitor to banks.
- How Upstart partners with banks
Annie: We partner with banks of various sizes, but I think the banks that have the most need for a product like ours are the more regional community banks who really have a desire to expand their footprint and expand their offering to acquire new customers, so that they can stay relevant and compete with some of the larger banks. They need a mechanism to do that at scale that they might not have internally. So if they partner with someone like Upstart who has a referral network, we can help them acquire new customers and we can help provide them needed technologies that their customers want. Let’s be honest, most customers are not dying to walk into a bank branch and sit with a teller anymore — most people want to be able to do their banking at home with their cup of coffee and in their pajamas. And so, you know, smaller banks, community banks need to be able to stay technically relevant. And Upstart is a way for them to do that if they partner with Upstart or with lots of fintechs that offer these sort of bank-facing products like that, so that they can offer technology that their customers really want and need and prevent their customer from going to a bigger bank that has more of the sort of digital offerings.
- Competitive advantage in the AI lending marketplace sector
Annie: I think our machine learning model is for sure a huge competitive advantage for us. As I mentioned, I think there’s gonna be a point in the future where everybody is using machine learning. We have a pretty significant head start as the largest company that’s been investing in this over the last decade. So, we have a big competitive advantage there just in terms of the technology that we’ve invested in building as well as the training data sets. ML, in order to work properly, needs a lot of training data. We have tons and tons of training data coming in every day, every time somebody makes a payment on a loan or misses a payment on a loan, that’s feeding those models. That competitive advantage is huge. The bank partnerships model is also a competitive advantage. Like I mentioned, there’s only so much you can do as a company if you are just serving your customers as opposed to having a broader outreach. So the more products that we can develop to solve consumer problems and struggles with banking that then banks can decide to adopt as an offer to their customers, I think the bigger the network becomes and the outreach becomes.
- How use of ML models varies between players in the industry
Annie: There’s a lot of different types of modeling techniques that can be used. There’s a lot of different types of data. There’s a lot of different types of applications for machine learning models. One company might decide to tackle one set of problems using ML; another company might decide to tackle an entirely different problem. For instance, I know some of the larger banks use ML for a lot of their customer contact center type of stuff — who to call, what questions to ask them, when to engage them. That’s an entirely different set of problems to apply ML to than the credit underwriting space where you’re trying to decide what price to offer somebody on a loan. So there are a lot of different use cases for AI models. And we’re just at the beginning of this. There’s a lot of different use cases today and in 10 years there’s going to be exponentially more as people uncover new problems that ML can help solve.
- What data sources different players use for ML underwriting models
Annie: There are some companies out there that are just relying on traditional credit bureau data. There are some companies that have experimented with things like cash flow data. There has been a lot of sort of research in that space and how cash flow data might be helpful in making more inclusive lending decisions. Obviously, Upstart is known for our use of education data, which is a data set that we believe is a really strong set of data to add into an ML model to get a more holistic view of an applicant’s circumstances. Not a lot of other players out there are using education data. So the type of data used definitely differs amongst companies.
- The economy and banking sector in 2023
Annie: For our partners themselves, what’s top of mind is liquidity issues (interest rate risk). Those are things that are core to banking risk management, so obviously, it’s important to our lending partners and something that they watch closely and we need to be helpful in. A nice thing about the Upstart program is that there are short-term loans and high yield loans. So, if you have interest rate risk because you have a lot of long-term assets, investing in some Upstart-powered loans can be a useful strategy for you as a lending partner.
Then there’s also this risk management hygiene or best practices about how you think as a company about business continuity planning. One thing with the SVB fallout that everybody was thinking about is, if SVB closed down on a Friday, we had a weekend where we didn’t know what was going to happen and we had this circumstance where on Monday morning there could have been a lot of companies that had to shut their doors because they didn’t have access to money anymore. Thankfully the FDIC stepped in and that didn’t happen. But if they hadn’t, a lot of the vendor supply chains that different companies rely on would have been shut off. Banks and FinTech providers should be thinking about “who are my critical third party relationships” and “what would I do if something disrupted their ability to provide that service to me?” That really comes down to a business continuity planning target type of program. So I think this is an unpleasant and unfortunate reminder of something that we should all be thinking about on a routine basis.
- Future outlook for the lending industry
Annie: From an aspirational answer, I would say that what I hope to see, and whether or not we’ll see it in the next couple of years remains to be seen, but one of the things that happens when we go into a down economy is banks sort of stop lending. So you have this really unfortunate circumstance where the economy is not looking so good, people need access to credit more than ever, and at that same time, banks have to adopt a risk-off sort of approach and they cut lending or they tighten their credit standards. I think that’s really unfortunate. What I would love to see happen is leveraging ML and AI, enabling lenders to have a future-proof lending strategy, even in a down cycle, so that they can keep the money flowing to the consumers that need access to credit, even during those uncertain times. That’s one thing that Upstart has thought a lot about. We developed and released this year a functionality called the Upstart Macro Index (UMI) that actually shows a historic view on the effects of the macroeconomic environment on the performance of credit. So if you can measure that historically and make better predictions about the future, then you can safely and responsibly continue to lend even when the economy is in a tough spot. I would really dream of a day that that is the circumstance, that banks don’t have to just stop lending when things get hard, and instead can again keep the money flowing to the people who need it. So I hope that the AI revolution will allow that.
- Impact of generative AI in lending
Annie: I think anybody who is not thinking about generative AI should start thinking about generative AI, especially if you’re a risk professional. It has been around for a while, but it became very widespread this year. So there’s just a lot more people that are thinking about it, using it, thinking about how it could help them. Over the next couple of years, I expect it really to sort of pick up steam and increase in the use cases that people have for it.
We don’t currently use any generative AI for any of our underwriting or application processing models, but I think all companies probably have employees who are using it for research and who are using it for, for example, editing email content. So I think that risk professionals like myself need to be right at the onset of this technology. So, right now, be thinking about “what do I want people using this for” versus “what do I not want them using it for?”. “What generative AI tools do I feel safe about them using?” versus the ones I would want them not using because I’m concerned about the sort of security implications. And then, “how do I do quality control around the outputs of the generative AI?” I’m sure you saw the situation where an attorney used generative AI to write a position statement and the generative AI fabricated a bunch of cases and case law for this position statement. That’s the type of use case I’m saying we need pretty robust quality control over, because it’s really sort of the power of Google, but magnified. We don’t want a situation where there’s a bunch of false information that’s being spread around, so companies really need to be thinking about how to put controls in place for their own institution to make sure that they don’t find themselves in an unfortunate situation like that attorney did.
Check out the Episode on the platform of your choice here: Spotify | Apple Podcasts | Soundcloud
About Upstart
Upstart is a leading artificial intelligence (AI) lending marketplace designed to improve access to affordable credit while reducing the risk and costs of lending for bank partners. By leveraging Upstart’s AI marketplace, Upstart-powered banks can offer higher approval rates and experience lower default rates, while simultaneously delivering the digital-first lending experience their customers demand. Upstart has originated more than $33B loans, and 84% of loans are fully automated.
About Annie Delgado
Annie is the Chief Risk Officer at Upstart. As part of her role, she has built a data-driven compliance program at Upstart, navigating the regulatory environment associated with the use of AI-powered credit modeling techniques.
About the Author
Kailee Costello is an MBA Candidate at The Wharton School, where she is part of the Wharton FinTech Podcast team. She’s most passionate about how FinTech is breaking down barriers to make financial products and services more accessible — particularly in the personal finance space. Don’t hesitate to reach out with questions, comments, feedback, and opportunities at kaileec@wharton.upenn.edu.
As always, for more FinTech insights and opportunities to collaborate, please find us below:
Wharton FinTech: Medium Blog | Twitter | Our Website | LinkedIn
