Building a fairer and more inclusive financial services industry for everyone
By Linda A. Lacewell, Superintendent of New York State Department of Financial Services
On Saturday morning, I read a Twitter thread from an Apple Card user — tech entrepreneur David Heinemeier Hansson — detailing how his card’s credit limit was considerably higher — twenty times — than that of his wife, despite his wife having a higher credit score. I responded, announcing that the New York State Department of Financial Services (DFS) would examine whether the algorithm used to make these credit limit decisions violates state laws that prohibit discrimination on the basis of sex. For the rest of the day, numerous Twitter users responded to David’s initial tweet, including Apple co-founder Steve Wozniak (who indicated his credit limit is ten times higher than that of his wife), describing similar instances where men received higher credit limits than women. Confounding this is the “black box” problem, in which consumers have little visibility into how a decision is made or why they have been rejected.
New York law prohibits discrimination against protected classes of individuals, which means an algorithm, as with any other method of determining creditworthiness, cannot result in disparate treatment for individuals based on age, creed, race, color, sex, sexual orientation, national origin, or other protected characteristics. We know the question of discrimination in algorithmic decisioning also extends to other areas of financial services. Just last week, DFS opened an investigation after reports regarding an algorithm sold by a UnitedHealth Group subsidy allegedly resulted in black patients receiving less comprehensive care than white patients.
Here in New York State we support innovation. From Buffalo to Brooklyn, we are home to entrepreneurs, start-ups, and welcome innovators from around the globe to bring their ideas and creativity. However, new technologies cannot leave certain consumers behind or entrench discrimination. We believe innovation can help solve many challenges, including making quality financial services more accessible and affordable. Yet, this can’t be accomplished without maintaining public confidence. For innovation to deliver lasting and sustained value, the consumers who use new products or services must be able to trust they are being treated fairly.
DFS regulates financial products and services offered in New York State. Financial service companies are responsible for ensuring the algorithms they use do not even unintentionally discriminate against protected groups. Our North Star is that the consumer comes first. We will work to investigate what may have gone wrong, and if the algorithm used by Apple Card did indeed promote unlawful discrimination we will take appropriate action. But this is not just about looking into one algorithm — DFS wants to work with the tech community to make sure consumers nationwide can have confidence that the algorithms that increasingly impact their ability to access financial services do not discriminate and instead treat all individuals equally and fairly no matter their sex, color of skin, or sexual orientation. We support and encourage innovation and know that most innovators have the best of intentions, which is why we welcome the opportunity to proactively engage with financial institutions and other stakeholders to ensure new products and services do not harm public confidence through discrimination.
We invite those who believe they have been mistreated or subject to discrimination to step forward and tell their stories (email us at consumers@dfs.ny.gov) and also invite experts and members of the tech community to reach out (email us at innovation@dfs.ny.gov) to create necessary standards to ensure new technologies can thrive without unfairly discriminating against consumers. At DFS we believe innovation and consumer protection can not only co-exist, but that innovation can lead to better products and services for consumers that help achieve a new standard of consumer protection. However, this will only be possible if we come together as a community — indeed we are all consumers — to solve this together. I hope we can all be part of this process to build a fairer and more inclusive financial services industry for everyone.
