Challenges of AI ethics in insurance
Using AI in insurance can lower prices, better detect fraud, and improve access, but what are the risks?
By Michelle Seng Ah Lee, DataKind UK Ethics Committee
At our most recent Data Ethics Book Club, hosted at Aviva’s London office, DataKind UK invited interested data experts and insurance fans to discuss the ethics of using artificial intelligence (AI) in personal insurance. The club’s reading list can be found here.
The UK Centre for Data Ethics and Innovation claims in their ‘AI and Personal Insurance’ snapshot paper that using AI could:
- reduce prices for policyholders
- lead to fairer outcomes by filtering out fraudulent claims
- open up insurance to new groups
- advise policyholders on how to reduce damage to people and property
- and incentivise take-up of insurance
AI has been used to speed up the processes for providing quotes and managing claims; find patterns between newly available individual data and specific risks; and help “nudge” and advise people on how to lead healthier and safer lives.
This all sounds too good to be true, so what are the drawbacks? The evening’s discussion touched on three types of potential risk: fairness and inequality, privacy and autonomy, and explanation and inference.
Fairness and Inequality
The first discussion was around the impact of the personalisation of insurance on existing inequalities. While AI has improved access to insurance for some people, it has started excluding others who are deemed too risky and so face unaffordable premiums. A book clubber suggested that the latter is a larger group that is less financially resilient, and thus more desperately in need of insurance in times of crisis. Another cited a report by the Institute and Faculty of Actuaries that claims insurance has a ‘poverty premium,’ with those who are poor paying more than those who are wealthy. The book club questioned whether, when insurance is intended to pool risk as a group, it could become so personalised and precise that it no longer serves those who need it.
The discussion moved on to unintended biases and discrimination. When asked what types of data should be off-limits to insurers for usage in decision-making, the first answer was “anything protected under the Equality Act,” including characteristics such as age, disability, or gender reassignment. This has been a source of controversy in the past, where insurers make assumptions based on what information they are given. Insurance companies have been accused of giving higher premium quotes to motorists with the name Mohammed. An algorithmic audit of Italian car insurance showed a driver born in Laos may be charged 1,000 Euros more than a driver born in Milan, all else being equal. In December 2012, the EU introduced new rules insisting that car insurance companies no longer discriminate on the basis of gender. The use of AI to fight insurance fraud has hit an all-time high, but attempts to geo-locate patterns of fraud can incorrectly flag claims in certain regions, exacerbated by the challenge in defining suspected fraud in the industry.
What about proxies of protected characteristics if the protected feature usage is banned? After the EU ruling, the gap between men and women’s insurance premiums rose rather than narrowing as expected. Men pay £101 more, compared to £27 more before the ruling. Instead of gender, it appears insurance pricing depends on occupation, and male-dominated occupations are correlated with high rates of driving under the influence. Women are, in fact, statistically less risky drivers, as they tend to drive fewer miles, and have fewer and less serious accidents.
This leads us to a discussion over fairness versus equality. The Guardian comments about car insurance “My guess is that women were actually paying too much before the ruling and are now paying premiums that more accurately reflect their risk. Car insurance may have become less equal. But it is more fair.” Perhaps by forcing insurers to ignore gender, the EU ruling pushed them to look for and rely on other risk indicators, such as the type of car. However, it is difficult to say whether it is fair that scaffolders pay higher premiums than midwives. Often, ‘statistical’ justification has been used to justify stereotypes — in the past, black people’s higher mortality rates and women’s financial dependence on men have both historically been used to rationalise differential pricing.
There is also a precedent of favouring equality over fairness. Flood Re in the UK ties flood insurance premiums to council tax bands to make it more affordable and accessible. It may not be ‘fair’ for other policyholders to face costs for those who live in homes with high flood risk, but it was a policy decision to prioritise improving access to flood insurance. One book clubber stated, “Flood Re shows insurance is a political issue.” Another commented that policies should protect those with limited control over where they live, for instance who cannot afford to move elsewhere, rather than those who chose to purchase a holiday home in a high-flood-risk zone.
The book club also brought up the disparity between actuarial fairness and behavioural fairness. The former refers to people paying an amount proportionate to their risk, while the latter refers to using data on how people behave rather than who people are. For example, in health insurance it is more important to know whether the customer is a smoker rather than where they come from.
Privacy and Autonomy
This led to a discussion over the practice of collecting more behavioural data about customers, and the resulting concerns over privacy and autonomy. The book club noted that a big change in insurance is the availability of more data. More companies are leveraging ‘alternative data’ and non-traditional sources to predict risk, such as our location data, social network, and public posts online. A book clubber described the landscape as a ‘Wild West’ in which some companies will keep collecting data until told to stop. These data are often collected without customers’ explicit consent or knowledge. Under GDPR, one book clubber claimed, as long as there is a legitimate interest and reasonable expectation from the customer that the data would be used that way (e.g. to combat fraud), the insurers do not need consent.
In the continued discussion on data that are off-limits for insurers, one book clubber noted a generic class of data: secrets with sensitivity issues. While difficult to define, it is generally what one would not tell others about openly. What is considered a secret can also change over time. Another book clubber noted that currently many people are deleting their period monitoring apps since the overturning of Roe v Wade. It is difficult to define what can be considered a ‘secret,’ and many secrets can be inferred through correlating data, such as shopping habits.
One book clubber stated that “autonomy is the great undefeated topic in personal insurance.” Beyond privacy, the increased usage of non-traditional and larger data sets reduces the control the customer has over his or her information. Using wearables or telematics to give insurance companies more data should be an informed choice. Another book clubber stated that power dynamics are greatly in favour of the insurer, and those who cannot afford insurance otherwise may feel unduly pressured to cede their privacy. One aspect that would help customers regain control would be greater transparency and explainability.
Explanation and Inference
Finally, the group touched in transparency. It is less well understood what drives insurance pricing, compared to credit scores, so it is difficult to control it. One of the issues is that the companies are using these data to make inferences, which may or may not be correct, and are not disclosed to the customer. Based on a customer’s supermarket loyalty card data, an insurer may infer that the individual exercises regularly. Should these assumptions be made accessible to the customers, so that they can challenge them? Some academics think so.
Some of the correlations are questionable in their causal mechanisms and accuracy. One book clubber referred to an insurer who claimed the ability to predict someone’s mental health from how much they were smiling. Another gave an example of the relationship between whether someone drinks bottled water or tap water and the motor insurance risk. The correlations (rather than causations) that drive these models are tenuous.
Key Takeaways
While the challenges of AI in personal insurance are complex and far from being resolved, the book clubbers remained cautiously optimistic. AI, in combination with human investigators, can more effectively combat organised fraud. Insurers can share data with customers to help them lead a safer and healthier lifestyle. While AI is bringing ethical issues in insurance to the forefront of board-level and policymakers’ agendas, many of the considerations are not new. The unique challenges of AI include the use of larger and non-traditional data sets, and thus the increasing potential for unintended biases and limiting explainable decision-making. Understanding these risks and designing appropriate mitigating actions should enable insurers to continue to innovate with confidence.
If you’re inspired with ideas for a future book club, we’re always looking for more topics and potential hosts — email us at contact@datakind.org.uk to start a conversation, and follow DataKind UK on EventBrite to get notified when we next hold one!
If you’d like to host your own Data Ethics Book Club, take a look at our Book Club in a Box, and find all of our previous reading lists on topics such as race, tech addiction, and fairness in AI on our GitHub.
