AI lending could make finance deals even more unfair for women—here's how this can be avoided

It's fairly well known that women receive less favorable loan terms from salespeople, on average, than men when they borrow money. One recent study into lending practices in US car dealerships recently confirmed as much. Equally, it has been found in bank lending and mortgages for many years around the world.

The academic literature suggests that salespeople may offer women worse terms in the belief they know less about the market, and so will be less capable of assessing whether they're getting a fair deal. It might also be that women are penalized for not being as assertive as men.

One increasingly pressing question is how this will be affected by artificial intelligence (AI) as it comes to play a bigger part in lending. Though banks and other lenders can be coy about the extent to which they're using machine learning and generative AI in lending, it's certainly already happening behind the scenes and is set to become much more important over the next couple of years.

You might think AI could reduce lending discrimination against women, perhaps by neutralizing the biases of sales representatives. In fact, a new study from my research group indicates that it has the potential to get worse. So why is this, and can it be avoided?

Our research looked at more than 50,000 car loans in Canada, and found yet more evidence of lending discrimination against women. In the field of credit research, the standard way of comparing loans is known as "expected utility."

This measures how much a loan benefits a borrower by considering factors like the interest rate, the likelihood of it being approved, and the amount of effort the salesperson allocates to the borrower. We found that the expected utility of loans was 68% lower for women than for men.

To see how AI might change the car industry, which is still only in the early stages of adopting it, we looked at how machine learning could optimize the commissions that lenders pay to salespeople for arranging loans for car buyers. Commissions play an essential role in car lending, influencing sales representatives' loan-pricing decisions and making up a substantial portion of dealership revenue.

Perhaps in an ideal world, incorporating AI into this process would mean you could automate loan pricing, remove salespeople's involvement and just do away with their commissions. In reality, there's enough competition between lenders, and the dealerships make so much money from the commissions, that they would probably just take their business elsewhere. The loan commission model is therefore unlikely to change—either in the car industry or in consumer lending more generally.

Instead, the opportunity for lenders is to use machine learning to optimize commissions so that sales representatives select loan rates that yield higher expected profits for the lender, and are motivated to devote enough effort to the customer to get them to agree to the deal. By doing this, we found that lenders would be able to boost their profits by 8%. This, of course, is at the expense of customers. We found that the expected utility of loans to customers falls by 20% in this scenario.

However, when we compared men and women borrowers, we found that the fall for women was 42% while it was only 17% for men. We didn't test for exactly what was happening, but it's a fair assumption that since the back data was "contaminated" with bad loan deals for women, the AI exacerbated this by assuming women are more tolerant of worse offers than men.

The workaround

This confirms longstanding fears among some industry watchers that AI could end up widening discrimination in lending, not only to women but to other groups who receive less favorable loan terms, such as certain ethnic minorities.

It could be argued that the prudent approach would be for lenders to steer clear of AI altogether. Yet we wondered whether a compromise might be possible. Could we encourage lenders to use AI more responsibly, to change the trade-off between profits and social justice?

We tested this in our study by programming the machine-learning algorithm to maximize profits while not worsening the expected utility of loans for women. In other words, utility only decreased for men. Under this restriction, we found that lenders could still increase their profits by 4%.

This indicates that when used thoughtfully, AI can both benefit lenders and protect disadvantaged groups at the same time. In response to those who would rather keep AI away from financial services, it might be better to accept its inevitability and instead use it as a tool to make lending fairer.

This article is republished from The Conversation under a Creative Commons license. Read the original article.