Podcast: Startup FairPlay to launch AI fairness index tool

FairPlay, a “fairness as a service” startup, is launching an index tool in Q3 of this year that shows financial institutions how their underwriting affects consumers.

Los Angeles-based Fairplay uses AI-powered data analytics software to help FIs assess the accuracy of their automated loan decision models and provides them with metrics to help identify potential biases, Kareem Saleh, founder and chief executive at FairPlay, tells Bank Automation News in this episode of “The Buzz” podcast.

Saleh was named a BAN executive to watch in 2024.

“Fundamentally, what we do is help financial institutions stress test their AI, identify blind spots in their AI and then correct those blind spots,” Saleh says. “And what we find is that something like 25% to 33% of the folks that financial institutions declined probably would have performed as well as the riskiest folks they approve.”

Since being founded in 2020, FairPlay has raised $14.5 million toward its tech, according to Crunchbase.

Keeping data in check

But even AI-powered decisioning algorithms require careful examination of the datasets they use, Saleh says.

“The conventional wisdom is that AI stands for ‘artificial intelligence,’ but it can sometimes also stand for ‘accidentally incorrect. If you don’t have a real clear-eyed view about this bias in the algorithms to overfit to the data, then you might miss the blind spots.” — Kareem Saleh, Founder & CEO, FairPlay

5 questions for compliance

The Consumer Financial Protection Bureau in June 2024 approved a rule requiring FIs that use algorithmic appraisal tools to ensure compliance with nondiscrimination laws.

Multiple lenders received fines from federal regulators for unfair lending practices in the past two years, including $2.6 trillion Bank of America, $2.4 trillion Citi and $560.5 billion TD Bank.

FairPlay’s software enables FIs to answer these questions to help ensure compliance:

• Is this algorithm fair?
• If not, why not?
• Could the algorithm be fairer?
• How could being fairer economically affect our business?
• Did we double-check declined loan applications for undeserved denials?

Three of the 10 largest U.S. banks are already using FairPlay fair lending analysis software, Saleh says, without disclosing their names. Its newest partner, $7.6 billion Pathward Financial, was added Feb. 18, he says.

“Banks that use our software are often able to increase their approval rates by 10%, increase their take rates by 13% and increase positive outcomes by 20%,” he says.

Listen to this episode of “The Buzz” podcast as Saleh discusses how banks can leverage AI to improve loan approval rates.

Register here for Bank Automation Summit 2025, taking place March 3-4 in Nashville, Tenn. View the full event agenda here. 

Subscribe to The Buzz Podcast on iTunes or Spotify,  or download the episode. 

The following is a transcript generated by AI technology that has been lightly edited but still contains errors.

Madeline Durrett 12:24:31
Hello and welcome to The Buzz bank automation news podcast. My name is Madeline Durrett, Senior Associate Editor at Bank automation news today. I’m joined by Karim Saleh, founder and CEO at fairness as a service company, fair play. Kareem, thanks so much for joining me today.

Kareem Saleh 12:24:49
Thanks for having me delighted to be here.

Madeline Durrett 12:24:53
FairPlay is a fairness as a service company that uses artificial intelligence. Would you elaborate in your own words for our listeners on what exactly you do and how AI fits in? Yeah.

Kareem Saleh 12:25:05
So as you point out, fair play is the world’s first fairness as a service company. We allow anybody using an algorithm to make a high stakes decision about someone’s life to answer five questions, is my algorithm fair? If not, why not? Could it be fairer? What’s the economic impact to our business of being fairer? And finally, did we give our declines the folks we rejected a second look to make sure we didn’t deny somebody an opportunity they deserve. Some of the biggest names in financial services use our tools to automate the testing of their AI systems for blind spots and to identify opportunities to be fairer, within their risk tolerance. That ends up being good for profits, good for people, and good for progress.

Madeline Durrett 12:25:55
So it helps everyone, not just customers, but banks as well.

Kareem Saleh 12:25:59
That’s right, banks that use our software are often able to increase their approval rates by 10% increase their take rates by 13% and increase positive outcomes by 20%

Madeline Durrett 12:26:14
and are you able to disclose or give us any hints on some of the banks you work with?

Kareem Saleh 12:26:18
Yes, of course. So we work with four of the top 20 banks. The most bank that we announced as a fair play partner is path word, formerly meta bank. Path word is a pioneer in the banking as a service ecosystem. Some of the biggest brands who originate through sponsor banks like h and r block and opportune originate through pathword. And so we’re delighted to be working with the folks at pathword and with several other major financial institutions to help realize the benefits of AI investments

Madeline Durrett 12:26:55
and fair play was founded in 2020 How have the banks you’ve worked with how? How have their needs evolved in maybe the past four or five years? Yeah,

Kareem Saleh 12:27:05
well, I think when we came to market five years ago, in 2020 our focus was really on the fintechs who were using complex machine learning and AI techniques in credit underwriting and for the most part, originating through sponsor banks. And they needed to prove to their sponsor banks and their sponsor banks regulators that the AI models they were using didn’t pose a threat either to the safety and soundness of those institutions or to the consumers they served. But then, of course, chatgpt comes along in 2023 and changes everything now, I think since the advent of large language models a few years ago, we’ve now started to see banks feel like they have no choice but to get into the AI game and relatively quickly. And so while we were focused primarily on fintechs in the early years of our business, the last several years, we have been helping major financial institutions, big household names that you would recognize, implement their AI underwriting systems in ways that allow them to get the benefits of those investments while also maintaining compliance with laws like the Equal Credit Opportunity Act, the Fair Housing Act and other applicable regulations,

Madeline Durrett 12:28:22
and to kind of build on that, what are some of The risks associated with AI powered decisioning, and how is fair play mitigating these risks?

Kareem Saleh 12:28:31
Yeah, so the conventional wisdom is that AI stands for artificial intelligence, but it can sometimes also stand for accidentally incorrect machine machine learning systems are capable of learning the wrong things. Just to give you one example, when we started doing this work over a decade ago, we didn’t have we didn’t have our own data set to get started, and so we went out and we bought a data set from a failed lender, and we trained up some AI models, and we were very proud of ourselves, and the AI models that we trained up came back and said, Hey, you should make loans in Arkansas. And it just so happens that my co founder and chief technology officer is from Arkansas, and he happened to know that the regulatory regime in Arkansas was extremely high. Hostile to these kinds of loans. And so we started asking ourselves, Well, why does the aI think we should make loans in Arkansas? And we started digging into the data, and we found that the data set that we had purchased didn’t include any loans in Arkansas, which meant that the data set didn’t include any defaults in Arkansas, which allowed the AI to come to the conclusion that loans never went bad in Arkansas. And so, you know, these systems are only as smart as the data that you train them on, and they have a natural tendency to over fit to the patterns that are in the data. And so if you don’t have a real clear eyed view about this bias in the algorithms to overfit to the data, then you might miss the blind spots in your algorithms. And so that’s fundamentally what we do is help financial institutions stress test their AI, identify blind spots in their AI and then correct for those blind spots. And what we find is that something like 25 to 33% of the folks that financial institutions declined probably would have performed as well as the riskiest folks they approve.

Madeline Durrett 12:30:42
It’s really interesting. And so, as you mentioned, you know some financial institutions, they’re reluctant to deploy AI at scale, partly due to the cost and also partly due to the risks. Other banks are already establishing AI task forces and hiring AI specialists. So at a certain point, will AI integration at scale be a requirement for banks to remain competitive. Yes,

Kareem Saleh 12:31:10
if your competitors can see customers that you can’t and seize opportunities that you can’t, then, over time, they’re going to outperform you. So this is like a little bit, we’re in a little bit of an AI arms race. You can’t afford for the bank down the street to know something that you don’t. And so I think it’s only a matter of time before all of these institutions are using AI across their businesses. If you look at, you know, the famous Jamie diamond annual letter from a few years ago where he says, basically, Silicon Valley is coming for banks. I think that you know, the premonitions that he set forth in that letter are largely coming true. These banks are transforming into technology companies. You see that most clearly at places like Capital One, like JP Morgan, like Goldman, when they were in the consumer business, and now I think the folks that are kind of in that middle market and lower middle market are now racing to catch up with some of the their bigger peers.

Madeline Durrett 12:32:16
So how much do you see AI usage by banks and credit unions increasing by year end compared to last year? Yeah,

Kareem Saleh 12:32:25
I think that AI adoption in financial services is going to increase probably on the order of 3x this year. We’re seeing it. We’re seeing AI being applied across the customer journey, whether it’s in marketing or fraud detection or income verification or identity verification or underwriting or pricing or line assignment, or, heaven forbid, account management, collections, loss met, claims administration. I think that it’s inevitable that AI is going to basically touch everything inside of these banks, whether it’s the front office, the middle office, the back office,

Madeline Durrett 12:33:05
and you kind of touched upon this already. But how are factors such as open banking shaping the AI landscape in the financial services sector?

Kareem Saleh 12:33:14
Yeah, AI, excuse me, open banking, which facilitates cash flow, underwriting, I think, is rapidly emerging as the state of the art in credit analytics, because cash flow and tends to be the truest measure of the consumers balance sheet. one of the things about cash flow underwriting is that there are so many transactions that you have to contend with, right? Because you’re basically looking at every credit and debit to a consumer’s bank account. And I don’t know about you, but I probably use my debit card, you know, 10 plus times a day, whether it’s at the coffee shop or the gym or whatever. And conventional underwriting techniques that most lenders use, like logistic regression, can only consume about 20 to 50 variables. Results. So if you really want to get the benefits of cash flow underwriting, where the number of variables you have to contend with can be many hundreds, sometimes 1000s, you really need a mathematical approach like AI that can consume an infinite amount of information and that can also be resilient to data that’s messy, missing or wrong. And so I think open banking is has put the industry on a trajectory towards kind of continuous underwriting. People will be underwritten all the time on the basis of data that is very, very current, and that on the basis of data that represents a more accurate portrait, a finer portrait, if you will, of a borrower’s ability and willingness to repay a loan, then perhaps conventional underwriting techniques, which can only consume a limited amount of data and tend To be based on credit reports which have necessarily a reporting lag and which may not fully reflect the consumer’s balance sheet,

Madeline Durrett 12:35:36
makes a lot of sense. So I want to pivot to some recent news. You were selected by MasterCard for its start PATH program, which aims to drive forward AI powered financial services. So how did Fair Play end up in this program? And what are you most excited about in being a part of it?

Kareem Saleh 12:35:56
Well, as you may know, MasterCard purchased finicity A few years ago to get into the cash flow underwriting attribute space, and so MasterCard is working very hard when it’s with its many 1000s of bank partners to bring the benefits of cash flow underwriting to the broader MasterCard ecosystem. And so we were delighted to be selected by MasterCard to participate in start path to really trade notes and identify areas of cooperation to accelerate the adoption of cash flow underwriting in the banking sector, both in the US and globally. So it’s still early days, but the way we typically work with cash flow underwriting attribute and score providers is to make sure that the data sets are representative, because, again, we’re trying to avoid blind spots. We’re trying to make sure that the scores developed on the basis of those cash flow underwriting is predictive and representative, and what we find is actually because cash flow underwriting is in some sense, the truest measure of the consumer’s balance sheet. The outcomes of cash flow underwriting are extremely fair, because you’re really measuring people on the basis of their ability to pay back a loan. So we’re really excited to be working with MasterCard and the 1000s of banks in their networks to try to realize some of the potential gains that are to be had from cash flow underwriting and AI, thank

Madeline Durrett 12:37:34
you, and you kind of answered this, but what? What are some of the advantages of being part of a consortium with other startups, and as you mentioned, other banks, especially when trying to stay ahead of the curve and identify industry needs as they arise.

Kareem Saleh 12:37:50
Well, the benefits of being in a consortium with MasterCard, with other startups, with 1000s of other banks, is the ability to trade best practices as this ecosystem develops. So I think we’re very early in the adoption of cash flow underwriting. That’s meant that there are kind of inconsistent standards, for example, with the with respect to the categorization of cash flow underwriting attributes and other questions related to kind of the appropriateness of certain data points and how they might be used to assess consumers. And so the great benefit of start path is being in a consortium with our peers, with our customers, with banks who are also grappling with these same issues, so that we can try to identify standards and best practices that’ll lift the whole ecosystem up.

Madeline Durrett 12:38:44
So what else is in the pipeline for fair play this year. What are some of your 2025, Business and Technology goals?

Kareem Saleh 12:38:50
Yeah, so one of the areas where we’re seeing a lot of growth is in the area of benchmarking. The lenders that we work with want to understand how the outcomes of their own underwriting compare to the outcomes of some of their peers, and to understand if there are geographies or populations, perhaps where that they’re missing or where they could do better. And so over the course of the next year, with one of the major credit bureaus and some of our major financial institution partners, we’re going to be launching the Fair Play fair ness index. Costs, which allows financial institutions to benchmark their underwriting outcomes, not just in mortgage, where those kind of benchmarks exist today, but in other non mortgage asset classes, like auto lending, installment lending, credit card lending, earned wage access, buy now, pay later, etc, so that we can bring more visibility into what the underwriting outcomes are for certain populations, certain geographies and across the credit spectrum.

Madeline Durrett 12:39:58
I really look forward to receiving updates on that. Well, this has been the buzz podcast. Thank you so much. Karim Saleh, founder and CEO at fairness as a service company, fair play for joining me today, please be sure to follow us on LinkedIn, and as a reminder, you can rate this podcast on your platform of choice. Thank you for your time, and be sure to visit us at Bank automation news.com, for more. Automation News,

Kareem Saleh 12:40:30
thanks, Madeline.

Transcribed by https://otter.ai