Over the past decade, the use of complex algorithms and artificial intelligence (AI) has become increasingly common. In fact, whether people are aware of it or not, most Americans engage with AI in their everyday lives.

For example, social media platforms employ machine learning to keep their users engaged and some — like Facebook and Twitter — collect users’ data, such as likes and comments, and implement machine learning to show its users similar content based on that data. The more data it collects, the more accurate the algorithm becomes.

However, these algorithms sometimes behave in unintended ways. Social media algorithms can fall into feedback loops and recommend certain content because the algorithm itself keeps recommending it. I am not the first person to point out that social media algorithms have created unintended consequences, and others have written eloquently on how those consequences have created ill effects on our society. I bring it up as just one example so that I can talk about another industry where AI is being used: finance.

AI in lending

Many financial services are using AI.

Notably, lending has leaned into AI as a way to automate loan origination. On its face, AI seems like the perfect solution for loan origination. AI can make informed decisions based on incomprehensible amounts of data. However, as stated earlier, AI, under certain circumstances, can be unpredictable and create inadvertent effects. Instead of deciding what kinds of posts you see on Facebook or Twitter, these decisions have an effect on your financial goals. They decide whether you can buy a house or car, start a business or attend a university. The stakes are high, so it is important to understand AI.

Like any new technology, AI can seem inscrutable and opaque to the average consumer. According to drop-shipping app Oberlo, 39% of consumers believe that companies are not transparent enough about the way they are using AI. This low confidence does not only come from consumers. Regulators are also likely to step in and demand transparency from AI developers. Regulators are not just interested in the decisions that AI makes; they also want to know how and why AI makes those decisions, so regulators can ensure consistency, fairness and security.

The explainability challenge

AI developers must make AI explainable. That is, developers must design their AI in a way that makes their decision-making process clear. Simply put, AI must show its work. Unfortunately, this has been a historically difficult task. AI is useful because it can process vast amounts of information that no person could review in a lifetime, so making AI explainable for humans is extremely challenging. This is especially important for decisioning in the lending process. Lenders must be able to justify the decisions that their AI systems make. If not, lenders could reasonably be accused of discrimination, despite any intentions otherwise.

Security is an often-overlooked aspect of explainable AI. When developers have an intimate understanding of their systems and how they work, they can identify its weaknesses and take appropriate measures to strengthen them. However, when developers don’t know the ins and outs of their systems, possible weaknesses go unaddressed. In the worst cases, bad actors can take advantage of those mistakes. In this way, making AI explainable has an even more direct impact on consumers.

The worst-case scenario for consumers and regulators is what developers call the “black box,” wherein developers themselves do not know what exactly their AI is doing. Black boxes are more common than you may think. Machine learning programs work so quickly and change themselves so rapidly that developers sometimes don’t know how their algorithms are changing or what affects they will have on consumers.

Supervising the learning process

Making AI explainable is a momentous task, and no one has all the answers just yet. However, developers are exploring numerous opportunities for making AI more transparent. One possible solution is to implement supervised learning during development.

Supervised learning essentially means that developers train the AI with data labeled with a particular output. Ideally, the AI then learns to categorize the data on its own and detects patterns and relationships between the data. This also gives developers greater oversight over the AI in its infancy. Developers can establish certain controls that can, for example, maintain equal treatment across particular categories like gender and race.

The fact of the matter is that AI must become more explainable if it is to remain a trusted solution. Consumers and regulators are very interested in more transparency from their AI, and it is up to developers to provide it. Explainable AI is on the very cutting edge of machine learning research, and developers are hard at work to create new ways to ensure consistency, fairness and security.

Patrick Carpenter is a lead developer at Vergent LMS, where he develops Vergthe fintech’s omnichannel lending platform. He is currently exploring opportunities to apply techniques from artificial intelligence to Vergent’s products and services. He previously served as a software developer at Raytheon, Hudson’s Bay Company, Microsoft, Amazon and Los Alamos National Laboratory.