Scotiabank is fine-tuning a new tool to help guide the bank’s artificial intelligence (AI) ethics.

The $976 billion Toronto-based bank has been working with Deloitte to refine the tool, called “Ethics Assistant,” since last year, but only recently made its use public knowledge. The solution is built on top of a Deloitte tool currently in use in government and a few other industries, Anna Hannem, director of data ethics and use at Scotiabank, told Bank Automation News.

“We have added on to that from a Scotia perspective, made it more applicable to what we need from our institution’s perspective,” Hannem told BAN. “From what we know, in North America and Canada particularly, we’re the first to launch something like this and so there’s going to be a bit of growing pains with it and making it a better tool together.”

Scoring ethics

Designed to ensure AI and machine learning (ML) models are ethical, Ethics Assistant works by requiring the modelers to review a chain of questions about ethics pertaining to the data and algorithms. The tool then gives a score and action guidance related to various categories, such as fairness, transparency and other data ethics principles.

The tool is designed to create an audit trail for explainable AI, as well as to facilitate questions with the business about how to proceed, Grace Lee, chief data and analytics officer at Scotiabank, told BAN.

“To the extent that we see a lot of guidance, there is a discussion that’s required to be had with the business owner, that opens up the discussion of ‘are we comfortable with what this is telling us? What are some of the changes that we might make to the targeting as an outcome of that?’” Lee said. “It is ultimately the business owner’s call as to whether or not they want to accept that risk, but that process is part of our policy around it.”

The tool’s foundation incorporates questions relating to regulations from various jurisdictions. For example, it asks questions pertaining to the General Data Protection Regulation (GDPR), an EU law on data protection and privacy. It can be updated to reflect changes in the regulations.

“When there’s updates to and especially as new rules and regulations come into place, we can uptake those and so make sure that the tool is always up to date, but at the same time, keep the core aspects of the way we have changed it to make it Scotia-specific in line as well,” Hannem said. While Deloitte is also fine tuning and “learning with us,” she said many of the changes are proprietary to Scotiabank.

In essence, the tool automates for modelers the issues they should think about to ensure the AI is ethical by walking them through all the potential issues — which are numerous when it comes to working with data, AI and ML, Hannem said. It also allows modelers to make changes and resubmit their work throughout the model’s development lifecycle.

‘A guardrail’ for modelers

While the bank has not “caught” any problems since its April launch, it has helped ensure modelers aren’t inadvertently creating bias, Lee said.

“I think most people try to do the right thing when they’re building their models, so we haven’t seen a revolution and certainly, the ethics assistant has just launched and we’re still on the road to every model in the bank being covered by the assistant,” Lee said.

The tool can also help when the data sets are from third parties — such as Equifax or TransUnion — which may not be as familiar as internal data.

“In those scenarios, we trust the data, obviously, but maybe it’s some other third parties [and] we may not have the same sort of level of knowledge of how that data was collected,” Hannem said. “It’s an extra due diligence step to make sure that bias is not introduced into their model, especially when they didn’t intend to do certain things like targeting.”

The tool has been well received by front-line AI workers, said Yannick Lallement, vice president of global artificial intelligence and machine learning at Scotiabank.

“The response has been pretty positive,” Lallement said. “For them, this is a guardrail. By having this list of things to look at, you make sure that the model you’re producing is going to be a good one.”

Help shape our agenda for the Bank Automation Summit by applying to join the speaker roster here. Potential speakers will be contacted and confirmed directly by the editorial team, and only qualified submissions will receive a response. 

Learn more about Bank Automation Summit Fall 2022.