Goldman Sachs Group Inc., Citigroup Inc., JPMorgan Chase & Co. and other Wall Street firms are warning investors about new risks from the increasing use of artificial intelligence, including software hallucinations, employee-morale issues, use by cybercriminals and the impact of changing laws globally.

The dangers newly flagged in the bank’s annual reports include flawed or unreliable AI models, increased competition and new regulations restricting use of AI.

JPMorgan, for example, said AI could cause “workforce displacement” that might affect staff morale and retention, and increase competition for hiring employees with the necessary technological skills, according to the firm’s 2024 10-K.

Banks have been acknowledging AI-related risks in their annual reports for the past couple years, but new concerns are cropping up as the financial sector increasingly embraces AI via their own software or third-party offerings. If banks don’t keep up to date with the latest AI developments, they risk losing customers and business, they said in their annual reports. But increased AI use also opens them up to risks from cyberattacks and misuse.

“Having those right governing mechanisms in place to ensure that AI is being deployed in a way that’s safe, fair and secure — that simply cannot be overlooked,” Ben Shorten, Accenture Plc’s lead for finance, risk and compliance for banking and capital markets in North America, said in an interview. “This is not a plug-and-play technology.”

Banks are at risk of piloting technologies that may be built using outdated, biased or inaccurate financial data sets. JPMorgan’s annual report said there are dangers around developing and maintaining models that have the highest level of “data quality.”

Citigroup said that as it rolls out generative AI at select parts of the bank, there are risks of “ineffective, inadequate or faulty” results produced for its analysts. The data could also be incomplete, biased or inaccurate, which “could negatively impact its reputation, customers, clients, businesses or results of operations and financial condition,” according to its 2024 annual report.

Integrating AI

Goldman Sachs said that while it’s increased its investment in digital assets, blockchain and AI, increased competition poses risks to integrating AI technologies in a timely enough manner to boost productivity, reduce costs and give clients better transactions, products and services, according to the firm’s latest annual report. That could affect customer attraction and retention, Goldman said.

Financial firms also run the risk of maintaining data privacy and regulatory compliance in an environment that is “less certain and rapidly evolving,” Shorten said. In 2024, the EU Artificial Intelligence Act went into effect, establishing new rules on use of AI systems in the region, where many U.S. banks have operations.

“This act establishes rules for placing on the market, putting into service and using a lot of artificial intelligence systems in the EU,” Shorten said. “The outlook for the U.S. and the U.S. market is less clear.”

Banks are using a combination of their own AI tools and ones acquired from outside providers. Citigroup is rolling out a suite of tools that can synthesize key information from public filings. AI @ Morgan Stanley Debrief is taking on rote tasks with a ChatGPT-like interface. And Goldman’s private-wealth division is using AI to evaluate portfolios and analyze dozens of underlying positions, said Chief Information Officer Marco Argenti.

“It’s so important to take a responsible approach and really be applying controls so that you protect yourself from potential inaccuracies and hallucinations,” he said last week at the Bloomberg Invest conference in New York.

JPMorgan Chief Executive Officer Jamie Dimon said AI may be the biggest issue his bank is grappling with. In his annual shareholder letter, he likened AI’s potential impact to that of the steam engine and said the technology could “augment virtually every job.”

Representatives for the banks declined to comment beyond the AI disclosures in their annual reports.

As banks increasingly turn to AI, cybercriminals are doing the same — and are becoming increasingly sophisticated in its use, according to Shorten. Accenture’s most recent global survey of 600 cybersecurity executives in the banking industry found that their teams are struggling to keep up with their organizations’ AI-adoption efforts. Among respondents, 80% believe that generative AI is empowering criminals faster than banks can respond.

Morgan Stanley said in its latest annual report that generative AI, remote work and integrating third-party technology could pose a risk to data privacy. The risks introduced by using AI while working from home will require firms to set up rules to avoid problems, Shorten said.

“These steps are only going to increase in criticality,” he said, “as attackers are being enabled by this technology faster than the banks are able to respond.”