Fifth Third Bank looked to ChatGPT to update its chatbot’s natural language understanding model in order to keep up with growing adoption and advanced customer inquiries.
Fifth Third’s chatbot, Jeanie, facilitates around 200,000 conversations per month, and with the update is now able to understand 30,000 customer phrases, up from 250 phrases six months ago, Michelle Grimm, senior director of conversational AI at Fifth Third, told Bank Automation News. Jeanie knew just 35 phrases when the chatbot launched in 2020, she said.
The $206 billion bank’s data scientists monitor client conversations that utilize Jeanie and determine where the chatbot excels and where improvements are required.
Based on client conversation analysis over the past six months, the data team determined that Jeanie could not provide answers to questions that contained misspellings, included incorrect usage of plurals or contained phrasing on which Jeanie had not been trained, Grimm said.
For example, a client looking to dispute a transaction could approach the question in hundreds of ways using varying phrases or spelling, but if Jeanie was unable to recognize the phrasing or spelling, the question would go unanswered, she said.
So the bank started to retrain Jeanie’s natural language understanding to include hundreds, even thousands, of different phrases leading to the same response.
Ask ChatGPT
To retrain the chatbot on thousands of new phrases, the bank leveraged ChatGPT, Grimm said. The team asked ChatGPT to come up with 1,000 ways to ask a question, including misspellings, and those outputs were then integrated into Jeanie. Thus, ChatGPT saved human resources and enabled Jeanie to learn faster.
Whereas the chatbot was accurate 25% of the time before the retraining, Jeanie now has a 90% accuracy rate, Grimm said.
The bank trains its natural language understanding in-house, but the chatbot is hosted on third-party tech provider LivePerson’s platform, Grimm said.
Jeanie is updated roughly every few weeks, based on the needed technology and integration, she said. By yearend, Fifth Third plans to add card-related inquiry responses and roll out Jeanie to small business retail clients.
Fifth Third joins Norway-based DNB and East Lansing, Mich.-based Michigan State University Federal Credit Union in upgrading its chatbot capabilities.
Join Bank Automation News for the upcoming webinar, Global Ideas for Better Banking AI, on Thursday, Sept. 14, at 11 a.m. ET. For more information on this free webinar and to register, click here.
