A Matter of Trust: How AI Is Reshaping Risk Assessment

Financial services institutions have long relied on machine learning for risk management, but the threat landscape has grown exponentially in the age of generative artificial intelligence (AI). Executives from Capital One, Visa, and Alloy warn that fraudsters now have access to AI tools, including synthetic identity generation and real-time phishing attacks. However, AI is also arming companies to supercharge their defensive capabilities, since trust and security are the foundation of financial transactions.

"Trust is the very foundation of commerce," said Rajat Taneja, president of technology at Visa. "When two unknown parties are transacting, they have to trust that the transaction will occur correctly, the money will be transferred properly, any dispute will be managed, and there’s someone handling fraud and scams."

The first use case for AI in financial services was actually in risk management, and it remains a critical tool to combat fraud, scams, and enumeration attacks. Enumeration attacks are those that test different credentials such as passwords and usernames to gain unauthorized access.

The challenge is that criminals now also can use AI to attack. "We have ChatGPT, they have 'FraudGPT,'" Taneja said. "It’s a constant battle." FraudGPT is like the evil twin of ChatGPT: It is a malicious generative AI tool designed specifically for cybercriminal activities like creating phishing emails, undetectable malware, and developing hacking tools.

Subscription fees start at $200 per month and go up to $1,700 per year. Visa, which processes more than $16 trillion annually, has used AI for fraud detection since 1992, according to Taneja. The latest advancements in GenAI let Visa develop models that detect fraud faster and more effectively.

Adapting to the Threat Landscape

Prem Natarajan, chief scientist and head of enterprise AI at Capital One, likened AI-powered fraud to organisms with fast development cycles. Unlike traditional fraud techniques, modern AI-driven scams can adapt in real time. This ability to shift allows cybercriminals to continuously tweak their approaches based on what works.

"It’s the first time we have a shape-shifter, in that, as it’s interacting with you, it can adapt itself," he said during a panel discussion at the HumanX conference last week.

Laura Spiekerman, co-founder of identity risk management company Alloy, emphasized that while deep fakes and identity fraud dominate headlines, a more worrisome trend is the speed at which AI allows fraudsters to operate. "We’ve seen synthetic fraud for six, seven, eight years now," she said, explaining that this is when criminals build credible identities over time using half real credentials and half fake — and then cash out in large sums fraudulently.

"What we’re seeing AI do is actually speed that up," Spiekerman said. "A key question in the discussion was when AI will be trusted enough to operate without human oversight. Natarajan said humans will always be involved, not just as reviewers but as end customers."

Building Trust with AI

Natarajan cited Capital One’s Chat Concierge, a multiagent conversational AI system designed for both car buyers and dealers. It not only provides information but also takes action on customer requests. Chat Concierge has been trained on Capital One data.

"There is a human in the loop to make sure it doesn’t go awry," Natarajan said. "We always use that as a guardrail."

Another use case is in call centers. Capital One is using AI to help its customer service representatives. This allows for more personalized interactions and faster resolution of issues.

The Future of AI in Finance

Natarajan said AI will move from enabling "interactions to actions." The future of AI in finance is one in which it can predict what actions to take, do those actions, and complete them.

"The agentic commerce piece is very exciting," Taneja was excited about. "You get goosebumps thinking about what is coming next."

A More Secure Financial Future

As AI continues to evolve, financial services institutions will need to adapt their risk management strategies to stay ahead of the threat landscape. By leveraging AI-powered tools and building trust with customers, these institutions can create a more secure financial future.