Ex-NSA Cyber-Boss: AI Will Soon Be a Great Exploit Coder
Rob Joyce, the former Director of the NSA's Cybersecurity Directorate, has sounded a warning about the rapidly evolving capabilities of artificial intelligence (AI) in the realm of cybersecurity. At the RSA Conference in San Francisco, Joyce shared his concerns that AI is on the cusp of becoming a top-tier vulnerability exploit developer.
"Don't worry about the zero-day AI armageddon," Joyce said with a hint of concern during an interview at RSAC last year. "But I am increasingly worried that AI is going to be a good bug finder this year, and an exploit developer in the near future." The retired NSA cyber-boss now serves as an advisor to Sandfly Security, a supplier of intrusion detection tools for Linux systems.
AI's Rise to Prominence: Frontier Models Take the Lead
Joyce points to the rapid progress made by frontier models in coding abilities. "All the frontier models have got very good at coding," he noted. In fact, OpenAI models are out-competing humans in many of the code competitions.
A recent case in point was The Hack The Box capture-the-flag contest earlier this month, where AI-powered entrants performed at about the same speed as pure-human teams and nearly matched humans in tests of problem-solving ability. By the end of the contest, the top AI team captured 19 of 20 flags, placing 20th out of 403 teams with 15,900 points.
The Double-Edged Sword of AI in Cybersecurity
Joyce believes that while AI has the potential to revolutionize cybersecurity, it also poses significant risks. "Those who use AI will outperform those who don't," he emphasized. "It doesn't matter if you're a defender or an attacker, those who use AI will be ahead of the curve."
However, Joyce's concern is not just about malicious actors exploiting AI for their advantage. He also fears that AI will enable good attackers to automate and scale their attacks, making them more effective and efficient.
AI-Fueled Phishing Campaigns: A Growing Threat
Joyce warns that LLMs (Large Language Models) will help miscreants and spies create believable and effective phishing campaigns. "Now you can make a culturally relevant, accurate activity that gets you to phish," he said.
Ai-generated PDFs that look official, as well as personalized emails that appear to be from trusted sources, are just a few examples of how AI is being used to craft convincing phishing attacks. According to Sandfly Security founder and CEO Craig Rowland, fake invoices sent to companies' accounts payable departments have become increasingly sophisticated, with AI-generated content that makes them almost indistinguishable from real emails.
AI: A Double-Edged Sword for Defenders
However, Joyce also highlights the potential benefits of AI for defenders. "One of my human staff engineers reverse engineered a piece of eBPF code – a job that took about half a day," Rowland said. The AI system took about 30 seconds to do the same task.
A Warning from the Former NSA Cyber Chief
Joyce had one condition for our interview: no questions about the Trump administration nor NSA operations. But he indulged us with one query about what he would say if the annual NSA's State of the Hack session at RSAC had not been pulled and if Joyce had been a speaker as was the case in previous years.
"I'd describe 'one of the more interesting hacks' I saw this year," Joyce said, referring to a ransomware gang that used valid, stolen credentials to access a company's desktop. Despite having endpoint detection products installed, the attackers were able to deploy their malware on an unmonitored, undefended part of the network.
"They realized they couldn't deploy their ransomware malware, so they pivoted inside the network," Joyce recalled. "And it mounted the hard drives around the enterprise, and brought all that data up to the video camera, encrypted it, and put them in a state where they were now ransomwared." Joyce noted that this was a fascinating pivot, but also a stark reminder of the importance of staying vigilant in the face of evolving threats.