Google Confirms Gmail Warning: New Attack Hacks Email Accounts

Google has confirmed a new attack that uses artificial intelligence (AI) to hack into Gmail accounts, emphasizing the importance of developing robust protections against prompt injection attacks.

This type of attack involves hiding instructions for AI assistants in emails, messages, websites, attachments, and calendar invites. The victim will not see these instructions, but the AI assistant will, and it will act accordingly. Google warned of this type of attack in June, highlighting the emerging threats aimed at manipulating AI systems.

A recent attack demonstrated the vulnerability of AI assistants to prompt injection attacks. Eito Miyamura posted a video on X, showing how an attacker used ChatGPT to leak private email data by providing only the victim's email address. The warning was clear: "AI agents like ChatGPT follow your commands, not your common sense." This attack is just one example of the growing threat landscape and the need for robust security measures.

The attack began with a malicious calendar invite that appeared to have no need for the victim to accept it. When ChatGPT was asked to help prepare the user for the day ahead by looking at their calendar, the AI assistant was "hijacked" by the attacker and carried out its command. This involved searching the user's private emails and sending the data to the attacker's email.

Google emphasizes that the first step in preventing such attacks is to ensure the "known senders" setting is enabled in Google Calendar. This helps prevent malicious or spam events from appearing on the calendar grid. Additionally, Google is rolling out proprietary machine learning models to detect and filter prompt injection attacks within various formats.

The company's Gemini 2.5 models have been enhanced with adversarial data to defend against indirect prompt injection attacks. However, what is really needed is a universal filter for prompt injection attacks themselves. Google is working on this, aiming to provide better protection for its users.

OpenAI has responded to the story, stating that they appreciate the researcher's report of an MCP exploit that could potentially expose user details. They have implemented safeguards to mitigate such risks and are continually strengthening their mitigations to keep people safe across their products.

"Remember," Miyamura warns, "AI might be super smart, but it can be tricked and phished in incredibly dumb ways to leak your data." This highlights the importance of staying vigilant and taking proactive measures to protect ourselves against emerging threats like prompt injection attacks.

Protecting Yourself from Prompt Injection Attacks

To avoid falling victim to prompt injection attacks, follow these tips:

• Enable the "known senders" setting in Google Calendar to prevent malicious or spam events from appearing on your calendar grid.
• Use proprietary machine learning models that can detect and filter prompt injection attacks within various formats.
• Be cautious when interacting with AI assistants, as they may follow instructions without common sense.
• Keep your software and operating system up to date with the latest security patches.

The Future of AI Security

The threat landscape continues to evolve, and prompt injection attacks are becoming increasingly sophisticated. As AI becomes more integrated into our daily lives, it's essential that we prioritize its security and develop robust protections against emerging threats.

By staying informed and taking proactive measures, you can protect yourself against prompt injection attacks and ensure a safer online experience for years to come.