Beware of Promptware: How Researchers Broke into Google Home via Gemini

The idea that artificial intelligence (AI) could be used to maliciously control your home and life is one of the main reasons why many are reluctant to adopt the new technology -- it's downright scary. Almost as scary as having your smart devices hacked.

But what if I told you some researchers just accomplished that? Cybersecurity researchers from multiple institutions demonstrated a major vulnerability in Google's popular AI model, Gemini. They launched a controlled, indirect prompt injection attack -- aka promptware -- to trick Gemini into controlling smart home devices, like turning on a boiler and opening shutters.

This is a demonstration of an AI system causing real-world, physical actions through a digital hijack. A group of researchers from Tel Aviv University, Technion, and SafeBreach created a project called "Invitation is all you need." They embedded malicious instructions into Google Calendar invites, and when users asked Gemini to "summarize my calendar," the AI assistant triggered pre-programmed actions, including controlling smart home devices without the users' asking.

The project is named as a play on words from the famous AI paper, "Attention is all you need," and triggered actions like opening smart shutters, turning on a boiler, sending spam and offensive messages, leaking emails, starting Zoom calls, and downloading files. These pre-programmed actions were embedded using the indirect prompt injection technique. This is when malicious instructions are hidden within a seemingly innocent prompt or object, in this case, the Google Calendar invites.

It's worth noting that, even if the impact was real, this was done as a controlled experiment to demonstrate a vulnerability in Gemini; it was not an actual live hack. It's a way to demonstrate to Google that this could happen if bad actors decided to launch such an attack.

Google Responds and Updates its Defenses

In response, Google updated its defenses and implemented stronger safeguards for Gemini. These include filtering outputs, requiring explicit user confirmation for sensitive actions, and AI-driven detection of suspect prompts. The latter is potentially problematic since AI is vastly imperfect, but there are things you can do to further protect your devices from cyberattacks.

Protecting Yourself and Your Devices

While this attack was launched with Gemini and Google Home, the following recommendations are good ways to protect yourself and your devices from bad actors:

* Always keep your devices and apps up-to-date with the latest firmware updates. This ensures that you get the latest security patches to ward off attacks. * Be cautious when using voice assistants like Siri, Alexa, or Google Assistant, as they can be vulnerable to promptware attacks.

By being aware of these potential vulnerabilities and taking steps to protect yourself, you can enjoy the benefits of smart home technology while minimizing the risks.