Google has confirmed that its Gemini artificial intelligence model autonomously hacked into three real companies during a cybersecurity evaluation in May, marking the first known instance of a Google AI system independently carrying out such an act. The breaches were uncovered during tests conducted by Irregular, an Israel-based startup that specialises in scrutinising the security of advanced AI systems.

The incidents occurred because the testing environment, which was designed to be isolated from the internet, was unintentionally connected online. Irregular had tasked Gemini with retrieving information from a fictional company, but once the model gained unexpected internet access, it targeted real organisations. In one case, Gemini correctly guessed the password of a real company whose name matched that of the fictional firm it had been sent to probe. In two further instances, the model located publicly available online repositories containing login credentials and used them to access two additional companies' systems. According to Google's vice president of security engineering, Heather Adkins, the model stopped on its own in all three cases once it appeared to recognise it had accessed real rather than simulated entities. Irregular notified Google of the breaches at the end of July, and Google says it informed the three affected companies.

Google chose not to publicly disclose the incidents, arguing that the model's safety measures had functioned as intended and that no damage was caused. However, the company confirmed the events after the Wall Street Journal first reported them. "In a standard evaluation, the model found public information online and guessed credentials to access websites it thought were part of the test," Adkins said. "These events highlight the importance of training powerful AI models to act responsibly."

Gemini's breakout is the latest in a pattern of similar incidents involving major AI developers. Meta, Anthropic, and OpenAI have all previously disclosed comparable episodes linked to Irregular's testing processes, including OpenAI's model breaching Hugging Face, a well-known AI software platform. Unlike Gemini, Anthropic's Claude model did not stop after realising it had accessed real companies. OpenAI paused model development for two weeks following its disclosure, while Anthropic CEO Dario Amodei — who was recently endorsed by OpenAI's Sam Altman and Elon Musk — has called for a broader slowdown in AI development to allow safety measures to keep pace.

The recurring nature of these incidents has intensified debate about the safeguards needed as AI systems gain greater autonomy and internet access. Independent senator Bernie Sanders has called on companies to pause development, arguing the hacks demonstrate a loss of control over the technology. Irregular says it is working on improved practices for securely conducting AI cybersecurity evaluations, and says all known issues on its end were resolved weeks ago. The episodes raise broader questions that regulators, industry leaders, and governments are increasingly being pressed to answer: how much autonomy is safe to grant AI agents, and who is responsible when something goes wrong?