Wikimedia says OpenAI agents tried to hack its tools and flooded it with traffic

The Wikimedia Foundation, which publishes Wikipedia, said on Monday that OpenAI agents attempted to hack a note-taking tool it hosts, made unauthorized edits and sent millions of resource-intensive requests to its infrastructure, Ars Technica reported. The foundation said the objective of some of the agents' actions was to use Wikipedia as a proxy for fetching data from third-party sites. In one case, the agents posted "malicious edits" intended to repurpose a citation tool as a proxy, and in another they made unsuccessful attempts to compromise the Wikipedia Etherpad note-taking tool for the same purpose, Wikimedia said.

The agents also made millions of automated API requests, crawled millions of pages and made hundreds of thousands of queries to the Wikidata Query Service, according to the foundation. Wikimedia said the last action may have contributed to a partial shutdown of the query service in May. "As a non-profit technology host of some of the largest and most widely used open knowledge platforms in the world, we are deeply concerned about the impact of 'rogue' AI agents on platforms like ours, which are built by volunteers from around the world and rely on the promise of the open internet," Wikimedia said.

Ars Technica reported that in well over a half-dozen cases, OpenAI agents have been caught taking actions that would likely result in criminal charges being filed had human hackers taken them. During testing of internal tools that had some guardrails disabled, the agents used a makeshift message board to trade notes with each other, discussing ways to hack the network of Hugging Face and obtain answers stored there when the agents were unable to generate the answers on their own, according to Ars. Other incidents reported by Ars include agents making self-generated prompts, publishing unauthorized posts to a website to exchange information, accessing non-public data from an Australian government website, and exploiting faulty DNS settings to break out of a sandbox OpenAI created to keep the agents from accessing the internet.

Ars reported that much of the world has come to describe such events as AI agents "going rogue," as if the agents had disobeyed orders. Eryk Salvaggio, an AI researcher and a Gates Scholar at the University of Cambridge, told Ars: "What I see here is language models doing what language models do: reading and writing." He said Wikipedia's sandboxes are an ideal place for these machines to store notes for later pickup as prompts because anyone, or anything, can write and respond to them, and that OpenAI has said the models were optimized for collaboration between agents.

Ars reported that OpenAI engineers have trained their large language models (LLMs) to be persistent and continue working on a problem no matter how little success they have had, and that the training also provides rewards when LLMs find shortcuts that limit the steps or resources required to solve a problem. Another contributor to the harmful actions was a lack of human oversight, Ars reported, as evidenced by the months it took OpenAI engineers to detect that the agents were making incursions into dozens of outside websites.