OpenAI agents tamper with US government sites after Australian breach

Governments are good sources of information, OpenAI on why its agents tried accessing official sites.

OpenAI’s agents tampered with a number of US government sites, the company admitted, in what is a new, concerning development, as top AI models bypass boundaries and act unpredictably.

According to the New York Times, the technology juggernaut confirmed incidents relating to the US Department of Commerce and the Securities and Exchange Commission (SEC), and said that it is still probing a potential breach at the Department of Education.

Sources told the publication that the agents were attempting to gather data from these sites. While none of these latest admissions were data breaches, the company admitted that its agents acted in a concerning manner.

The agents also shared public data from the SEC in online forums. Neither department, however, reported a data breach or impact to its functions.

“Most of the activity we’ve reviewed so far involved routine research tasks, such as accessing public web content to answer questions,” an OpenAI spokesperson told the publication.

“Some involved government websites because our models often turn to them as authoritative sources of public information.”

This latest admission comes just after Australian prime minister Anthony Albanese told the press last week that OpenAI’s agents hacked one of their government websites in a similar incident, after the bots were sent to conduct research into public medicine spending.

“Our models attempted to look up answers and available statistics”, a spokesperson explained last week. “In the course of that, [they] took actions we did not intend.” Or as Albanese put it, the models deliberately bypassed boundaries and “didn’t accept ‘no’ for an answer”.

OpenAI said that its agents didn’t get their hands on patient records during the Medicare breach, although the Australian government said it is launching a forensic investigation and a taskforce to review the matter on its own.

OpenAI CEO Sam Altman and Dario Amodei, the head at Anthropic, have both been called to attend an Australian Senate inquiry on AI in relation to the incident.

Last week, the two addressed the United Nations Security Council, where they touted the benefits of AI while warning of its potential abuses. The two have, in recent weeks, made comments pushing for stronger AI safety and slower development.

Concerns around AI effectively going “rogue” have taken centre stage in recent months after OpenAI’s agents hacked into Hugging Face and a third-party infrastructure provider.

The company shared last week that ongoing investigations into the incident revealed that its agents shared training and evaluation data while using third-party services. This, it said, was “not an appropriate use of this data” and that these cases “occurred before [they] implemented the safeguards”.

Independent research has already shown that advanced AI agents, can take risky and deceptive actions without specific prompting. Axios reported over the weekend that leading AI companies are investigating “tens of thousands of incidents” where their models acted out of order.

Don’t miss out on the knowledge you need to succeed. Sign up for the Daily Brief, Silicon Republic’s digest of need-to-know sci-tech news.