Artificial intelligence, explained.
Nvidia CEO Jensen Huang, on stage at the Consumer Electronics Show in Las Vegas, Nevada, in January, is one calling for accelerated development of AI (Patrick T. Fallon/AFP via Getty Images)
Slowing AI development is unrealistic – building the capacity to detect and report when it misbehaves is not.
Subscribe to The Informer for monthly expert analysis, and to Events for advance notice of visiting world leaders and distinguished guests.
You may unsubscribe from Lowy Institute newsletters at any time. For information on our privacy practices and how to unsubscribe, see our Privacy Policy.
The AI alarm is loud. The incident response is too quietThe AI alarm is loud. The incident response is too quiet
A sombre Anthony Albanese stood at a rostrum in New York on 24 September with a warning to computing giant OpenAI.
“This situation is obviously unacceptable”, Australia’s Prime Minister said (Opens in new window).
Albanese disclosed that an OpenAI agent, on the lookout for statistics, had broken into an Australian government website, gaining unauthorised access to files in a statistics portal containing data from Australia’s healthcare scheme, Medicare. The most astonishing thing about the incident is that the Australian government only received notification about the intrusion a full three months after it happened. Albanese criticised both the delay and the way the notification had been sent – to “a general inbox of an Australian government agency (Opens in new window)”.
The story sounds like the opening scene of a B-grade AI disaster movie. Before panicking, however, note that currently the actual consequences of this intrusion are still unknown. The portal held aggregate Medicare statistics and was separate from systems handling individual claims and payments. Australian officials said there was no evidence that personal medical records had been accessed. The investigation is still ongoing.
Both facts deserve attention. First, an AI agent entered a system without authorisation. Second, that the evidence available so far does not point to a sweeping compromise of Australians’ medical data. The far more useful question for policymakers than the doomsday narrative making the rounds is not “Has AI escaped our control?” but rather a more pragmatic and realistic approach: “If our controls to stop the AI from escaping failed, how quickly can we detect an AI created incident, establish what happened and tell the people who need to respond?”
Some (but not all) of the biggest AI titans have been unexpectedly united in calling for a slowdown in AI development. Dario Amodei, the Anthropic CEO, published a widely publicised essay titled “We Must Pace the Frontier (Opens in new window)”, in which he warns that the speed of AI capability development is outpacing the industry’s capacity to keep it under control. This concern was immediately echoed by others including OpenAI’s Sam Altman, Google DeepMind’s Demis Hassabis, and Elon Musk, who all publicly backed Amodei’s sentiment.
It is a question of who blinks first in a staring contest – and it looks like neither will.
Despite Amodei’s essay and Altman’s impassioned plea at the UN Security Council (Opens in new window), the slowdown is very unlikely to happen. Both the Chinese government and President Donald Trump, in a rare example of consensus, albeit for vastly different reasons, have dismissed the calls to slowdown. China does not want to slow down because it sees the call as an attempt to preserve US tech dominance. None of the Chinese AI companies are likely to go against what their government says publicly. Trump, on the other hand, does not want to slow down because of fears of the US yielding ground in AI development to China. It is a question of who blinks first in a staring contest – and it looks like neither will.
Europeans have also dismissed a go-slow. A recent Politico (Opens in new window) article on Europe’s technology industry (Opens in new window) reflects a growing resistance to apocalyptic accounts of AI. But their calls to reject the hype are accompanied by demands to further sharpen scrutiny.
The currently known (at time of writing this piece) impact of the Medicare breach in Australia may be modest, yet the notification sequence was a serious weakness. OpenAI did not report it until three months after the incident and then only to a mailbox at a government agency, with the email opened five days after receipt. Australia has established a taskforce to review the incident (Opens in new window) and whether its existing processes are suited to AI-enabled cyber intrusions into its government websites.
OpenAI has since acknowledged that its handling was “not good enough.” (Opens in new window) Appearing before an Australian parliamentary committee on 6 October, strategy chief Jason Kwon said the June incident should not have occurred and authorities should have been notified immediately, before every detail was known. In an internal evaluation, the agent sought Victorian public-spending data on skin-disease medicines. When blocked, it entered non-public areas, executed commands, viewed internal files and credentials, and saved files. OpenAI said it found no evidence that medical records or individual patient data were accessed.
The Australian episode came soon after the widely reported Hugging Face incident (Opens in new window) which happened in July. During internal evaluations, OpenAI agents found ways past restrictions intended to isolate them. The agent reached outside systems and compromised parts of Hugging Face’s infrastructure. OpenAI says the activity was primarily driven by an internal research model that exceeded its mandate, taking shortcuts to find an answer on its own.
A few days after the Australian announcement, came the news from OpenAI that its AI agents had accidentally posted links to over 50 images that had previously been uploaded by its subscribers to third-party image hosting sites. In addition, its agents had “interacted with several US government websites in unexpected ways (Opens in new window)”, including the Securities and Exchange Commission and the US Census Bureau. Another report mentions a possible hack also originating from rogue OpenAI agents on a US Department of Education website (Opens in new window).
Nor was Australia the only apparent government target. Transluce (an AI research firm) said (Opens in new window) agents made nearly 900 requests to Canadian government sites in May and June, including apparently unsuccessful hacking attempts. The firm disclosed the activity on 28 September and said the tactics resembled other agent activity it had associated with OpenAI, but it could not confidently attribute the attempts to the company. Canadian officials said there was no indication government systems were compromised. OpenAI said it was reviewing the findings and had briefed Canadian authorities.
Could it be that the strong responses against the calls for slowdown from the Chinese government, the US President and European companies may be the reason why these prominent incidents involving AI agents are now coming to light? Like a sort of “I told you so!”?
These incidents involve identifiable systems, permissions, vulnerabilities and reporting decisions and therefore can be investigated thoroughly, just as if these intrusions had been perpetrated by human hackers. They can also teach governments and organisations how to spot similar activity earlier. This is a more productive basis for policy than treating every unexpected agent action as evidence of impending catastrophe.
Australian Prime Minister Anthony Albanese disclosed that an OpenAI agent, on the lookout for statistics, had broken into an Australian government website (Ludovic Marin/AFP via Getty Images)
A year ago, I wrote an op-ed “Who’s Accountable When AI Agents Go Rogue? (Opens in new window)”, where I argued that AI agents expand the range of ways organisations can be attacked as they interact with people, software and other agents. I also argued for oversight tailored to how much freedom a system actually has. That point becomes relevant when translated into a practical question: if an agent exceeds its instructions, who will observe the transgression, who will receive the report and who will bear responsibility for any transgressions?
These are questions that need to be worked on, rather than building doomsday scenarios.
Not all US tech titans agree with the call to slow down. Nvidia’s Jensen Huang, for instance, has instead espoused acceleration of the development of AI. Speaking at a recent conference (Opens in new window), he advocated focusing on preventing dangerous AI enabled incidents by advancing the development of AI safely through innovation. His solution – AI companies should not release products until sure they are safe instead of asking for regulatory intervention. In other words, the AI companies should regulate themselves.
In addition to self-regulation, AI companies should improve their incident response capabilities and outlook. In 1736, concerned about the danger of fires in rapidly growing Philadelphia, Benjamin Franklin helped organise the Union Fire Company, a voluntary association whose members agreed to respond to fires and maintain equipment such as buckets, ladders and fire hooks. It became Philadelphia’s first organised volunteer fire company, and other similar companies soon followed. Benjamin Franklin did not wait for government to create a formal fire service. That is a classic example of self-initiated and efficient first stage incident response.
Another lesson from the Australian case is not that AI has suddenly become uncontrollable. It is that basic cyber hygiene still matters.
Incident response, simply put, is how an organisation would detect, investigate, contain and recover from a cyber intrusion or data breach. Effective incident response requires clear lines of responsibility, continuous monitoring, rapid escalation, preservation of evidence, communication with affected parties and regulators, and documentation of lessons learned after the event. Just as Benjamin Franklin’s intention was to respond to fires, not prevent them, incident response cannot eliminate every cyber risk. It is more about ensuring that when something does go wrong, it is spotted quickly, reported transparently and contained effectively.
Incident response processes for agentic AI transgressions cannot depend on someone noticing a bizarre answer on a screen or reading an email five days after it was sent to generic mailbox like what happened in Australia. Organisations that were targeted need records of what an agent was asked to do, which tools it used, what information it accessed and what actions those tools carried out.
This is an emerging field, not a solved engineering problem. In a March 2026 report (Opens in new window), the US National Institute of Standards and Technology (NIST) identified fragmented logging, difficulties detecting deceptive behaviour and weak arrangements for sharing incident information among the challenges of monitoring deployed AI. NIST has also convened work on common definitions and practices for AI incident response. These efforts matter because developers, customers and governments need enough shared language to recognise when they are discussing the same kind of event.
Another lesson from the Australian case is not that AI has suddenly become uncontrollable. It is that basic cyber hygiene still matters. OpenAI’s delay in notifying Australian authorities is serious, but so is the fact that the intrusion apparently went undetected inside Australian government systems for three months. Regulation can set expectations, but it cannot compensate for weak monitoring, poor visibility or slow incident detection.
That should shift the policy debate away from AI panic and towards institutional capability. Governments need systems that can spot unusual behaviour quickly, generate reliable audit trails and trigger clear reporting obligations when something goes wrong. They also need to avoid piecemeal AI deployments that sit outside normal security and governance structures. The priority should be straightforward: build security and accountability into public sector technology from the start and make sure agencies can detect and report incidents before a 90-day blind spot becomes the norm.
The latest incidents do not require policymakers to choose between AI panic and complacency. They require a capacity that is less theatrical and more demanding: to watch agents in action, prevent them from crossing boundaries, recognise swiftly when they do, and report the facts quickly enough for someone to respond. That is how a disturbing incident becomes a lesson rather than a spectacle.
About the author
Asha Hemrajani
Asha Hemrajani is Senior Fellow at the Centre of Excellence in National Security (CENS) at RSIS.
The most-pressing world events explained by Lowy Institute experts and global contributors, in your inbox, every Thursday.
You may unsubscribe from The Interpreter at any time. For information on our privacy practices and how to unsubscribe, see our Privacy Policy.
