Australia is the first known nation to have its government data breached by a rogue AI agent, following recent revelations that an OpenAI model gained access to Medicare statistics in June.
It was later revealed OpenAI also breached multiple government websites in the US.
After AI giants signed a voluntary accord to develop their own safety standards following a meeting with US President Donald Trump, eSafety commissioner Julie Inman Grant said the companies could not enforce measures themselves.
"Clearly, they haven't policed themselves or we wouldn't have had the OpenAI Medicare hack or the Hugging Face hack or anything else," she told reporters in Brisbane on Wednesday.
"They've had time - the end of self-regulation is over. And it's great to see that we, along with others, want to take this on and create better online futures for our people."
The commissioner said the calls for self regulation from AI companies mirrored similar requests at the birth of the internet and social media.
Ms Inman Grant said further controls needed to be placed on the industry, with Australia in a "perilous place" as AI companies had not built adequate safety mechanisms.
She held concerns about people feeding AI private data, which would likely include information needed for speeches.
"We have to take control now. We, of course, will always rather work with the industry rather than against them, but something has to change," she said.
Prime Minister Anthony Albanese has long touted the benefits of AI, encouraging Australians and his own party to use it, albeit carefully.
"This is a complex area. AI can transform our economy, boost productivity, and lead to revolutionary breakthroughs in health care, innovation, and science," he told reporters on Wednesday.
"But there are risks there as well, and we need to make sure that we mitigate the risks."
OpenAI on Tuesday issued an apology for the breach, offering to set up an independent task force to investigate privacy concerns, alongside one launched by the government.
The company and its competitor Anthropic have both said they would slow production of their latest AI models.
The government has also ordered all departments to audit their old technology and patch any holes that could allow an AI agent to access data.
The home affairs department-driven push to mothball old systems announced on Wednesday tracks with a sweeping cybersecurity overhaul slated for completion by 2030.
Abigail Bradshaw, the Australian Signals Directorate's director-general, said her team had been working with non-publicly available AI software for more than six months to strengthen government systems - something many other developed countries had not been able to do.
The directorate had access to private security measures to ensure the 2026 census was secure, and was now helping other government departments, Ms Bradshaw said on Wednesday.
"Australia can participate in developing new norms, new behaviours, and new protocols which are beneficial to Australians and are shaping norms which will impact people around the world," she told ABC Television.
Her directorate had been working with other Five Eyes nations since June to ensure Australia was on the frontier of security movements.
19°C