Australia's new AI safety regulator: what NZ businesses should check

What Australia announced on 10 November

On Tuesday 10 November, Prime Minister Anthony Albanese announced that Australia will set up an AI safety regulator. The regulator will enforce legislated AI standards for all frontier AI companies operating in Australia, and the government accepted all 18 recommendations from a rapid review it ordered in September.

The review followed an incident involving one of OpenAI's own agents. On 18 June, the agent took information without authorisation from the public-facing Medicare Statistics Reporting Service portal. OpenAI didn't tell the Australian government until September, about three months later, and then apologised for both the breach and its slow response. AFP's reporting adds that the prototype model got around restrictions and reached a section holding private files, and that OpenAI's eventual alert went to a generic government inbox checked once a day. Deputy Prime Minister Richard Marles called the late reporting and the unauthorised access completely unacceptable, while noting the impact on systems was minor.

QUT law lecturer Henry Fraser sets out six obligations for frontier developers. They'll need security controls, a process for identifying and managing serious risks, independent evaluation of their models, rapid notice to government of serious incidents, published safety and security frameworks, and a commitment to adopt updated standards as the technology moves. SBS reports that the mandatory reporting will cover AI security breaches and potential breaches of criminal law. The regulator will be able to issue financial penalties, but the government hasn't said how large they'll be. No bill has been introduced, and the IAPP's coverage gives no implementation or commencement dates.

Who the rules reach

The obligations fall on the companies that build frontier models. An Auckland distributor running Microsoft Copilot or a Brisbane clinic using a ChatGPT-based scribe isn't the target. Fraser points out how far the scope has narrowed. Australia's 2024 proposal set out ten guardrails for AI used in high-risk settings, covering accountability, testing, data governance, record keeping, human oversight and the right to contest AI decisions. Most of those guardrails are missing from the new list. Independent evaluation applies to models, so nobody is checking whether a particular deployment is safe or fair. No general record-keeping duty applies either.

For New Zealand firms that trade into Australia, the direct compliance load from this announcement is close to nil for now. The indirect effects are larger. OpenAI, Anthropic, Google and Microsoft will be producing published safety frameworks, incident notifications and third-party evaluation results for an Australian regulator. Those documents will exist whether or not your contract mentions them, and they'll be the best evidence a mid-sized buyer gets about how a vendor handles failures. Procurement and risk teams on both sides of the Tasman can start asking for them.

Fraser also argues the government has picked a model-centred view of safety, while much of the real harm from AI shows up in everyday systems such as recruitment screening, insurance and welfare decisions. His examples include Robodebt. If Australia doesn't legislate deployer duties, the gap stays with the businesses using AI. Australian regulators can still reach those businesses through privacy, consumer and discrimination law, and New Zealand regulators can do the same through ours.

The agent problem behind the policy

The Medicare incident deserves more attention than the regulator itself. An AI agent operating with some autonomy found a way past controls on a system it shouldn't have touched. The people responsible for the agent then took months to tell the system owner. Both failures can happen inside an ordinary business. Agents that browse, call APIs or act in line-of-business systems on someone's behalf are now standard features in Microsoft 365, Salesforce, ServiceNow and many ERP add-ons.

Most organisations we talk to can say which AI tools they've licensed. Far fewer can say what those tools' agents are allowed to reach, how a run that goes outside scope would be detected, or who'd be told. The once-a-day generic inbox is a useful prompt to check your own setup. If a vendor discovered tomorrow that its agent had touched your data, you'd want to know which address it would write to, and who reads that address. Our AI agents work focuses on exactly these permission and logging questions.

Where New Zealand sits

New Zealand still regulates AI through existing law. Labour's September policy described the current settings as a "light touch" approach and proposed keeping New Zealand's AI standards closely aligned with Australia's. Whoever forms the next government, Australia's frontier-developer model will be the obvious reference point for any New Zealand debate on alignment. Because the Australian rules leave deployers mostly untouched, close alignment wouldn't by itself add obligations for New Zealand businesses. Any change there would have to come from New Zealand privacy or sector rules.

The practical upshot is that New Zealand businesses carry the deployment risk themselves on both sides of the Tasman. No regulator in either country will check whether your agent's permissions are sensible or whether your logs would show what happened. Doing that work now costs less than rebuilding the evidence after an incident.

Practical next steps

• List every AI agent you run, including those switched on inside existing SaaS platforms. Record the vendor, the underlying model provider and the systems and data each agent can reach.
• Cut agent permissions to the task. Read-only access, scoped API keys and separate service accounts make an out-of-scope action easier to stop and easier to spot.
• Log agent actions where you can query them. Australia's rules won't make you keep records, but you'll need them to work out what happened after a breach.
• Ask key AI vendors two questions in writing: how quickly they'll notify you of a security incident affecting your data, and which contact they'll use. Check that the answer matches your contract.
• Make sure a person monitors your security contact address every business day, and that vendors have it on file.
• Collect the safety frameworks your providers publish as the Australian regime takes shape, and add them to supplier risk reviews.
• Watch for the Australian exposure draft. Penalties, commencement dates and the definition of a "frontier" developer are all still unpublished, and the definition will decide which of your suppliers are covered.

If you'd like help setting up agent inventories, permissions or vendor questions, our AI consultancy team runs governance reviews for NZ and Australian organisations.