INCIDENT REPLAY · 2025-09-18

Optus Triple Zero outage during firewall upgrade

From about 00:30 to 13:30 AEST on 18 September 2025 (almost 14 hours) emergency calls from Optus customers in South Australia, Western Australia, the Northern Territory and parts of NSW intermittently failed while ordinary calls kept working. Optus first said about 600 calls were affected; the independent Schott review found about 75 percent of 605 attempts failed (The Register: 455 calls did not go through). The outage was initially linked to three and later four deaths; accounts of the review tie two deaths to failed calls. Optus's monitoring excluded Triple Zero calls so no alarm fired, and five customer warnings to the call centre were not escalated. A Senate inquiry hearing was held on 3 November 2025, and Wikipedia reports the ACCC began Federal Court action in July 2026.

A reconstruction, not the operator's network. Hammerhead's authors rebuilt the change from the public record as a small network of their own. No configuration here is the operator's. What follows is what the tool reported on that reconstruction; it is not a claim that the tool would have prevented the outage.

What happened

Who: Optus (Singtel Optus), Australia; upgrade executed by contractor Nokia

The summary at the top is paraphrased from the sources below. Hammerhead's catalog rates its confidence in this account high.

Sources

What the public record says

From about 00:30 to 13:30 AEST on 18 September 2025 (almost 14 hours) emergency calls from Optus customers in South Australia, Western Australia, the Northern Territory and parts of NSW intermittently failed while ordinary calls kept working. Optus first said about 600 calls were affected; the independent Schott review found about 75 percent of 605 attempts failed (The Register: 455 calls did not go through). The outage was initially linked to three and later four deaths; accounts of the review tie two deaths to failed calls. Optus's monitoring excluded Triple Zero calls so no alarm fired, and five customer warnings to the call centre were not escalated. A Senate inquiry hearing was held on 3 November 2025, and Wikipedia reports the ACCC began Federal Court action in July 2026.
Who: Optus (Singtel Optus), Australia; upgrade executed by contractor Nokia

What the reconstruction assumed

Decided by Hammerhead's authors to build a network that runs. None of it describes the operator's network.

Everything below is ours.

The family template (tools/replays/gen_family_policy.py): MOBILE-CORE-1 (the mobile core sending emergency calls) reaches E911-GW-1 (the emergency gateway) through FW-16 (the firewall isolated for its upgrade), which filters inbound on its outside interface. The intent file names the flow: emergency calls reach the emergency gateway on udp/5060.
A firewall was isolated for an upgrade without diverting its traffic first, and one traffic class had no surviving path. Modelled as the isolated firewall's ACL closing while the second firewall in the pair, which carries the other classes, has no permit for emergency signalling; the intended change is the same upgrade with the diversion in place (the second firewall permits the class).
Cisco IOS syntax and stand-in address space throughout. A reconstruction, not any operator's configuration.

What Hammerhead said

Each run below is pinned by Hammerhead's replay tests at commit 0ee9732: the verdict and the quoted text are checked against the tool's output, and a run that disagrees fails the test. Reason lines are copied verbatim; a pinned line can be part of a longer reason. Every preflight run uses --max-blast-radius 100, so a change is never counted as caught merely because it touched many devices of a small network.

the change as intended

now: pass

On the reconstruction, hammerhead preflight on a before and an after snapshot with --policy @optus_triple_zero_2025/policy.yaml --allow-flow-changes returned pass.

First run, before the fixes this batch prompted

first: pass

Pinned in tests/gauntlet/replays-policy.yaml.

the change as pushed

now: fail

On the reconstruction, hammerhead preflight on a before and an after snapshot with --policy @optus_triple_zero_2025/policy.yaml --allow-flow-changes returned fail, for the reasons below.

Reason text pinned (some reason contains each line)

policy gate: high/critical failure(s) on the 'after' network: emergency calls reach the emergency gateway
NOT reachable (expected reachable)

First run, before the fixes this batch prompted

first: fail

policy gate: high/critical failure(s) on the 'after' network: emergency calls reach the emergency gateway

Pinned in tests/gauntlet/replays-policy.yaml.

What changed in Hammerhead because of it

From the “What was fixed” table in Hammerhead's replay notes, where each fix landed after a failing test. “Found by” is copied as written; a family name means the batch this incident belongs to.

A flow rule names its protocol and port

Every reachability rule probed TCP/80, so an intent about SIP could not tell the push from the intended change: 7 of 11 intended changes failed on main and the one that dropped only the named port passed. protocol: and dst_port: on the rule; the policy gate's reason now carries the violation, not only the policy's name.

Found by: Required-flow family

Limits

The configurations are a reconstruction. Where the record is silent, the assumptions above fill the gap; if one is wrong, the replay is wrong in that respect.
A verdict describes the reconstructed network's converged state. Timing, traffic load, and anything a device does that no configuration states, such as software reacting to a health check or a firmware bug, are outside it.
A pass is reported as a pass. A fail says a gate fired on the reconstruction, nothing more.
Why the catalog counts this outage as partly in scope: Modelling the change (isolate firewall 16 with no prior traffic diversion) would show a traffic class left with no surviving path, which is a lost-redundancy finding; but only emergency-call signalling failed to re-route, which points to voice-core call routing that a router/firewall config model may not capture.

Imported from docs/replays/optus-triple-zero-2025/FACTS.md and the replay manifests in the Hammerhead repository at commit 0ee9732.