Optus says fatal triple-zero outage was caused by 'human error'

Optus' major triple-zero outage last week was caused by a team failing to follow step one of an upgrade procedure, the head of the telco has revealed.

Optus chief executive Stephen Rue has said a firewall upgrade was botched due to "human error", leading to the issue that affected more than 631 emergency calls and was linked to three deaths over 13 hours on Thursday.

"Preliminary investigations have determined that on the first night of the upgrade, the steps taken on past successful upgrades of a similar nature were not followed," he told the media today.

"On this occasion of the upgrade of 18 September, the first step in the process was not followed." 

Rue said step one of the upgrade is to divert calls away from the relevant part of the core network to a separate part of the core network, which would have allowed calls to go through as normal. 

He said the team was comprised of both Optus and third-party Nokia staff in Australia and India.

"The team is based both here and it's also based in Chennai," he said. 

Rue knocked back any suggestions that the issue was caused by taking jobs offshore or job cuts, instead maintaining that it was an issue with the upgrade procedure. 

"This is not about Nokia. This is about Optus's accountability to ensure its network operates. That's what this is about," he said.

Optus admitted that five people had called their customer service line to report an issue affecting triple-zero calls on Thursday, but the matter was not escalated and the telco remained unaware of the error.

Rue said that 86 of the 631 customers who failed to connect to triple zero did get through, with about 65 reaching emergency services by camping on to another carrier like Telstra or TPG.

"But early internal investigations have therefore determined there were approximately 480 customers who did not get through to triple zero," he said.

"The independent review, as well as our ongoing investigation, will be examining why this happened and will determine whether there are any patterns that exist to the core failures."

Rue admitted that Optus had "absolutely" suffered reputational damage.

"There are no words that can express how sorry I am about the very sad loss of the lives of four people who could not reach emergency services in their time of need," he said.

Optus commissions independent review 

Optus has commissioned an independent review, led by Kerry Schott, into the outage.

Schott is currently a Director of AGL, Chair of the Carbon Market Institute and Chair of the Competition Review Panel.

The review will examine Optus' management of triple zero calls and actions in response to the incident in adherence to policies, procedures and legal requirements.

The report is expected to be completed before the end of the year. 

Optus Board Chairman John Arthur said the board was working with Rue and promised to prevent any further repetition of this incident, which is Optus' second major triple-zero outage in about two years.

"What has occurred is clearly unacceptable, and on behalf of the Board, I apologise for this failure in our accountability to all those who relied on our triple-zero services," he said in a statement.

"I particularly express my deep condolences to the families and friends of the four people who passed away."

Optus paid a $12 million fine after a technical issue left more than 2000 people unable to ring triple zero in 2023, but failed to implement a third of the recommendations from an investigation into the incident.

Last week's outage has left the axe looming over Rue, who was hired in November after his predecessor Kelly Bayer Rosmarin stepped down following fallout from the 2023 outage.

The Australian Media and Communications Authority is also investigating the incident, with Communications Minister Anika Wells vowing that Optus will face "significant consequences" and be "held to account".

But any potential penalties the telco faces will largely be financial, rather than criminal.

Wells also hinted at a wider review into the telco industry and emergency service network, as Optus is not the only provider to oversee triple-zero outages.

Optus owner is 'deeply sorry'

The Singapore-based owner of Optus has issued an apology over last week's fatal triple-zero outage.

Singtel Group chief executive Yuen Kuan Moon said he was "deeply sorry" to learn about the incident.

"Our hearts go out to the families and friends of those who have passed away and we know that Optus will get to the bottom of this matter," he said in his first statement since the outage.

"We are working with the Optus board and management to ensure a thorough investigation of this incident to prevent any future recurrence.

"Optus has been cooperating fully and transparently with all relevant government agencies and regulatory bodies while the matter is being investigated and will share the findings in due course."

Singtel has invested over $9.3 billion into Optus over the past five years.

A large portion of that funding has been for building network infrastructure across Australia. 

DOWNLOAD THE 9NEWS APP: Stay across all the latest in breaking news, sport, politics and the weather via our news app and get notifications sent straight to your smartphone. Available on the Apple App Store and Google Play.