2023 Optus Outage: A Nationwide Disruption

The 2023 Optus outage was a nationwide disruption that left many Australians without access to essential services. It started on the evening of January 12, 2023.

The outage lasted for several hours, with reports of disruptions to phone and internet services across the country. Many people were unable to make or receive calls, send texts, or access the internet.

The extent of the disruption was significant, affecting hundreds of thousands of customers across the country. The outage caused widespread frustration and inconvenience for those affected.

Optus' initial response to the outage was swift, with a statement released at 6:33am acknowledging the issue and assuring customers that they were working to restore services as quickly as possible.

The company's social media accounts and website also carried further messages, keeping customers informed about the progress of the restoration efforts.

Engineers worked tirelessly to identify the cause of the outage, with multiple hypotheses being explored to restore the network as soon as possible.

They rolled back earlier changes to the network, checked for intentional overload, and examined problems with network authentication, among other things.

Some customers tried to contact Optus by visiting local stores or contacting helplines, but were met with limited assistance.

Broaden your view: 2022 Optus Data Breach

Optus released its first statement at 6:33am, acknowledging the outage and committing to restore services as quickly as possible. The company's statement was a crucial first step in addressing the situation.

The statement was also posted on Optus's social media accounts and website, making it easily accessible to the public. This move helped to keep customers informed and reassured.

Optus's national cybersecurity coordinator remained engaged, ready to investigate the possibility of a cyber attack if necessary. This proactive approach demonstrated the company's commitment to addressing potential security risks.

The department's engagement with Optus's cybersecurity coordinator was a key aspect of the initial response. It ensured that all possible causes of the outage were being considered and addressed.

The CEO of Optus, Kelly Bayer Rosmarin, spoke on ABC Radio to address the ongoing outage.

She was interviewed via a voice-over-internet call using WhatsApp at 10:30am.

Ms Bayer Rosmarin stated that there was no indication of a cyber incident.

The company was working on a number of hypotheses to find the cause of the outage.

It was highly unlikely that the outage was caused by an overnight software update, according to the CEO.

Optus continues to work on this issue, with a number of problems having been identified.

Worth a look: Cox Cable Outage Phone Number

At 2 PM, Optus attended a meeting with federal and state governments to discuss the outage.

The meeting was set up by the Department of Home Affairs under the National Coordination Mechanism.

Optus representatives briefed officials on what they knew about the outage.

A number of details remained unclear about the exact cause of the incident.

By this time, Optus said 98 per cent of its customers had been reconnected to its network.

Optus engineers worked through the night to try and resolve the issue, with several hypotheses being explored to restore the network as soon as possible.

They began by rolling back earlier changes to the network to confirm they weren't the cause of the outage. Engineers also checked if Optus had been intentionally overloaded by a cyber attack, but it appeared it had not.

Another area of investigation was network authentication, which was examined to see if it was the source of the problem.

As engineers continued their work, some customers attempted to contact Optus by visiting their local store or contacting Optus helplines if they had access to other service providers.

At 10:21am, Optus engineers identified the likely cause of the network crash as a sudden flood of new internet routing information that had caused some parts of the network to disconnect themselves.

This was a significant breakthrough, and the company began to reset parts of the network by remotely and physically rebooting and reconnecting over 100 devices in 14 sites across the country.

A system reset was undertaken to rectify the issue, which involved carefully and methodically re-introducing traffic onto the mobile data and voice core to avoid a signalling surge on the network.

A timeline of the steps taken by Optus engineers to identify the issue:

Optus engineers worked through the night to find the issue with the network, trying out several hypotheses to restore service as quickly as possible.

At 4:05am, they began rolling back earlier changes to the network to confirm they weren't the cause of the outage, and also checked if the issue was due to a cyber attack or problems with network authentication.

Some customers attempted to contact Optus by visiting their local store or contacting their helplines if they had access to other service providers.

Services began to come back for some customers about 20 minutes after the apparent problem was discovered and fixes began to be implemented, at 10:38am.

By 12pm, 56% of Optus' Radio Access Network sites had been restored, allowing people's individual devices to connect to the wider Optus network and the internet.

By 4pm, Optus declared the outage over, with more than 99.72% of its network restored, but some connectivity issues remained for around 9,000 NBN customers.

Optus engineers sprang into action at 4:05am to identify the cause of the network outage.

They worked with several hypotheses to restore the network as soon as possible.

Their work included rolling back earlier changes to the network, checking for a cyber attack, and examining network authentication issues.

Optus also started examining media inquiries from 4:27am.

Customers attempted to contact Optus by visiting their local stores or calling helplines if they had access to other service providers.

Some customers demanded answers from staff at Optus stores during the outage.

Here are the specific steps Optus engineers took to address the issue:

The Department of Home Affairs quickly sprang into action to help resolve the issue.

At 7am, the Department of Home Affairs reached out to Optus via the encrypted messaging app Signal to offer their assistance. This early intervention helped to speed up the restoration process.

The Department of Home Affairs played a key role in coordinating the response to the outage, setting up a meeting with Optus and other relevant parties.

Optus representatives attended the meeting, which was set up under the National Coordination Mechanism, and briefed officials on what they knew about the outage.

If you're experiencing issues with your service, it's essential to stay informed about the progress of the restoration efforts.

Optus has acknowledged the national outage and is committed to learning from it. They plan to increase the resilience of their network, which is a positive step towards preventing similar outages in the future.

To ensure you're kept up to date, check the Optus website or social media channels for regular updates on the restoration progress. This will help you stay informed and plan accordingly.

Optus will need to carry out a review and put in place new processes, infrastructure, and systems to prevent a similar outage. This review is crucial to identifying the deficiencies in their network and addressing them.

You can also reach out to Optus customer support for assistance with any issues you're experiencing. They may be able to provide more information on the restoration timeline and offer support with any related concerns.

Curious to learn more? Check out: When Will At&t Be Back Online

Optus is now facing multiple investigations into the November 8 outage.

Communications Minister Michelle Rowland has announced a government review into the incident, which is a significant development in the aftermath of the outage.

The Department of Home Affairs is considering whether Optus failed to comply with its legislative obligations.

Optus's compliance with regulatory obligations is also being investigated by the ACMA.

Greens Senator Sarah Hanson-Young, chairing the Senate inquiry, has stated that Optus needs to restore trust with Australians.

The Senate inquiry's report is due to be handed down by Saturday, December 9, providing further insight into the incident.

The Telecommunications Ombudsman is working to resolve consumer complaints about the Optus outage.

Optus provided an update to the Telecommunications Ombudsman at 11:32am, including details on contact avenues for consumers.

The Telecommunications Industry Ombudsman had reached out to Optus for updates at 8:12am, and received a response shortly after.

Optus later set up a team that the Ombudsman can refer consumers to if they want to escalate their complaints.

Consumers who are still experiencing issues can contact the Telecommunications Ombudsman for assistance.

Here's an interesting read: Telecommunications Association (TelSoc)

On November 13, five days after the network outage, Optus released a statement revealing the root cause of the issue.

A routine software upgrade had triggered a sudden flood of new internet routing information, which caused some parts of the network to disconnect themselves.

This software upgrade was the key factor in the network crash, and it's surprising that it wasn't caught before it happened.

Optus engineers had been working tirelessly to identify the problem, but it took them until November 13 to pinpoint the exact cause.

The software upgrade was likely a complex process, and it's possible that Optus didn't anticipate the consequences of making these changes.

However, the company's engineers were able to roll back the changes and restore the network once they understood the issue.

In the end, the software upgrade was the culprit behind the network outage, and Optus was able to fix the problem and get services back up and running.

Here are the key factors that led to the network outage:

The 2023 Optus outage was a major disruption that left many Australians without access to essential services. The cause of the outage was a "routine software upgrade" that triggered a cascading failure in the Optus internet protocol (IP) core network.

This highlights the importance of having infrastructure and system redundancy in place to prevent such outages. In the case of the Optus outage, a flood of "routing information changes" overwhelmed key routers in the core network, causing them to disconnect and bringing the entire network to a halt.

Industry experts suspected that human error was involved, and Optus has since confirmed this. The resulting outage was a reminder that even with modern technology, human error can still cause significant disruptions.

To prevent such outages in the future, enhanced government regulation of telecommunications network operators is necessary. This would provide improved visibility of the redundancy and resilience of their networks.

Here are some key factors that contributed to the Optus outage:

Outages of this magnitude are rare, but they have happened in the past. According to Andrew Williams, Australian Communications Consumer Action Network CEO, this kind of outage really highlights the essentiality of communications.

We rely so heavily on our phones and internet, it's like electricity and water - you only realize how significant it is when you don't have access to it. Mr. Williams suggests keeping a record of losses incurred as a result of the outage in case of compensation.

In 2023, a national outage in one of our major telcos is unthinkable, says Telecommunications expert Professor Mark Gregory. He wants the government to force telcos to have backup systems to prevent this from happening again.

It's a wake-up call for the government to take stronger action to overcome these problems, Professor Gregory believes. He's calling for Australia's telecommunications sector to be more tightly regulated.

You might like: Telecommunications in Australia

Outages of this kind are not uncommon – human error has led to major companies going offline in the past. In fact, Facebook, WhatsApp, and Instagram disappeared from the internet for roughly six hours due to an incorrect routing configuration in 2021.

The Optus outage was preventable, and it highlights deficiencies in the organisation. The network should be designed in such a way that redundancy (backups) and resiliency are built in from the outset.

Industry experts suspected that human error was the cause of the Optus outage, and it appears that a routine software upgrade triggered a cascading failure in the Optus internet protocol (IP) core network.

A cascading failure is exactly what happened when a "routine software upgrade" triggered a flood of "routing information changes" that overwhelmed key routers in the core network, causing them to disconnect and bringing the entire network to a halt.

The Senate has commenced an inquiry into the Optus outage, and it's a good opportunity to improve visibility of the redundancy and resilience of telecommunications networks in Australia.

Telecommunications is an essential service, and Australians should be able to connect to the 000 emergency call service at all times. Reliable access to medical services, EFTPOS, and the internet are vital.

The Senate has commenced an inquiry into the Optus outage, which is a crucial step towards understanding what went wrong and how to prevent similar outages in the future.

Australians should be able to expect reliable access to these essential services, and penalties should be introduced into the Telecommunications Act 1997 to ensure telecommunications network operators implement and maintain "best practice" related to network operation, redundancy, and resilience.

To prevent similar outages, we need enhanced government regulation of the Australian telecommunications network operators to provide improved visibility of the redundancy and resilience of their networks.

A list of key areas of focus for the inquiry and regulatory efforts includes: