A global technology outage caused by a faulty software update grounded flights, including some at the Ottawa International Airport, knocked media outlets offline, and disrupted hospitals, small businesses and government offices on Friday, highlighting the fragility of a digitized world dependent on just a handful of providers.
Porter Airlines cancelled all flights until noon, including those out of Ottawa. By 3 p.m. Friday afternoon, the airline had posted an update saying its operations were “in the early stages of returning to normal.”
The airline said its website would be restored “when all systems are back online and automated passenger rebooking has begun.”
OBJ360 (Sponsored)
Ontario East is driving recruitment success in a new era and adapting to workforce changes
The last decade has ushered in seismic change for the workforce in Eastern Ontario, and around the world. As a result, employers are discovering that traditional methods of workforce recruitment
Eastern Ontario’s largest not-for-profit home care and community support organization, was undergoing a significant organizational change. The goal was to better serve clients, staff, and communities by unifying three independently
All of Porter’s departures from YOW from 4 p.m. onward were listed as “on time,” with the exception of a 4:05 p.m. flight to Winnipeg and a 5:10 p.m. flight to Toronto’s Billy Bishop airport that were cancelled. Several Porter flights arriving from Vancouver, Toronto, Newark, Orlando and Boston were also cancelled.
Ottawa airport spokesperson Krista Kealey told OBJ in an email that YOW is “monitoring the situation closely.” She added that all other airlines besides Porter were still flying.
“We will continue to watch the situation in the event that there are any ripple effects with operations elsewhere,” Kealey said. “As always, we recommend that passengers check with their airlines before coming to the airport in case there are schedule changes.”
The trouble with the update issued by cybersecurity firm CrowdStrike affected customers running Microsoft Windows. It was not the result of hacking or a cyberattack, according to CrowdStrike, which apologized and said a fix was on the way.
Paul Vallée, founder and CEO of Ottawa cybersecurity software provider Tehama, said the outage is likely to be costly.
“I wouldn’t be surprised if, on a holistic economic analysis, this wasn’t the most expensive single bug of all time,” he told OBJ in an email Friday. “I think it might have cost more than the Y2K bug, especially since remediation, while expensive, also drove a systems renewal which had lasting positive economic and technological impacts. No equivalent good thing is likely to emerge from this incredibly expensive systems error.”
He added that this is the kind of incident that would not have been seen under previous software structures.
“This incident speaks to the challenge of the modern trend towards centralized cloud software infrastructure,” he said. “In this case, a single vendor, Crowdstrike, became so ubiquitously adopted that when they released a buggy update much of the world’s mission critical infrastructure went down. This sort of single bug breaking everything is a consequence of modern cloud architecture and was unthinkable under traditional architectures that are now thought of as antiquated.”
Local health-care institutions were among those hit by the outage. The Children’s Hospital of Eastern Ontario issued a statement saying it has been affected but was continuing to provide services “using workarounds where needed.”
“We do not anticipate any impact on our services offered to children and their families today,” the hospital said.
The Queensway Carleton Hospital said its electronic health record system was disrupted from 1:30 to 5:45 a.m. It said the system has “since been restored to normal operations” and no other services have been affected.
The Ottawa Hospital, meanwhile, said its operations were not disrupted by the outage, adding it would continue to keep a close eye on the situation.
The Canadian Federation of Independent Business is “monitoring the situation closely,” a spokesperson told OBJ, as members across the country report disruptions in payment processing and payroll software. Some businesses have also been having challenges depositing employee pay, said Ryan Mallough, vice-president of legislative Affairs for Ontario with CFIB.
“Additionally, we are concerned about what airport delays may mean for the shipping of goods across the country. Hopefully these issues are resolved soon,” he said in a statement. “The longer the outage lasts, the larger the impact will be on business owners, their employees, and their customers.”
Meanwhile, the disarray continued – and escalated – around the world.
Long lines formed at airports in the U.S., Europe and Asia as airlines lost access to check-in and booking services at a time when many travellers are heading away on summer vacations.
News outlets in Australia – where telecommunications were severely affected – were pushed off air for hours. Hospitals and doctor’s offices had problems with their appointment systems, while banks in South Africa and New Zealand reported outages to their payment system or websites and apps.
Some athletes and spectators descending on Paris ahead of the Olympics were delayed as was the arrival of their uniforms and accreditations, but Games organizers said disruptions were limited and didn’t affect ticketing or the torch relay.
“This is a very, very uncomfortable illustration of the fragility of the world’s core internet infrastructure,” said Ciaran Martin, a professor at Oxford University’s Blavatnik School of Government and former Head of Britain’s National Cyber Security Centre.
Ian Lee, associate professor at Carleton University’s Sprott School of Business, says that while the outage illustrates the widespread dependency on technology, “outliers” like this are “a price to pay.”
“This is not an argument that we have become too dependent on these systems. It is inevitable because of the complexity of the modern world in which we live,” Lee told OBJ. “I’m sure this company is going to have lessons learned … and a ‘what went wrong’ analysis.
“It really does wreak havoc, I’m not trying to trivialize it,” said Lee. “But the idea that we go back to the Stone Age, to stone tablets or quill and paper is just too absurd.”
Instead, he said “outliers” like this “extremely unusual meltdown” do not justify arguments to return to “the good old days.”
“The old days was a terrible system,” he said. “The idea we’ll go back to (that) system isn’t true, it’s not possible, so instead, we’ll say, ‘From time to time, the system will fail. How can we create a temporary backup system?’
“There’s real money involved, there will be real money involved, so it’s going to motivate these companies to problem-solve to make sure we don’t experience something like this again of this magnitude,” Lee continued. “But companies are enormously adaptive and they will adapt.”
DownDectector, which tracks user-reported disruptions to internet services, recorded that airlines, payment platforms and online shopping websites across the world were affected – although the disruption appeared piecemeal and was apparently related to whether the companies used Microsoft cloud-based services.
Cyber expert James Bore said real harm would be caused by the outage because systems people have come to rely on at critical times are not going to be available. Hospitals, for example, will struggle to sort out appointments and those who need care may not get it — and it will lead to deaths, he said.
“All of these systems are running the same software,” Bore said. “We’ve made all of these tools so widespread that when things inevitably go wrong – and they will, as we’ve seen – they go wrong at a huge scale.”
The head of Germany’s IT security agency, Claudia Plattner, said that “the problems will last some time – we can’t expect a very quick solution.” A forecast for when exactly all systems will be up and running is difficult, but “it won’t be hours,” she added.
Toronto-based cybersecurity and technology lawyer Brent Arnold says the scope of the global outage is enormous.
Arnold says the disruption — caused by a software update gone wrong rather than a security breach — could the largest of its kind.
He says the outage is a good reminder of just how interconnected the technology industry is, and the costs could be substantial given the number of companies affected.
CrowdStrike said in a recording on its customer service line that the problem was related to “the Falcon sensor,” referring to one of its products used to block online attacks. The company says it has 29,000 customers.
During an interview on NBC’s “Today Show” Friday, CrowdStrike CEO George Kurtz apologized for the outage, saying the company was “deeply sorry for the impact that we’ve caused to customers, to travellers, to anyone affected by this, including our companies.”
“We know what the issue is” and are working to remediate it, Kurtz said.
“It was only the Microsoft operating system” that was affected, though it didn’t happen on every Microsoft Windows system, he said.
Shares of the company, which is based in Austin, Texas, fell nearly 10 per cent on Friday. Microsoft’s stock price fell more than three per cent.
Though the outage’s impact could be felt far and wide, the forecasting firm Capital Economics said it was likely to have little impact on the world economy.
Cybersecurity experts said those affected by the outage also needed to be wary of bad actors reaching out claiming they can help. “Attackers will definitely prey on organizations as a result of this,” said Gartner analyst Eric Grenier.
Meanwhile, governments, officials and companies across the world scrambled to respond.
Across Canada, the outage snarled traffic at border crossings and caused delays at airports.
Police in Windsor, for example, reported there are long delays at the Canada-U.S. border, both at the Detroit-Windsor Tunnel and the Ambassador Bridge, due to the worldwide outage.
Meanwhile, Pearson International Airport in Toronto said flight delays and cancellations were to be expected as airlines at Canada’s largest airport experienced technology disruptions.
Some companies were reporting their operations were unaffected by the tech outage.
Rogers Communications said there have been no impacts to its phone and internet networks, nor the media outlets it owns.
The Canadian Bankers Association says its members are reviewing the situation based on updates from their technology partners.
It says Canada’s banking system is well-protected and that current effects on banking services would be temporary.
The industry group represents 60 domestic and foreign banks in Canada.
Air Canada said there were no major impacts to its operations, but the airline is monitoring the situation closely.
TMX Group, which operates the Toronto Stock Exchange, said its core trading and clearing systems were operating as normal. It is continuing to monitor for any potential impacts.
South of the border, the White House said U.S. President Joe Biden was briefed on the outage and his team has been touch with the company and other impacted entities.
New Zealand’s acting prime minister, David Seymour, said on X that officials in the country were “moving at pace to understand the potential impacts,” adding that he had no information indicating it was a cybersecurity threat.
The issue was causing “inconvenience” for the public and businesses, he added.
On the Milan stock exchange, the FTSE MIB index of blue-chip Italian stocks could not be compiled for an hour, though trading continued.
Major delays reported at airports grew on Friday morning, with most attributing the problems in booking systems of individual airlines.
In the U.S., the FAA said the airlines United, American, Delta and Allegiant had all been grounded.
Airlines and railways in the U.K. were also affected, with longer than usual waiting times.
In Germany, Berlin-Brandenburg Airport halted flights for several hours due to difficulties in checking in passengers, while landings at Zurich airport were suspended and flights in Hungary, Italy and Turkey disrupted.
The Dutch carrier KLM said it had been “forced to suspend most” of its operations.
Amsterdam’s Schiphol Airport warned that the outage was having a “major impact on flights” to and from the busy European hub. The chaotic morning coincided with one of the busiest days of the year for Schiphol.
Widespread problems were reported at Australian airports, where lines grew and some passengers were stranded as online check-in services and self-service booths were disabled – although flights were still operating.
In India, Hong Kong and Thailand, many airlines were forced to manually check in passengers. An airline in Kenya was also reporting disruption.
While the outages were being experienced worldwide, Australia appeared to be severely affected by the issue. Disruption reported on the site DownDetector included the banks NAB, Commonwealth and Bendigo, and the airlines Virgin Australia and Qantas, as well as internet and phone providers such as Telstra.
National news outlets – including public broadcaster ABC and Sky News Australia – were unable to broadcast on their TV and radio channels for hours. Some news anchors went on air online from dark offices, in front of computers showing “blue screens of death.”
Britain’s National Health Service said the outage caused problems at most doctors’ offices across England. NHS England said in a statement said the glitch was affecting the appointment and patient record system used across the public health system.
Some hospitals in northern Germany canceled all elective surgery scheduled for Friday, but emergency care was unaffected.
Israel said its hospitals and post office operations were disrupted.
In South Africa, at least one major bank said it was experiencing nationwide service disruptions as customers reported they were unable to make payments using their bank cards in stores. The New Zealand banks ASB and Kiwibank said their services were down as well.
Shipping was disrupted too. A major container hub in the Baltic port of Gdansk, Poland, the Baltic Hub, said it was battling problems resulting from the global system outage.
– With additional reporting from the Canadian Press and the Associated Press.
