By now, plenty has been written in the media about Southwest, but I wanted to understand this all a bit more deeply, so I did what analysts do — I dove into the situation, reading articles (such as this one from the Los Angeles Times), social media, message boards, and blog posts. I found, among other sources, a person claiming to be a technology engineer for the airline and a pilot who posted his views on Facebook. What did these sources have in common? They each blame different parts of the organization for the failure, which tells me that the issue was systemic, not isolated. I also found references to software failures with an application called SkySolver and one called Crew. To help you learn the lessons from Southwest, here is my assessment of what happened and why.
Southwest Airlines Has Been An Innovation Leader
I was an A-lister on Southwest for many years. It was the easiest way to get from my home in Southern California to many places, including Forrester’s San Francisco office and Texas, where my family lives. As a business, Southwest appeared to have it all, including an industry-leading Net Promoter Score℠ (NPS) and a reputation as an innovator. Just this year, it announced a plan to invest $2 billion in new customer experience and digital upgrades, including improved Wi-Fi, larger overhead bins, and power ports, as well as new functionality for the carrier’s digital platforms and airport kiosks. The airline has also been a leader in employee experience, according to Forrester research. So why did Southwest flight operations suddenly grind to a halt in the middle of the busiest travel season of the year, stranding thousands of its customers? It begins by understanding Southwest’s point-to-point (P2P) flight network, how great it was, and what a vulnerability it has become.
Southwest’s P2P Innovation Depended On Aging Technology
Transportation networks are living systems, where one flight depends on a daisy chain of other flights, each with dependencies on crew, parts, and terminal availability, to name a just few. Most airlines have hubs in their networks; hubs make transportation networks easier to manage because most things move through a few locations. The downside to hubs is greater travel times between destinations and the increased risk of delays.
A P2P network has no hubs: Flights move from one city to the next in a carefully choreographed sequence. Southwest managed to create a P2P network serving over 100 cities and 4,000 flights daily. This lets it offer more nonstops, get flyers to destinations more quickly, and have fewer delays. P2P networks are more sensitive to disturbances, however, and have less room for errors. But if operated correctly, they can also be more resilient, especially when hubs go down.
To keep its system optimized, Southwest depends on a complex suite of analytics and software. An analytics application called SkySolver likely provides crew routing recommendations to the Crew system, which can move crew members from one location to another to staff flights. Multiple sources also mentioned Southwest’s antiquated process for tracking crew locations: using the telephone.
The dependence on information technology and voice communications turned Southwest’s greatest innovation into its biggest liability:
- Technology systems and processes were neglected. The pilot mentioned serious problems in technology, infrastructure, and staffing that were occurring and being ignored. He said that leadership had finally recognized these problems, but “two decades of neglect takes several years to overcome.” The software engineer corroborated this story of neglect: “[The software] went offline due to its outdated software packages and overutilized server resources, aka CPU, memory, and disk space.”
- SkySolver likely exceeded its design capacity and began to fail. The type of problems being crunched by optimizers such as SkySolver quickly grow more difficult as increasing constraints are placed on the problems that they are solving. SkySolver was probably designed to handle a certain number of flight delays and cancellations that happen due to air traffic, weather, and maintenance issues. This year’s extreme weather caused more delays and cancellations, which forced more constraints and calculations to keep the network operating. Aging servers, networks, and storage systems would eventually begin to give out.
- Small issues cascaded until Southwest’s network collapsed. You can imagine what happened next. As Crew received bad recommendations from SkySolver, employees probably began to miss movements to their new flights, creating even more demand on the system. Phone calls for directions clogged up phone lines. This interfered with other crew members who could not report their locations, introducing data issues. Where were the pilots and flight attendants? This continued to increase the load on systems, causing more errors until massive flight cancellations were needed. Southwest’s systems didn’t know where its crews were. “The frontline employees were ready and on station. We were properly staffed. We were at the airports. Hell, we were on the airplanes. But our antiquated software systems failed, coupled with a decades-old system of having to manage 20,000 frontline employees by phone calls. No automation had been developed to run this.”
- The problems went on for days. This also explains why the cancellations rolled on for days — once crews were out of place, the airline likely had trouble manually recalculating and repositioning enough people in time, causing more delays, which affected the following day, and so on.
Southwest Was A Victim Of Its Own Success
The pilot we found pointed at a series of C-level execs who were finance-oriented accountants, not tech-savvy operators. He said that leadership rode a wave of strong benefits for eight years as the C-suite focused on “ […] stock buybacks and Wall Street.” He said reduced flight operations during the pandemic further masked the cracks in the plaster. The software engineer corroborated a lack of technical understanding at the leadership level and neglect by saying, “Southwest Airlines thinks of itself as an airline first and not a technology company … ”
What Southwest failed to recognize, while it was raking in the profits and buying back stock, was that the industry was changing and the software that it built its business on was not keeping up. Between 2004 and 2019, the number of airline flights increased 69%. Even now, the number of flights per year is up 45% from 2004. The industry is also growing more complex. Other airlines such as American, Delta, and United have been investing in partnerships and a significant amount in technology innovation to deal with these issues. While Southwest does invest in customer experience, it has continued to function as a discount airline and neglected its back-end operations infrastructure — all of that has just caught up with the company.
We will have more to say in future blogs on topics including future fitness, the importance of listening to employees, and how to structure your technology leadership. Our goal: to help you learn the lessons from the Southwest crisis so that this does not happen to you. Stay tuned.