In the initial days and weeks after Boeing’s test flight of its new spacecraft went awry, the company and NASA went to great lengths to highlight the positives of the mission – how, as NASA administrator Jim Bridenstine said then, “a lot of things went right.”
But more than two months after the test mission was cut short by what both Boeing and NASA now acknowledged were potentially catastrophic software errors, the space agency is being far more blunt about the poor performance of one of its most trusted contractors and dictating the steps Boeing must take to fix the serious problems that have been uncovered so far.
In a call with reporters Friday, NASA officials said an independent investigation of the marred test flight of Boeing’s Starliner spacecraft has produced 61 corrective actions and identified 49 gaps in Boeing’s testing procedures. A decision on whether Boeing will be allowed to proceed with flying astronauts or have to redo the test mission without humans on board may be months away, they said.
“We could have lost a spacecraft twice during this mission,” said Doug Loverro, NASA’s associate administrator for human exploration and mission operations. “So clearly this was a close call.”
Along with SpaceX, Boeing is under contract to build a spacecraft to fly NASA’s astronauts to the International Space Station as part of NASA’s “commercial crew” program. NASA hasn’t had the ability to fly astronauts since the space shuttle was retired in 2011, and has faced numerous delays and setbacks in its attempt fly humans again from U.S. soil.
Given that lives are on the line, Loverro added that, “I want to make sure everybody understands that we at NASA are talking this very seriously . . . And we’re going to make sure that at the end of the day, we can fly astronauts safely on Starliner.”
The test of Boeing’s Starliner spacecraft ran into trouble almost from the moment it was hoisted into space shortly before Christmas. The spacecraft’s on board computer was off by 11 hours – a significant software problem that went undiscovered because Boeing’s preflight testing was cut short and used a faulty computer simulator.
While Starliner was in flight, Boeing uncovered another software problem that should have been unearthed by testing on the ground – one that could have caused the service module to crash into the crew module before the spacecraft was reentering the atmosphere.
“It’s important to remember we went into this flight. . . with a test plan,” said Jim Chilton, Boeing’s senior vice president for space and launch. “We had all agreed to that plan, and we executed the plan. And it wasn’t good enough.”
As a result, NASA is now planning on embedding some of its software experts into Boeing’s team to more rigorously oversee its work and testing. Examples of the types of corrective actions include fully testing all outcomes of the software instead of just the most likely, as well as strengthening oversight of the software teams.
“We had delegated too much authority to the software board to approve changes,” Loverro said, referring to the engineering team reviewing software processes.
Meanwhile, NASA’s probe of Boeing and its processes continues, as the space agency tries to figure out when it will allow Boeing to try again.
“We’ll evaluate the results of their work,” Loverro said. “And we’ll be in a position to decide whether or not we need another test flight or not. We are still a ways away from that. And I can’t even tell you what the schedule is for making that decision because it’s very dependent upon what we see as Boeing’s corrective action plan.”
Chilton said that Boeing has no “intent to avoid [another test flight]. We stand ready to do it.”
But a repeat of the test would come at an enormous cost in a program that is already unusually rigid and governed by what’s known as a “firmed fixed-price cost” contract. In case it does have to repeat the test, Boeing has taken a $410 million charge, it said during its most recent earnings call.
Boeing has been under enormous financial strain since the grounding of its 737 Max airplane fleet after two fatal crashes killed 346 people. Both the Max and Starliner problems were tied to software problems, and Chilton said Friday that the issues discovered during the Starliner investigation have been shared with the commercial airplane division.
“Certainly we have what we consider a strong, and in fact we are strengthening our central and core engineering organizations both around software and other things,” he said. “These learnings have been fed to those teams, and I know are being applied across our enterprise. I’m not aware of anything common or relevant to the 737 Max out of this.”
The call with NASA came as House investigators on Friday released a damning report that concluded the mistakes on the 737 Max were the result of “technical design failures, lack of transparency with both regulators and customers and efforts to obfuscate information about the operation of the aircraft.”