Log-in here if you’re already a subscriber
A longstanding difference of opinion on the appropriate design standard for electric vertical take-off and landing aircraft has become an acute point of contention as U.S. and European regulators vie for leadership in the space and eVTOL developers face the prospect of unequal hurdles on their paths to market.
At issue is the theoretical likelihood of a catastrophic failure condition, as computed through numerical analyses used to support the certification of aircraft systems. The European Union Aviation Safety Agency wants developers of commercial air taxis to substantiate that likelihood to ten to the minus nine (10-9) or one in a billion flight hours, the same standard to which transport category aircraft including jetliners are held.
Related: Autonomous aircraft startups are plotting an evolution, not a revolution
However, the Federal Aviation Administration is open to a more flexible approach that matches the level of design rigor to the size and complexity of the aircraft, consistent with its safety continuum for small airplanes and helicopters. That could mean certifying some systems to a standard of 10-8, one in 100 million, or 10-7, one in 10 million flight hours, probabilities that by most measures are still highly unlikely.
Until the FAA began accepting comments on its proposed airworthiness criteria for Joby Aviation’s eVTOL aircraft, these varying safety objectives were seen as a sticking point between the regulators, but not necessarily one that needed to be addressed immediately. However, in the closing days of 2022, as the deadline for comments neared, EASA and European eVTOL developers barraged the FAA with comments insisting on a 10-9 standard for urban air mobility operations, despite the fact that numerical safety objectives are typically addressed in means of compliance, not high-level airworthiness criteria.
Advocates of 10-9 say they simply want electric air taxis to be as safe as possible and that anything less would be cutting corners. Yet, supporters of the FAA’s approach believe that insisting on the highest level of design rigor could actually be counterproductive for safety, and can point to historical examples where the 10-9 standard prevented the widespread adoption of safety-enhancing technologies.
Meanwhile, the choice of numerical safety objective could have a significant impact on the time and cost involved in certifying an eVTOL aircraft, which would disproportionately impact startups with limited resources. Although the safety implications of 10-8 versus 10-9 are debatable, the higher standard would hand a clear advantage to legacy players like Airbus and Boeing with deep pockets and no particular rush to certify an eVTOL.
The backdrop for the debate is a general aviation landscape that looks very different on each side of the Atlantic, with entrenched disparities in operating rules reflecting philosophical differences between the FAA and EASA that extend well beyond eVTOLs. In a special investigative feature, The Air Current dives deep into the arguments for and against 10-9, and how regulators’ decisions now could shape the future of this emerging market.
The spectrum of ‘extremely improbable’
There are many things that can go wrong with an aircraft, and it is the job of aircraft designers to anticipate what those things are. They are not entirely on their own in this task. Over the course of decades, the aviation industry has developed processes and methods for rigorously evaluating the safety of aircraft and aircraft systems, which are now spelled out in standards like SAE International’s ARP4761.
The safety assessment process typically begins with a functional hazard assessment: a comprehensive examination of aircraft and system functions to identify potential failure conditions and their likely effect on safety. Failure conditions are classified by severity. At the low end of the scale are failure conditions that have no effect on safety, and minor failures that reduce safety margins only slightly. At the other end are hazardous failure conditions that cause large reductions in safety margins, and catastrophic failures that are likely to result in loss of the aircraft and multiple fatalities.
Related: Batteries are a looming certification challenge for electric aviation hopefuls
As anyone who has had to implement a safety management system knows, risk is the combination of the likelihood of a negative outcome and its severity. A minor failure condition that occurs frequently usually poses less risk than a hazardous failure that happens rarely. A basic principle of aircraft certification is that the probability and severity of each failure condition should be inversely related.
Regulators will generally accept minor failure conditions that are likely to happen during the life of an individual aircraft, but they all agree that catastrophic failures should be extremely improbable. The term extremely improbable, now enshrined in regulation, implies that a failure condition is unlikely to occur during the full operational life of the entire fleet.
According to John Vincent, CEO of the International Federation of Airworthiness, the aviation industry began adopting a numerical approach to probabilities starting in the 1960s, as the increasingly complex systems on large airplanes began to warrant objective standards and more rigorous approaches to complying with them. In a presentation last year, Vincent explained that 10-9 as a system safety objective for catastrophic failures on transport category aircraft can be seen as having its origin in the accident rate for large airplanes in the 1970s, which was roughly four per million flight hours, or 4 x 10-6.
Assuming that 10% of those accidents were due to aircraft systems failures yielded a systems-related accident rate of four per 10 million flight hours, 4 x 10-7. On the principle that new technology should improve safety, not reduce it, a safety objective of less than one systems-related accident per 10 million flight hours, 1 x 10-7, was established. Assuming that a transport category airplane has around 100 catastrophic failure conditions required a safety objective for any single failure condition of one per billion flight hours, or 10-9.
Related: Crash safety comes to the fore with a new generation of electric aircraft
The FAA describes a version of this logic for small airplanes in Advisory Circular 23.1309. Historically, single-engine airplanes under 6,000 pounds had a higher fatal accident rate to begin with, estimated at one per 10,000 flight hours (10-4) or 10-5 for the 10% of accidents caused by systems failures. But because they were so much simpler than jetliners, each aircraft was assumed to have just 10 catastrophic failure conditions, not 100. That resulted in a safety objective for any single failure condition of 10-6 rather than 10-9.
In this way, the numerical probability associated with “extremely improbable” varies with both the complexity of the aircraft and the societally accepted accident rate. Small aircraft typically have fewer possible failure conditions than larger aircraft, and society tolerates them crashing more frequently. Today, regulators interpret extremely improbable as 10-6, 10-7, 10-8 or 10-9 depending on things like aircraft weight, number of occupants and engine type. Engineers use various numerical methods to substantiate these probabilities, drawing on data from component service histories, testing and simulations.
Meanwhile, the certification of aviation software also links the level of rigor required in software development to the severity of a possible failure condition. Established by the standards organization RTCA, DO-178 is the primary means for obtaining approval of aviation software and defines five development assurance levels (DALs), A through E, based on the consequences should the software malfunction. For transport category aircraft, catastrophic failure conditions are associated with the highest level of rigor, DAL A. However, just as authorities accept different numerical probabilities for smaller aircraft, they also accept lower DALs, B and C, for critical software on small airplanes and rotorcraft.
One number to rule them all
There’s no question that the probabilistic approach to aircraft safety has been enormously successful. On balance, the reliability of civil aircraft has continuously improved since the 1970s even as their systems have become spectacularly complex. While there have been some conspicuous failures (such as the 737 Max) many of these have been traced to faulty assumptions and poor implementation of system safety processes, rather than inherent flaws in the processes themselves. Meanwhile, these processes continue to evolve to become more rigorous and robust.
Related: Boeing’s MCAS on the 737 Max may not have been needed at all
Although 10-9 is rooted in very rough estimates of accident rates and catastrophic failure conditions, its selection as a safety objective for transport category aircraft has pushed the industry toward system design architectures that have proven themselves in service. Broadly speaking, this means more redundancy and independence — up to a point — for critical components. An individual flight control computer can have less than 10-9 reliability if there are two independent backup computers that combine to push the system over the threshold.
When EASA began writing its Special Condition for Small Category VTOL Aircraft (SC VTOL), first published in July 2019, it included the boilerplate requirement for any aircraft system — “each catastrophic failure condition is extremely improbable” — and emphasized that such conditions must not result from a single failure. However, it also chose to make early decisions about the numerical probabilities and functional development assurance levels (FDALs) it would associate with extremely improbable as part of a means of compliance.
On one hand, most eVTOLs are small aircraft that lack the internal combustion engines, mechanical transmissions, hydraulic and other systems that can fail catastrophically on conventional aircraft. On the other, they are also fly-by-wire aircraft with complex integrated digital systems and novel forms of propulsion and energy storage. Perhaps more significantly, their developers were promising that urban skies would soon be crowded with thousands of the vehicles, which would carry millions of passengers per year over some of the world’s most congested metropolitan areas.
Ultimately, EASA chose to establish two categories for eVTOL certification: Basic and Enhanced. Basic eVTOLs for private use away from urban areas have a sliding scale of safety objectives based on their maximum seating configuration, similar to the sliding scale established for small airplanes. However, Category Enhanced aircraft, which carry commercial passengers and fly over congested areas, must satisfy 10-9 and FDAL A criteria for catastrophic failure conditions, whether they carry two people or nine.
European eVTOL developers haven’t pushed back. “It was a commitment from day one, to go [up] against the same certification criteria as a civil [airliner] and not to lower any of the standards towards safety,” said Volocopter CEO Dirk Hoke, whose two-seat VoloCity multicopter is being held to the same level of design rigor as an Airbus A350. “One failure and we all go back to step one, and I think we cannot take the risk to play around with the trust of the population.”
The perils of being too stringent
While there is broad consensus on the overall benefits of a probabilistic approach to aircraft safety, there is less agreement on how meaningful numerical estimates are once they move into the realm of the highly theoretical. No aircraft type has collectively flown a billion flight hours, thus there is no service record to substantiate estimates at that level (the nearest contender is the Boeing 737 family, which as of early 2022 had logged more than 401 million flight hours over more than 50 years of operation). Does a catastrophic failure condition pose substantially more risk if it is projected to occur twice rather than once in a billion flight hours? Probably not, but a strict application of 10-9 wouldn’t allow it.
The pitfalls of numerical probability estimates have been recognized for as long as they’ve been applied to aircraft certification. As the FAA highlights in its System Safety Handbook, a probability indicates that a failure condition is possible over a given series of operations, but it can’t say when during that series the failure will occur. Most probabilities rely on statistics obtained from past experience, but seemingly minor changes to manufacturing processes or operating conditions can nullify that statistical data. And reliability estimates for complex systems often have relatively low confidence levels because they’re necessarily based on small samples.
The mathematical argument for applying 10-9 to commercial eVTOLs is that eventually air taxi operations will reach such a scale that any higher frequency of catastrophic failures will be unacceptable. However, regulators do not typically allow any aircraft type to fail at a predictable tempo. When a catastrophic failure condition occurs despite having been deemed extremely improbable — whether at the 10-6, 10-7, 10-8 or 10-9 level — regulators will usually ground the model and/or mandate corrective action. If two or three eVTOLs crash before a fix is implemented, the cost in human life and property damage will still be vastly less than if a single 150-passenger jetliner crashes under the same circumstances.
All of these are reasons why some eVTOL developers are reluctant to accept a blanket 10-9 and DAL A requirement for catastrophic failure conditions, which can impose steep costs on a small aircraft project for a questionable increase in safety. Some of them believe their resources would be better invested in developing software that makes their aircraft dramatically easier to fly, given that loss of control is a leading cause of fatal accidents, especially in small aircraft. Today, automation can be certified to lower levels of reliability in transport category aircraft with the assumption that the human pilots will manage adequately if the system fails. However, the 737 Max and other accidents illustrate that assumption is not always valid.
“When we talk about design assurance level or, for example, a 10-9 probability of a catastrophic failure, what is the overall safety performance of that aircraft and operation when the system reverts to a pilot?” Walter Desrosier, vice president of engineering and maintenance for the General Aviation Manufacturers Association, wrote in an email to The Air Current. “If an additional capability such as envelope protection or stall avoidance with design level of 10-8 or 10-7 which supports the pilot could be added, overall safety performance could be improved.”
Related: What a safety debate about self-driving cars means for self-flying aircraft
The FAA is generally open to such arguments because of its longstanding embrace of the safety continuum, which is a philosophy that the level of safety established by regulation, guidance and oversight should vary for different aircraft types based on risk and societal expectations of safety. “While too little safety oversight is a clear threat to safety, too much rigor can prove to be an equal threat,” the FAA has explained in associated literature. “If certification requirements and oversight are overly stringent, safety can be jeopardized because the burden of certification will prevent the adoption of safety enhancing technologies.”
The consequences of “too much rigor” can be seen in the FAA’s 2001 decision to attach the 10-9 standard to failure conditions in normal category helicopters that would result in a hazardously misleading indication of attitude, airspeed and altitude in instrument meteorological conditions. Previously, single-engine helicopters had routinely been certified for instrument flight rules, but the new safety objective made it cost-prohibitive to pursue IFR certification for all but the most expensive multi-engine helicopters.
The result was a move toward VFR-only operations that was later blamed for a culture of pushing into poor weather and a high rate of VFR-into-IMC accidents. In 2017, responding to industry pressure, the FAA adopted a formal safety continuum for normal category rotorcraft that lowers the safety objectives for single-engine helicopters, similar to its existing safety continuum for Part 23 airplanes. Yet, the legacy of its 2001 decision endures in the relative infrequency of IFR helicopter operations in the U.S. and the persistence of VFR-into-IMC accidents, such as the 2020 crash that killed basketball star Kobe Bryant.
Peter Lyons, head of certification at the California-based eVTOL developer Overair, endorses the concept that there is an optimal level of design rigor that maximizes the adoption of new technologies that increase operational safety.
“I truly believe that the eVTOLs being developed today will be the safest class of aircraft in history,” he said in an email. “We have so much opportunity for huge advancements in operational safety. To name a few: the robustness, redundancy and resilience built into these aircraft, the elimination of single points of failure, the replacement of fuel-burning engines with electric engines that have far fewer parts and failure modes, their operation in much more controlled environments with more software-based envelope protection features.”
Philosophical differences
Unlike EASA, the FAA has not established public safety objectives for eVTOL aircraft; these are being negotiated individually with each applicant in their means of compliance. Multiple sources have indicated to TAC that different systems may have different safety objectives, with some flight control and propulsion systems subject to at least 10-8 and DAL B, a step above what is required for small airplanes and rotorcraft with comparable seating capacities.
In a statement to TAC, the FAA said: “Manufacturers have not applied to have their aircraft certified as transport category aircraft. Accordingly, the FAA will use certification standards applicable to the size and complexity of aircraft and the types of operations envisioned. The FAA is working closely with the European Aviation Safety Agency to harmonize certification standards.”
The FAA and EASA are actually fairly aligned on high-level requirements for eVTOL safety; for example, both EASA’s SC VTOL and the FAA’s proposed airworthiness criteria for Joby indicate that catastrophic failure conditions should be “extremely improbable”. Because 10-9 and FDAL A are specified in means of compliance, EASA could theoretically choose to accept alternate means of compliance with different safety objectives.
However, while EASA indicated to TAC that it would consider proportionate safety levels for Category Basic aircraft, it emphasized that it has chosen to implement a “very stringent standard” for Category Enhanced eVTOLs “similar to other urban commercial aircraft operations.” Recently posted comments on Joby’s airworthiness criteria by the European eVTOL developers Airbus, Lilium, Vertical Aerospace and Volocopter all echo that 10-9 is the only acceptable standard for commercial passenger flights over congested areas.
In its own comment, the U.K. Civil Aviation Authority, which has elected to adopt SC VTOL as its basis for certifying eVTOLs, likewise criticized the FAA for failing to apply Part 23 Class 4 (commuter category) safety objectives, which are 10-9 and DAL A for catastrophic failure conditions.
“The FAA states that the [Joby] JAS4-1 will be used in Pt-135 operations,” the CAA wrote, referring to the FAA certification that encompasses both on-demand and commuter operations. “However, Pt-135 does not exist in the UK and as such CAA considers that the intended type of operation would fall under Commercial Air Transport (Pt-121 equivalent). This aligns with category Enhanced under SC.VTOL.”
The CAA’s comment highlights what may be the fundamental conflict between the FAA and EASA: the former’s generally more permissive attitude toward operations other than scheduled Part 121 air transport. For example, in the U.S. and many other parts of the world, single-engine turboprops have long been allowed to conduct commercial operations at night and in IMC. However, they were prohibited from doing so in Europe until 2017, and it took EASA and its member states more than a decade of study and consultation to accept that modern single-engine turboprops could be as safe or safer than older twin-engine planes that were flying people around in the clouds.
Today, single-engine helicopters certified at the level of 10-6 to 10-8 can operate freely over most U.S. cities in Part 135 operations, but in Europe, the privilege of flying over congested areas is typically restricted to multi-engine helicopters certified to 10-9, with the associated costs resulting in fewer users. It would be inconsistent for European authorities to expect lower safety objectives in electric air taxis that operate over congested areas, just as it would be inconsistent for the FAA to demand much higher standards for eVTOLs that fly over cities in the U.S.
Can harmonization wait?
While everyone agrees that perfect harmonization between the FAA and EASA on eVTOL certification requirements would be ideal, opinions vary on how important it is to achieve concurrence now, when both regulators are still learning about eVTOL technology and it is not clear which of their approaches is superior. In recent testimony before the U.S. Senate Commerce Committee, Stéphane Fymat of Honeywell — an investor in and supplier to Lilium and Vertical Aerospace — presented it as an urgent imperative, asking Congress to use the upcoming FAA Reauthorization Act to mandate harmonization with EASA and the UK CAA by 2024.
“If the FAA wants a safety continuum from 10-7 [to] 10-9, that’s fine. But the FAA should make sure their 10-9 standards are harmonized with EASA such that U.S. manufacturers have a global standard. The FAA should automatically validate 10-9 certified aircraft, so that European manufacturers are absolutely assured of a straightforward path to the U.S. market,” Fymat told TAC by email.
Lilium Chief Airworthiness Officer Bhavesh Mandalia said a lack of harmonization could undermine confidence in specific eVTOL models and the industry more broadly. “I think if, for example, you’re allowed to operate one product within one region, and not other regions, that’s making a statement,” he said, suggesting that it would be “better headlines” for the FAA to move to higher safety objectives than for EASA to move to lower ones.
Yet, while U.S. eVTOL developers that certify to lower safety objectives could be disadvantaged in international markets, a more flexible certification path could also get them to market much sooner, which is the first and most critical hurdle for pre-revenue startups. Once in service, gaining real operational experience, they could start proving their safety case or iterating their product for foreign regulators.
So much of the argument for 10-9 hinges on the assumption that the urban air mobility market will play out exactly as forecast in a 2021 investor deck — that the first generation of certified eVTOLs will be moving passengers on the scale of commercial airlines by the end of the decade, rather than progressively expanding the existing market for vertical flight alongside helicopters. Boeing’s joint venture Wisk, for example, has confidently projected that its self-flying air taxis will be performing 14 million flights per year within five years of certification.
“We believe that when operating air taxi aircraft at scale, over dense urban environments, it is imperative to design air taxi aircraft to the highest levels of safety,” Wisk stated. “That means designing to meet a 10-9 or equivalent level of safety for safety-critical systems.”
Of course, this future vision remains highly speculative, and if eVTOL developers can’t overcome the hurdle of initial certification with a reasonably affordable product, it will never happen at all. At the end of the day, the traveling public won’t particularly care whether their air taxi is certified to 10-8 or 10-9 — they will care about whether it gets them to their destination safely. Accomplishing that will require a holistic approach toward safety in design, manufacturing, maintenance and operations, of which system safety objectives are just one part.
Write to Elan Head at elan@theaircurrent.com
Subscribe to continue reading...Subscribe to The Air Current
Our award-winning aerospace reporting combines the highest standards of journalism with the level of technical detail and rigor expected by a sophisticated industry audience.
- Exclusive reporting and analysis on the strategy and technology of flying
- Full access to our archive of industry intelligence
- We respect your time; everything we publish earns your attention
