So what’s HOT?

Highly Optimised Tolerance (HOT) is a recent theory developed as part of the study of complex systems. HOT theory is derived from the study of biological and engineered systems and emphasises that such systems are typified by both highly structured, unique, self-dissimilar internal architectures and robust yet fragile external behaviour. 

If HOT theory holds true then we should be able to see a change in the distribution of the severity of adverse events as the design paradigm for a family of systems moves from the, ‘just make it work’ stage to the ‘optimise for robustness’ stage. This is something we can actually test through observation of real world systems.

Is commercial aviation a HOT system? The US commercial aviation aircraft fleet is a logical candidate for investigation as to whether it is a HOT system. For a start there is a set of readily available multi-decade accident data and most importantly a common set of system design requirements, i.e. Part 121 of the Federal Aviation Regulations. As an initial investigation we can partition NTSB accident records into rough decades and then plot the cumulative probability of X ≥ x event severity (2) for each decade (Fig. 1) to compare the shape of distributions.

The proponents of HOT (Carlson & Doyle inter alia) argue that this complexity both delivers robustness but also makes HOT systems vulnerable to catastrophic failure sequences triggered by minor perturbations.  As a result HOT systems exhibit heavy tail distributions (1). They believe this reflects the inherent trade-off between favouring small losses for common events (i.e. robustness for a specified fault hypothesis), at the expense of large losses when subject to rare perturbations (when the specified fault hypothesis is violated).

Fig. 1: NTSB accidents per decade

Fig. 1: NTSB accident severity and frequency per decade

There is a striking difference between the 1962-1972 data plot of Fig 1. and the subsequent data plots, with the 1962-1972 data plots decaying rapidly at the 100+ mark while the other plots exhibit tails extending to the right (3). The shift to the right in the tails of the curve can be explained by the growth in carrying capacity of aircraft across the sets of data (3) establishing a greater asymptote for each of the curves (4).

What’s interesting is that the 62-72 data appears to have the strongest cutoff with the other decades exhibiting weaker ones. In order to provide an insight into overall improvement of robustness we can also plot the total number of accidents per decade (Fig. 2). As Fig. 2 shows the total number of accidents per decade has trended down significantly from the 1962 to 2006 (Fig. 2) (5).
 
 NTSB FAR 121 aircraft accidents per decade

Fig. 2: NTSB FAR 121 aircraft accidents per decade

But is there a power law? The answer to this is, maybe sort of. Accident severity cannot behave as a pure power law as there is maximum upper size to all such events so, as the scaling factor approaches this cutoff value, the size of accidents must taper off. So from the data we do appear to have a truncated power law with an exponential cut-off.

And now a word of caution. All the above is boldly stated without conducting a formal statistical analysis. Hey, I’m writing a blog here! So to rigorously check my assertion requires firstly a formal estimation of the scaling factor, then defining where the scaling region finishes followed by checking the actual goodness of fit and, finally, comparing alternative distributions (such as Weibull, Log Normal)  to see if they’re a better fit. For those of steely will Clauset et al (Clauset, Shalizi, Newman 2007) provide an excellent discussion of the techniques required.

But does it really matter? As the proponents of HOT (Carlson, Doyle 2002) point out the fundamental point is that a HOT system doesn’t exhibit Gaussian or Exponential distributions, to that extent the data as presented confirms this assertion.

Footnotes

  1. Although a power law relation is stated by HOT’s proponents, this is not seen as being a critical element of the HOT theory (Carlson, Doyle 2002), rather they emphasise that the tails should not be exponential or gaussian.
  2. Severity of each accident is expressed by the total number of fatalities.
  3. The 1962-1972 period saw the capacities in mixed class layouts reach 200+ (727) then climb past 400+ (747) in the early seventies.
  4. Based on the worst credible aviation accident posited. For example the mid-air collision of two fully laden commercial aircraft, as occurred at Tenerife.
  5. This neatly illustrates the problem we have with proving safety as the robustness of systems increases. As robustness increases, the amount of empirical data actually decreases.

Further Reading

Carlson, J.M., Doyle, J., Complexity and Robustness, Proc. of the National Academy of Sciences, 19 February, 2002, vol. 99 suppl. 1 pg 2545.

Clauset, A., Shalizi, C.R., Newman, M. E. J., Power-law distributions in empirical data, 2007, arXiv:0706.1062v1, URL http://arxiv.org/abs/0706.1062v1.