Systems Research and Behavioral Science

Syst. Res. 17, 543–559 (2000)

& Research Paper

On a Wing and a Prayer?

Exploring the Human Components
of Technological Failure
Denis Smith*
Centre for Risk and Crisis Management, Sheffield University Management School, Sheffield, UK

This paper attempts to explore the human factors and systems dynamics of human±
machine interaction by reference to the Kegworth aircraft accident. The paper seeks to
move beyond the more traditional human factors literature to include research findings
from both systems research and crisis management in an attempt to examine the
relationships between active and latent failures. The role of human error within
complex technical systems is explored and particular emphasis is placed upon the
dynamics of latent errors within the management of such systems. The main thesis
developed here is that, while there are important cognitive processes at work within
accident causation, attention needs to be moved away from the level of the operator to
the wider managerial and social frameworks within which individuals work. Copyright
# 2000 John Wiley & Sons, Ltd.

Keywords Kegworth air crash; organizational resilience; sense making; crisis management

INTRODUCTION human operator and machines and the sub-

The study of human reliability is to a large
extent motivated by the need to prevent
unwanted consequences from erroneous
actions Ð in other words to prevent accidents
from happening. (Hollnagel, 1993, p. 12)
The Chapter of Accidents is the Longest
Chapter in the Book. (John Wilkes)
The academic literature has long recognized both
the importance of the interaction between the
* Correspondence to: Denis Smith, Centre for Risk and Crisis
Management, Sheffield University Management School, 9 Mappin
Street, Sheffield S1 4DT, UK.
sequent impact of organizational factors on this
relationship. In more recent years, a series of
major disasters have provided a powerful illus-
tration of the complex dynamics of the failure
process and have pointed to the dominant role of
latent error within the chain of causality for such
events. As technical systems have become more
complicated, it appears that they may have begun
to exceed the abilities of their human operators to
control them effectively. This is particularly
apparent when elements of the system are work-
ing in a degraded mode. Under these conditions,
managers and operators tend to engage in trade-
offs, thus eroding organizational defences to the
point of vulnerability (see Turner, 1978).

Copyright # 2000 John Wiley & Sons, Ltd. Received 15 December 1997
Accepted 2 June 1999
RESEARCH PAPER
Because of the importance of human±machine
interactions within the failure of systems,
designers have sought to factor multiple safe-
guards into their designs in an attempt to
provide more defences against the risks of
negative human intervention. However, in so
doing, they may also increase the complexity
and opacity of the system. This in turn will
further erode the capacity of operators to recover
the process at the point when it is close to
collapse, especially in those cases where the
automated controls can no longer cope with the
problem. This has, in effect, relegated the role of
operator to that of a monitor under steady state
conditions, but requires a rapid role change
when the system moves into a degraded mode.
Under these conditions, operators may become
isolated from the control of the process and
denied effective feedback from the computer
until the point of failure is reached (see Wiener,
1985). At this point they are then required to
make sense of the situation and develop appro-
priate intervention strategies to prevent the
problem escalating. The result has been the
creation of a complex working environment,
for operators and managers, within which
both the probability of human error may have
been increased, and the consequences associated
with such error heightened. The study of error
within systems failure has, therefore, widened
from an initial concern with the role of human
operators to include a broader understanding of
both managers and systems designers in pre-
cipitating systems and organizational failures
(see Turner, 1976, 1978; Reason, 1990a, 1990b,
1997).
Our aim in this paper is to examine the
relationships between systems vulnerability and
human factors issues by reference to the
Kegworth air crash. Of particular interest here
is the manner in which human operators make
sense of the problems that face them under con-
ditions where the system has become vulnerable.
However, before examining this accident in more
detail it is necessary to provide some clarity
on the use of the term `system' and to outline
some of the current research on vulnerability and
sense making which has relevance to our
discussions.
Copyright *
c 2000 John Wiley & Sons, Ltd.
544
Syst. Res.
There is considerable ambiguity in the use of
the term `system' (see Checkland, 1981). Much
of this ambiguity stems from the common usage
of the term, compared to its meaning within
academic research, and this has prompted some
to argue for the use of other terms in order to
provide greater clarity. The key characteristics of
a system can be considered as:
the abstract idea of a whole having emergent
properties, a layered structure and processes
of communication and control which in
principle enable it to survive in a changing
environment. (Checkland and Scholes, 1990,
p. 22)
Irrespective of the semantics used, the central
notion of system is that it is an abstract means of
describing a complex phenomenon in its entirety,
rather than taking a narrower, reductionist
approach. It is the interactions between the
parts of the system and the emergent properties
that arise as a result, which give the system
characteristics that are often overlooked when we
aggregate smaller elements together (see Bignall
and Fortune, 1984; O'Connor and McDermott,
1997). Almost by definition, it is the various
aspects of emergence that create problems for the
operators of high-technology systems. These
emergent properties are invariably not covered
by operating protocols and, as such, require the
operators to make sense of what is happening at
the point of failure and then to develop strategies
to cope with those problems.
Within systems thinking, there are a number of
common elements which have a particular
importance here. For example, Checkland
(1981) highlights the notions of communication,
control, emergence and hierarchy as central
concepts within systems thinking and which
clearly have implications for an understanding
of systems failure. If we add the broad category
of human factors to these issues then a complex
mosaic for analysis can be seen to develop. For
the purposes of this paper, there are two main
issues that emerge from this mosaic which are
worthy of further discussion, namely vulner-
ability and sense making (as an element of
Weltanschauung). It is to a discussion of these
concepts that our attention must now turn.
Syst. Res. 17, 543–559 (2000)
Denis Smith
Syst. Res.
SYSTEMS, VULNERABILITY AND
SENSE MAKING
Fortune and Peters (1995) describe the notion of
vulnerability in terms of a system's susceptibility
to the adverse consequences of a trigger event.
As such, it relates closely to Turner's (1976, 1978)
notion of incubation and Reason's (1990a, 1997)
`resident pathogen' metaphor. The resilience of a
system can be seen to exist across its human,
organizational and technological (HOT) (Shri-
vastava et al., 1988) components. Consequently, it
is important to think in terms of a matrix of
vulnerability incorporating these HOT factors,
along with the important notions of space and
time (Giddens, 1990) as these also have implica-
tions for sense making as well as vulnerability.
The resultant `matrix' can be expressed in such a
way that we can conceptualize pathways of
vulnerability within a system which occur
because of the impact of the emergent properties
upon the interaction of the spatial and temporal
dynamics of these HOT factors (see Figure 1).
The capability of managers and operators to
cope with emergent properties will obviously be
a function of their ability to make sense of the
issues that they face. This leads us into the
second issue, which is relevant to a discussion of
failure, namely the concept of `Weltanschauung'.
This notion of world-view is a central component
Figure 1. Pathways of vulnerability
Copyright *
c 2000 John Wiley & Sons, Ltd.
On a Wing and a Prayer?
RESEARCH PAPER
of systems thinking because, as Fortune and
Peters argue.
mismatches in expectations may arise from
difference in Weltanschauung but because
each party takes their own assumptions as
`given' and assumes they are shared by others
the differences may not emerge in advance of
the failure being perceived. Indeed, without
specific attempts to apply the concept of
Weltanschauung they may never be revealed.
(Fortune and Peters, 1995, pp. 240±241)
It is clear then that different expectations about
the likely performance of a system may create
problems of interpretation, especially when that
system is operating in a degraded mode. Because
of the nature of complex technologies, organiza-
tions have devised protocols to be used when the
system moves outside of its normal operating
envelope. While the use of these protocols is
often an important training tool for operators
and may help condition them to respond to
certain events, they are weaker when dealing
with emergent properties. Some have argued
that the reliance on protocols can impair the
problem-solving capabilities of operators (see
Degani and Wiener, 1993), particularly as the
emergent properties of the system will invariably
not be covered by such protocols. This raises an
important issue for the training of operators in
safety critical system which, if not adequately
addressed, may lead to difficulties for the
operators when the system begins to fail during
normal operations (see Diehl, 1991; Edwards,
1990; Green, 1990; Hurst and Hurst, 1982; Jensen,
1995; Nagel, 1988). It is this challenge to the
dominant world-view concerning systems beha-
viour that is an important dynamic of the sense-
making process, particularly when dealing with
conditions of failure.
Within the context of systems failure, sense
making can also be linked to the concept of
resilience (Weick, 1993) and therefore vulner-
ability. For Weick, sense making as a process
is tested to the extreme when people encoun-
ter an event whose occurrence is so implau-
sible that they hesitate to report it for fear they
will not be believed. In essence these people
Syst. Res. 17, 543–559 (2000)
545
RESEARCH PAPER
think to themselves, it can't be, therefore, it
isn't. (Weick, 1995, p. 1)
Under conditions of failure, our ability (or
otherwise) to make sense of the emergent pro-
perties associated with that failure will be
important in dealing with systems vulnerability.
Given Weick's (1990, 1993) assertion that sense
making is concerned with
contextual and negotiated agreements that
attempt to reduce confusion (Weick, 1993,
p. 636)
and that
The basic idea of sense-making is that reality
is an ongoing accomplishment that emerges
from efforts to create order and make retro-
spective sense of what occurs. (Weick, 1993,
p. 635)
then it is clear that there is considerable scope for
error within this process, particularly for complex
and uncertain problems. Of particular importance
here is the role of the organization as a controlling
construct. This should allow for the provision of
some sort of order on the various conflicting
environmental and task demands that impinge
upon those who are making sense of systems
problems (Weick, 1993). The social dynamic of
sense making is also an important factor in
explaining the manner in which cues are inter-
preted, communication is shaped and behaviours
are affected (Weick, 1995). Weick (1993) has also
observed that both role structures and interacting
routines are important factors in framing organ-
izations and their behaviour. When these routines
and role structures break down they erode
organizational resilience which may, in turn,
impact upon the likelihood and significance of
human error. When the system operates in a
degraded mode it is likely that routines and role
structures can break down or, alternatively, they
may compound the problem by exaggerating the
impact of emergence.
Resilience in the face of any technological
failure will invariably be a function of organiza-
tional and human factors, as well as those fail-safe
mechanisms which are built into the technology.
While these may all mitigate the negative impacts
Copyright *
c 2000 John Wiley & Sons, Ltd.
546
Syst. Res.
of the failure, there remains a requirement that the
human component of the system is adept at
making sense of these problems. In addition,
there is a need to ensure that role disintegration
does not take place. This is especially important
when the system is operating in degraded mode
(Weick, 1988, 1993, 1995) as the human operator
may prove to be the last line of defence, with the
formal systems defences proving to be an
``armour of complacency''.
HUMAN INTERVENTION: ROOT CAUSE OR
LAST LINE OF DEFENCE?
What holds organisations in place may be
more tenuous than we realise. (Weick, 1993,
p. 638)
Considerable research has focused on the gener-
ation of `human error' amongst operators, al-
though there is an increasing recognition that the
notion of a single root failure is an inadequate
construct in explaining failure generation (see
Reason, 1997). The incidence of failure attributed
to human causes has risen from an estimated
20% in the 1960s to some 80% in the 1990s
(Hollnagel, 1993; Meshkati, 1991). This change is,
to some extent, a reflection of the increasing
complexity of technical and organizational
systems and the resultant inability of humans at
all stages in the design, build and operating
process to exercise control over that system. In
virtually every case, the generation of failure
events cannot simply be attributed to a single
source but rather results from the complex
mosaic of interactions that occur between
elements. This complexity necessitates that
operators are able to deal with a range of `non-
design emergencies' (Reason, 1990a) which lie
outside of the known failure envelope designated
by systems designers. While contingency plans
may be available for many events, there will
always be the unforeseen cascade through the
system which was either not considered, or
possibly even dismissed, by designers as being
so improbable as to be impossible (see Perrow,
1984). It is these events that operators and
Syst. Res. 17, 543–559 (2000)
Denis Smith
Syst. Res.
Figure 2. Points of intervention for the management of human error. Source: Reason (1990a, p. 202).
managers have to make sense of and it is here
where the greatest potential for error lies.
In developing a framework for analysing
accidents, Reason (1990a, 1990b, 1997) identifies
a number of points of intervention for the
management of human error and these are
shown in Figure 2. Reason argues that too little
attention is focused on line management
deficiencies and the psychological precursors of
unsafe acts within accident analysis. Too much
weight is given in many post-accident analyses to
the point at which the accident occurred Ð that
is, the unsafe act Ð and this gives a distorted
view as to the nature of human error. The notion
that errors arise from the actions of operators
ignores a whole series of decisions taken by
managers and systems designers in the construc-
tion and maintenance of the system within which
the operators work. A series of faulty assump-
tions on the part of managers may actually result
in the incubation of disaster potential, which only
becomes realized when the disaster or accident
occurs. In many cases, there is evidence to suggest
that the operators made mistakes because their
operational environment encouraged them to do
Copyright *
c 2000 John Wiley & Sons, Ltd.
On a Wing and a Prayer?
RESEARCH PAPER
so. Similarly, the role of the organization (and its
associated culture) in shaping the pattern of
individual behaviour within a crisis event is
often ignored by managers.
In discussing the sense-making process in-
volved in the Mann Gulch fire, Weick elaborates
on the role of `organization', arguing that we can
see a `recipe for disorganization' which
reads, Thrust people into unfamiliar roles,
leave some key roles unfilled, make the task
more ambiguous, discredit the role system,
and make all of these changes in a context in
which small events can combine into some-
thing monstrous. (Weick, 1993, p. 638)
This notion of disorganization is an important
concept within any discussion of systems failure.
For non-design emergencies, operators will be
placed into unfamiliar role situations and the
associated stress of the event may lead to cogni-
tive narrowing taking place, thus weakening the
abilities of the operators to manage the problem
still further. In particular, the effect of cognitive
narrowing would inhibit the problem-solving
capabilities of those operators and managers
Syst. Res. 17, 543–559 (2000)
547
RESEARCH PAPER
who are interacting with the technology at the
point of failure, thereby increasing the problems
of system emergence. Their interpretation of
events will create a set of extracted cues and
these will then shape subsequent behaviour
(Weick, 1988, 1995). When the technical system
is also tightly coupled and interactively complex
(Perrow, 1984), then any failure will quickly
escalate from an incident to a major event, thus
creating severe time constraints for operators. If
this is combined with role ambiguity and
communication breakdown within the organiza-
tion, then the recipe for catastrophe is clear. It is
this downward spiral of failure that operators
and managers need to make sense of. These
aspects of the `recipe' for a downward spiral
would fall within Reason's (1990a) stages of
fallible decisions, line management deficiencies
and psychological precursors. They would have
a significant impact upon the cognitive processes
that underpin the unsafe act itself and would
also go some way towards undermining the
defences that the organization has in place.
Miller (1988) argues that an oversimplification
of blame within accident investigation results in
the oft-quoted cause of human error and neglects
further investigation of the wider issues ident-
ified here. Miller also argues that accidents
invariably result from a sequence of events in
which a causal chain is present and that, in all
cases, the principle of `known precedent'a applies
(Miller, 1988). This raises an interesting question
for those who manage complex technologies,
namely: how do we ensure that the `known
precedent' is dealt with in the sense-making
process so that systems vulnerability can be
reduced? It is now necessary to examine some
of these issues in more detail by reference to the
Kegworth air crash.
SAFETY CRITICAL SYSTEMS AND THE
MANAGEMENT OF ERROR: THE CASE OF
THE KEGWORTH AIR CRASH
As aircraft become more sophisticated, they
create problems of interpretation for pilots
when non-routine events occur. Modern aircraft
a c
Miller defines this notion as meaning that `it is rare, if ever, that new
accident causal factors appear' (Miller, 1988, p. 60).
Copyright *
c 2000 John Wiley & Sons, Ltd.
548
Syst. Res.
have automated many of the tasks previously
undertaken by flight engineers and pilots and
this has, in turn, resulted in a reduction of flight
deck staff. Such reductions have served to erode
slack in the system, which would have been a
valuable resource if the aircraft experienced a
major failure. Similar `economies' have been
made by reducing the number of engines needed
for normal flights to two. One of these modern
twin-jet aircraft involved in a fatal accident was
the British Midland Boeing 737±400 which
crashed on the M1 motorway in January 1989
with the loss of 47 lives (Air Accidents Investi-
gation Branch, 1990; Carter, 1994). This accident
occurred because the pilots, who allegedly
became confused by their instruments and the
nature of the event, shut down the wrong engine
following an engine fire and the aircraft crashed
on its approach to the airport. While the pilots
carried much of the blame for the accident, and
subsequently had their employment terminated,b
the accident was much more complex than a
simple case of pilot error (Smith, 1992). Although
there is little doubt that human intervention was
a principal cause of the disaster, the question of
whether the pilots were as culpable in causing
the accident, as many accounts suggest, is open
to debate. It can be argued that there are a range
of factors that precipitated the event and these
were subsequently compounded by the actions
of the pilots as they tried to make sense of the
problem.
The accident took place when the aircraft was
en route from London Heathrow to Belfast.c The
aircraft was relatively new, having logged only
521 airframe hours, and had come into service
with British Midland in 1988 (Air Accidents
Investigation Branch, 1990). This aircraft differed
from the 300 Series in a number of ways. The
most noticeable changes related to the new
engines, which were an upgraded version of
engines fitted to earlier models, and the pro-
vision of a `glass cockpit' in the cabin. Both of
these factors were major contributors to the
accident. The Captain and First Officer could be
b
There seems little doubt that this decision to terminate their
employment was a result of the accident.
For a full account of the accident see Air Accidents Investigation
Branch (1990), Carter (1994) and Smith (1992).
Syst. Res. 17, 543–559 (2000)
Denis Smith
Syst. Res.
considered as experienced pilots and had logged
a total of 13,176 and 3290 flying hours respect-
ively (Air Accidents Investigation Branch, 1990).
However, of these hours, the Captain had only
logged 763 hours on B-737 aircraft, of which 23
hours had been on the 400 Series, and the First
Officer had 192 hours on the B-737, with 53 on
the 400 Series (Air Accidents Investigation
Branch, 1990). Both the Captain and the First
Officer completed their conversion course for the
400 Series aircraft on the 17 October 1988 (Air
Accidents Investigation Branch, 1990). The pilots
claim that this conversion course for the 400
Series took the form of a lecture and tape±slide
presentation, which did not give them a com-
plete understanding of the aircraft's character-
istics. Unfortunately, there was apparently no
simulator training for the 400 Series aircraft
available at that time in the company, as this
would have allowed the pilots to gain valuable
experience with the new instrumentation prior to
making commercial flights. The event which
faced them that evening, namely an engine
failure and compressor stalls, would have been
the first time that they had been confronted with
that particular set of instrument conditions. In
addition, the pilots have argued that the vibra-
tion meter, which was in a secondary position,
did not provide them with a sufficiently clear
warning that a problem existed. Again, simu-
lated training may have given them more
familiarity with the instrumentation's character-
istics.d
The aircraft engine initially failed some 20 min-
utes after take-off. As the aircraft climbed
through 28,300 feet, part of a fan blade detached
itself in the port (left-hand) engine and caused
some damage to the engine (Air Accidents
Investigation Branch, 1990). The pilots became
confused by the combination of cues Ð namely,
heavy vibration and associated noise, shudder-
ing and a smell of burning Ð and they believed
that the right-hand engine was on fire. After
throttling the engine back, the captain decided to
shut it down. The pilots took the decision to
divert to the airline's home base at Castle
Donnington near Nottingham (East Midlands
d
British Midland introduced their B737-3/4/500 simulator in 1990
(Aviation Information Resources, 1995).
Copyright *
c 2000 John Wiley & Sons, Ltd.
On a Wing and a Prayer?
RESEARCH PAPER
Airport) rather than return to Heathrow. Upon
attempting to land at East Midlands Airport, the
port engine lost power and the pilots received a
fire warning. Immediate attempts were made to
restart the right-hand engine but these were
unsuccessful and the plane crashed on the M1
motorway.
The first indication that the pilots had of a
problem with the engine was when the plane
vibrated heavily and, according to the Captain,
smoke came through the air conditioning system.
The First Officer informed the Captain that there
was a fire and that, after some hesitation, stated
that it was in the right (no. 2) engine. The
Captain then ordered the throttling back of the
supposedly damaged engine which, he held
later, was common procedure, before closing it
down. This action gave the indications that their
actions in identifying the failure had been correct
as the vibration ceased. However, by shutting
down an engine, the pilots had overridden the
fuel management system and this allowed more
fuel to be provided to the damaged engine,
thereby reducing the juddering to a negligible
level (Air Accidents Investigation Branch, 1990).
Had the pilots checked the vibration indicator,
which was in a secondary position amongst the
engine instrumentation, they might have been
more readily able to identify the source of the
problem. However, the position and size of this
instrument resulted in it being neglected by the
First Officer when he made his initial diagnosis
of the problem. This initial error was not
corrected later when the pilots tried to evaluate
their decision making. Nothing seemed to alert
them to the real cause of the problem or to the
mistake that they had made and there were no
apparent warnings provided by those who had
observed the fire.
In this case, the error did not arise because
there were not available contingencies available
to the pilots but because a relatively bad rule,
that of throttling back on the supposedly
damaged engine, compounded the pilots' initial
judgemental error and reinforced that false
judgement. Had the pilots been able to visually
confirm the status of the engine, either through
the provision of external cameras or by a physical
check through the passenger cabin windows,
Syst. Res. 17, 543–559 (2000)
549
RESEARCH PAPER
then their initial mistake would have been
recognized. The problem was compounded
further by the fact that the cabin crew did not
alert the pilots to the fact that several passengers
had witnessed smoke and flames come out of the
left hand (no. 1) engine.
The AAIB Report stated that the cause of the
crash was due to the fact that
the operating crew shut down the No 2 engine
after a fan blade had fractured in the No 1
engine. This engine subsequently suffered a
major thrust loss due to secondary fan
damage after power had been increased
during the final approach. (Air Accidents
Investigation Branch, 1990, p. 2)
In addition, the report listed five factors that
compounded this erroneous decision by the
pilots. These were: that the combination of vibra-
tion, noise, smell and shuddering were outside of
the pilots' experience; that, contrary to their
training, they reacted too quickly to the event;
the pilots did not fully assimilate the information
provided by their instruments before throttling
back the no. 2 engine; in throttling back the no. 2
engine, the vibration, noise and shuddering
ceased and this reinforced their view that they
had made a correct decision; and finally, the
pilots were not informed that many passengers
and some crew had seen flames coming out of
the no. 1 engine (Air Accidents Investigation
Branch, 1990). In addition, the report made 31
safety recommendations associated with the
accident, many of which related to technical
and design issues.
DISCUSSION
The accident at Kegworth illustrates many of the
problems surrounding the process of error within
an organizational setting. There is a general
recognition within the literature that failures in
technical systems are all too often attributed to
human causes (see, amongst others, Norman,
1990; Hollnagel, 1993). However, there is also a
view that such an attribution of blame is some-
what misplaced as there are a multitude of
factors which interact to generate failure (see
Copyright *
c 2000 John Wiley & Sons, Ltd.
550
Syst. Res.
Turner, 1976, 1978; Perrow, 1984; Wagenaar et al.,
1990; Smith, 1995; Reason, 1997). In dealing with
this issue, Connors et al. (1994) suggest a number
of reasons why failure is disproportionately attri-
buted to operators. These reasons are considered
to be as follows: a reluctance by managers to
consider the various design-related issues
(because of the irradiation effect that this would
have on other systems of the same type); that
blaming the operator may deflect some financial
liability for the event away from the organiz-
ation; the disproportionate power relationships
within organizations and the general helpless-
ness of operators within the legitimation process
tend to result in a culture of blame; and that there
is a tendency amongst operators to blame
themselves when dealing with complex tech-
nologies (Connors et al., 1994).
Such arguments have relevance for the accid-
ent at Kegworth. Had the root cause of the crash
been given as a faulty engine, rather than the
focus on operator error that ensued, then this
may have resulted in all Boeing 737±400 aircraft
being grounded until the fault was rectified.
Given that two other aircraft experienced a
similar engine failure after the Kegworth crash,
although with no loss of life (Air Accidents
Investigations Branch, 1990), then such concerns
over a possible flaw in the engine may have been
considered to be well founded. Indeed, there are
a number of other technical factors that may have
a causal bearing on this accident or may have
impacted upon the casualties that ensued. These
include: the failure to fit external cameras,
following the engine fire in a Boeing 737 aircraft
at Manchester Airport; the apparent reluctance
on the part of manufacturers to strengthen cabin
floors and seat provision; the failure on the part
of operators and manufacturers to fit rear-facing
seats in aircraft; and the nature of overhead
storage bins and their impact upon upper body
injuries during a crash-landing. What is clear is
that operator error, while an important active
failure within this accident chain, served to
expose a whole series of latent failures within
the system and these made a major contribution
to the event (Smith, 1992) both in terms of
causality and consequence. These errors can be
categorized according to the various points of
Syst. Res. 17, 543–559 (2000)
Denis Smith
Syst. Res.
intervention outlined by Reason and are shown
in Table 1.
What is clear from Table 1 is that the unsafe act
of switching off the wrong engine was preceded
by a whole series of latent failures, which com-
pounded its impact. In addition, there were a
number of systems-related problems and inade-
quate defences which failed either to prevent this
unsafe act from occurring in the first place or to
mitigate the consequences of its impact once the
error was made. Within safety critical systems, it
is important that visual and auditory warnings
are given to operators in order that they may
clearly identify the source of any problem. It
would appear from this accident that such clarity
was not afforded to the pilots from their instru-
mentation. Herein lies the danger for automated
systems. In this context, Wiener has observed
that
automated devices, while preventing many
errors, seem to invite other errors. In fact, as a
generalisation, it appears that automation
tunes out small errors and creates opportu-
nities for large ones. (Wiener, 1985, p. 83)
Indeed, in discussing the monitoring role of
pilots on the modern flight deck, Wiener goes on
to argue that
we are still making the same seemingly
contradictory statement: a human being is a
poor monitor, but that is what he or she
should be doing. (Wiener, 1985, p. 87)
In the case of the Kegworth accident, it is
apparent that the pilots were not good monitors
of the system; although one can also point to the
failure of the flight services crew to communicate
eyewitness accounts of the engine fire to the
pilots, as being a major compounding factor in
the accident. Similarly, the ability to make a
visual check on the engines from the cockpit via
external cameras would also have been an
invaluable feedback mechanism. The accident
also raises issues surrounding the role and
effectiveness of checklists in fault diagnosis, an
issue that has again attracted the attention of
researchers in human factors (see Degani and
Wiener, 1993). This event suggests that the use of
checklists proved to be an inadequate method of
Copyright *
c 2000 John Wiley & Sons, Ltd.
On a Wing and a Prayer?
RESEARCH PAPER
focusing the pilots' attention on the likely source
of the problem.
MAKING SENSE OF THE EVENT
If we try to model the dynamics of a cockpit and
the interactions between pilots and the machine,
then a series of potential error sources emerge.
These are shown in Figure 3, which is based on
the accident at Kegworth. There are a number of
important issues surrounding the interaction
between human operators and the technology
that they are working with, as well as issues
relating to the crew resource management
process (see, for example, Weick and Roberts,
1993). The distraction caused by external com-
munications from Air Traffic Control, the
inability of the checklist to help the pilots
successfully identify the root cause of the
problem and correct the error, the systemic
barriers to communication within the aircraft
and the role of instrumentation in providing
valuable feedback to the pilots are all considered
to be important characteristics of the accident
chain for this event.
A key aspect of this accident was the role of the
aircraft's instrumentation in both confusing the
pilots and failing to provide them with a clear
indication of their initial error. In this case the
error did not arise because there were no availa-
ble contingencies but because a relatively bad
rule Ð that is, throttling back on the supposedly
damaged engine Ð compounded the initial error
and reinforced their flawed judgement. A num-
ber of questions remain unanswered by the
findings of the official report and, while some
blame does inevitably rest with both the pilots
and other crew members, there are further issues
that proved to be important in compounding the
problems faced on the flight deck.
Firstly, one has to question the effectiveness of
the team that was operating the aircraft. This
obviously includes the two pilots and their
relative abilities in that task, but also includes
the interaction between the flight services crew,
who had knowledge about the event but failed to
pass it on to the pilots. It also raises questions
concerning the reasons why two pilots with less
Syst. Res. 17, 543–559 (2000)
551
RESEARCH PAPER Syst. Res.

Table 1. Points of intervention for error generation applied to the Kegworth air crash
Points of intervention
Fallible decisions
. Manufacturer's decision not to test the engine in a ¯ying test bed
. No out-of-range markings provided on the vibration meter (attention-getting
device)
. The cabin services crew did not pass on vital information concerning direct
observations of events to the pilots
. The decision to divert to East Midlands Airport may have increased the
workload for the pilots due to the time factor and additional load. This may have
prevented a full assessment of events
. Co-pilot's determination of the source of the engine ®re was inaccurate (the left
engine was the faulty one and not the right engine)
. The pilots were deemed to have acted too quickly to the situation and did not
assimilate information from their instruments prior to making a decision

Line management . Lack of effective communication between the cabin crew and the pilots prevented
de®ciencies passenger accounts of the ®re being passed up to the ¯ight deck
. Awareness raising needed for the improved accuracy of vibration meters
. The nature of the training provided to cabin staff for emergency situations may
be insuf®cient to cope with such an event
. Need for greater simulator training for new aircraft types
. The presence of a ¯ight engineer might have led to an effective diagnosis of the
fault (aircraft designed for a two-person crew) by adding slack to the decision
team
Psychological precursors of . The combination of smoke/smell of burning created a confused perception of
unsafe acts incident causality for the pilots due to their knowledge of the systems design
. The engine was throttled back and this gave the pilots the indication that the
problem was solved
. Possible effects of an authority gradient within the aircraft may have impaired
the quality of communication, decision making and problem solving (hurried
decision and diagnosis)
. Failure event was outside the experience of the pilots
. Competing radio transmission traf®c caused some measure of distraction for the
pilots as they were trying to work through their checklist
. Problems of information processing due to the con®guration of the engine
instruments (vibration meter in a secondary position)
Unsafe acts . Pilots shut off the wrong engine having initially throttled it back

Inadequate defences and
systems de®ciencies
Data drawn from Air Accidents Investigation Branch (1990); Carter (1994).
. Engine failed on the ®nal approach to land: pilots unable to correct the problem
by restarting the second engine due to time constraints
. Instrumentation inadequate to provide a suf®ciently clear warning of the
problem and its root cause
. Warning mechanisms inadequate for the pilots to take effective action
. No external cameras ®tted to provide pilots with a visual check on engine status
. Aircraft could have been further strengthened to exceed current design
requirements for crash testing which might have reduced the extent of the
fatalities and casualties (cabin ¯oor, seating and overhead stowage bins)
. The speed of interaction between the fan blade shattering and poor engine
performance was rapid
. Emergency procedures for engine failure and shutdown inadequate to cope with
the demands of the event ( pilot overload)
. The emergency checklist procedure appears to have been inadequate to allow the
pilots to diagnose the fault

Copyright *
c 2000 John Wiley & Sons, Ltd. Syst. Res. 17, 543–559 (2000)

552 Denis Smith

Syst. Res.
Figure 3. Possible cockpit-related interactions on board the Kegworth aircraft
than 76 hours on the 400 Series were rostered on
the same flight. However, given the absence of
simulator training for this type it was probably
inevitable that most crews would have limited
experience of the aircraft at the time of this
accident. Secondly, the training that the pilots
had been provided with to allow them to transfer
to the new model of 737 can also be questioned.
At the time of the accident the conversion course
consisted of a tape±slide presentation and
associated question and answer sessions. There
was no available simulator available for the
pilots at that time. Indeed, it appears that British
Midland may have only introduced the first
available simulators for this model of aircraft into
the UK as late as 1990e (Aviation Information
Resources, 1995). Thirdly, the ergonomics of the
glass cockpit and the switch to LED instruments
appear to have played a major role in precipitat-
ing the disaster. The vibration indicator, which
was deemed to be of importance within the
e
Aer Lingus introduced its B737-3/4/500 simulator in 1989 and
British Airways in 1991 (Aviation Information Resources, 1995).
Copyright *
c 2000 John Wiley & Sons, Ltd.
On a Wing and a Prayer?
RESEARCH PAPER
accident sequence, was both small and placed in
a secondary position within the bank of instru-
ments. Fourthly, the lack of external cameras,
which had been a recommendation from the
earlier Manchester Airport fire involving a 737,
prevented the pilots from visually checking the
status of the engines from the cockpit. Similarly,
the lack of a flight engineer on this type of
aircraft also compounded the problem, as that
individual would have been in a position both to
visually check upon the status of the engine and
to assess the instrument readings. These com-
ments, however, do not preclude the fact that the
pilots could have checked with the flight services
crew concerning their observation of the event,
or that the latter should have been active in
communicating with the pilots. This cultural
dynamic is an important contributor to many
accidents which are deemed to have human error
at their core. Fifthly, there was considerable
`interference' and `distractions' from Air Traffic
Control and this combined with the difficulties
that the pilots appeared to have with the
Syst. Res. 17, 543–559 (2000)
553
RESEARCH PAPER
Figure 4. Weick's seven properties of sense making
on-board navigational computer in diverting to
East Midlands Airport to inhibit their effective
evaluation of the decision. Finally, the conven-
tion of throttling back a supposedly damaged
engine seems a somewhat difficult rule to justify
in the light of this particular event.
MAKING SENSE OF CONFUSION
Those who forget that sensemaking is a social
process miss a constant substrate that shapes
interpretations and interpreting. Conduct is
contingent on the conduct of others, whether
those others are imagined or physically pre-
sent. (Weick, 1995, p. 39)
Weick (1995) sees sense making as a process with
seven main elements (see Figure 4). Firstly, he
argues that sense making is invariably grounded
in the notion of the self Ð the individual is an
integral part of the process. Consequently, our
determination of the problem and our ability to
define what is `out there' will ultimately be a
function of our notions of self. In this context,
sense making becomes
an ongoing puzzle undergoing continued
redefinition, coincident with presenting some
self to others and trying to decide which self is
appropriate. (Weick, 1995, p. 20)
Weick's second element concerns the retrospec-
tive nature of sense making. Here the actions of
those trying to make sense of the phenomena
will invariably become an integral part of the
problem being investigated: `An action can
Copyright *
c 2000 John Wiley & Sons, Ltd.
554
Syst. Res.
become an object of attention only after it has
occurred' (Weick, 1995, p. 26). Under the con-
ditions of emergence associated with systems
failure, it is clear that this creates considerable
problems for those trying to manage complex
technologies. Given the nature of coupling and
complexity and the role of human intervention in
shifting the pattern of the event, this process of
retrospective sense making is both complex and
often poorly understood by those caught up in a
disaster. This relates, in turn, to Weick's third
element which sees the process as being `enactive
of sensible environments' (Weick, 1995, p. 30).
Put another way, the environment is a social
construction and this then becomes an integrated
part of the problem being defined. In dealing
specifically with crises, Weick observes that
from the perspective of enactment, what is
striking is that crises can have small, volitional
beginnings in human action. Small events are
carried forward, cumulate with other events,
and over time systematically construct an
environment that is a rare combination of
unexpected simultaneous failures. (Weick,
1988, p. 309)
Weick (1988) also argues that the process of
enactment will be dependent upon three major
factors. These are: the commitment that those
involved in the sense making process have
towards their actions; the capacity that the actors
have to deal with the problem in terms of their
skills and perceptions; and the assumptions and
expectations held by decision makers (Weick,
1988). One can argue that Rasmussen's (1983,
1992) distinction between skills, rules and knowl-
edge will be of importance in shaping this
process of enactmentf.
Within this whole process, discourse and
conversation are central components of sense
making and they comprise Weick's fourth group.
Sense making is a social process and it is
inevitably dependent upon the discourse that
exists between those who are trying to come to
terms with the problem facing them. This
f
By segmenting their training programmes to deal with each of these
three aspects of an operator's task requirements, then it should allow
for a more holistic approach to the identification and management of
error and sensemaking issues.
Syst. Res. 17, 543–559 (2000)
Denis Smith
Syst. Res.
element of sense making is of particular import-
ance within aviation as flight crews attempt to
solve problems which fall outside of their
protocols (see Johnson, 1997; Linde, 1988; Cush-
ing, 1994). The fifth element sees sense making as
an ongoing process. As sense makers are
presented with more stimuli and information
then they will continue to re-evaluate the nature
of the problem or, perhaps, reject data that does
not fit into their current world-view. The
processes of insight and reflection are important
here, and the whole process is deemed to be tied
into those emotional factors and relationships
that exist between the various sense makers
(Weick, 1995). This is closely related to Weick's
sixth element, which concerns the extraction of
cues and the breaking up of the flow of events in
order to provide for an interpretation of the
event. This process runs the risk of becoming
reductionist, particularly when it becomes con-
textualized within organizational settings. In the
aftermath of a crisis event, this extraction of cues
will be important in allowing for the reconstruc-
tion of events, with the possible elevation in
significance of certain acts and stimuli over
others. The final element in the process is the
notion that sense making is driven by plausibility
rather than accuracy, for as Weick observes:
sensemaking is about plausibility, coherence,
and reasonableness. Sensemaking is about
accounts that are socially acceptable and
credible . . . It would be nice if these acceptable
accounts were also accurate. (Weick, 1995,
p. 61)
If the interpretation of the event is plausible
enough, then people will act upon the cues that
emerge and base their subsequent actions on that
interpretation. Under conditions of crisis, both
the time and the information required to ensure
greater accuracy are often not available to
decision makers. As a result, they then tend to
base their interpretation on the credible nature of
events. Under such conditions, this issue of
plausibility may lead to cognitive narrowing as
decision makers reject interpretations that do not
meet their world-view. Indeed, if the event
results from the emergent properties of the
system then it is likely that the decision makers
Copyright *
c 2000 John Wiley & Sons, Ltd.
On a Wing and a Prayer?
RESEARCH PAPER
may not recognize the pattern of events and will
go with the `solution' that fits within their
dominant paradigm. In drawing these issues
together, Weick details the overall sense-making
process as follows:
Once people begin to act (enactment), they
generate tangible outcomes (cues) in some
context (social), and this helps them discover
(retrospect) what is occurring (ongoing), what
needs to be explained ( plausibility), and what
should be done next (identity enhancement).
Managers keep forgetting that it is what they
do, not what they plan, that explains success.
(Weick, 1995, p. 55)
If we apply these seven elements to the crew
actions on the Kegworth plane then we can gain
some insights into the manner in which they
sought to make sense of the engine failure and
the resultant emergency.
The main patterns of sense making that were
undertaken by the Kegworth crew are shown in
Figure 5. The initial cues that were presented to
the pilots came from the noise, smoke, fumes and
smell associated with the engine fire. They
received no clear or effective warning from
their instruments to alert them to the source of
the problem. From this point on, the event
assumes characteristics that are influenced by
the First Officer's determination that the problem
was with the right-hand (no. 2) engine. This
error, combined with a failure of the aircraft's
warning systems to alert the pilots to their false
assumption, set in train a sequence of events that
would result in the accident.
The action of throttling back on the no. 2
engine and the disengaging of the auto-throttle
resulted in a reduction of the vibration and
smoke which gave the pilots an illusion of
normality. After some time, the Captain took
the decision to shut down the no. 2 engine as it
seemed plausible that they had correctly diag-
nosed the nature of the problem. This would
later create problems on the approach to East
Midlands Airport, as the crew could not restart
the healthy engine when the no. 1 engine began
to fail under the additional power requirements
of landing. Despite the severity of the situation
the pilots were apparently not made aware of the
Syst. Res. 17, 543–559 (2000)
555
RESEARCH PAPER
Figure 5. Patterns of sense making in the Kegworth aircraft
fact that a number of passengers and some cabin
staff had seen flames coming out of the no. 1
engine. The passengers made no comment when
the Captain announced that they had a problem
with the no. 2 engine, making the assumption
that the pilots were fully aware of the problem.
This may, in part, reflect on the social roles that
exist within aircraft and the possible authority
gradients (see Alkov et al., 1992) that can exist
within such social settings. For their part, some
of the cabin staff claimed that they had not heard
the Captain's reference to the right-hand engine,
although a number of the passengers clearly
heard the announcement. The communications
process was also central to the prevention of
effective review for the pilots. When they tried to
review the cues that they had received, they were
interrupted by legitimate communications from
Air Traffic Control.
There are a number of organizational issues
that also had an impact upon the event. The most
notable was the decision to roster pilots together
with little collective time on the 400 Series
(accepting the difficulties caused by a lack of
Copyright *
c 2000 John Wiley & Sons, Ltd.
556
Syst. Res.
simulator provision) and the usefulness of the
conversion training that was given to the pilots
in moving them to a glass cockpit aircraft. Given
the nature of the 400 Series cockpit layout, it
might have been expected that this conversion
course would have been more extensive to take
account of these changes. It can be suggested that
a greater familiarity with the instrumentation
and its significance might have helped the
Kegworth pilots in their sense making for the
event. Certainly the Captain believed that
the vibration meter was unreliable and, given
his experience on other aircraft, did not give it
the attention that it demanded. While the First
Officer made the initial determination of the
source of the problem, he could not remember
what prompted him to make that decision.
Clearly then, the instrumentation and warning
systems did little, if anything, to aid the pilots in
their decision making, although this may have
been as much an issue with the pilots themselves
as with the layout. This was compounded further
by the fact that the checklists did not alert the
pilots to their error in decision making and so
Syst. Res. 17, 543–559 (2000)
Denis Smith
Syst. Res.
their initial determination of the problem re-
mained plausible and, therefore, dominant. It re-
mains an issue for further research as to whether
the extensive use of checklists can impair the
effective sense-making process, especially given
the nature of emergence. While the process of
sense making continued for the short duration of
the flight to East Midlands airport, the constant
distractions and the time constraints imposed
upon the pilots prevented their initial mistake
from being identified. It is difficult to assess the
importance of role identity within this case,
although there would seem to be important
issues around the relative status of aircraft staff
and the possible effects of this upon communi-
cation. This issue would require further research
in a simulated environment before its impact on
the sense-making process could be evaluated.
CONCLUSIONS
Our actions are always a little further along
than is our understanding of those actions,
which means that we can intensify crises
literally before we know what we are doing.
Unwitting escalation of crises is especially
likely when technologies are complex, highly
interactive, non-routine and poorly under-
stood. The very action which enables people
to gain some understanding of these complex
technologies can also cause those technologies
to escalate and kill. (Weick, 1988, p. 308)
The accident at Kegworth illustrates the complex
interactions that precipitate crisis events. While
the root cause of the crash is held to be a result of
human error, there are clearly a number of
mitigating factors that need to be taken into
account when assessing causality. This paper has
sought to outline and review some of the main
issues surrounding the role of human error in
such events. However, the constraints of space
prevent a full discussion of the human factors
elements inherent in aircraft operations. It is clear
that further research in this area is required; in
particular, work is needed to assess the role of
modern glass cockpits in creating problems of
interpretation and sense making for pilots. This
Copyright *
c 2000 John Wiley & Sons, Ltd.
On a Wing and a Prayer?
RESEARCH PAPER
is likely to become more of an issue as a new
generation of glass cockpit and fly-by-wire
aircraft come into service. As these technologies
become more advanced, it is likely that the
human operators will be faced with further
sense-making problems, especially as these tech-
nologies become ever more increasingly con-
trolled by computers. The Kegworth accident
pointed to the need for clear diagnostics within
the instrumentation and that these systems
should provide pilots with unambiguously
clear information concerning the root cause of
problems with the aircraft. The design and layout
of instrumentation have, once again, proved to
be a significant factor in affecting the accident
chain, as previous evidence had already
suggested (see Mann and Schnetzler, 1986). On
this occasion, the position of the instruments,
and the pilots' perception of their accuracy and
reliability, combined with the available circum-
stantial evidence to generate a false assessment
of the problem. Again, this is a well-researched
area, although the new generation of cockpits
will create new and more complex scenarios for
researchers to deal with, and further work on the
sense-making process is required here. What is
also clear is that further attention needs to be
given to the processes by which organizations
learn from crisis events of this nature. Clearly,
sense making is an effective precursor to this
learning process and its core principles need to
be incorporated into accident investigation.
While a study of single-case accidents is
important, it is necessary to validate the theo-
retical frameworks outlined in this paper across a
range of accidents. In particular, it is important
to combine post-accident analysis of this nature
with other forms of data capture, in an attempt to
ensure that these frameworks are robust. Of
importance in this respect is the study of error
generation across aircraft types, or within the
context of a single organization where a safety
culture may not be in evidence. Further work is
clearly required in these areas, particularly those
involving the new generations of aircraft in
which the computer assumes greater control of
the operation than before. Such technological
developments have significant implications for
both the selection and training of pilots and the
Syst. Res. 17, 543–559 (2000)
557
RESEARCH PAPER
management of the interface between the soft-
ware engineers and the end users of the tech-
nology. The core assumptions of each group of
`experts' may not be the same and this may
increase the potential for error. It is in this area
that the broader concepts of crisis management
will be of importance in providing frameworks
for the assessment of error potential across the
organization, rather than simply at the opera-
tional level.
It is clear that human factors research has a
considerable role to play in preventing accidents
involving complex technologies. What is also
apparent is the need for such research to move
beyond the traditional cockpit-based assessment
of the problems and further into the wider
organizational context. It is here that a broader
understanding of cognitive processes within
latent error incubation would provide consider-
able scope to improve the reliability of organiza-
tions and the systems that they operate. This
would occur not only by improving the quality
of the systems themselves, but also by changing
the current paradigms that are prevalent within
management theory.
ACKNOWLEDGEMENTS
The author is grateful to a number of colleagues
who have made comments on earlier versions of
this paper. These include Dominic Elliott, Ray
Hudson, Rebecca Lawton, Jo McCloskey and
James Reason. In addition, the author is grateful
for the comments made by two anonymous
reviewers. Inevitably, the author is responsible
for all errors of omission and commission within
the paper.
REFERENCES
Air Accidents Investigation Branch (1990). Report on the
accident to Boeing 737-400 G-OBME near Kegworth,
Leicestershire on January 8, 1989. Department of
Transport, Aircraft Accident Report 4/90, HMSO,
London (Internet copy at http://roof.ccta.gov.uk/
aaib/gobme/gobmerep.htm).
Alkov, R. A., Borowsky, M. S., Williamson, D. W., and
Yacavone, D. W. (1992). The effect of trans-cockpit
Copyright *
c 2000 John Wiley & Sons, Ltd.
558
Syst. Res.
authority gradient on Navy/Marine helicopter mis-
haps. Aviation, Space, and Environmental Medicine
63(8), 659±661.
Aviation Information Resources (1995). Civil simu-
lators directory. Flight International 147(4468), 33±34,
36±38, 42±44, 46±50, 52±56.
Bignall, V., and Fortune, J. (1984). Understanding
Systems Failures, Manchester University Press,
Manchester, UK.
Carter, R. (1994). A major disaster Ð the M1 plane
crash Ð how it occurred. In Wallace, W. A., Rowles,
J. M., and Colton, C. L. (eds), Management of Dis-
asters and their Aftermath, BMJ, London, pp. 10±28.
Checkland, P. (1981). Systems Thinking, Systems Prac-
tice, Wiley, Chichester.
Checkland, P., and Scholes, J. (1990). Soft Systems
Methodology in Action, Wiley, Chichester.
Connors, M. M., Harrison, A. A., and Summit, J.
(1994). Crew systems: integrating human and
technical subsystems for the exploration of space.
Behavioral Science 39, 183±212.
Cushing, S. (1994). Fatal Words: Communication Clashes
and Aircraft Crashes, University of Chicago Press,
Chicago.
Degani, A., and Wiener, E. L. (1993). Cockpit check-
lists: concepts, design and use. Human Factors 35(2),
345±359.
Diehl, A. E. (1991). Human performance and systems
safety considerations in aviation mishaps. Inter-
national Journal of Aviation Psychology 1(2), 97±106.
Edwards, D. C. (1990). Pilot Mental and Physical
Performance, Iowa State University Press, Ames.
Fortune, J., and Peters, G. (1995). Learning from Failure:
The Systems Approach, Wiley, Chichester.
Giddens, A. (1990). The Consequences of Modernity,
Polity Press, Cambridge, UK.
Green, R. (1990). Human error on the flight deck.
Philosophical Transactions of the Royal Society of London
B 37, 503±512.
Hollnagel, E. (1993). Human Reliability Analysis: Context
and Control, Academic Press, London.
Hurst, R., and Hurst, L. R. (eds) (1982). Pilot Error: The
Human Factor, Jason Aronson, New York.
Jensen, R. S. (1995). Pilot Judgment and Crew Resource
Management, Avebury Aviation, Aldershot, UK.
Johnson, C. W. (1997). The epistemics of accidents.
International Journal of Human±Computer Studies
47(5), 659±688.
Linde, C. (1988). The quantitative study of commu-
nicative success: politeness and accidents in aviation
discourse. Language in Society 17(3), 375±399.
Mann, T. L., and Schnetzler, L. A. (1986). Evaluation of
formats for aircraft control/display units. Applied
Ergonomics 17(4), 265±270.
Meshkati, N. (1991). Human factors in large-scale
technological systems' accidents: Three Mile Island,
Bhopal, Chernobyl. Industrial Crisis Quarterly 5(2),
133±154.
Syst. Res. 17, 543–559 (2000)
Denis Smith
Syst. Res.
Miller, C. O. (1988). System safety. In Wiener, E. L.,
and Nagel, D. C. (eds), Human Factors in Aviation,
Academic Press, San Diego, pp. 53±80.
Nagel, D. C. (1988). Human error in aviation opera-
tions. In Weiner, E. L., and Nagel, D. C. (eds),
Human Factors in Aviation, Academic Press, San
Diego, pp. 263±303.
Norman, D. A. (1990). The problem with automation:
inappropriate feedback and interaction not over-
automation. Philosophical Transactions of the Royal
Society of London B 37, 585±593.
O'Connor, J., and McDermott, I. (1997). The Art of
Systems Thinking: Essential Skills for Creativity and
Problem Solving, Thorsons, London.
Perrow, C. (1984). Normal Accidents: Living with High-
Risk Technologies, Basic Books, New York.
Rasmussen, J. (1982). Human errors: a taxonomy for
describing human malfunction in industrial installa-
tions. Journal of Occupational Accidents 4, 311±335.
Rasmussen, J. (1983). Skills, rules, knowledge: signals,
signs and symbols and other distinctions in human
performance models. IEEE Transactions: Systems,
Man and Cybernetics 13, 257±267.
Reason, J. (1990a). Human Error, Cambridge University
Press, Cambridge, UK.
Reason, J. (1990b). The contribution of latent human
failures to the breakdown of complex systems.
Philosophical Transactions of the Royal Society of London
B 37, 475±484.
Reason, J. (1997). Managing the Risks of Organisational
Accidents, Ashgate, London.
Copyright *
c 2000 John Wiley & Sons, Ltd.
On a Wing and a Prayer?
RESEARCH PAPER
Shrivastava, P., Mitroff, I., Miller, D., and Miglani, M.
(1988). Understanding industrial crises. Journal of
Management Studies 25(2), 283±303.
Smith, D. (1992). The Kegworth aircrash: a crisis in
three phases? Disaster Management 4(2), 63±72.
Smith, D. (1995). The dark side of excellence: manag-
ing strategic failures. In Thompson, J. (ed.), Handbook
of Strategic Management, Butterworth-Heinemann,
London.
Turner, B. A. (1976). The organizational and inter-
organizational development of disasters. Adminis-
trative Science Quarterly 21, 378±397.
Turner, B. A. (1978). Manmade Disasters, Wykeham
Press, London.
Wagenaar, W. A., Hudson, P. T. W., and Reason, J. T.
(1990). Cognitive failures and accidents. Applied
Cognitive Psychology 4, 273±294.
Weick, K. E. (1988). Enacted sense-making in crisis
situations. Journal of Management Studies 25, 305±317.
Weick, K. E. (1990). The vulnerable system: an analysis
of the Tenerife air disaster. Journal of Management 16,
571±593.
Weick, K. E. (1993). The collapse of sense-making in
organisations: the Mann Gulchj disaster. Admini-
strative Science Quarterly 38, 628±652.
Weick, K. E. (1995). Sense-Making in Organisations,
Sage, Thousand Oaks, CA.
Weick, K. E., and Roberts, K. H. (1993). Collective
mind in organisations: heedful interrelating on flight
decks. Administrative Science Quarterly 38, 357±381.
Wiener, E. L. (1985). Beyond the sterile cockpit. Human
Factors 27(1), 75±90.
Syst. Res. 17, 543–559 (2000)
559