Safety Science 57 (2013) 2734

Contents lists available at SciVerse ScienceDirect

Safety Science
journal homepage: www.elsevier.com/locate/ssci

Strategies in coping with complexity: Development of a behavioural marker system for air trafc controllers
Tom Kontogiannis a,, Stathis Malakis b
a b

Department of Production Engineering & Management, Technical University of Crete, Chania, Greece Hellenic Civil Aviation Authority, Diagoras International Airport, Rhodes, Greece

a r t i c l e

i n f o

a b s t r a c t
To meet the increasing air trafc volume, organizations seek better tools to assess the trafc handling capacity of air trafc control (ATC) systems. This effort requires a better understanding of how complex situations are related to controller strategies and how controllers intervene to maintain control. Recent empirical studies and reviews have shown that controllers cope with complexity by adapting priorities, managing their cognitive resources, and regulating their own performance. This study discusses the development of a behavioural marker system to evaluate and provide feedback on the strategies that controllers use to cope with complexity. An extensive literature review provided the basis for an initial classication of strategies for coping with complexity which was tested in an observational study for reliability. After three iterations of adaptation to the taxonomy, two independent raters were able to reach acceptable levels of reliability in classifying video segments of simulated trafc scenarios. A potential application of the study regards the design of refresher training enriched with the principles of error management and the assessment of new technologies and controller tools on the handling capacity of ATC systems. 2013 Elsevier Ltd. All rights reserved.

Article history: Received 21 July 2011 Received in revised form 4 December 2012 Accepted 20 January 2013

Keywords: Complexity Cognitive strategies Teamwork Behavioural markers Error management Air trafc control

1. Introduction The continued growth of civil aviation and the introduction of new air trafc management systems (e.g., free ight) have being changing the demands on air trafc control (ATC) services. The continuing growth of trafc requires an increase in the capacity of the airspace which can be achieved through the adaptation of airspace design (e.g., sector boundaries), controller tools (e.g., conict resolution), and operating procedures (e.g., reduced separation minima). To assess the appropriateness of design interventions, however, it is important to understand how a given air trafc situation is related to the cognitive difculty in controlling this situation and the associated workload (Mogford et al., 1995; Pawlak et al., 1996). Studying how controllers adapt their behaviour to cope with complexity is very important if we are to understand how modern technology and new demands may affect system performance. A comprehensive review of complexity in several domains showed that many denitions of complexity rely on the size or number of parts of the system which is not sufcient to account for the richness of what is meant by complexity (Edmonds, 1999). Another characteristic of complexity regards the connections between components and their dependencies that make it difcult to project system behaviour into the future. Xing and

Corresponding author.
E-mail address: konto@dpem.tuc.gr (T. Kontogiannis). 0925-7535/$ - see front matter 2013 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.ssci.2013.01.014

Manning (2005) proposed that complexity should be understood as a multidimensional construct with attributes encompassing the number and variety of elements as well as their relationships. On the basis of an extensive literature survey, a Eurocontrol report (Hilburn, 2004) identied several metrics of complexity based on regression models that weight task demands according to their predictive power. A well-known set of metrics is the dynamic density metrics (Laudeman et al., 1998; Masalonis et al., 2003) that attempt to predict change of mental workload over time. However, a unied dynamic density metric (Kopardekar and Magyarits, 2003) was found to account for less than half of the variance in self-ratings of mental workload. Empirically derived metrics, such as the dynamic density metrics, focus on task demands and fail to model the capacity of controllers to respond; this may explain why a signicant portion of variance remains unexplained (Loft et al., 2007). Recent studies have shown that the relationship between complexity and performance is not linear but it is an emergent property of the complex interaction between controllers and trafc situations (Athenes et al., 2002; Histon and Hansman, 2002; Mogford et al., 1995; Loft et al., 2007). This approach reects earlier views of Sperandio (1978) that the relationship between complexity and performance outcome can be better understood by considering how controllers adapt their strategies to manage their cognitive resources and regulate their workload. Brooker (2003) has postulated an adaptable function of performance over complexity that has been supported by earlier re-

28

T. Kontogiannis, S. Malakis / Safety Science 57 (2013) 2734

search. As complexity increases, controllers may change their priorities and the quality of service may become less important in favour of maintaining control of the whole stream of aircraft (i.e., fewer variables are taken into account). It is only above an upper trafc density that operational errors would become more frequent and general performance would deteriorate. It seems that there is little evidence to suggest that any sudden and uncontrolled fall of performance occurs, except at very high trafc ows. A critical review of the literature (Loft et al., 2007) adopted a systems control model to examine the relationship between complexity and performance (Fig. 1). On the one hand, performance can be adjusted by explicit control of the airspace (shown in the outer feedback loop) to reduce complexity. In this sense, controllers can take action to change future task demands fed back through the system (e.g., by accepting aircraft early or by putting aircraft in a holding pattern). On the other hand, complexity can be managed by reorganizing priorities and choosing a different strategy (see dashed lines for feedforward and inner feedback loops). Looking at feedforward (at left), controllers may become aware that a large number of aircraft are about to enter the sector and thus adjust their strategy so that the communication load with aircraft on frequency becomes less. Looking at feedback (at right), controllers may become aware of potential conicts and thus adjust their priorities toward achieving safety at the expense of quality of service. In these ways, the model in Fig. 1 shows that controllers monitor both the goal-state discrepancies and their own capacity to respond in order to develop their self-regulation strategies. The present study aims to classify strategies used by controllers to regulate their performance and maintain resilience despite high levels of complexity. The focus of this study has been on the inner feed-forward and feedback loops used by controllers to anticipate the work to be done and assess the work in progress. Attention should be paid, not only to individual strategies, but also to coordination and communication patterns that reduce workload and mitigate complexity. 2. Aims and structure of the study The current study aims to examine how controllers adapt their performance to complex trafc scenarios. A critical review of the literature was undertaken to develop a classication of strategies that controllers use to mitigate complexity such as, adapting cycles of recognition and anticipation (Histon and Hansman, 2002), replanning (Brooker, 2003; Amaldi and Leroux, 1995), handling uncertainty, team coordination and restructuring (Stein et al., 2006; Koros et al., 2006). The classication of complexity mitigation strategies should be enriched with several behavioural mark-

ers that use domain specic language to exemplify performance concepts and facilitate the evaluation of strategies. Behavioural markers are observable, non-technical behaviours that contribute to superior or substantial performance within a work environment (Klampfer et al., 2001, p. 10). It is important that behavioural markers describe observable behaviours and have a causal relationship with the performance outcome. Markers should exemplify concepts in a clear manner and relate to each other in a meaningful way (e.g., they may relate to a theoretical model of performance). Although behavioural markers cannot capture every possible aspect of performance (Flin et al., 2008), an effort was made in this study to identify those that have a direct inuence on performance outcome. To test the reliability of the taxonomy, an observation study of experienced approach controllers used four simulated scenarios that were specically designed to create complex trafc situations. Two independent raters have been presented with a large number of video segments of the simulated scenarios and applied the taxonomy. The evaluation process was iterative so that improvements could be made until a satisfactory level of agreement could be reached between the raters. A behavioural marker system of complexity mitigation strategies can be used to assess new technologies and foresee weaknesses that may lead to delays and errors in conict resolution. It can also provide designers with useful knowledge to design exible tools and emerging technologies that match the strategies of controllers. The article presents several sources of complexity in air trafc control, models of the relationship between complexity and performance, and intervening factors (e.g., self-regulating strategies) in the introductory section. Section 2 presents the aim and structure of the study in order to develop an initial taxonomy of complexity mitigation strategies and make it more consistent and reliable in a following observational study. A literature review is undertaken in Section 3 to develop an initial taxonomy which is tested in an observational study of expert approach controllers (Section 4) in coping with abnormal situations and emergencies as part of an informal refresher training program. The results show how renements in the categories have improved inter-rater reliability in a set of four ATC simulated scenarios (Section 5). Finally, we discuss the results in the context of adaptation of controller strategies and conclude with implications for error management training and design of ATC systems (Sections 6 and 7). 3. Complexity mitigation strategies in the ATC literature The typical method for the initial development of behavioural marker systems is to carry out a literature review of domain specic research, followed by interviews and critical incident methods

Anticipating work to be done

Meta-cognitive strategies

Inner feed- forward loop

Inner feedback loop

Assessing work in progress

Input: Work assigned Task demands

Choice of work method Mediating factors

Output: Work done Performance

Outer feedback
Fig. 1. A cognitive model of controller activities (adapted from Loft et al., 2007).

T. Kontogiannis, S. Malakis / Safety Science 57 (2013) 2734

29

in order to extract strategies used by controllers in coping with complex scenarios (e.g., OConnor and Long, 2008, study of navy deck ofcers). In the current study, the behavioural markers were extracted from an extensive literature review of the ATC domain and a previous study of enroute controllers (both experts and novices) in two training programmes (i.e., Malakis et al., 2010a, 2010b). An evaluation of the reliability of the initial taxonomy is made in the next section, using data from observations of expert controllers in simulated scenarios. The current section reviews and develops a classication of strategies specically related to how controllers adapt their performance to handle complex trafc scenarios. The classication scheme (Table 1) is presented in terms of four broad classes of mitigation strategies. Adjusting monitoring involves making subtle adaptations to recognize and anticipate potential conicts as trafc patterns increase in intensity. Replanning involves making changes in goal priorities to sustain increasing levels of complexity without compromising safety, modifying plans on the y, and critiquing mental models to cope with information uncertainty. Restructuring refers to changes in managing workload and task allocation between sectors, especially at the highest levels of complexity. Finally, changes in communication and coordination are important teamwork strategies that support the previous adaptations at all levels of complexity. Some mitigation strategies may be difcult to record by external observers (e.g., identifying hot-points of converging trafc) because they refer to internal mental processes that require further introspection. Others may be easier to record as they involve external actions and communications (e.g., proactive coordination). An effort has been made to present several exemplars of mitigation strategies to illustrate how controllers may adapt their strategies to the escalation of complexity.

gards processing aircraft in groups or streams which allows controllers to work with more aircraft simultaneously and use fewer control actions (Amaldi and Leroux, 1995; Histon and Hansman, 2002; Redding et al., 1991). As complexity increases, controllers can become overwhelmed with trafc and their behaviour becomes reactive. Coping with complex situations requires being proactive and staying ahead of trafc. For instance, controllers not only have to plan when to give instructions to aircraft, but they also have to ensure that they are able to so without problems when the right time comes. Another strategy to avoid potential threats, such as perception errors, has to do with reducing complexity on the radar screen (Malakis et al., 2010a; Kallus et al., 1999) e.g., by fading out irrelevant maps, by ltering out lower levels of trafc and by removing overlapping labels on the radar screen.

3.2. Re-planning Experts become increasingly adept at handling complex events by making logical leaps beyond the procedures to quickly reach a solution (Wickens et al., 1997). In making decisions, practitioners tend to run a mental test to determine the time available and their priorities (Cohen et al., 1996). The criteria for choosing a solution are rst of all safety, then trafc efciency and workload of sectors. Solutions have to meet these criteria, depending on the time that controllers spend on thinking about them. Hence, by changing their priorities, controllers are able to control their workload and maintain a safe trafc environment (Brooker, 2003). Several studies have pointed out the uncertainty surrounding the planning of trafc (Amaldi and Leroux, 1995; Weitzman, 1993). At any time, controllers must make judgments whether to leave the situation and see how it develops, or else make an early action - which may turn out to be needless. In busy periods, controllers shift their criteria for classifying conicts, becoming more conservative so that they intervene to ensure separation if there is any uncertainty regarding future separation between aircraft (Kallus et al., 1999). Emergencies and abnormal situations generate uncertainty as ight crews may be reluctant to provide conclusive information. In an emergency descent scenario, Malakis et al. (2010a) reported that controllers noticed that an aircraft was initiating a descent without any prior information from the ight crew. In this case, controllers had to assemble a mental model of the situation (e.g., turbulence, cabin decompression, engine failure) without increas-

3.1. Adjusting monitoring patterns Trafc monitoring strategies enable controllers to detect signs of impending conicts, build a model of the situation and play out mentally the progression of trafc (Redding et al., 1991; Seamster et al., 1993). The mental model allows controllers to structure information in the airspace into categories (e.g., aircraft heading to same destination, converging points of trafc or hotspots, and nonstandard ows) and hence, reduce complexity associated with monitoring trafc (Reynolds et al., 2002; Histon and Hansman, 2002) Another strategy that reduces monitoring requirements reTable 1 Complexity mitigation strategies from the literature review. Classes Adjusting monitoring and anticipation Complexity coping strategies 1. 2. 3. 4. 5. 6. 7. 8.

Relies on past experience and abstract ows of standard practice to identify streams of aircraft and converging trafc Selects actions that require less monitoring (e.g., V-separation involves less monitoring) Looks ahead and anticipates times of heavy trafc Reduces complexity by simplifying picture on radar (e.g., identies aircraft that may be a factor later) Underplays criteria of efciency and adopts more conservative response criteria Decides how soon to resolve conicts (i.e., response criterion, uncertainty reduction) Uses and stretches procedures Critiques mental models to minimize interruptions to ight crews

Re-planning and managing uncertainty

Restructuring tasks over time and sector

9. Selects actions that result in lower workload (e.g., defensive vectoring) 10. Manages task interruptions (e.g., slows down operation to give priority to a higher one) 11. Adapts or expedites handoff procedures 12. Coordinates with adjacent sectors to resize area of regard temporally 13. Decides on efcient ways to communicate to minimize interruptions (e.g., maintain control of frequency, do not offer unnecessary information) 14. Resorts to proactive coordination (e.g., prepares handouts for the EC, lists down all units to be informed in a suspect emergency) 15. Enquiries all necessary information together rather than in a piece meal fashion 16. Selects actions that require less coordination and uses hand gestures or pointing out to potential conicts on radar screen

Communication and coordination

30

T. Kontogiannis, S. Malakis / Safety Science 57 (2013) 2734

ing the workload of the ight crew with extra queries. Expert controllers can build and critique their mental models without undue questions to ight crews. 3.3. Restructuring distributing tasks over time and sector As trafc complexity increases, so does the number of tasks competing for attention, the pace of work, and the number of interruptions resulting from team communication. A usual coping strategy regards changing the distribution of tasks over time and sector. For example, Kirwan and Flynn (2002) identied various heuristics that controllers use to resolve conicts, such as: (a) using as few control actions as possible, (b) giving aircraft initial level changes early and ne-tuning later, (c) using solutions that require less coordination, and (d) using vertical separation for complex conicts. These solutions have to do with how controllers distribute their tasks over time and space to regulate their pace of work and off-load tasks to adjacent sectors. Earlier studies have shown that controllers adapt their separation methods to control their workload and cope with complex situations (Amaldi and Leroux, 1995; Willems et al., 1999). Every new event has to be considered in the context of work and this may change the order of priorities. Workload is not a constant parameter but follows the changing pattern of the situation as it escalates; hence, task sequences and priorities can be altered as complexity increases. Controllers tend to handoff aircraft early to enable the adjacent colleagues to take control of aircraft according to the plans as soon as possible. In busy periods, expediting handoffs sometimes can be a good strategy for reducing workload. A more drastic strategy to mitigate complexity is to change the airspace boundary to accommodate more trafc in adjacent sectors (Stein et al., 2006). 3.4. Changes and adaptation in teamwork As the complexity of scenarios increases, so does the workload on team communication. Several studies have found that anticipating information for future activities and predicting the workload of others can make team communication more efcient, especially at high tempos of work (e.g., Entin and Serfaty, 1999). Expert controllers are able to communicate effectively without unnecessary elements that garble communications. They are able to appreciate major attributes of information (i.e., criticality and timelines) and judge the level of workload and interruptibility of other team members. Other patterns of adaptation to complexity may regard changes in gathering and communicating information e.g., enquiring all necessary information together rather than in a piece meal fashion (Koros et al., 2006). In an airport diversion scenario, Malakis et al. (2010b) reported that the controllers rst wrote down a list of all relevant issues related to candidate diversion airports before starting communications. They minimized communications by avoiding many calls for individual aspects of the problem; instead they made all enquiries to nd a diversion airport at the beginning.
Table 2 A description of the scenarios practiced in the informal refresher training program.

Emergency scenarios may increase team coordination by requiring more contacts and route changes with a larger number of aircraft. In coping with complexity, controllers seem to choose options that require less coordination (e.g., issue a drastic turn of a single aircraft instead of a smooth turn of a set of aircraft so that fewer future contacts are made to route the aircraft to destination). Emergencies can also increase complexity by restricting the available time window for response. Controllers may adapt their coordination patterns and reduce workload by warning colleagues of imminent problems. In the airport diversion scenario (Malakis et al., 2010b), the controllers would coordinate with the nearest airport and alert colleagues of a potential aircraft diversion without specifying the precise nature of the problem.

4. Method To apply the taxonomy of complexity mitigation strategies in practice, data for the analysis were generated with an accredited ATC simulator. Four scenarios were designed by two On-the-Job Training Instructors (OJTIs) that represented situations of high complexity in the Approach Control Center (see Table 2). The scenarios were practiced by four controller teams in the executive and planner positions. The practice sessions were video-taped by the simulator software so that they could be re-played by independent raters. The two OJTIs who designed the ATC scenarios were also used as independent raters to apply the taxonomy to the replay of the four scenarios. Prior to the validation session, the simulator replays were viewed by the authors who marked a large number of instances or video segments that could be brought to the attention of the raters later on. In each validation session, the raters viewed the marked segments in the simulator replays and attempted to assign controller behaviours to the categories of the taxonomy. The two OJTIs were responsible for refresher training and were highly experienced in their eld. However, they had no prior experience with this type of behaviour rating and received a 3 h training in the use of the taxonomy. The participating controllers were asked by the OJTIs to verbalize as much as possible their trafc plans to their colleagues so that their behaviour becomes more observable to the external raters. Participation was on a volunteer basis as part of a new policy to facilitate knowledge sharing inside the organization All participants had access to the recorded scenarios so that they could learn from the experience of their colleagues. This created an open atmosphere so that controllers could verbalize their thoughts without fear of criticism from their OJTIs and supervisors. A number of after-action-review sessions followed so that additional questions could be addressed to the participants in order to collect operational feedback and improve the specication of the items of the classication scheme in the next iteration cycle. Our research setting was a busy local Approach Control Centre where operating teams comprised two controllers who would resolve potential separation losses between aircraft to prevent

Gradual escalation of trafc. In this scenario, air trafc is gradually increased and reaches a maximum of 15 aircraft, all demanding attention at the same time. This scenario lasts for 40 min and the controllers have to handle 28 aircraft in total. There are no emergencies or adverse weather conditions Sudden escalation of trafc. In this scenario, trafc escalates suddenly to 8 aircraft with a complex conict geometry and an emergency aircraft demanding priority for landing Closing weather and strong winds. This is a high uncertainty scenario as clouds are moving closer to the airport restricting the trafc ow and requiring diversions to other airports. Complexity is increased by the need to respond to a ight emergency and give priority to an aircraft to land rst Emergency descent scenario. Due to a system failure, an aircraft is compelled to initiate an emergency descend. The controllers have to recognize the subtle signs of the emergency descent from its onset, provide trafc information to pilots and keep other affected aircraft clear. The escalation pattern for this emergency is very steep as the descending aircraft crosses the airspace in less than 3 min

T. Kontogiannis, S. Malakis / Safety Science 57 (2013) 2734

31

mid-air collisions. The Executive Controller (EC) was responsible for the direct control of aircraft in the sector and the implementation of the overall plan. The Coordinator Controller (CC) or planner was responsible for establishing the overall plan for the entry and exit of aircraft in the sector and for assisting EC in his/her tasks. Air trafc controllers maintain situation awareness by scanning the radar screen and the ight progress strips (FPSs) constantly (Kallus et al., 1999). In the EC position, the radar screen seems to be the most important source of information and receives almost constant attention. Requests from adjacent sectors and pilots are a valuable source of verbal information that updates the mental model of controllers. Finally, controllers attend to verbal loops in the control room by overhearing to conversations between pilots and controllers because there could be something happening that may affect them. The purpose of this research was to evaluate whether the taxonomy developed from the literature review can be used to reliably classify complexity mitigation strategies in ATC scenarios. It was anticipated that this process would be iterative, and the raters would need to make changes to the taxonomy in order to achieve acceptable reliability. 5. Results on the categorization of video enactments of simulated scenarios Iteration 1. A total of 36 video enactments of simulated scenarios were presented to the two raters who independently categorized them using the taxonomy developed from the literature review (see Table 1). Cohens kappa was used to calculate the inter-rater reliability between the raters. An inter-rater reliability of k = 0.56 was produced (moderate agreement; Landis and Koch, 1977). The two raters discussed the areas of agreement or disagreement, and examined overlapping areas between the categories. On the basis of the discussion, the following changes to the taxonomy were made in order to improve inter-rater reliability:  In the rst class, simplifying picture on radar was not frequently observed in this study and was considered to serve the needs of anticipating heavy trafc in future. As a result, this category was merged with the previous category of looking ahead and anticipating times of heavy trafc.  In the second class, the categories stretching procedures and critiquing mental models were removed as they were difcult to observe with adequate reliability. The expert raters advised the inclusion of two other categories that were easier to observe and provided additional markers. The new category 7 (Table 3) refers to the level of precision or detail in instructions given to ight crews so that more scope for changes is allowed later; for instance, route instructions may focus on aircraft direction rather than precise headings. The other category 8 refers to drastic decisions (i.e., sacricing decisions) that sacrice economy aspects of a specic ight in order to reduce complexity; for instance, an aircraft is instructed to change completely its ight plan in order to avoid entering a busy sector which reduces potential conicts.  Signicant changes were made to the class of restructuring tasks. In specic, selecting actions that result in lower workload and managing task interruptions were merged into a single category because observed actions could serve either categories (see item #9 in Table 3). In addition, adapting handoffs and resizing area of regard were assigned to the communication and coordination class. The expert raters advised the inclusion of two other categories related to creating groups of aircraft with similar action requirements and tightening up the sequences of tasks in order to manage heavy trafc (see

#10 and #11, Table 3). Finally, a new category 12 was inserted in Table 3 that considered the cost of change and its risk implications; for instance, a new plan may increase communications or may place aircraft in a sequence that is difcult to undo in future.  In the communication and coordination class, selecting actions that require less coordination was merged with resorting to proactive coordination as there was signicant overlapping. Furthermore, enquiring complete information and avoiding piece meal information was judged by the raters to be a behaviour that served the needs of monitoring although oral information was utilized. As a result this category was inserted in the monitoring and anticipation class (see item 4, Table 3). Iteration 2. Another set of 38 video enactments from the second simulated scenario were independently analyzed by the two raters using the revised taxonomy (see Table 3). An inter-rater reliability of k = 0.68 resulted (substantial agreement; Landis and Koch, 1977). In general, there was very good agreement for the elements of the monitoring and replanning classes between the two experts. However, the workload management class appeared to overlap with the former classes. In specic, selecting actions that result in lower workload (item #9, Table 3) seemed to overlap with selecting actions that require less monitoring(item #2, Table 3); in many cases, less monitoring implies lower workload too. As a result the two categories were merged into a single category and assigned to the workload management class (see item #8, Table 4). Another overlap appeared between tightening up the sequences of tasks and leverage actions and sacricing decisions (items #11 and #8 in Table 3) as both categories refer to aspect of control actions and re-planning that reduce workload and complexity. Therefore, the two categories were merged as a single category in the area of re-planning (see item #7, Table 4). Iteration 3. The raters were presented with another set of 32 video enactments from the third scenario and tested the reliability of the second revision of the taxonomy. Table 4 shows the revised taxonomy with individual behavioural markers for each category. An inter-rater reliability of k = 0.76 resulted. In general, there was substantial agreement for all classes and perfect agreement for the communication and coordination class. A discussion with the raters indentied two cases of confusion and overlapping. In the rst case, there was some degree of overlapping between categories 1 and 9 (Table 4) since grouping aircraft in similar patterns could serve both the needs of monitoring and reducing workload in future. New markers and instructions were written down to clarify that grouping aircraft could be either an element of perception or an element of action where aircraft are advised to change their ight pattern on purpose to form a grouping pattern. In the second case, leverage actions and sacricing decisions (category 7) was perceived to be a similar but more stringent requirement than underplaying criteria of efciency (category 4); as a result, a decision was taken to remove the element of sacricing decisions from category 7 and insert it to category 4 (Table 4) within the same class of managing uncertainty. Finally, the raters were presented with another set of 30 video enactments from the fourth scenario and tested the reliability of the taxonomy. Inter-rater reliability increased to k = 0.81 as a result of the proposed clarication in the markers and the previous category renement.

6. Discussion Efforts to introduce new systems of work (e.g., free ight, new airspace capacity) by aviation providers and regulatory bodies

32 Table 3 Revised taxonomy of complexity mitigation strategies.

T. Kontogiannis, S. Malakis / Safety Science 57 (2013) 2734

Adjusting monitoring and anticipation 1. Relies on past experience and abstract ows of standard practice to identify streams of aircraft and converging trafc 2. Selects actions that require less monitoring of trafc to identify conicts 3. Looks ahead to anticipate contingencies and times of heavy trafc; simplies picture on radar in anticipation of heavy trafc 4. Reduces uncertainty by collecting complete information prior to assessing the problem Re-planning and managing uncertainty 5. Underplays criteria of efciency and adopts a more conservative stance 6. Decides how soon to resolve conicts (i.e., response criterion, uncertainty reduction) 7. As uncertainty goes up, precision of instructions is reduced to allow more scope for changes later on (i.e., commitment and management of uncertainty) 8. Identies leverage actions that would reduce signicantly conicts and makes sacricing decisions Managing workload and changes (errors) 9. Selects actions that result in lower workload communications and manages interruptions (e.g., do not interrupt a priority task to provide a service) 10. Creates groups of a/c with similar patterns and action requirements 11. Tightens-up sequences of tasks and trajectories of aircraft 12. Weighs-up the cost of change (Undo effect) as efciency can be traded-off for higher workload and risk (e.g., Undo errors, forgotten a/c, interruptions) Communication and coordination 13. Decides on most efcient ways to communicate and minimizes information garbling 14. Resorts to proactive coordination to support colleague 15. Adapts or expedites handoff procedures with other trafc sectors 16. Asks support from supervisor and others in order to change allocation of team roles (in some cases, resorts to re-sectorization)

Table 4 Prototype taxonomy of complexity mitigation strategies. Examples Adjusting monitoring and anticipation 1. Relies on past experience and abstract ows of standard practice to reduce categories and options 2. Looks ahead to anticipate contingencies and times of heavy trafc; simplies picture on radar in anticipation of heavy trafc 3. Reduces complexity by collecting complete information prior to assessing the problem Re-planning and managing uncertainty 4. Underplays criteria of efciency, adopts a more conservative stance Groups a/c as cleared for ILS landing vs left on holding pattern; notices some a/c converging in small space (e.g., between clouds); classies a/c in a trail Visualizes or writes-down possible conicts from the start before setting a plan; does not leave tasks for long periods since trafc may soon get heavy; Gathers complete information from the crews; avoids piece meal information as it requires more route changes; Puts some a/c on holding pattern; gives priority to a/c diversion rather than economy of routes; issues large diversions to one or two a/c to reduce conicts rather than issuing small changes to many a/c; Resolves potential conicts as soon as possible to turn attention to other things; issues instructions earlier to put ights on preferred routes and reduce uncertainty Route instructions focus on direction rather than precise headings; remains careful with commitment to initial instructions as they may have to be changed later on; A number of a/c are urged for landing before handling the emergency; reduces space between a/c in same trail or group; decides to by-pass a/c to nal route to avoid entering a busy sector VOR procedure for landing is preferred to ILS landing as crews can rely on manuals to execute procedure which relieves controllers; read-back may not be seen as necessary A number of a/c are placed in trail or put on holding or placed close together but separated by altitude Does not agree to last minute suggestions for changes; new plan may increase workload of communications; new plan may place a/c in a sequence that is difcult to undo; new plan may result in errors difcult to recover Inicts with voice to indicate busy; issues instructions to multiple aircraft; maintains control of frequency CC prepares handouts for EC and writes a list with all units to be notied in a suspected emergency An extra controller takes over all a/c heading for landing (e.g., ILS landing); a sector is divided in two parts Expedites handoffs in order to place a/c in preferred altitude or heading; requires adjoining sector to handoff a/c in trail

5. Decides how soon to resolve conicts (i.e., response criterion, uncertainty reduction) 6. As uncertainty goes up, precision of instructions is reduced to allow more scope for changes later on (i.e., commitment and management of uncertainty) 7. Tightens-up task sequences (or a/c trajectories), takes shortcuts, and identies leverage actions that would reduce signicantly conicts and complexity (i.e., sacricing decisions) Managing workload and changes (errors) 8. Selects actions that result in lower workload communications and manages interruptions (e.g., do not interrupt a priority task to provide a service) 9. Creates groups of a/c with similar patterns and action requirements to reduce workload and need for continuous changes 10. Weighs-up the cost of change (Undo effect) as efciency can be traded-off for higher workload and risk (e.g., Undo errors, forgotten a/c, interruptions) Communication and coordination 11. Decides on most efcient ways to communicate and minimizes information garbling in team 12. Resorts to proactive coordination to support colleague 13. Asks support from supervisor and others in order to change allocation of team roles (in some cases, resorts to re-sectorization) 14. Adapts or expedites handoff procedures with other trafc sectors

should be assessed in terms of the new workload demands imposed upon the controllers and ight crews (Hollnagel, 2007). A review of literature (Loft et al., 2007) showed that existing models of task demands and workload have not sufciently addressed how controllers regulate their performance to meet increases in trafc complexity. It is important that more elaborate models and classications of complexity mitigation strategies are developed so that

researchers can explore better the effects of new systems of work on controller performance. In this sense, our study contributes to our knowledge of complexity mitigation strategies and the relationship between complexity and performance. The ndings of the present study point out that controllers not only react to increasing task demands but also regulate their strategies proactively so that their performance degrades gracefully

T. Kontogiannis, S. Malakis / Safety Science 57 (2013) 2734

33

with the escalation of complexity. Even at low complexity levels, controllers seem to simplify problems that may be encountered in future (e.g., perceive aircraft as groups, reduce clutter on the radar screen) and anticipate threats (e.g., weather, frequency congestion problems). As the complexity increases, controllers may tend to reduce the quality of service offered to ight crews by adapting planning and performance criteria to the situation. Experienced controllers adopt a critical stance and critique their understanding on a regular basis in order to adapt their plans to the changing requirements of the situation. In order to regulate performance and pace of work, controllers may select options that result in lower workload (e.g., choose vertical over horizontal separation, when both options were valid) and manage interruptions according to task priorities. Finally, controllers tend to expedite communications by resorting to proactive strategies in communication (e.g., handle notes to colleagues for essential information) and coordination (e.g., liaise with nearest airport for a potential diversion problem without knowing the precise nature of problem). As complexity increases, controllers would choose to handoff aircraft to adjacent sectors early and turn down requests for early transfer from other sectors. The reliability analysis showed that the nal taxonomy seemed to have a consistent internal structure, although some difculties in classifying its content were experienced. With regard to between-class distinctions, the workload management class appeared to have some overlap with the other classes (see results of iteration 2). This may be because keeping workload under control is also an important aspect of other coping strategies such as, monitoring, planning and coordination. Difculties in making within-class distinctions had to do with the granularity of descriptions. For instance, a coping response can be seen both at a high level of planning (e.g., adapting priorities) and at a low level of planning (e.g., making task sequences more tight). It is anticipated that these sources of difculty can be tackled with additional rater training and elaboration of behavioural markers. In the present study, the two raters had only 3 h of training in the use of the taxonomy. The intention of this training was not to improve the reliability of raters per se, but to give them basic training in observing and rating behaviours in the controller work positions. With the progression of practice and iteration cycles, higher levels of reliability were achieved which reects higher standards of consistency of the proposed taxonomy. It should be noted that the raters were allowed to recommend changes in the taxonomy during the after-action-review sessions which resulted in some degree of calibration of the raters to establish common standards. However, this may have reduced the degree of independence of the phases of tool development and evaluation which requires further tests by additional practitioners and raters, not involved in the development process. The prototype taxonomy of complexity mitigation strategies (see Table 4) provides a structure for evaluating the performance of controllers in complex trafc scenarios. This is the rst stage of development and further work is required to ensure that the system can be reliably used by evaluators. Yule et al. (2008) for instance, proposed that behavioural markers should also be tested for sensitivity (i.e., the level of accuracy between raters and reference experts), inter-rater reliability using Likert type scales, and usability in the actual work environment. (e.g., can others use the taxonomy? can it be applied on line in the controller work positions?) Further development is required so that the prototype behavioural marker system has clear implications for improving safety and performance on ATC services. Specically, further work is needed to examine whether the taxonomy can be effectively used by other practitioners who have not been involved in the development phase and whether the taxonomy can be applied when observing controllers live on their job rather than from vi-

deo clips. Finally, an empirical validation is needed to examine whether the taxonomy can distinguish between the performance of experts and novices and whether the recorded behavioural markers are closely related to external measures of performance (e.g., task duration, delays, and inappropriate plans). It is hoped that a more complete study of complexity mitigation strategies will provide a basis for researchers to develop better models for assessing complexity. For instance, trafc factors in complexity should be extended beyond the number of aircraft and potential conicts to include possibilities for grouping aircraft into chunks and predictability of hot-spots. Operational constraints should also consider possibilities for alternative options, new priorities and proactive coordination that result in lower workload. Finally, airspace factors should consider the effects of early sector handoffs and dynamic re-sectorization. These adaptations on trafc factors, operational constraints and airspace factors could take into account the exibility of controller strategies and improve the modelling of complexity.

7. Practical implications and concluding remarks The benets of the present study should be sought in developing appropriate forms of operational aids, training programs and system design that would support controllers in coping with the complexity of future trafc environments. Controller activities are mainly driven by meta-cognitive or self-regulation strategies that adapt their recognition, planning and coordination activities to match the tempo and intensity of work. Meta-cognitive or self-regulation strategies can be viewed as control mechanisms that operate in parallel to the execution of ordinary activities (e.g., monitoring the screen, communicating with pilots, and updating ight strips). Building and maintaining meta-cognitive activities requires additional time and cognitive resources which further increases the actual workload from the execution of ordinary activities. A case in point for reducing cognitive burden is error management training where meta-cognitive activities are mastered in the context of technical skills that are practiced in simulator training. Recent studies in the aviation domain (Naikar and Sauders, 2003; Thomas and Petrilli, 2004) have taken a similar approach where trainees are required to master technical skills in a more varied context of practice that allows errors to occur but provides opportunities for error detection. The traditional training of controllers to become awless in technical skills can increase pre-occupation with success and may deprive them from opportunities to practice meta-cognitive skills for regulating workload and hence, reduce the likelihood of errors. Error management approaches shift the emphasis from mastering technical skills to building up a repertoire of self-regulation strategies. Error management training should also address several teamwork functions related to proactive coordination, cross-checking of team members and backing them up in periods of high workload. Finally, training should address issues related to changes in task distribution, such as resizing the area of regard and adapting handoff procedures. Problems related to controller familiarity with the obstacles and constraints of new sectors should be identied in advance so that re-sectorization policies are pre-specied and adequate training is provided to avert the risks of temporal workload increases during the transition period of resizing sectors. Previous research has suggested that situational awareness may be better preserved by displays that are compatible with controllers mental model of the airspace and by the integration of all necessary information (Wickens et al., 1997; Weitzman, 1993; Koros et al., 2006). Results from the current study may hold implications for system design and automated presentation tools. In the same sense, an understanding of how controllers mentally structure

34

T. Kontogiannis, S. Malakis / Safety Science 57 (2013) 2734 Klampfer, B., Flin, R., Helmreich, R., Hausler, R., Sexton, B., Fletcher, G., Field, P., Staender, S., Lauche, K., Dieckmann, P., Amacher, A., 2001. Enhancing Performance in High Risk Environments: Recommendations for the use of 9 rich, Behavioural Markers. Report from the Behavioural Markers Workshop, Zo June. Berlin: Damler Benz Foundation. Kopardekar, P., Magyarits, S., 2003. Measurement and prediction of dynamic density. In: 5th USA/Europe ATM R & D Seminar, Budapest, Hungary. Koros, A., Della Roco, P.S., Panjwani, G., Ingurgio, V., DArcy, J.F., 2006. Complexity in Airport Trafc Control Towers: A Field Study. Part 2. Controller Strategies and Information Requirements. Report No. DOT/FAA/TC-06/22 National Technical Information Service (NTIS), Springeld, Virginia. Landis, J.R., Koch, G., 1977. The measurement of observer agreement for categorical data. Biometrics 33 (1), 159174. Laudeman, I., Shelden, S., Branstrom, R., Brasil, C., 1998. Dynamic Density: An Air Trafc Management Metric (NASA-TM-1988-11226). NASA Ames Research Center, Moffet Field, CA. Loft, S., Sanderson, P., Neal, A., Mooij, M., 2007. Modeling and predicting mental workload in en-route air trafc control: critical review and broader implications. Human Factors 49, 376399. Malakis, S., Kontogiannis, T., Kirwan, B., 2010a. Managing emergencies and abnormal situations in air trafc control (part I): taskwork strategies. Applied Ergonomics 41, 620627. Malakis, S., Kontogiannis, T., Kirwan, B., 2010b. Managing emergencies and abnormal situations in air trafc control (part II): teamwork strategies. Applied Ergonomics 41, 628635. Masalonis, A.J., Callaham, M.B., Wanke, C.R., 2003. Dynamic Density and Complexity Metrics for Real Time Trafc Flow Management. MITRE, McLean. Mogford, R.H., Guttman, J., Morrow, S.L., Kopardekar, P., 1995. The Complexity Construct in Air Trafc Control: A Review and Synthesis of the Literature (No. DOT/FAA/CT-TN95/22). Federal Aviation Administration, William Huhes Technical Center, Atlantic City, NJ. Naikar, N., Sauders, A., 2003. Crossing the boundaries of safe operation: an approach for training technical skills in error management. Cognition, Technology and Work 5, 171180. OConnor, P., Long, W.M., 2008. The development of a prototype behavioural marker system for US Navy ofcers of the deck. Safety Science 49, 13811387. Pawlak, W.S., Brinton, C.R., Crouch, K., Lancaster, K.M., 1996. A framework for the evaluation of air trafc control complexity. Presentation at the AIAA Guidance, Navigation and Control Conference, San Diego, CA. Redding, R.E., Ryder, J.M., Seamster, T.L., Purcell, J.A., Cannon, J.R., 1991. Cognitive Task Analysis of en Route Air Trafc Control: Model Extension and Validation. FAA Report, (ERIC Document Reproduction Service No. ED 340 848) McLean, VA. Reynolds, T.G., Histon, J.M., Davison, H.J., Hansman, R.J., 2002. Structure, intent and conformance monitoring in ATC. In: Proceedings of the Air Trafc Management (ATM) Workshop on ATM System Architectures and CNS Technologies, Capri Italy, 2226 September 2002. Seamster, T.L., Redding, R.E., Cannon, J.R., Ryder, J.M., Purcell, J.A., 1993. Cognitive task analysis of expertise in air trafc control. International Journal of Aviation Psychology 3, 257283. Sperandio, J.C., 1978. The regulation of working methods as a function of workload among air trafc controllers. Ergonomics 21, 195202. Stein, E.S., Della Rocco, P.S., Sollenberger, R.L., 2006. Dynamic Re-sectorization in Air Trafc Control: A Human Factors Perspective. Technical Report DOT/FAA/TC TN06/19, Administration Technical Center. Thomas, M.J.W., Petrilli, R., 2004. Error management training: An investigation of expert pilots error management strategies during normal operations and ight crew training. ATSB Aviation Safety Project 2004/0050, Centre for Applied Behavioural Science, University of South Australia. Weitzman, D.O., 1993. Identifying information processing requirements for air trafc control problem solving: Designing for diversity. In: Proceedings of the Human Factors and Ergonomics Society 37th Annual Meeting Human Factors and Ergonomics Society, Santa Monica CA, pp. 103107. Wickens, C.D., Mavor, A.S., McGee, J.P., 1997. Flight to the Future: Human Factors in Air Trafc Control. National Academy Press, Washington, DC. Willems, B., Allen, R.C., Stein, E.S., 1999. Air Trafc Control Specialist Visual Scanning: Task Load, Visual Noise, and Intrusions into Controller Airspace (DOT/ FAA/CT-TN99/23). Federal Aviation Administration, Atlantic City, NJ. Xing, J.M., Manning, C., 2005. Complexity and Automation Displays of Air Trafc Control: Literature Review and Analysis. FAA Civil Aeromedical Institute, Report, DOT/FAA/AM-05/4. Yule, S., Flin, R., Maran, N., Rowley, D., Youngson, G., Patterson-Brown, S., 2008. Surgeons nontechncial skills in the operating room: reliability testing of the NOTSS behavior rating system. World Journal of Surgery 32, 548556.

their airspace, how they simplify clutter on their radar screens to mitigate complexity and how they manage pending crew requests would enable designers to produce decision support systems that closely match the information and strategic requirements of controllers. The complexity mitigation strategies can provide useful input in the design of free ight environments where the separation assurance function is shared between controllers and pilots. Any changes in the planning priorities and response criteria of controllers introduced by new designs should be carefully studied in advance to avoid contradictions with current controller practices. In the free-ight environment, for instance, controllers may monitor passively the ight crews who would have the primary responsibility for conict resolution. This change in task allocation can cause many difculties to controllers who would prefer to resolve conicts early and move on rather than wait for ight crews to resolve conicts or wait to be called upon for assistance unexpectedly. The new task allocation between ight crews - controllers and the conditions for task switchover should be considered within the context of controller strategies in coping with complexity. Finally, the present study opens new research agendas in exploring the way that practitioners adapt their strategies to complexity. The adaptive function of performance with complexity invites further research into how practitioners switch their strategies with increasing grades of complexity and how changes in communication and coordination can interact with re-planning and restructuring of controller activities. Futures studies are also needed to develop comprehensive models that show how control strategies regulate workload and change task demands, especially in future work environments. References
Amaldi, P., Leroux, M., 1995. Selecting relevant information in a complex environment: The case of air trafc control. In: Norros, L. (Ed.), 5th European Conference on Cognitive Science Approaches in Process Control. VTT Automation, Espoo, Finland, pp. 8998. Athenes, S., Averty, P., Puechmorel, S., Delahaye, D., Collet, C., 2002. Complexity and controller workload: Trying to bridge the gap. In: Proceedings of the 2002 International Conference on HumanComputer Interaction in Aeronautics. Massachusetts Institute of Technology, Cambridge MA, pp. 5660. Brooker, P., 2003. Control workload, airspace and future systems. Human Factors and Aerospace Safety 3 (1), 123. Cohen, M.S., Freeman, J.T., Wolf, S.P., 1996. Meta-recognition in time stressed decision making: recognizing critiquing and correcting. Human Factors 38, 206219. Edmonds, B., 1999. Syntactic Measures of Complexity. PhD Thesis. University of Manchester, Manchester UK. Entin, E.E., Serfaty, D., 1999. Adaptive team coordination. Human Factors 41, 321 325. Flin, R., OConnor, P., Crichton, M., 2008. Safety at the Sharp End: Training Nonspecic Skills. Ashgate Publishing, Aldershot. Hilburn, B., 2004. Cognitive Complexity in Air Trafc Control: A Literature Review. EUROCONTROL Report. Brtigny-sur-Orge CEDEX, France. Histon, J.M., Hansman, R.J., 2002. The Impact of Structure on Cognitive Complexity in Air Trafc Control (Rep. No. ICAT-2002-4). MIT International Center for Air Transportation, Cambridge, MA. Hollnagel, E., 2007. Flight decks and free ight: where are the system boundaries? Applied Ergonomics 38, 409416. Kallus, K.W., Van Damme, D., Dittman, A., 1999. Integrated job and task analysis of air trafc controllers: Phase 2. Task analysis of en-route controllers (European Air Trafc Management Programme Rep. No. HUM.ET1.ST01.1000-REP-04). EUROCONTROL, Brussels, Belgium. Kirwan, B., Flynn, M., 2002. Towards a Controller-based Conict Resolution Tool: A Literature Review (European Air Trafc Management Programme Rep. No. ASA.01.CORA.2.DEL04-A.LIT). EUROCONTROL, Brussels, Belgium.