Proceedings of the Human Factors and Ergonomics Society Annual Meeting http://pro.sagepub.

com/

Cracking the Bullwhip: Team Collaboration and Performance within a Simulated Supply Chain
Simon Banbury, Shaun Helman, James Spearpoint and Sbastien Tremblay Proceedings of the Human Factors and Ergonomics Society Annual Meeting 2010 54: 1620 DOI: 10.1177/154193121005401955 The online version of this article can be found at: http://pro.sagepub.com/content/54/19/1620

Published by:
http://www.sagepublications.com

On behalf of:

Human Factors and Ergonomics Society

Additional services and information for Proceedings of the Human Factors and Ergonomics Society Annual Meeting can be found at: Email Alerts: http://pro.sagepub.com/cgi/alerts Subscriptions: http://pro.sagepub.com/subscriptions Reprints: http://www.sagepub.com/journalsReprints.nav Permissions: http://www.sagepub.com/journalsPermissions.nav Citations: http://pro.sagepub.com/content/54/19/1620.refs.html

>> Version of Record - Sep 1, 2010 What is This?

Downloaded from pro.sagepub.com at Alexandru Ioan Cuza on December 10, 2013

PROCEEDINGS of the HUMAN FACTORS and ERGONOMICS SOCIETY 54th ANNUAL MEETING - 2010

1620

Cracking the Bullwhip: Team Collaboration and Performance within a Simulated Supply Chain
Simon Banbury1
1

Shaun Helman2

James Spearpoint3
2

Sbastien Tremblay4

CAE Professional Services, Kanata, Canada. 3 QinetiQ, United Kingdom.

Transport Research Laboratory, United Kingdom. 4 Universit Laval, Qubec, Canada.

The current study explored the role of collaboration in team performance using a computer-based simulation of a supply chain called the Beer Game developed by the Massachusetts Institute of Technology (MIT). In SCM simulations, as in real life, a bullwhip effect leads to a drop in profitability of the supply chain. The inclusion of Human Factors knowledge within the domain of SCM provides a rich source of understanding of bullwhip-related phenomena experienced by managers. In this paper we describe a technique called Cognitive Network Tracing which is used to examine the processes by which supply chain members make decisions and engage in communication in such scenarios. We examined the influence of different levels of Situation Awareness (SA) information given to supply chain members, and the influence of individual- or team-focused instructions, on a variety of measures of performance, communication, and SA. Results showed that team-focused groups of participants achieved better supply chain management performance than individual-focused groups of participants, but only when they were given information about current demand level in the supply chain. It is concluded that Management Flight Simulators, such as the Beer Game, have validity as tools to examine team collaboration and performance in management scenarios. INTRODUCTION As a result of globalization and the proliferation of multinational companies, changes to the business environment have contributed to the widespread development of supply chain networks. In tandem to this trend, there has also been interest in the evaluation of supply chains as an avenue for cost-cutting and increasing competitiveness, as well as complementing Just-In-Time, Lean Manufacturing and Agile Manufacturing practices. A typical supply chain consists of retailers, wholesalers, distributors, and factories/manufacturers, connected by transportation, information and financial infrastructure. In this environment the retailer faces customer demand, and places an order with the wholesaler to replenish stock sold. The wholesaler then makes a replenishment decision as to what to order from the distributor, and so on. Many difficulties arise in the management of supply chains due to the presence of multiple decision makers and the dynamic complexity of the domain (Sterman, 1989). Crucially, supply chain management is rendered complex because of: (i) ordering and shipping delays between levels; (ii) the decision making of each member within the supply chain has an impact on the decision making of others; and, (iii) the potential for divergent or conflicting agendas between members of different organizations within the supply chain (Steckel, Gupta and Banerji, 2004). Analysis of supply chains has found that the bullwhip effect is a common symptom of a poorly performing supply chain (Sahin and Robinson, 2002). In these cases, orders to the suppliers in the chain tend to have a larger variance than sales to the customers; an effect that is transmitted along the supply chain in an amplified form (Steckel et al, 2004). Once the bullwhip effect has taken hold of a supply chain it is not uncommon for the components of the supply chain to be placing orders that are 20 to 50 times what is actually needed to correct shortfalls of inventory (Sterman, 1989). Long periods of backlogs are typically followed by managers being swamped with excess inventory. The Role of Collaboration in Supply Chain Management There are many contributory factors to the bullwhip effect, both structural and human-behavioral (Disney, Naim and Potter, 2002), and opinions differ concerning which of these should be the focus of improvement interventions. The underlying issue is that all levels of the supply chain act locally whilst being part of a larger, only dimly-perceived system (Senge, 1990). In other words, supply chains that are able to collaborate effectively so that all members are working towards common goals and a shared awareness of the dynamics of the system, should be less susceptible to the bullwhip effect. Although the concept of Situation Awareness (SA; Endsley, 1995) has not been identified explicitly within the domain of management, there is a fair deal of evidence to implicate its importance. Similar to pilots sampling information from many sources to gain an understanding of the situation, a supply chain must acquire and share an awareness of materials, money and manpower in order to anticipate the consequences of their replenishment decisions. In addition, when each level of the supply chain seeks to maintain stock at a target level, it is necessary to account for time lags between making a decision and experiencing its effects (Sterman, 1989). Wickens (2002) describes higher levels of SA as the ability to deal with time lagged systems; indeed, supply chains must also be able to account for possible time lags between making a replenishment decision and observing its effects. Clearly, in terms of SA-related

Copyright 2010 by Human Factors and Ergonomics Society, Inc. All rights reserved. 10.1518/107118110X12829370089524

Downloaded from pro.sagepub.com at Alexandru Ioan Cuza on December 10, 2013

PROCEEDINGS of the HUMAN FACTORS and ERGONOMICS SOCIETY 54th ANNUAL MEETING - 2010

1621

issues, there is a great deal of commonality between the management domain, and domains more commonly associated with SA research (e.g. aviation). The distortion of demand caused by the bullwhip effect implies that unless a decision maker has information regarding the true demand pattern at the retail end of the supply chain, they are likely to be misled by an amplified pattern (Lee, Padmanabhan and Whang, 1997). Lee et al (1997) argue that allowing the sharing of inventory status data between members of the supply chain (i.e. collaboration processes to share SA) should alleviate problems relating to distortions of perceived demand. The present study uses collaboration technology (MSN Messenger) to allow supply chain members to share information relating to their current inventories and their anticipated future demand. Such collaboration should create a less distorted perception of the actual demand pattern, leading to better overall supply chain performance. The Beer Game: Using a Supply Chain Simulation to Study Team Performance and Collaboration The present study used the Cognitive Network Tracing (Banbury and Howes, 2001) approach to assess the processes of communication, leadership, and planning employed by teams as they were trying to achieve goals within the simulation. The approach describes the deliberate propagation of information tokens, or seeds, in the collaborative working environment and the subsequent observation of their trajectory within that environment. Clearly, the information seeds must be both critical enough to demand action by team members, and salient enough for the experimenter to observe their subsequent affect on team members behavior. The efficiency of how this information seed is propagated through the team can be examined in a number of ways: for example, by measuring the time taken for an alternative course of action to be taken, or examining which members of the team received the information seed (i.e. trajectory). The current study explored the role of collaboration using a computer-based simulation of a supply chain called the Beer Game developed by the Massachusetts Institute of Technology (MIT). Supply Chain Management (SCM) game scenarios are complex because of the almost universal occurrence of the bullwhip effect. In SCM simulations, as in real life, the bullwhip effect leads to a drop in profitability of the supply chain as each player is penalized for every unit held in their inventory (i.e. warehousing costs) and for every order unfulfilled (i.e. backorder costs). The information seeds in the current study were levels of demand, and changes in demand given to the first stage in the supply chain. Using a number of quantitative and qualitative measures, we attempted to observe how these

information seeds propagated through the communications between team members in the supply chain, and how they influenced team collaboration and performance on the SCM task. The present study also examined the impact of different levels of SA provided in the information seed given to the first stage in the supply chain. Current demand information was contrasted with Future demand information. Furthermore, the impact of different player goals determined by the instructions given to participants (team versus individual goals) was examined. Ideally, to maximize performance the supply chain should be seamless; that is all levels of the supply chain should think and act as one entity (Disney et al., 2002). The intention of this manipulation was to examine the extent to which explicitly promoting team collaboration improves the overall performance of the supply chain. METHOD Materials The Massachusetts Institute of Technology (MIT) has developed an online test-bed for SCM training and evaluation known as The Beer Game (see Figure 1).

Figure 1: A representation of the Beer Game simulation. Participants take a role in a supply chain; either as retailer, wholesaler, distributor, or factory. When a game is in progress, the controller of each facility receives orders from the facility immediately up the chain, and attempts to fill them with stock from its inventory. Each facility replenishes its stock by placing orders with the facility immediately down the chain. This software is ideally suited to the present study, since players can interact with one another through chat software (in this case MSN Messenger) while playing, thus enabling examination of real-time communication and its influence on performance (Figure 2).

Downloaded from pro.sagepub.com at Alexandru Ioan Cuza on December 10, 2013

PROCEEDINGS of the HUMAN FACTORS and ERGONOMICS SOCIETY 54th ANNUAL MEETING - 2010

1622

Several dependent variables were employed. Firstly, Performance, comprising objective measures of supply chain performance, calculated from the overall costs, orders and back-orders incurred during the game for the whole chain. In addition, Bullwhip size, which is an objective measure of the magnitude of the bullwhip, was calculated for each facility using an equation used commonly in the literature (Disney et al, 2002). Secondly, Deviation SA was calculated as the difference between participants estimate of the demand they would face the following week, in two weeks time and in three weeks time from the facility immediately up the chain, and the actual demand. The lower this deviation score, the better participants are at correctly anticipating future demand (i.e. possess high levels of SA for future demand). Thirdly, from the MSN Messenger logs of communication between participants, the total number of communications (of various types and between various facilities) was calculated. Figure 2: Actual screenshot of Beer Game simulation (as seen by participant playing the factory facility) with MSN Messenger windows for Retailer, Wholesaler and Distributor in the same relative spatial positions as the people playing those facilities in the laboratory. Experimental Design Sixty-four participants took part in the experiment and were tested in groups of four. A 2 X 2 betweenparticipants design was used. Groups were pre-assigned to one of the four possible conditions that resulted from the two twolevel independent variables. The first independent variable was Information Seed (two levels: Current and Future). The second independent variable was Player goal (two levels: Individual and Team). Information Seed was manipulated by varying the SA level of information about demand given to the retailer at the beginning of the game. In the Current condition, the retailer received the following instruction, which provides only information about how to use current demand information: Each week you will see the demand you are experiencing from customers this week. You should use this information as you see fit in order to achieve the game goals. In the Future condition, the instruction gave information on how to use current demand information (i.e., Current), and also information about an increase in demand that was due to happen in week 13 onwards (i.e., Future): Each week you will see the demand you are experiencing from customers this week. You should use this information as you see fit in order to achieve the game goals. Furthermore, your company is currently constructing an extension to your existing store. The new extension will be operational from week 13 onwards. Be prepared for a significant increase in customer demand following its opening. Player goal was manipulated by a simple variation in instructions. In the Individual condition, participants were instructed to play the game so as to minimize their own facilitys costs; whereas in the Team condition, participants were instructed to play the game so as to minimize costs for the supply chain as a whole. Procedure Following an extensive training sessions, participants played through a 23-week simulation of the Beer Game. Each week, the Retailer, then the Wholesaler, then the Distributor, and finally the Factory placed their order within a 45 second timeframe (timed by the experimenter with a stopwatch). Each turn, before the Factory player clicked the submit order button (but only after they had decided what to order by typing their order into the relevant field), all participants were asked to fill in their SA sheets, to predict the demand placed on them for the next week, 2 weeks hence, and 3 weeks hence. Then the Factory placed its order and the following turn started with the Retailer again. Participants were not allowed to talk to one another at all during the experiment, but were allowed to communicate freely via MSN Messenger, at any time. This software enables two-way instant communication between each pair of players via typing text in a window on the computer screen. Communication logs were saved for later analysis. Each facilitys screen had the Beer Game simulation open in one window, and three MSN Messenger windows also open for communication with the three other facilities. RESULTS Table 1 shows the means (and standard deviations) for the four key performance measures of Mean Total Costs, Mean Orders, Mean Back Orders, and Mean Bullwhip. Table 1: Performance Results
Mean Total Costs ($) Information Seed Current Future Indiv Instructions 1999.50 (251.4) 2206.80 (973.1) Team Instructions 1451.50 (393.8) 2198.00 (455.0) Mean Orders (Units) Information Seed Current Future Indiv Instructions 8.55 (1.3) 8.77 (2.1) Team Instructions 7.17 (1.7) 8.76 (1.7) Mean Number of Back Orders (Units)

Downloaded from pro.sagepub.com at Alexandru Ioan Cuza on December 10, 2013

PROCEEDINGS of the HUMAN FACTORS and ERGONOMICS SOCIETY 54th ANNUAL MEETING - 2010

1623

Indiv Instructions Team Instructions

Indiv Instructions Team Instructions

Information Seed Current Future 11.1 (514) 13.7 (7.4) 8.06 (5.5) 11.09 (3.5) Mean Bullwhip (Units) Information Seed Current Future 23.73 (9.4) 28.13 (45.5) 8.20 (5.1) 33.1 (29.1)

Although sample sizes in this project are too small to permit reliable inferential statistical analysis, there are clear trends in the data that allow preliminary and specific interpretation. The data indicate that performance was better when instructions stressed the importance of team performance over individual performance, but only when Current SA information was provided. The provision of Future SA information seemed to confuse team members, resulting in worse performance; especially in the Team condition. Table 2: Situation Awareness Results
Mean SA Deviation score Information Seed Current Future 4.15 (3.7) 4.38 (2.4) 3.40 (0.9) 4.82 (1.5)

Figures 3 and 4: Total number of communication events between facilities, with direction indicated. Numbers on arrows represent the number of discrete occurrences of communication between the facilities linked, in the direction indicated. Numbers in red indicate deviation SA, where zero is perfect, and higher scores indicate less accurate SA. DISCUSSION Testing only a relatively small number of participants (N=64 in 16 teams), the project demonstrated that a number of measures of system management performance, a measure of prediction SA, and measures of communication frequency and quality, can all be influenced by experimental manipulations that have relevance to SCM scenarios. By carrying out an objective analysis of the communication behavior between teams (quantitative and qualitative), in addition to measuring team performance, we are able to begin to unpack the processes by which these experimental manipulations influence supply chain performance. We propose the following tentative conclusions and directions for future work to take. Firstly, it was found that team-focused instructions appear to improve supply chain performance measures (when information seeds given to the team contained current demand information). At first sight this finding may seem obvious. However, this reinforces the need, especially when pulling together teams who are not in the same organization, to remind team members to work towards a common goal, rather than their own local measures of performance. Secondly, future demand seeds, although containing future information that should have helped team performance and collaboration actually had detrimental effects on both. We can only speculate as to why this was the case. One possibility is that the information about future rises in demand was non-specific. It merely suggested a rise in demand, without specifying a level of the increase. This may have resulted in team members paying attention to the future information, but being unable to use this information to actually inform replenishment decisions due to the confusion over what level of demand increase there would be. We are currently carrying out more analyses of the communication data to assess whether this may have been the case, and have also planned future work examining this issue directly through experimental manipulation of the level of detail in the projection SA information seed. For now it is worth noting that this is another example of a counter-intuitive

Indiv Instructions Team Instructions

Table 2 shows the level of SA achieved (averaged over 1 week hence, 2 weeks hence and 3 weeks hence deviation of predicted from actual demand where zero is perfect, and higher scores indicate less accurate SA). Again Current SA instructions coupled with a focus on team performance seemed to benefit performance. Table 3 shows the mean frequency of communication by SA Seed and Player Goal (Instructions). Table 3: Communication Results
Mean Frequency of Communication Information Seed Current Future Indiv Instructions 3.16 (1.1) 4.60 (1.6) Team Instructions 3.10 (1.2) 4.67 (1.9)

When viewed overall and within the context of the SA results shown in Table 2, the level of communication does not seem to be important in the maintenance of SA. However, when a more qualitative pass is made at the data, evidence can be found that suggests a clear link between certain levels and types of communication and SA (see Figures 3 and 4). In the worst performing team (Individual instructions / Future demand information seed), there is very little communication past the wholesaler, while in the best performing team (Team instructions / Current demand information seed) there are rich communication lines between all facilities. Crucially, there is something akin to Electronic Point of Sale (EPOS) in the best performing team, with the Retailer providing the Factory and Distributor with regular demand information.

Downloaded from pro.sagepub.com at Alexandru Ioan Cuza on December 10, 2013

PROCEEDINGS of the HUMAN FACTORS and ERGONOMICS SOCIETY 54th ANNUAL MEETING - 2010

1624

finding that merits further study. We anticipate that through applying the Cognitive Network Tracing technique to this issue, we should be able to deduce what levels of specificity are required for certain types of information to be useful in complex team collaborative tasks. Results also permit other qualitative conclusions to be drawn with regard to performance and communications. The beneficial effects of team-focused instructions and current demand seed information helped cost-based performance by making ordering more efficient. Rather than just by having people make simple strategy shifts that would also reduce costs (e.g. keeping lots of stock to avoid backorders, which are more expensive), participants were ordering less, but also managing to have fewer backorders. This increased hold on ordering strategy led to a smaller bullwhip effect, reducing that factor traditionally blamed for many of the inefficiencies in badly run supply chains. Thus we can speculate that team-focused instructions and good quality SA information seeds may have beneficial effects on team collaborative tasks in terms of allowing team members to adopt optimum strategies. The communications between team members also permit good qualitative analysis. By examining best- and worst-performing teams in detail, it was possible to identify key features of communication that led to better supply chain performance. Good performance in the Beer Game would appear to rely on the Retailer providing information the Distributor and Factory, rather than just to the Wholesaler, their immediate neighbor in the chain. Again, this kind of advice may have ramifications for many different tasks, and makes the Cognitive Network Tracing technique a useful tool in optimizing team communications and process. This spontaneous behavior adopted by the better-performing supply chains in this study, bears a striking resemblance to Electronic Point of Sale (EPOS) technologies that provide regular Retailer demand information to all levels of the supply chain. CONCLUSIONS It has been argued that the symptoms applicable to supply chain management can be seen in the real estate industry, which is also characterized by a Boom and Bust pattern. For example, developers capitalizing on low vacancy rates and high property values initiate many new projects without taking into account the projects that are already in progress. In addition, researchers in the field of economics have been concerned with the bullwhip observed in the general economy; whereby overexpansion in the economy leads to undesirable compensatory employment and production cuts to normalize the system (Sterman, 1989). The inclusion of Human Factors knowledge within the management domain provides a rich source of understanding of the bullwhip-related phenomena experienced by managers, and insights into how managers might be trained to be more effective. Management Flight Simulators such as the Beer Game, allow the consequences of management decisions to be observed more clearly than in real-world

settings. Identifying sources of poor performance in these settings can assist in the development of training to capitalize on these lessons learnt. This training can improve managers understanding of how their environment works; resulting in improvements in the quality of their decision making. In addition, specific deficiencies in a managers understanding of the supply chain can be identified and could be suggestive of particular design requirements for support tools. ACKNOWLEDGEMENTS The authors would like to acknowledge the invaluable contribution of Russell Bond and Stephanie Applegate to this manuscript. REFERENCES Banbury, S. and Howes, A. (2001). Development of generic methodologies for the evaluation of collaborative technologies. DERA Technical Report (Contract CU0052927). Disney, S.M., Naim, M.M. and Potter, A.T. (2002) The impact of e-business on supply chain dynamics. Proceedings of the 12th International Working Seminar on Production Economics, Igls, Austria, 18th-22nd February 2002. Endsley, M.R. (1995). Toward a theory of situation awareness in dynamic systems. Human Factors, 37, 3264. Lee, H. L., Padmanabhan, V., and Whang, S. (1997). Information distortion in a supply chain: The bullwhip effect. Management Science, 43(4), 546-558. Sahin, F. and Robinson, E.P. (2002). Flow Coordination and Information Sharing in Supply Chains: Review, Implications, and Directions for Future Research, Decision Sciences, 33, 505. Steckel, J.H., Gupta, S. and Banerji, A. (2004). Supply Chain Decision Making: Will Shorter Cycle Times and Shared Point of Sale Information Necessarily Help? Management Science, 50(4), 458-468. Sterman, J. (1989). Modelling managerial behavior: Misperceptions of feedback in a dynamic decision making experiment. Management Science, 35(3), 321339. Wickens, C.D. (2002). Multiple resources and performance prediction. Theoretical Issues in Ergonomics Science, 3(2), 159-177.

Downloaded from pro.sagepub.com at Alexandru Ioan Cuza on December 10, 2013