Measuring Business Excellence

Emerald Article: An experiment in the usefulness of a strategy map Nopadol Rompho

Article information:
To cite this document: Nopadol Rompho, (2012),"An experiment in the usefulness of a strategy map", Measuring Business Excellence, Vol. 16 Iss: 2 pp. 55 - 69 Permanent link to this document: http://dx.doi.org/10.1108/13683041211230320 Downloaded on: 15-10-2012 References: This document contains references to 100 other documents To copy this document: permissions@emeraldinsight.com

Access to this document was granted through an Emerald subscription provided by Saudi Digital Library For Authors: If you would like to write for this, or any other Emerald publication, then please use our Emerald for Authors service. Information about how to choose which publication to write for and submission guidelines are available for all. Please visit www.emeraldinsight.com/authors for more information. About Emerald www.emeraldinsight.com With over forty years' experience, Emerald Group Publishing is a leading independent publisher of global research with impact in business, society, public policy and education. In total, Emerald publishes over 275 journals and more than 130 book series, as well as an extensive range of online products and services. Emerald is both COUNTER 3 and TRANSFER compliant. The organization is a partner of the Committee on Publication Ethics (COPE) and also works with Portico and the LOCKSS initiative for digital archive preservation.
*Related content and download information correct at time of download.

An experiment in the usefulness of a strategy map

Nopadol Rompho

Nopadol Rompho is Assistant Professor in the Department of Operations Management, Faculty of Commerce and Accountancy, Thammasat University, Bangkok, Thailand.

Summary Purpose The objective of this study is to investigate the effect of a strategy map on the quality of decision making. Design/methodology/approach An experimental design is used in this study. A total of 24 participants were randomly assigned into two groups a control group and a treatment group. Each participant completed two rounds of the experiment covering 24 decisions using a simulation game. The multiple regression technique is used as the analysis tool in this study. Findings The results show that strategy maps do not affect the performance of participants decision making. Research limitations/implications This research employs the laboratory experimental design. Thus the external validity (the generalizability) is limited. Practical implications Results obtained from this research suggest that a strategy map alone is not an effective tool to help managers make better decisions. More details, such as a correlation between strategic measures in the strategy map or guidelines for the use of the map, could help make a strategy map more useful. Originality/value This study tests the cause and effect relationship between using a strategy map and the results of decisions made. Unlike many studies that employ the survey method in which confounding variables cannot be controlled, results from this experiment suggest that a simple strategy map alone does not improve the quality of decision making. This nding can thus be valuable to managers who are considering using a strategy map as a management tool. Keywords Strategy map, Balanced scorecard, Performance measurement systems, Experimentation, Decision making, Business performance Paper type Research paper

1. Introduction
It is widely accepted that a performance measurement and management framework plays a very important role in managing an organization and can improve business performance (Sharma et al., 2005). The interest in this topic has increased signicantly in the last 20 years (Taticchi, 2008) and it is attracting increasing attention in business and management literature (Neely, 1999; Avella et al., 2001; Unahabhokha et al., 2006). Both for-prot and nonprot organizations have invested substantially in the attempt to set up a framework that will over the course of time lead to successful management of the organization. Over the last two decades, substantial effort has been put into the design and implementation of performance measurement systems (Eccles and Pyburn, 1992; Kaplan and Norton, 1993, 1996, 2000; Neely et al., 1996, 2000; Olve et al., 1999) and the assessment of its uses (Dixon et al., 1990; Bititci et al., 1998). Despite the effort spent to set up these frameworks, many organizations fail to realize the benet gained by employing these tools (Meyer and Gupta,

The author received a research grant from the Faculty of Commerce and Accountancy, Thammasat University.

DOI 10.1108/13683041211230320

VOL. 16 NO. 2 2012, pp. 55-69, Q Emerald Group Publishing Limited, ISSN 1368-3047

MEASURING BUSINESS EXCELLENCE

PAGE 55

1994; Meekings, 1995; Bierbusse and Siesfeld, 1997; Lewy and Du Mee, 1998; McCunn, 1998; Schneiderman, 1999; Bourne et al., 1999, 2000; 2002, 2003a,b; Townley et al., 2003) The balanced scorecard has been found to be the most popular performance and management framework (Silk, 1998; Malmi, 2001). Gomes et al. (2004) also found that the balanced scorecard is most cited performance measurement system. A study found that 44 percent of organizations in North America (Rigby, 2001) and 35 percent of large US rms (Marr et al., 2004) use the Balanced Scorecard. It is also reported that over 40 percent of organizations worldwide (57 percent in the UK, 46 percent in the US, and 28 percent in Germany and Austria) have adopted this tool (Rigby, 2001; Speckbacher et al., 2003). In a more recent work, it has also been reported that 40 percent of Fortune magazines top 1,000 companies used the balanced scorecard in 2007 (Thompson and Mathys, 2008). This framework, originating in research conducted by Professor Dr Robert S. Kaplan and Dr David P. Norton in 1992, proposed that an organization should pay attention to both nancial and non-nancial performance from four perspectives: nancial, customer, internal processes, and learning and growth (Kaplan and Norton, 1992). Each perspective in the balanced scorecard has four main components that need to be claried. The rst component is strategic objective, which is the outcome the manager expects in that particular perspective if the strategy is implemented successfully. Once strategic objectives are identied, the measure of each objective (the second component) must then be proposed. The third component is a target value. This is the numerical value that is set up as a guideline to judge the level of success of each objective. Finally, strategic initiatives must be proposed (the fourth component), i.e. the project or activity that should be performed to help the organization achieve its objective. Thus it can be argued that the balanced scorecard can translate strategy (strategic objectives) into action (initiatives). The balanced scorecard is not a method of categorizing measures into four perspectives. Those measures must be related to organization strategy and related to each other (Hoque and James, 2000). The strategic objective for each perspective must show a cause and effect relationship. This cause and effect linkage is made explicit by the illustration of a strategy map, which is one of the strongest points of the balanced scorecard (Norreklit, 2003; Bourguignon et al., 2004).

2. Strategy maps
The strategy map is the tool Kaplan and Norton proposed to help make strategy more explicit than a vague presentation in some passage (Kaplan and Norton, 2000, 2001a, 2004a; Kaplan et al., 2010). It is suggested that the graphic presentation of the strategy map may contribute to rapid adoption and implementation of the balanced scorecard through its exibility, rationalizing management processes and empowering management action (Free and Qu, 2011). This concept is generally accepted to be useful for strategy implementation, regardless of whether or not the balanced scorecard is used (Eccles, 1991; Copeland et al., 1996; Young and OByrne, 2001). A strategy map is a logical and comprehensive tool for describing strategy by identifying the critical elements and their linkages within an organizations strategy (Kaplan and Norton, 2001b). For a for-prot organization, it starts at the learning and growth perspective, which will have a direct effect on the organizations internal processes and lead to customer satisfaction. Once customers are satised, a good nancial outcome will follow. According to Kaplan and Norton (2004b), a strategy map is based on ve principles: 1. strategy balances contradictory forces; 2. strategy is based on a differentiated customer value proposition; 3. value is created through internal business processes; 4. strategy consists of simultaneous, complementary themes; and 5. strategic alignment determines the value of intangible assets.

PAGE 56 MEASURING BUSINESS EXCELLENCE VOL. 16 NO. 2 2012

A strategy map not only makes strategy more explicit, it can also be used as a tool to help managers review their organizations performance. At the end of the year (or the period each organization uses as its review period), managers can review the organizations performance against the strategy map. The map will explicitly illustrate the cause and effect linkage. If one area is found to be underperforming, the cause can be easily identied. This is essentially an operational control loop that can assist managers to focus on the real issue and not blame everything on unrelated factors. Many times, even if a clear explanation is not demonstrated, a strategy map could stimulate managers to ascertain whether the current strategy is applicable to the current situation and eventually lead to a revision of current strategy. This use of strategy map is a strategic learning loop. Thus a strategy map is a double-loop learning tool (Kaplan and Norton, 2001a,b). Despite the use of strategy map as previously described and the belief that strategy maps can be use as a double-loop learning tool, it has been criticized as not discriminating among logical and causal linkages, not considering time lag effect among measures, too simple and difcult to validate, and lacking a feedback loop (Norreklit, 2000, 2003; Ahn, 2001; Malina et al., 2007; Kunc, 2008). It also does not provide a concrete technique to consolidate individual perspective, so every step has to be done subjectively (Abrain and Buglioni, 2003). Other scholars who support the resources-based view of a company (Grant, 1991; Petergraf, 1993; Barney et al., 2001; Barney, 2001) also note that the company consists of a bundle of resources in which the different assets depend on each other to create value and thus the valuation of intangible assets on a stand-alone basis is impossible (Lev, 2001). This means that the strategy map, designed to show only a unidirectional cause and effect linkage, is then not conceptually correct. These arguments have raised serious doubts as to the usefulness of a strategy map. There have been attempts to suggest a way to create a strategy map. For example, Quezada et al. (2009) proposed a simple tool that helps in the creation of the strategy map. However, the proposed methodology does not indicate whether the decision makers gain benet by using the strategy maps. Some studies addressed the effect of performance measures and strategy maps on decision making. For instance, Lipe and Salterio (2000) found that decision makers tend to compare the measures that are common across different units and ignore the measures that are unique to individual business units. These results were conrmed by the study of Banker et al. (2004) in which an experiment was conducted to test the judgmental effects of performance measures linked to strategy. It was found that the performance evaluators are more inuenced by strategically linked measures than non-linked measures only when details about strategy, including the strategy map, are given to evaluators. These ndings indicate a strategy map does have an effect on evaluator decision-making. Although a number of studies show a positive relationship between strategic performance measures and a rms nancial performance (Anderson et al., 1994; Anderson et al., 1997; Banker et al., 1993; Banker et al., 2000; Ittner and Larcker, 1996, 1998a; De Geuser et al., 2009), those studies do not directly address the issue of whether the use of strategy maps contribute to the success. Kaplan and Norton (2000) emphasized that the better employees understand strategy, the better they can implement strategies and they thus claim that strategy maps are a tool to help employees better understand strategy - but again, they do not provide concrete evidence to support that claim. In the work of Wilkes (2005), it was found that companies with scorecards and strategy maps outperformed other companies. This nding was supported by the work of Marr (2005), indicating that rms using a causal model had better performance than those that did not use it. In a more recent work, Lucianetti (2010) also found that employees in companies that use strategy maps perceive more benet from the balanced scorecard than those who do not have a strategy map. Still, none of these ndings prove a causal linkage between use of a strategy map and performance since there are number of uncontrollable factors; also, these ndings could be interpreted in another way as showing that successful companies have the luxury of time and thus tend to pay more attention to a strategy map than unsuccessful companies that are ghting for survival and might not have time for this tool.

VOL. 16 NO. 2 2012 MEASURING BUSINESS EXCELLENCE PAGE 57

There are very few studies, if any, that address a linkage between the use of strategy maps and decision-making leading to successful performance. This might be due to the fact that it is very difcult to test the cause and effect of each strategic objective in the map taking into account the time lag effect, i.e. no one knows exactly how long since the achievement of one objective will have an effect on the achievement of the other objective. It is even harder to test this cause and effect linkage if the situation is constantly changing and nothing can be controlled. Even if a strategy map is assumed to be valid, very few studies examine whether it has any effect on decisions made by managers and eventually leads to the correct decision. Thus this study tries to close the gap by testing the effect of strategy maps on the quality of decision-making. However this study does not address how to test the validity of a strategy map in an organization: it is aimed only at answering the question if the strategy map is valid, can it help managers make better decisions? The methodology of study is presented in the following section.

3. Research methodology
This study employs the experimental research design as it aims to prove the causal relationship between using a strategy map and the results of making decisions. To answer this research question, 24 MBA students were recruited to participate in the experiment. Of the participants, 12 were randomly assigned as the control group and the other 12 as the treatment group. Random assignment was employed to ensure that the two groups were similar in age, grade point average, and grade in BA 701 (the course that is related to performance measurement). These three factors were controlled in the belief they may have an effect on decision-making. Studies have shown that age was found to inuence decision-making (Taylor, 1975): older people used less information, spent more time viewing and reviewing less information than younger people (Johnson, 1990). Risk-taking has also been shown to diminish as age increases (Deakin et al., 2004). Grade point average was used to indicate the business knowledge that MBA students gained during the time of the experiment and the nal grade for BA701 (performance management and value creation) represents knowledge of performance measurement systems in general. Random assignment in this study was used to reduce selection bias. Participants in both the control and treatment groups attended a session at which the researcher explained the research methodology. In this study, researcher constructed a simulation game called Managing through measures (MTM) and used this as a tool for the experiment. This game is specically designed by the researcher for the purpose of this research. The reason to use the simulation game instead of a real-life scenario is that in a real-life setting, there can be a signicant amount of uncontrollable factors that can inuence the outcome. Thus, it becomes more difcult to test the cause and effect relationship between the usage of strategy map and the result of decision making in a real-life scenario compared to a simulated setting. The idea of using the simulation game, instead of a real-life scenario, to test the performance of decision making is not new. It has been found in literature for many years (see for example, Hogarth and Makridakis, 1981; Gladstein and Reilly, 1985; Anderson and Morrice, 2000; Anderson, 2005; Cantor and Macdonald, 2009; Capelo and Dias, 2009). This game ran for 12 quarters. In each quarter, each participant needed to decide in which strategic initiative(s) he/she wants to invest. In each quarter each participant has a specied bankroll for investment (US$10 million for quarters 1-4, US$30 million for quarters 5-8, and US$60 million for quarters 9-12). Each participant was told to use this bankroll to invest in any initiative in any amount or to not invest at all. Each quarter, any investment would immediately be deducted as expense in that quarter. The unused bankroll could not be carried over to another quarter. The effect of each investment would be for that quarter only. Each participant was able to see the performance result of their decisions at the end of each quarter. Table I shows the list of initiatives and performance measures. These measures were selected to include both leading and lagging indicators among perspectives in the balanced scorecard (see also in Figure 1). These measures are considered to be equally important in the experiment; thus, they are equally weighted. Note

PAGE 58 MEASURING BUSINESS EXCELLENCE VOL. 16 NO. 2 2012

Table I List of initiatives and performance measures

Initiative Customer relationship management project Performance measure Customer satisfaction index Type of measure Leading indicator for number of repeat customers/total number of customers Lagging indicator for percent of defects in production process and percent on-time delivery Leading indicator for total revenue Lagging indicator for number of new customers/total number of customers and number of repeat customers/total number of customers Leading indicator for percent market share Lagging indicator for number of new products developed per quarter Leading indicator for percent market share Lagging indicator for customer satisfaction index Leading indicator for customer satisfaction index and reputation score Lagging indicator for employee capability index Leading indicator for customer satisfaction index and reputation score Lagging indicator for employee capability index Leading indicator for number of new customers/total number of customers Lagging indicator for employee capability index Leading indicator for number of new customers/total number of customers Lagging indicator for employee satisfaction index and percent of defects in production process Leading indicator for reputation score and employee capability index Leading indicator for number of new products developed per quarter, percent on-time delivery, and percent of defects in production process Lagging indicator for employee satisfaction index

Advertising campaign

Percent market share

Promotion campaign

Number of new customers/total number of customers Number of repeat customers/total number of customers Percent on-time delivery

Value customer project Six sigma project

Total quality management project

Percent of defects in production process Number of new products developed per quarter Reputation score

Innovation project

Public relations campaign

Bonus bank project Training

Employee satisfaction index Employee capability index

Figure 1 Strategy map used in the experiment

Profit Revenue Expense

Financial

Customer
Market Share Customer Acquisition Customer Retention Customer Satisfaction

Organizational Reputation New Product Development

On time delivery Quality of Product

Internal Process

Employee Satisfaction

Employee Capabilities

Learning & Growth

that, the type of measure (leading/lagging indicator) as presented in Table I was not shown to the participants during the experiment. Each participant was also informed that each performance measure would be affected by investment in a particular initiative as shown in Table I. For example, if the participant invested in the customer relationship management project, the customer satisfaction index would be improved. At the same time, the customer satisfaction index can be changed by

VOL. 16 NO. 2 2012 MEASURING BUSINESS EXCELLENCE PAGE 59

other factors. For example even if a participant did not invest in the customer relationship management project, the customer satisfaction index would improve if other performance measures are improved. All of these performance measures are linked with each other and also have an effect on either total revenue or total expense, which nally lead to prot the desired outcome of the game. However the cause and effect relationship is not made explicit at the beginning of the experiment; the participant was only told the mechanism of the game as previously indicated. Each quarter, three additional measures were also reported: total revenue, total expenses and net prot. Each participants objective was to maximize accumulated prot (the sum of prot over all 12 quarters). The reason to set prot as a single objective is that the aim of this study is to test the quality of decision making with or without the strategy map. Thus, by using only one dependent variable, participants in the test will only have to focus on one thing. When including more than one objective it becomes more difcult to measure how good participants decisions are and can lead to inconclusive results. The economic conditions set for each quarter also differed. This condition was expressed by an economic index, a number ranging from 0.7-1.2. This index is the constant number input by the researcher in each quarter. A higher economic index meant that the participant could get a better result, as observed via the performance measures, from a smaller investment than when the economic index was lower. Before used, the simulation game was tested for validity and reliability. For a validity test, the game was used as a tool to teach students enrolled in an undergraduate level performance management course. It was tested successfully. The prot, the desired outcome, reected the right decisions, which resulted from the analysis of the performance measures. The game was also reviewed by two experts in this eld. As a result, its validity was conrmed. Its reliability was also tested for every initiative. If the same amount of investment was input into the same initiative in the same situation (economic index), the prot and other performance measures were found to be exactly the same. Thus, the reliability of the game was conrmed. There were two rounds in this experiment. In the rst round, every participant, in both the control and treatment groups, was asked to make decisions each quarter. Once the decision was made, the simulation game was run by the researcher and the result of the performance measures, including net prot, was reported to participant immediately. These results were not shown in the four perspectives in the balanced scorecard format, only as a list of thirteen separate measures. In the second round, only those in the treatment group were given additional information, i.e. the strategy map. The reason to introduce the strategy map in the second round of the experiment is to ensure that both control and treatment groups have similar abilities, as it can be observed in the difference of performance of both groups in the rst round, when those two groups had the same information. If the abilities of both groups are at the same level, then the difference of the performance of both groups in the rst round should be minimal. This strategy map was created based on the relationship that the researcher created in the simulation game. This is thus not a hypothesized strategy map but a true cause and effect relationship. The strategy map given to those in the treatment group is shown in Figure 1. The participants in the control group were given nothing and all conditions and rules in the second round were the same as in the rst. Once the second round experiment nished, the researcher debriefed the participants and showed them the results. Note that each Table II The design of the experiment
Group Control Treatment First round (pre-test) O1 O3 Strategy map (X) Second round (post-test) O2 O4

PAGE 60 MEASURING BUSINESS EXCELLENCE VOL. 16 NO. 2 2012

participant was given approximately ten minutes to make a decision each quarter. Thus the two rounds of 12 quarters each took approximately four hours. The design of the experiment is summarized in Table II. O1 O4 in Table II refers to net prot that results after the participant makes a decision. X refers to the existence of a strategy map in the experiment (only participants in treatment group were given the strategy map). If a strategy map is a tool that helps improve decision-making, the difference between O4 and O3 (O4 O3) must be larger than the difference between O2 and O1 (O2 O1), or in other words (O4 O3) (O2 O1) must be positive. Since the conditions in each quarter are different (economic index and investment bankroll), the value of (O4 O3) and (O2 O1) were calculated for each quarter (from quarter 1 to 12) for each participant. As there were 12 participants in the control group and 12 participants in the treatment group, there were one hundred and forty-four (O2 O1) and 144 (O4 O3). These gures (O2 O1) and (O4 O3) are termed D Prot and are treated as a dependent variable. The independent variable in this experiment is the group of participants, which is treated as a dummy variable (0 control group and 1 treatment group). The control variables included in the analysis are age, GPA, each participants grade for BA701, investment bankroll and economic index. The latter two controlled variables are included because both varied from quarter to quarter in the simulation game. Thus it might affect D Prot. Multiple regression technique was employed as the tool for analysis of this study. Before being used, all assumptions for multiple regression analysis were tested and the results showed that all assumptions are met. The regression equation is as follows: DProfit a b1 Treatment b2 Age b3 GPA b4 BA701 b5 Investment b6 Economic 1 Where D Prot (O2 O1) for control group or (O4 O3) for treatment group, a constant, Treatment 0 for control group and 1 for treatment group, Age age of participant, GPA grade point average of participant, BA701 participant grade for BA701, Investment investment bankroll (US$10 million for Q1-Q4, US$30 million for Q5-Q8 and US$60 million for Q9-Q12), Economic economic index for each quarter, and 1 error in estimation.

4. Results and discussion

For the 24 participants, the average D Prot is US$3,117,933, which is statistically greater than 0 (t-score 8.32, p-value , 0.01). This was not surprising, as for each quarter, prot in the second round of the experiment was better than in the rst round. This nding could be due to the greater experience and knowledge for each participant when making decisions in the second round than in the rst round, when the game is new. When no strategy map was presented to either group, it was also found that the difference of prot between control group and treatment group in the rst round was insignicant (t-score 0.728, p-value 0.467), thus conrming that the abilities of participants in those two groups were at the same level.

Table III Descriptive statistics for variables in the study

Control (treatment 0) Standard Average deviation 3,118,750 29.17 3.54 3.67 33,333,333 0.983 5,951,669 2.98 0.16 0.30 20,619,768 0.16 Treatment (treatment 1) Standard Average deviation 3,117,115 27.25 3.58 3.83 33,333,333 0.983 6,764,736 1.30 0.15 0.26 20,619,768 0.16 Total Average 3,117,933 28.21 3.56 3.76 33,333,333 0.983 Standard deviation 6,360,077 2.49 0.16 0.29 20,583,814 0.16

Variable D Prot (US$) Age GPA BA 701 Investment (US$) Economic

VOL. 16 NO. 2 2012 MEASURING BUSINESS EXCELLENCE PAGE 61

However the improvement in prot on average is similar between the treatment group and the control group. Table III shows the descriptive statistics for the variables in this study. It is interesting to note that the average age of the participants in the control group is greater than that of the treatment group, while GPA and nal grade for BA701 are greater for the treatment group than for the control group. The average of the investment bankroll is the simple average of the US$10, US$30, and US$60 million according to the rules of the simulation game. Finally, since the economic index was set in the simulation game and not changed for the second, the average of the index is identical for both the control and treatment groups. In order to ascertain the variables that could explain the change in prot, further analysis was performed via multiple regression and treating D Prot as a dependent variable. This revealed that none of the independent and control variables used in this study could be used to explain the variance of D Prot (the regression model r 2 0.039 and p-value for the overall model 0.538). This means that strategy map (treatment) had no effect on the results of the decision-making. Table IV shows the results of the regression analysis. Although a strategy map is said to be a visual representation of the cause and effect relationships within a companys critical objective (Kaplan and Norton, 2004a) and might help users better understand strategy because a graphical presentation makes it easier to identify relationships and produces better comprehension (Wittrock, 1974; Kourilsky and Wittrock, 1987; Glenberg and Langston, 1992), the ndings in this study indicate that a strategy map does not have any relationship with performances resulting from decisions made. This can be explained by the fact that the strategy map used in this study, which is similar to those used in many organizations, has performance measures that are not clearly dened. Thus, participants might not be very clear about the denition of each performance measure used in this study. This result is consistent with the ndings obtained from many scholars that highlighted the inadequate denition and utilization of the performance indicators as a main drawback of the balanced scorecard (Lingle and Schiemann, 1996; Stivers et al., 1998; Ittner and Larcker, 1998b; Lipe and Salterio, 2000; Malmi, 2001; Speckbacher et al., 2003). Participants in the treatment group in this study who were given a strategy map might see each measure as independent and as not mirroring the cause and effect logic. They may simply ignore the strategy map when making decisions. This possible explanation was also conrmed by interviewing all participants in the treatment group. Some participants even stated that they did not believe that the strategy map actually helped them make better decisions and that the only information they needed to make decisions was the result of each performance measure, not the linkage in the strategy map. As a result, the information in the strategy map provided to them did not help them make better decisions. Additionally, participants did not test the causal effect in the strategy map quantitatively. This phenomenon is consistent with the ndings of Ittner and Larcker (1998b) who reported that only 23 per cent of the 157 organizations survey tested the causal model of strategy maps. This number supports the study of Speckbacher et al. (2003), in which half of the companies adopting balanced scorecard did not develop a causal model of their strategies and a study by Davis and Albright (2004) that found that 77 per cent of the companies adopting the Table IV Results from the regression analysis
Standardized coefcients (Constant) Treatment Age GPA Grade in BA 701 Investment Economic Index t 1.462 2 0.615 2 0.666 2 0.405 2 1.021 1.455 2 0.306 p-value 0.146 0.540 0.506 0.686 0.309 0.148 0.760

2 0.064 2 0.073 2 0.043 2 0.116 0.128 2 0.027

PAGE 62 MEASURING BUSINESS EXCELLENCE VOL. 16 NO. 2 2012

balanced scorecard fail to develop the casual model of their strategies. These ndings explain why many organizations that use a balanced scorecard fail to improve their performance, as evidenced in the study of Ittner et al. (2003) and Hendricks et al. (2004). Nevertheless, it is interesting to note that the strategy map used in this study can be too simplistic to inuence the decision makers. A more detailed strategy map illustrating the exhaustive relationships among the different measures can also yield different results as it is more difcult for a manager to make a decision without the help of the map. The second possible explanation for this nding is mental model of the participant. In the decision-making process, people use a mental model to describe, explain and predict a systems behavior (Craik, 1943; Johnson-Laird, 1983). They will make better decisions if their mental models are similar to reality (Rowe and Cooke, 1995; Stout et al., 1996; Wyman and Randel, 1998; Davis and Yi, 2004; Mathieu et al., 2000, 2005). Managers also build their mental models as they engage in business decisions (Capelo and Dias, 2009). Thus in our case, taking into account that the participants in this study are middle managers who are currently studying for their MBA degrees, it is possible that the participants in the control group, who did not have a strategy map when making decisions, were able to build a mental model that closely aligns to a strategy map. They then do not need a documented strategy map to help them make decisions. This could explain why the results of their decisions are not statistically different than for the participants in treatment group. It can be argued that the decision to introduce the strategy map for the treatment group in the second round might have inuenced the ndings of the experiment. By then, managers in both groups would have already known the linkage between one initiative and another, resulting in the insignicant difference between the performances of the two groups. If this argument is true, it even conrms that the strategy map does not provide any additional information. Managers can learn from their own mental models without using a strategy map at all. The nal possible explanation of what was found in this study is due to the quality of feedback. Managers make decisions and learn about their decisions in the context of feedback loops (Forrester, 1961). After managers discover the results of their decisions, they use that feedback to modify their mental model (Argyris, 1976). Quality of feedback has been found to denitely affect the quality of the decision-making process (Sterman, 2000). In this study, there are twelve decisions to be made in each of the two rounds. In every simulation game quarter, every participant received feedback in terms of performance measures. This feedback may have had a high enough quality to help them make better decisions without using the strategy map. Only the results of performance measures reported after decision-making would seem to be adequate if this was the case.

5. Conclusion
Based on the ndings of this study, it can be implied that a strategy map alone does not improve decision-making. A strategy map is simple enough that managers can build their own mental model similar to the map. For example, without the map, managers can imagine that higher customer satisfaction can lead to higher customer retention or more market share can cause more revenue. A simple cause and effect relationship is thus not sufcient to provide understanding of complex relationships (Voelpel et al., 2006). As a result, in addition to the strategy map, managers should be given more details, such as correlation between each measure in the map and a detailed explanation of how to interpret the map. In the end, a strategy map must provide information that managers cannot obtain simply by using their own mental model. Steps must be taken to convince managers that strategy maps provide useful information and can be a very powerful tool that helps them make better decisions. This includes the training of what a strategy map is and how to use it effectively. The successful case studies can also be good examples for managers to demonstrate how to use a strategy map. Otherwise, managers may ignore the map on the grounds that the map provides them with nothing more than a graphical representation of organizational strategies.

VOL. 16 NO. 2 2012 MEASURING BUSINESS EXCELLENCE PAGE 63

Additionally, a strategy map must be reviewed consistently. It should be updated when strategies are changed or every time that performance is reviewed. Once there is enough data for each performance measure, a strategy map should be statistically tested for validity. This step will ensure that the strategy map is valid and up-to-date and can be used to help managers make better decisions. This study is not without limitations. Since the research approach employed in this study is a laboratory experiment, which, while it yields high internal validity (proof of the cause and effect relationship), is not in a natural setting. Therefore the decision-making process may vary from real life situations. This limits the external validity (generalizability) of this study. In order to enhance the external validity by conducting the experiment in a natural setting, the assumption of time lag between measures needs to be considered. For example, the time between enhancing customer satisfaction until it reects the improvement of customer retention should be estimated. Then, the effect of using a strategy map can be tested empirically in a real-life setting. The experiment in this study was also designed to test the use of a strategy map at one level of management, i.e. the manager himself making the decision leading to results. However, one of the strengths of the use of a strategy map is its ability to communicate articulated strategies to subordinates in a hierarchy or to peers of other units. The strategy map could diagrammatically show how one strategy leads to outcomes. Subordinates and peers of other divisions could well see how they contribute to the performance or achievements of an organization as a whole. However, this was not tested in this experiment. More research on this topic is clearly needed. A eld experiment could be employed by asking managers to make decisions with and without a strategy map in a real work setting. A strategy map with detailed guidelines and correlation between each measure could thus be tested as to whether it has an effect on a managers decision making. The usefulness of the strategy map and a method of using it can be investigated based on managers perception. It has also been observed in this study that none of independent variables used in this study can be used to explain the variance of improvement in prot, which is the result of decision making. Thus, it is interesting to include more independent variables and test whether the strategy map will have a relationship with the performance of decision making. Finally the characteristics of successful strategy maps can be identied in order to help managers create the right strategy map for their organizations.

References
Abrain, A. and Buglioni, L. (2003), A multidimensional performance model for consolidating balanced scorecards, Advances in Engineering Software, Vol. 34, pp. 339-49. Ahn, H. (2001), Applying the balanced scorecard concept: an experience report, Long Range Planning, Vol. 34 No. 4, pp. 441-61. Anderson, E., Fomell, W. and Lehman, D. (1994), Customer satisfaction, market share, and protability: ndings from Sweden, Journal of Marketing Research, Vol. 58, pp. 53-66. Anderson, E., Fomell, W. and Rust, R. (1997), Customer satisfaction, productivity, and protability: differences between goods and services, Marketing Science, Vol. 16, pp. 129-45. Anderson, E.G. and Morrice, D.J. (2000), A simulation game for teaching service-oriented supply chain management: does information sharing help managers with service capacity decisions?, Production and Operations Management, Vol. 9, pp. 40-55. Anderson, J.R. (2005), The relationship between student perceptions of team dynamics and simulation game outcomes: an individual-level analysis, Journal of Education for Business, Vol. 81 No. 2, pp. 85-90. Argyris, C. (1976), Increasing Leadership Effectiveness, Wiley, New York, NY. Avella, L., Fernandez, E. and Vazquez, C.J. (2001), Analysis of manufacturing strategy as an explanatory factor of competitiveness in the large Spanish industrial rm, International Journal of Production Economics, Vol. 72, pp. 139-57.

PAGE 64 MEASURING BUSINESS EXCELLENCE VOL. 16 NO. 2 2012

Banker, R., Potter, G. and Schroeder, R. (1993), Reporting manufacturing performance measures to workers: an empirical study, Journal of Management Accounting Research, Vol. 5, pp. 33-55. Banker, R., Potter, G. and Srinivasan, D. (2000), An empirical investigation of an incentive plan that includes non-nancial performance measures, The Accounting Review, Vol. 75 No. 1, pp. 65-92. Banker, R.D., Chang, H. and Pizzini, M.J. (2004), The balanced scorecard: judgmental effects of performance measures linked to strategy, The Accounting Review, Vol. 79 No. 1, pp. 1-23. Barney, J.B. (2001), Is the resource-based view a useful perspective for strategic management research? Yes, Academy of Management Review, Vol. 26 No. 1, pp. 41-56. Barney, J.B., Wright, M. and Ketchen, D.J. Jr (2001), The resource based view of the rm: ten years after 1991, Journal of Management, Vol. 27 No. 6, pp. 625-43. Bierbusse, P. and Siesfeld, T. (1997), Measures that matter, Journal of Strategic Performance Measurement, Vol. 1 No. 2, pp. 6-11. Bititci, U.S., Carrie, A.S. and Turner, T. (1998), Diagnosing the integrity of your performance measurement system, Control, pp. 9-13. Bourguignon, A., Malleret, V. and Norreklit, H. (2004), The American balanced scorecard versus the French tableau de bord: the ideological dimension, Management Accounting Research, Vol. 15, pp. 107-34. Bourne, M., Neely, A.D., Mills, J.F. and Platts, K. (2003a), Implementing performance measurement systems: a literature review, International Journal of Business Performance Management, Vol. 5 No. 1, pp. 1-24. Bourne, M., Neely, A.D., Mills, J.F. and Platts, K. (2003b), Why some performance measurement initiatives fail: lessons from the change management literature, International Journal of Business Performance Management, Vol. 5 Nos 2/3, pp. 245-69. Bourne, M., Neely, A.D., Mills, J.F., Platts, K. and Wilcox, M. (2000), Designing, implementing and updating performance measurement systems, International Journal of Operations & Production Management, Vol. 20 No. 7, pp. 754-71. Bourne, M.C.S., Neely, A.D., Platts, K.W. and Mills, J.F. (2002), The success and failure of performance measurement initiatives: the perceptions of participating managers, International Journal of Operations & Production Management, Vol. 22 No. 11, pp. 1288-310. Bourne, M.C.S., Mills, J.F., Bicheno, J., Hamblin, D.J., Wilcox, M., Neely, A.D. and Platts, K.W. (1999), Performance measurement system design: testing a process approach in manufacturing companies, International Journal of Business Performance Measurement, Vol. 1 No. 2, pp. 154-70. Cantor, D.E. and Macdonald, J. (2009), Decision-making in the supply chain: examining problem solving approaches and information availability, Journal of Operations Management, Vol. 27 No. 3, pp. 220-32. Capelo, C. and Dias, J.F. (2009), A system dynamics-based simulation experiment for testing mental model and performance effects of using the balanced scorecard, System Dynamics Review, Vol. 25 No. 1, pp. 1-34. Copeland, T., Koller, T. and Murrin, J. (1996), Valuation: Measuring and Managing the Value of Companies, John Wiley & Sons, New York, NY. Craik, K. (1943), The Nature of Explanation, Cambridge University Press, Cambridge. Davis, S. and Albright, T. (2004), An investigation of the effect of balanced scorecard implementation on nancial performance, Management Accounting Research, Vol. 15 No. 2, pp. 135-53. Davis, F. and Yi, M. (2004), Improving computer skill training: behavior modeling, symbolic mental rehearsal, and the role of knowledge structures, Journal of Applied Psychology, Vol. 89, pp. 509-23. De Geuser, F., Mooraj, S. and Oyon, D. (2009), Does the balanced scorecard add value? Empirical evidence on its effect on performance, European Accounting Review, Vol. 18 No. 1, pp. 93-122. Deakin, J., Aitken, M., Robbins, T. and Sahakian, B.J. (2004), Risk taking during decision-making in normal volunteers changes with age, Journal of the International Neuropsychological Society, Vol. 10 No. 4, pp. 590-8.

VOL. 16 NO. 2 2012 MEASURING BUSINESS EXCELLENCE PAGE 65

Dixon, J.R., Nanni, A.J. and Vollmann, T.E. (1990), The New Performance Challenge Measuring Operations for World-class Competition, Dow Jones-Irwin, Homewood, IL. Eccles, R. (1991), The performance measurement manifesto, Harvard Business Review, Vol. 69, pp. 131-7. Eccles, R.G. and Pyburn, P.J. (1992), Creating a comprehensive system to measure performance, Management Accounting, October, pp. 41-4. Forrester, J. (1961), Industrial Dynamics, Pegasus Communications, Waltham, MA. Free, C. and Qu, S.Q. (2011), The use of graphics in promoting management ideas: an analysis of the balanced scorecard, 1992-2010, Journal of Accounting & Organizational Change, Vol. 7 No. 2, pp. 158-89. Gladstein, D.L. and Reilly, N.P. (1985), Group decision making under threat: the tycoon game, Academy of Management Journal, Vol. 28 No. 3, pp. 613-27. Glenberg, A. and Langston, W. (1992), Comprehension of illustrated text: pictures help to build mental models, Journal of Memory and Language, Vol. 31, pp. 129-51. Gomes, C.F., Yasin, M.M. and Lisboa, J.V. (2004), Literature review of manufacturing performance measures and measurement in an organizational context: a framework and direction for future research, Journal of Technology Management, Vol. 15 No. 6, pp. 511-30. Grant, R.M. (1991), The resource-based theory of competitive advantage: implications for strategy formulation, California Management Review, Vol. 33 No. 3, pp. 14-35. Hendricks, K., Menor, L. and Wiedman, C. (2004), The balanced scorecard: to adopt or not to adopt?, Ivey Business Journal, November/December, pp. 1-7. Hogarth, R.M. and Makridakis, S. (1981), The value of decision making in a complex environment: an experimental approach, Management Science, Vol. 27 No. 1, pp. 93-107. Hoque, Z. and James, W. (2000), Linking balanced scorecard measures to size and market factors: impact on organizational performance, Journal of Management Accounting Research, Vol. 12 No. 1, pp. 1-15. Ittner, C. and Larcker, D. (1996), Measuring the impact of quality initiatives on rm nancial performance, Advances in the Management of Organizational Quality, Vol. 1, pp. 1-37. Ittner, C. and Larcker, D. (1998a), Are non-nancial measures leading indicators of nancial performance? An analysis of customer satisfaction, Journal of Accounting Research, pp. 1-35. Ittner, C. and Larcker, D. (1998b), Innovations in performance measurement: trends and research implications, Journal of Management Accounting Research, Vol. 10, pp. 205-38. Ittner, C., Larcker, D. and Randall, T. (2003), Performance implications of strategic performance measurement in nancial services rms, Accounting, Organizations and Society, Vol. 28, pp. 715-41. Johnson, M.S. (1990), Age differences in decision making: a process methodology for examining strategic information processing, Journal of Gerontology, Vol. 45 No. 2, pp. 75-8. Johnson-Laird, P. (1983), Mental Models: Towards a Cognitive Science of Language, Inference and Consciousness, Harvard University Press, Cambridge, MA. Kaplan, R.S. and Norton, D.P. (1992), The balanced scorecard measures that drive performance, Harvard Business Review, Vol. 70 No. 1, pp. 71-9. Kaplan, R.S. and Norton, D.P. (1993), Putting the balanced scorecard to work, Harvard Business Review, Vol. 71 No. 5, pp. 134-42. Kaplan, R.S. and Norton, D.P. (1996), The Balanced Scorecard Translating Strategy Into Action, Harvard Business School Press, Boston, MA. Kaplan, R.S. and Norton, D.P. (2000), Having trouble with your strategy? Then map it, Harvard Business Review, Vol. 78 No. 5, pp. 167-76. Kaplan, R.S. and Norton, D.P. (2001a), The Strategy-Focused Organization: How Balanced Scorecard Companies Thrive In The New Business Environment, Harvard Business School Press, Boston, MA.

PAGE 66 MEASURING BUSINESS EXCELLENCE VOL. 16 NO. 2 2012

Kaplan, R.S. and Norton, D.P. (2001b), Transforming the balanced scorecard from performance measurement to strategic management: part I, Accounting Horizons, Vol. 15 No. 1, pp. 87-104. Kaplan, R.S. and Norton, D.P. (2004a), Strategy Maps: Converting Intangible Assets Into Tangible Outcomes, Harvard Business School Press, Boston, MA. Kaplan, R.S. and Norton, D.P. (2004b), The strategy map: guide to aligning intangible assets, Strategy & Leadership, Vol. 32 No. 5, pp. 10-17. Kaplan, R.S., Norton, D.P. and Rugelsjoen, B. (2010), Managing alliances with the balanced scorecard, Harvard Business Review, Vol. 88 No. 1, pp. 114-20. Kourilsky, M. and Wittrock, M. (1987), Verbal and graphical strategies in the teaching of economics, Teaching & Teacher Education, Vol. 3 No. 1, pp. 1-12. Kunc, M. (2008), Using systems thinking to enhance strategy maps, Management Decision, Vol. 46 Nos 5-6, pp. 761-78. Lev, B. (2001), Intangibles: Management, Measurement, and Reporting, The Brookings Institution, Washington, DC. Lewy, D. and Du Mee, C. (1998), The ten commandments of balanced scorecard implementation, Management Control and Accounting, April, pp. 34-6. Lingle, J. and Schiemann, W. (1996), From balanced scorecard to strategic gauges: is measurement worth it?, Management Review, Vol. 85, pp. 56-61. Lipe, M.G. and Salterio, S. (2000), The balanced scorecard: judgmental effects of common and unique performance measures, The Accounting Review, Vol. 75 No. 3, pp. 283-98. Lucianetti, L. (2010), The impact of the strategy maps on balanced scorecard performance, International Journal of Business Performance Management, Vol. 12 No. 1, pp. 21-36. McCunn, P. (1998), The balanced scorecard: the eleventh commandment, Management Accounting, April, pp. 34-6. Malina, M.A., Norreklit, H.S.O. and Selto, F.H. (2007), Relations among measures, climate of control, and performance measurement models, Contemporary Accounting Research, Vol. 24 No. 3, pp. 935-82. Malmi, T. (2001), Balanced scorecards in Finnish companies, Management Accounting Research, Vol. 12 No. 2, pp. 207-20. Marr, B. (2005), Business performance measurement: an overview of the current state of use in the USA, Measuring Business Excellence, Vol. 9 No. 3, pp. 56-62. Marr, B., Neely, A., Bourne, M., Franco, M., Wilcox, M., Adams, C., Mason, S. and Kennerley, M. (2004), Business performance measurement what is the state of use in large US rms, Proceedings of Performance Measurement and Management: Public and Private, Centre for Business Performance, Craneld University, Craneld, pp. 627-34. Mathieu, J., Goodwin, G., Heffner, T., Salas, E. and Cannon-Bowers, J. (2000), The inuence of shared mental models on team process and performance, Journal of Applied Psychology, Vol. 85, pp. 273-83. Mathieu, J.E., Heffner, T.S., Goodwin, G.F., Cannon-Bowers, J.A. and Salas, E. (2005), Scaling the quality of teammates mental models: equinality and normative comparisons, Journal of Organizational Behavior, Vol. 26, pp. 37-56. Meekings, A. (1995), Unlocking the potential of performance measurement: a guide to practical implementation, Public Money & Management, October-December, pp. 1-8. Meyer, M.W. and Gupta, V. (1994), The performance paradox, Research in Organizational Behavior, Vol. 16, pp. 309-69. Neely, A. (1999), The performance measurement revolution: why now and what next, International Journal of Operations & Production Management, Vol. 19 No. 2, pp. 205-28. Neely, A.D., Mills, J.F., Platts, K., Gregory, M.J. and Richards, H. (1996), Performance measurement system design: should process based approaches be adopted?, International Journal of Production Economics, Vol. 46, pp. 423-31.

VOL. 16 NO. 2 2012 MEASURING BUSINESS EXCELLENCE PAGE 67

Neely, A.D., Mills, J.F., Platts, K., Richards, H., Gregory, M.J., Bourne, M. and Kennerley, M.P. (2000), Performance measurement system design: developing and testing a process-based approach, International Journal of Operations & Production Management, Vol. 20 No. 10, pp. 1119-45. Norreklit, H. (2000), The balance on the balanced scorecard a critical analysis of some of its assumptions, Management Accounting Research, Vol. 11 No. 1, pp. 66-88. Norreklit, H. (2003), The balanced scorecard: what is the score? A rhetorical analysis of the balanced scorecard, Accounting, Organizations and Society, Vol. 28 No. 6, pp. 591-619. Olve, N.G., Roy, J. and Wetter, M. (1999), A Practical Guide to Using the Balanced Scorecard, Wiley, Chichester. Petergraf, M.A. (1993), The cornerstones of competitive advantage: a resource-based view, Strategic Management Journal, Vol. 14, pp. 179-88. Quezada, L.E., Cordova, F.M., Palominos, P., Godoy, K. and Ross, J. (2009), Method for identifying strategic objectives in strategy maps, International Journal of Production Economics, Vol. 122, pp. 492-500. Rigby, D. (2001), Management tools and techniques: a survey, California Management Review, Vol. 43 No. 2, pp. 139-60. Rowe, A. and Cooke, N. (1995), Measuring mental models: choosing the right tools for the job, Human Resource Development Quarterly, Vol. 6, pp. 243-55. Schneiderman, A. (1999), Why balanced scorecards fail, Journal of Strategic Performance Measurement, January, pp. 6-11. Sharma, M.K., Bhagwat, R. and Dangayach, G.S. (2005), Practice of performance measurement: experience from Indian SMEs, International Journal of Globalisation and Small Business, Vol. 1 No. 2, pp. 183-213. Silk, S. (1998), Automating the balanced scorecard, Management Accounting, Vol. 79 No. 11, pp. 38-44. Speckbacher, G., Bischof, J. and Pfeiffer, T. (2003), A descriptive analysis on the implementation of balanced scorecards in German-speaking countries, Management Accounting Research, Vol. 14, pp. 361-87. Sterman, J.D. (2000), Business Dynamics: Systems Thinking and Modeling for a Complex World, Irwin McGraw-Hill, New York, NY. Stivers, B., Covin, T., Hall, G. and Smalt, S. (1998), How nonnancial performance measures are used, Management Accounting, Vol. 44, pp. 46-9. Stout, R., Salas, E. and Kraiger, K. (1996), The role of trainee knowledge structures in aviation team environments, International Journal of Aviation Psychology, Vol. 7, pp. 235-350. Taticchi, P. (2008), Business performance measurement and management: implementation of principles in SMEs and enterprise networks, PhD thesis, University of Perugia, Perugia. Taylor, D.N. (1975), Age and experience as determinants of managerial information processing and decision making performance, The Academy of Management Journal, Vol. 18 No. 1. Thompson, K.R. and Mathys, N.J. (2008), The aligned balanced scorecard: an improved tool for building high performance organizations, Organizational Dynamics, Vol. 37 No. 4, pp. 378-93. Townley, B., Cooper, D.J. and Oakes, L. (2003), Performance measures and the rationalization of organizations, Organization Studies, Vol. 24 No. 7, pp. 1045-71. Unahabhokha, C., Platts, K. and Tan, K. (2006), A framework for developing and using a predictive delivery performance measurement system, International Journal of Manufacturing Technology and Management, Vol. 8 No. 4, pp. 308-29. Voelpel, S.C., Leibold, M. and Eckhoff, R.A. (2006), The tyranny of the balanced scorecard in the innovation economy, Journal of Intellectual Capital, Vol. 7 No. 1, pp. 43-60. Wilkes, J. (2005), Leading and lagging practices in performance management, Measuring Business Excellence, Vol. 9 No. 3, pp. 41-6. Wittrock, M. (1974), Learning as generative processes, Educational Psychologist, Vol. 11, pp. 87-95.

PAGE 68 MEASURING BUSINESS EXCELLENCE VOL. 16 NO. 2 2012

Wyman, B. and Randel, J. (1998), The relation of knowledge organization to performance of a complex cognitive task, Applied Cognitive Psychology, Vol. 12, pp. 251-64. Young, S. and OByrne, S. (2001), EVA and Value-Based Management, McGraw-Hill, New York, NY.

Further reading
Ittner, C. and Larcker, D. (2003), Coming up short on non-nancial performance measurement, Harvard Business Review, November, pp. 88-95.

About the author

Dr Nopadol Rompho joined Thammasat Business School in 2003. His primary research interest is in the area of performance measurement and operations management. Before joining the university, he worked in oil and gas companies as an engineer and business analyst. He obtained a Bachelors degree in Chemical Engineering (2nd Class Honors) from Chulalongkorn University, Thailand in 1995, Master of Science in Chemical Engineering from Oregon State University, USA in 1997, Master of Business Administration from Thammasat University, Thailand in 2001, and Doctor of Philosophy from University of Glasgow, UK in 2006. Nopadol Rompho can be contacted at: nrompho@tu.ac.th

To purchase reprints of this article please e-mail: reprints@emeraldinsight.com Or visit our web site for further details: www.emeraldinsight.com/reprints

VOL. 16 NO. 2 2012 MEASURING BUSINESS EXCELLENCE PAGE 69