The Balanced Scorecard: The Effect of Strategy Information on Performance Evaluation Judgments

Transcription

1 JOURNAL OF MANAGEMENT ACCOUNTING RESEARCH Vol pp American Accounting Association DOI: /jmar The Balanced Scorecard: The Effect of Strategy Information on Performance Evaluation Judgments Kerry A. Humphreys Ken T. Trotman University of New South Wales ABSTRACT: Judging divisional performance using the balanced scorecard is a complex task, with prior research finding a bias toward measures common to two divisions when managerial performance is evaluated. We conduct two experiments investigating the role of strategy information and strategically linked performance measures in eliminating this bias and establish a boundary condition for the common measures bias. We demonstrate that when strategy information is provided to managers and only some measures are strategically linked, the common measures bias exists (consistent with Banker et al. 2004). We find that when all the performance measures are strategically linked, but no strategy information is provided, the common measures bias also exists. However, if strategy information is provided and all measures are strategically linked (a condition which did not exist in previous research), the common measures bias is eliminated. Keywords: balanced scorecard; common measures bias; strategy information. INTRODUCTION Strategic performance measurement systems, such as the balanced scorecard, aim to improve performance evaluation judgments by providing a set of enhanced financial metrics and future-oriented, non-financial measures (Sprinkle and Williamson 2006). For this reason, it is suggested the balanced scorecard be used in management review decisions (Kaplan and Norton 1996). However, evaluating performance using the balanced scorecard is a complex task (Merchant 2006), where the decision maker is presented with multiple measures across four perspectives, often without guidance on how to weight and combine information cues in an overall judgment and without normative criterion for evaluating the accuracy of the performance evaluations that result (Langfield-Smith 2006; Dilla and Steinbart 2005; Lipe and Salterio 2000). Consequently, suboptimal evaluations are apparent in the literature (Luft and Shields 2009). We are grateful to Mandy Cheng, Anne Farrell, Susan Krische, Joan Luft, and Steve Salterio for their comments on previous versions of the paper. We also thank brown bag participants at the University of Illinois at Urbana Champaign, research seminar participants at The University of New South Wales, and conference participants at the Global Management Accounting Research Symposium Sydney and AFAANZ Annual Conference Sydney, especially our discussants Mike Shields and Anna Stamatelatos. We gratefully acknowledge the financial support from an Australian Research Council grant to Ken Trotman. Published Online: December

2 82 Humphreys and Trotman Adaptive decision theory (Payne et al. 1993) suggests that in complex settings, a trade-off between the desire to maximize decision quality and the limited processing capacity of human decision makers will lead to decision makers seeking to minimize the cognitive effort required to make a decision. Consistent with this theory, Lipe and Salterio (2000, 293) found that superiors employed a natural simplifying strategy when making managerial performance evaluation judgments using the balanced scorecard. This simplifying strategy, known as the common measures bias, results in managerial performance evaluations that reflect a greater weighting of measures common to two divisions and less weighting of measures unique to each division. Over the past decade, a number of studies have found the existence of the common measures bias (Dilla and Steinbart 2005; Banker et al. 2004; Libby et al. 2004; Ittner et al. 2003a; Lipe and Salterio 2000). The common measures bias may be due to a lack of cognitive effort to incorporate the unique measures (Libby et al. 2004) and/or the fact that managers have concerns about the strategic relevance of the unique measures, given they are not used by all divisions (Banker et al. 2004; Libby et al. 2004). The balanced scorecard is designed to facilitate a performance evaluation that reflects all performance measures linked to a division s strategy (both common and unique) and, importantly, the unique measures often indicate what individual business units need to achieve in order to succeed (Libby et al. 2004). Consequently, limited use of unique measures in performance evaluation judgments decreases the potential benefits of using the balanced scorecard. The main aim of this paper is to test the generalizability of the findings on the common measures bias by examining under what circumstances the provision of strategy information, and the extent to which measures are strategically linked, eliminates the common measures bias on performance evaluation tasks. Both strategy information and strategically linked measures are typically associated with the balanced scorecard. The translation of strategy into a coherent set of performance measures is a distinguishing feature of strategic performance measurement systems, including the balanced scorecard (Burney et al. 2009; Chenhall 2005). In the case of the balanced scorecard, strategically linking performance measures encourage managers to focus on delivering strategic initiatives that will positively impact these measures (Kaplan and Norton 2004, 2006). Hence, it is necessary to strategically link measures in a balanced scorecard to make strategies more meaningful and actionable (Kaplan and Norton 2008, 85). One other study in the accounting literature employs strategy information to investigate balanced scorecard judgment issues. Banker et al. (2004) found that the common measures bias continues to exist even after strategy information was provided. When Banker et al. (2004) examined the common measures bias for each measure type (linked and non-linked) contained in their balanced scorecards, they found that when strategy information was provided, managers exhibit the bias for linked measures, but not in non-linked measures. 1 Based on these results, it is not clear that making more of the measures linked will reduce the common measures bias. Research subsequent to Banker et al. (2004) has demonstrated the information processing benefits of providing a causal model to decision makers (Tayler 2010; Farrell et al. 2007; Vera-Muñoz et al. 2007). Of particular relevance to the current task, Tayler (2010, 12) suggests that presentation of a causal model to managers should reduce the cognitive complexity of evaluation tasks where multiple measures are provided. When managers review balanced scorecards where all the measures presented are strategically linked, we expect that providing strategy information will assist them to identify the strategic context for each performance measure (Ariely 2008). Being able to determine that all measures are strategically relevant will reduce the cognitive 1 The manipulation employed in Banker et al. (2004) meant that each scorecard provided in the task had eight strategically linked measures and eight performance measures intentionally not linked to the divisional strategy.

3 The Balanced Scorecard: The Effect of Strategy Information on Judgments 83 complexity of the evaluation task and increase the likelihood that participants use all measures provided in evaluating performance (Slovic and MacPhillamy 1974). For this reason, we hypothesize that the common measures bias will be eliminated when strategy information is provided and all performance measures are strategically linked. These issues are addressed in two related experiments. We find that the Banker et al. (2004) result, where the common measures bias exists even when strategy information is provided, replicates only in a situation where some non-linked measures are included in the balanced scorecard (see Experiment 1). We also find that when all performance measures are strategically linked, but no strategy information is provided, the common measures bias remains (see Experiment 2). However, if strategy information is provided and all measures are strategically linked, we find that the common measures bias is eliminated (see both Experiments 1 and 2). In summary, the common measures bias is eliminated only when all performance measures are linked to divisional strategy and strategy information is provided. Our study makes the following contributions. First, it examines the common measures bias in situations where all balanced scorecard measures are strategically linked, as suggested by Kaplan and Norton (2004), compared to previous research where only some measures were strategically linked (Banker et al. 2004). In both experiments, we find the common measures bias to be absent when managers receive performance measures that are strategically linked and strategy information is provided. This finding is important, because while all the performance measures in a normative balanced scorecard are expected to be strategically linked (Kaplan and Norton 1996, 2004, 2008), in practice, multiple performance measurement systems (including the balanced scorecard) sometimes do not link all the measures to strategy (e.g., Ittner et al. 2003b). We provide evidence of the judgmental effect of this variation in practice. Second, our study investigates the impact on the common measures bias of communicating strategy information (specifically narrative strategy and a strategy map depicting the causal linkages between strategic objectives). We show that to make a balanced performance evaluation judgment, communication of strategy information is necessary, even when all performance measures are strategically linked. While the benefits of effectively communicating strategy information within organizations when using a balanced scorecard have been discussed in the literature (Nørreklit and Mitchell 2007; Kaplan and Norton 2000, 2004; Malina and Selto 2001; Nørreklit 2000; Ittner and Larcker 1998), this is the first time their beneficial role in eliminating the common measures bias has been demonstrated empirically. The structure of our two experiments is presented in Figure 1. Experiment 1 examines the impact of the absence/presence of detailed strategy information on the common measures bias, when performance measures are half linked/half non-linked (refer to cells 1 and 2 of Figure 1; as in Banker et al. 2004). In addition, it compares two treatments where detailed strategy information is provided in one treatment, the performance measures are half linked/ half non-linked (see cell 2 of Figure 1); while in the other treatment, all performance measures are linked to divisional strategy (refer to cell 4 of Figure 1). This latter condition has not been examined in previous research. In Experiment 2, we examine the impact of the absence/presence of detailed strategy information, when all measures are strategically linked (refer to cells 3 and 4, respectively, of Figure 1). Hypothesis Development EXPERIMENT 1 Payne et al. (1993, 51) suggest that the common measures bias occurs due to the cognitive ease of comparison of common dimensions. In the context of the balanced scorecard, this bias has been described as a simplifying strategy (Lipe and Salterio 2000) employed to minimize the

4 84 Humphreys and Trotman FIGURE 1 Key Findings Regarding the Common Measures Bias (CMB) from the Present Study and Related Extant Literature Provision of Strategy Information Extent of Strategic Linkage Half the measures linked to strategy All measures linked to strategy No Strategy Information 1. CMB present (Banker et al and Present study: Experiment 1 a ) 3. CMB present (Present study: Experiment 2) Detailed Strategy Information 2. CMB present (Banker et al and Present study: Experiment 1 a ) 4. CMB absent (Present study: Experiments 1 and 2) a In Banker et al. (2004) and for managers in these cells in Experiment 1, half the measures received were strategically linked and half the measures were non-linked. Different measures and targets were employed in the two studies; however, performance was balanced across common, unique, linked, and non-linked measures. cognitive effort required to evaluate divisional performance. When managers have strategy information, one potential method of overcoming this common measures bias is to provide them with performance measures that are all strategically linked. The linking of performance measures and business unit strategy has been identified as a major strength of the balanced scorecard (Otley 1999) and a key feature that distinguishes it from other performance measurement systems (Kaplan and Norton 2004; Ittner and Larcker 1998). Banker et al. (2004), while still finding the common measures bias when strategy information was provided, speculate on the possibility of strategy information being of some benefit. Importantly, when Banker et al. (2004) examined the impact of strategy information on the common measures bias, half the scorecard measures were strategically linked and half were not. In Experiment 1, we manipulate whether the measures are all strategically linked or only half are strategically linked, when strategy information is provided. We expect the common measures bias to persist when managers receive performance measures of which half are strategically linked and half are not (as was the case in Banker et al. 2004). The combination of non-linked items and strategically linked items means managers cannot match all performance measures to the strategy information provided, which increases the coordinative complexity and thus the cognitive effort required to complete the performance evaluation task. This is expected to encourage managers to look for an easier cognitive processing option. Divisional comparison focused on common measures is cognitively easier, as it allows the importance factor to cancel out (Slovic and MacPhillamy 1974, 174). This will result in the common measures bias. In contrast, when strategy information is provided and all measures are strategically linked, the strategic context for each performance measure can be identified by managers (Ariely 2008). As managers are able to conclude on the strategic relevance of each performance measure, cognitive complexity is reduced and managers are more likely to make use of all cues in evaluating performance (Slovic and MacPhillamy 1974), i.e., the common measures bias is eliminated. Hence, we hypothesize that: H1: In a situation where managers receive strategy information, those reviewing fully linked performance measures will make a more balanced evaluation of performance (reflecting both unique and common measures) than those reviewing half linked and half non-linked performance measures.

5 The Balanced Scorecard: The Effect of Strategy Information on Judgments 85 Overview of Experiment 1 To develop this case, we referred to the case materials from Lipe and Salterio (2000) and Banker et al. (2004), as well as the Kenyon Stores internal scorecard (Kaplan and Norton 1996, 111) from which the Lipe and Salterio (2000) case was developed. The Lipe and Salterio (2000) and Banker et al. (2004) cases included some performance measures that were not explicitly linked to the specific strategic objectives for each division outlined in the tasks. 2 In the current experiment, some managers review fully linked performance measures; thus, we adapted these cases to include the additional strategically linked measures required. The two divisions referred to specifically in the case are General Jeans, a retailer of youth fashion apparel, and Captain Kids, an established children s clothing retailer. Consistent with Banker et al. (2004), we have two profit-focused strategic themes for each division. The strategy map developed for General Jeans is presented on the left side of Figure 2. For Captain Kids, the Improve price mark-ups side of the strategy map is the same (common theme), but the Grow new-store sales side is replaced by Grow existing-store sales (unique theme). On the right side of Figure 2 are the 16 strategically linked measures for General Jeans. The half linked/half nonlinked measures for both divisions are shown in Figure 3. Participants A total of 92 Executive M.B.A. students from a major Australian university completed the experiment. All were enrolled in an introductory accounting course. Participants had an average of 11.2 years of full-time work experience, and 65 percent were male. Research Design This experiment employs a (2) mixed fractional factorial design with data collected for six of the eight possible cells. The three between-subjects factors are extent of strategic linkage, provision of strategy information, and divisional out-performance pattern, while the within-subjects factor is division. These independent variables are combined in six cells from the above design, comprising two sets of four cells with two overlapping cells of data (shown as Cells 3 and 4 in Figure 4). Hence, our results are analyzed using two (2) designs (see Kadous et al. 2005). As shown in Figure 4, Cells 1, 2, 3, and 4 are used to replicate the Banker et al. (2004) results (see Panel B), while Cells 3, 4, 5, and 6 are used to test H1 (see Panel C). Extent of strategic linkage is manipulated at two levels, being: (1) half linked/half nonlinked where half the measures in the balanced scorecard are linked to the divisional strategy, while the other half are not (this was the design employed by Banker et al. 2004); and (2) fully linked where all the measures in the scorecard are linked to the divisional strategy. The second between-subjects variable is provision of strategy information, which is manipulated at the same two levels as Banker et al. (2004), being: (1) no strategy information no description or map of strategic objectives are provided; and (2) detailed strategy information consisting of a narrative description of divisional strategic objectives and a strategy map communicating causal linkages between strategic objectives provided with the 2 Banker et al. s (2004) experimental design required that each scorecard contain both strategically linked and non-strategically linked measures, where each perspective contained a common and a unique measure linked to the division s strategy, and another common and unique measure that was not linked to the strategy. Lipe and Salterio (2000) indicated that the divisional performance measures provided in their task captured the strategies of those divisions, but the linkage was considerably less explicit for participants compared with the strategy information treatments in the present study and Banker et al. (2004).

6 86 Humphreys and Trotman FIGURE 2 The Fully Linked Strategy Map and Balanced Scorecard for General Jeans Employed in Experiment 1 balanced scorecard (see Figure 2 for General Jeans strategy map). We manipulate this variable to replicate the Banker et al. (2004) finding that the common measures bias persists when measures are half linked/half non-linked, irrespective of whether or not strategy information is provided. This finding offers additional assurance that our participants and task are sufficiently similar to this earlier research to allow comparable results. The third between-subjects variable is the divisional out-performance pattern. Divisional outperformance refers to the situation where the performance of one division consistently exceeds the performance of another division across a particular set of performance measures (Lipe and Salterio 2000). 3 Two patterns of divisional out-performance where the common measures bias is observable are manipulated in the current experiment, being: (1) Captain Kids on common measures (and Captain Kids out-performs General Jeans on unique measures); and (2) Captain Kids on unique measures (and Captain Kids out-performs General Jeans on common measures). 3 Lipe and Salterio (2000), and much of the subsequent research in this area (Dilla and Steinbart 2005; Roberts et al. 2004), manipulated all combinations of divisional out-performance on common measures and on unique measures, resulting in four combinations for out-performance. However, as Libby et al. (2004, 1082) highlight, the only combination of results that allows for the possibility of common measures bias is where performance on common measures favors one division and performance on unique measures favors the other division and vice versa.

7 The Balanced Scorecard: The Effect of Strategy Information on Judgments 87 FIGURE 3 Common and Unique Performance Measures for the Half Linked/Half Non-Linked General Jeans and Captain Kids Balanced Scorecards in Experiment 1 Type Measure Financial Perspective Common (linked) Sales margin Common (non-linked) Debt-to-assets ratio Unique (linked): General Jeans Percentage of sales from new stores Unique (non-linked): General Jeans Operating cash flow as a percentage of sales Unique (linked): Captain Kids Average sales growth per store Unique (non-linked): Captain Kids Inventory turnover Customer Perspective Common (linked) Customer satisfaction rating Common (non-linked) Price relative to competitors price Unique (linked): General Jeans Sales to new customers Unique (non-linked): General Jeans Number of new items for which first to market Unique (linked): Captain Kids Average sale per customer Unique (non-linked): Captain Kids Number of credit card customers per store Internal Business Process Perspective Common (linked) Retail Industry Association sales staff knowledge and attitude rating Common (non-linked) Time to process customer returns Unique (linked): General Jeans Advertising campaign awareness rating (/10) Unique (non-linked): General Jeans Vendor rating (/30) Unique (linked): Captain Kids Brand recognition rating Unique (non-linked): Captain Kids Number of stock-outs Learning and Growth Perspective Common (linked) Employee satisfaction Common (non-linked) Store computerization Unique (linked): General Jeans Retail experience of new marketing managers (years) Unique (non-linked): General Jeans Sales staff with university degrees Unique (linked): Captain Kids Average hours of brand manager training Unique (non-linked): Captain Kids Employee suggestions per year The within-subjects factor is division (General Jeans and Captain Kids) with participants evaluating the performance of both divisional managers. Potential order effects related to the presentation of the performance measures are controlled, with the order of common and unique measures alternated in each perspective and counterbalanced for half the participants in each cell. 4 Potential order effects related to division are also controlled, with half the participants completing the General Jeans performance evaluation first and the remaining participants beginning with the Captain Kids performance evaluation. 5 4 Half the participants in each cell receive a balanced scorecard with the following order of common (C) and unique (U) measures: Financial perspective C 1 U 1 C 2 U 2 ; Customer perspective U 1 C 1 U 2 C 2 ; Internal business process perspective C 1 U 1 C 2 U 2 ; and Learning & Growth perspective U 1 C 1 U 2 C 2. For the remaining participants in the cell, the order in each perspective is reversed hence: Financial perspective U 1 C 1 U 2 C 2 ; Customer perspective C 1 U 1 C 2 U 2 ; Internal business process perspective U 1 C 1 U 2 C 2 ; and Learning & Growth perspective C 1 U 1 C 2 U 2. 5 No order effects were observed for either division or performance measure presentation (p. 0.10).

8 88 Humphreys and Trotman FIGURE 4 Experimental Design for Experiment 1 Panel A: Overall Experimental Design No strategy information Detailed strategy information Captain Kids on common measures Captain Kids on unique measures Captain Kids on common measures Captain Kids on unique measures Half Linked/Half Non-Linked Measures Cell 1 Cell 2 Fully Linked Measures NA NA Cell 3 Cell 5 Cell 4 Cell 6 Panel B: Design for Replication of Banker et al. (2004) a General Jeans Out-Performs Captain Kids on Common Measures General Jeans Out-Performs Captain Kids on Unique Measures No strategy information Cell 1 Cell 2 Detailed strategy information Cell 3 Cell 4 Panel C: Design for Test of H1 b Captain Kids on common measures Captain Kids on unique measures Half Linked/Half Non-Linked Measures Fully Linked Measures Cell 3 Cell 5 Cell 4 Cell 6 a In Panel B, both divisions have half linked/half non-linked measures. b In Panel C, all participants receive detailed strategy information. Research Instrument Consistent with Lipe and Salterio s (2000) case, our case describes the balanced scorecard concept (including listing the four categories of measures) and indicates that the firm s Chief Financial Officer recently attended a symposium on the balanced scorecard and has decided to implement it in two of the company s major divisions being General Jeans and Captain Kids. A brief overview of the two divisions is provided. Participants are then presented with a balanced scorecard for the first division, which provides annual actual versus target performance (together with percent better than target) for 16 measures. Half the participants also receive the detailed strategy information. The approach to assigning performance targets developed by Lipe and Salterio (2000) was adopted here with performance data selected to ensure common and unique measures had the same level of performance above target in total. Changes were then made to the Lipe and Salterio (2000) percent better than target figures for some performance

9 The Balanced Scorecard: The Effect of Strategy Information on Judgments 89 measures to ensure that not only did the common and unique items have the same level of performance above target in total, but this was also the case for the strategically linked and nonlinked items. 6 All participants are asked to evaluate the manager s performance in delivering the strategy of the first division on a scale from 0 100, with seven descriptive labels ranging from Reassign to Excellent performance (Libby et al. 2004; Lipe and Salterio 2000, 2002). The balanced scorecard (and detailed strategy information, where applicable) is then presented for the second division, and participants are asked to evaluate the divisional manager s performance. Consistent with Libby et al. (2004), the dependent variable we measure is the difference in participant evaluations for the two divisions (where a significant positive difference from 0 indicates a common measures bias). Results We begin by replicating a Banker et al. (2004) finding that the common measures bias exists when measures are half linked/half non-linked, irrespective of whether or not strategy information is provided. We used a (2) ANOVA 7 (see Panel B of Figure 4) to test whether there are any significant differences in the use of common and unique measures for those with no strategy information compared to those with detailed strategy information, when half the performance measures are strategically linked and half are not. Consistent with Banker et al. (2004), no significant difference is found between these two treatments with respect to the use of common and unique measures. The common measures bias is exhibited by both treatments with the mean difference for both the no strategy information and detailed strategy information treatments significantly different from zero (mean = 5.4, t = 3.392, p = and mean = 3.4, t = 2.134, p = 0.041; two-tailed, respectively), as shown in Panel A of Table 1. Accordingly, in Panel B of Table 1, we find a significant difference for OutPerformance (F = , p, 0.001) indicating the greater weight placed on common measures compared to unique measures for these treatments, and no significant interaction between StrategyInformation and OutPerformance (F = 0.717, p = 0.401). These results are consistent with Banker et al. (2004) who find the common measures bias exists for both the no strategy information and detailed strategy information treatments, when managers receive half linked and half non-linked measures. This finding provides additional assurance that the participants and task in the present study are sufficiently similar to prior research to allow comparable results. To test H1, we used a (2) ANOVA (see Panel C of Figure 4) to investigate the differences between the half linked/half non-linked treatment and the fully linked treatment, when strategy information is provided to both treatments. The mean difference in divisional performance evaluations for the half linked/half non-linked treatment is 3.4, which is significantly different from 6 The sum of performance above target across all common measures remained at for the better division (with total above target performance of for both common, linked and common, non-linked measures) and for the worse division (comprising total above target performance of for common, linked measures, and for common, non-linked measures). The sum of excess performance across all unique measures was between and for the better division (comprising total above target performance for unique, linked measures of between and 42.54, and for unique, non-linked measures of between and 42.54) and between and for the worse division (comprising total above target performance for unique, linked measures of between and 25.97, and for unique, non-linked measures of between and 25.98). 7 As the dependent variable reflects the combined effect for the two divisions (difference in divisional performance evaluations for the two divisions), the results of our (2) ANOVA are presented as a ANOVA in Panel B of Table 1. This also applies to the results of the (2) ANOVA used to test H1 (see Panel C of Table 1).

10 90 Humphreys and Trotman TABLE 1 Experiment 1: Experimental Results for Managers Divisional Performance Evaluations with Half Linked/Half Non-Linked Measures (for both No Strategy Information and Detailed Strategy Information) versus Fully Linked Measures Panel A: Mean Evaluations (Standard Deviations) of the Performance of the General Jeans and Captain Kids Divisional Managers by Experimental Condition a Detailed Strategy Information No Strategy Information Divisional Out-Performance Pattern Half Linked/Half Non-Linked Measures Fully Linked Measures Half Linked/Half Non-Linked Measures Common Measures Favor General Jeans (9.4) (10.9) (8.7) Favor Captain Kids (9.8) (13.3) (9.0) Average (9.4) (12.0) (8.6) Unique Measures Favor General Jeans (10.6) (13.2) (11.0) Favor Captain Kids (9.8) (10.9) (12.3) Average (10.0) (11.9) (11.3) Difference b Average Common less Unique (8.8) (9.2) (8.9) t = t = t = p = p = p = n = 30 n = 30 n = 32 Panel B: Results of a ANOVA on the Difference in Managers Divisional Performance Evaluations by Experimental Condition (for Treatments with Half Linked/Half Non-Linked Measures only) df SS F p-value StrategyInformation OutPerformance , StrategyInformation 3 OutPerformance Error 58 Panel C: Results of a ANOVA on the Difference in Managers Divisional Performance Evaluations by Experimental Condition (for Treatments with Detailed Strategy Information) df SS F p-value StrategicLinkage OutPerformance StrategicLinkage 3 OutPerformance Error 56 (continued on next page)

11 The Balanced Scorecard: The Effect of Strategy Information on Judgments 91 TABLE 1 (continued) a All participants made two performance evaluation judgments, one for the divisional manager of General Jeans, and the other for the divisional manager of Captain Kids. In half the experimental conditions, performance on common measures favors General Jeans, while performance on unique measures favors Captain Kids. For the remaining experimental conditions, performance on common measures favors Captain Kids, while performance on unique measures favors General Jeans. b A significant positive difference from 0 indicates a common measures bias (two-tailed tests used). Variable Definitions: StrategyInformation = whether strategy information is provided (no strategy information; detailed strategy information); StrategicLinkage = extent to which the performance measures are linked to strategy (half linked/half non-linked, or fully linked); and OutPerformance = pattern by which the performance of one division exceeds the other division. 0(t = 2.134, p = 0.041; two-tailed), indicating that a significant common measures bias exists in this condition (see Panel A of Table 1). Also, as shown in Panel A of Table 1, the mean difference in divisional performance evaluations for the fully linked treatment is 1.4 (t = 0.820, p = 0.419; two-tailed), indicating the common measures bias does not exist. Consistent with the above results, a significant interaction is found between StrategicLinkage and OutPerformance, as shown in Panel C of Table 1 (F = 4.132, p = 0.047). Thus H1 is supported. Overview and Hypothesis Development EXPERIMENT 2 In Experiment 1, the only treatment where the common measures bias was eliminated was for those managers receiving fully linked measures and detailed strategy information. However, because the managers receiving fully linked measures also received strategy information, it is not possible to tell whether the absence of the common measures bias was entirely due to the choice of measures or whether the presentation of strategy information is also required to eliminate the bias. To resolve this uncertainty, in Experiment 2 we compare a treatment with detailed strategy information with a no strategy information treatment, when all performance measures are strategically linked. In addition, to expand the generalizability of our findings, Experiment 2 replaces the common, profit-based strategic theme with an asset productivity-focused theme (the unique, profit-based strategic themes are unchanged) to test the generalizability of earlier studies to a change in strategic theme focus. We examine this asset productivity-focused common strategic theme in a situation where the common measures bias has previously been found, namely, when no strategy information is provided. While Banker et al. (2004) included some performance measures that were not linked to the strategic objectives of each division in their task, for the fully linked treatment in Experiment 1, all the performance measures were linked to the divisional strategies provided. In situations where all measures are strategically linked, the provision of strategy information will assist managers to identify the strategic relevance of each performance measure (Ariely 2008) and accordingly reduce the cognitive complexity of the evaluation task. In contrast, those managers without strategy information will be unable to match all performance measures to the strategy. The need for additional cognitive effort will result in the use of a simplifying strategy that is, to rely on measures that are common to both divisions. Thus, we hypothesize the following:

12 92 Humphreys and Trotman H2: Managers who receive strategy information will produce a more balanced evaluation of performance (reflecting both unique and common measures) than managers who receive no strategy information. Finally, in the cases developed by Lipe and Salterio (2000) and Banker et al. (2004), managers were asked to consider common and unique performance measures based on two profit-focused themes. However, in the extract from Kenyon Stores internal scorecard (Kaplan and Norton 1996) from which these cases were developed, one strategic theme focuses on increasing revenue via greater market penetration (that is, profit-focused); while another focuses on increasing productivity via greater asset utilization (that is, asset productivity-focused). In practice, it is not uncommon for major retail organizations to have an asset productivity focus, in addition to a profit-focused strategic theme. 8 To examine whether the common measures bias finding is robust across the focus of the common strategic theme, we consider the following research question in a situation where neither treatment receives any strategy information: RQ1: Do managers reviewing performance measures based on one profit-focused and one asset productivity-focused theme make a more balanced evaluation of performance (reflecting both unique and common measures) than managers reviewing performance measures based on two profit-focused themes? Method A total of 103 Australian Executive M.B.A. students completed the follow-up experiment. All were enrolled in an introductory accounting course. 9 Participants had an average of 10.9 years of full-time work experience. We employ a (2) mixed fractional factorial design with data collected for six of the eight possible cells. The three between-subjects factors are provision of strategy information, focus of the common theme and divisional out-performance pattern, and the within-subjects factor is division. These independent variables are combined in six cells from the above design, developed to investigate H2 and RQ1. The design comprises two sets of four cells with two overlapping cells of data (shown in Figure 5 as Cells 1 and 2). Consistent with Kadous et al. (2005), our results are analyzed using two (2) experimental studies. Cells 1, 2, 3, and 4 of Figure 5 (see Panel B) are used to test H2; while cells 1, 2, 5, and 6 are used to examine RQ1, in a situation where participants do not receive strategy information. Provision of strategy information is manipulated at the same two levels, as was the case in Experiment 1 ( no strategy information and detailed strategy information ). The second between-subjects variable is focus of the common theme, which is manipulated at two levels, being: (1) asset productivity-focused common theme the common measures in the balanced scorecard provided are based on an asset productivity-focused strategic theme; and (2) profit-focused common theme consistent with Experiment 1 and extant research, the common measures in the balanced scorecard are based on a profit-focused strategic theme. The strategic objectives and performance measures for the profit-focused common theme condition receiving detailed strategy information are shown in Figure 2. For participants in the asset productivity-focused common theme condition receiving detailed strategy information, the strategic theme culminating in 8 Retailers with asset productivity-focused strategies include Gap Inc. (2007 annual report) and Target Corporation (2006 annual report) in North America; and David Jones Limited and Just Group Limited (2007 annual reports) in Australia. 9 Participants completing Experiment 2 were enrolled in the same introductory accounting course, but one year apart from those who completed Experiment 1.

13 The Balanced Scorecard: The Effect of Strategy Information on Judgments 93 Improve price mark-ups (see Figure 2) was replaced with a strategic theme culminating in Improve asset utilization. 10 The third between-subjects manipulation is the divisional out-performance pattern (on either common or unique measures); this is the same independent variable as Experiment The withinsubjects factor of division (where participants evaluate the performance of both the General Jeans and Captain Kids divisional managers) also remains the same as Experiment 1. Consistent with Experiment 1, we control for potential order effects relating to the presentation of the common and unique performance measures. 12 Participants complete the same task as Experiment 1. Results We used a (2) ANOVA 13 to test H2, comparing the difference in divisional scores for the two strategy information treatments (see Panel B of Figure 5). The mean difference in divisional performance evaluations for the no strategy information and detailed strategy information treatments are 3.6 and 1.1, respectively (see Panel A of Table 2). The mean difference for the no strategy information treatment is significantly different from zero (t = 2.223, p = 0.033; twotailed), indicating that a significant common measures bias exists (see Panel A of Table 2). There is no common measures bias for the detailed strategy information treatment (t = 0.831, p = 0.412; two-tailed). The absence of the common measures bias for the treatment provided with detailed strategy information, when all the measures are strategically linked, is consistent with our findings in Experiment 1. Consistent with the above results, a significant interaction is found between StrategyInformation and OutPerformance, as shown in Panel B of Table 2 (F = 4.786, p = 0.032). Hence, H2 is supported. To address RQ1, we compared the difference in divisional scores for the asset productivityfocused and profit-focused common themes (as per Panel C of Figure 5). The common measures bias is exhibited in both treatments with the mean difference for both the asset productivity-focused and profit-focused common theme treatments significantly different from zero (mean = 3.6, t = 2.223, p = and mean = 5.3, t = 3.728, p = 0.001; two-tailed, respectively) as shown in Panel A of Table 2. From our (2) ANOVA on the differences in managers divisional performance evaluations (with CommonTheme and OutPerformance as independent variables), we 10 For participants in the asset productivity-focused common theme treatment, the following common measures were provided in each perspective: (1) Financial Asset turnover (times) and Inventory turnover (times); (2) Customer Mystery shopper product availability rating (/10) and Customer satisfaction rating; (3) Internal Business Process Outofstockratioand Correct deliveries from suppliers; and (4) Learning and Growth Information systems projects delivered on time and New IT investment ($m). All percent better than target figures were the same as for the profit-focused common theme. Target and actual performance figures also generally remained the same, albeit with two exceptions first, for Asset turnover (times) and Inventory turnover (times), the target and actual figures changed from percentages (e.g., 12.0 percent) to whole numbers (e.g., 12.0 times); second, for New IT investment ($m), one decimal place was moved (e.g., $6.60m became $66.0m) to provide realistic targets. 11 The approach to assigning performance targets developed by Lipe and Salterio (2000) was adopted here with performance data selected to ensure common and unique measures had the same level of performance above target in total. The sum of performance above target across all common measures was for the better division and for the worse division; while the sum of excess performance across all unique measures was between and for the better division and between and for the worse division. As was the case in Lipe and Salterio (2000), the slight variation for unique measures results from different units for the measures in each division. 12 No order effects were observed for either division or performance measure presentation (p. 0.10). 13 As the dependent variable reflects the combined effect for the two divisions (difference in divisional performance evaluations for the two divisions), the results of our (2) ANOVA are presented as a ANOVA in Panel B of Table 2. This also applies to the results of the (2) ANOVA used to test RQ1 (see Panel C of Table 2).

14 94 Humphreys and Trotman FIGURE 5 Experimental Design for Experiment 2 Panel A: Overall Experimental Design No strategy information Detailed strategy information Captain Kids on common measures Captain Kids on unique measures Captain Kids on common measures Captain Kids on unique measures Asset Productivity- Focused Common Theme Profit-Focused Common Theme Cell 1 Cell 5 Cell 2 Cell 6 Cell 3 Cell 4 NA NA Panel B: Design for Test of H2 a General Jeans Out-Performs Captain Kids on Common Measures General Jeans Out-Performs Captain Kids on Unique Measures No strategy information Cell 1 Cell 2 Detailed strategy information Cell 3 Cell 4 Panel C: Design to Address RQ1 b Captain Kids on common measures Captain Kids on unique measures Asset Productivity-Focused Common Theme Profit-Focused Common Theme Cell 1 Cell 5 Cell 2 Cell 6 a In Panel B, both divisions share an asset productivity-focused common theme. b In Panel C, all participants receive no strategy information. find a significant difference for OutPerformance (F = , p, 0.001) in Panel C of Table 2. This indicates the greater weight placed on common measures compared to unique measures across treatments. With regard to our research question, there is no significant interaction between CommonTheme and OutPerformance (F = 0.638, p = 0.427), as shown in Panel C of Table 2. Thus, we find no significant difference for the use of common and unique measures between the asset productivity-focused and profit-focused common themes. DISCUSSION AND CONCLUSION A well-established finding related to the use of balanced scorecards for performance evaluation is the existence of the common measures bias (Banker et al. 2004; Libby et al. 2004; Lipe and Salterio 2000). The limited use of unique measures in performance evaluation detracts from the benefits of using balanced scorecards for these judgments. We test the generalizability of the

15 The Balanced Scorecard: The Effect of Strategy Information on Judgments 95 TABLE 2 Experiment 2: Experimental Results for Managers Divisional Performance Evaluations with No Strategy Information (for both an Asset Productivity-Focused and Profit-Focused Common Theme) versus Detailed Strategy Information (all with Fully Linked Measures) Panel A: Mean Evaluations (Standard Deviations) of the Performance of the General Jeans and Captain Kids Divisional Managers by Experimental Condition a Divisional Out-Performance Pattern Asset Productivity-Focused Common Theme No Strategy Information Detailed Strategy Information Profit-Focused Common Theme No Strategy Information Common Measures Favor General Jeans (7.6) (10.7) (7.3) Favor Captain Kids (10.2) (11.0) (9.9) Average (8.8) (10.7) (8.7) Unique Measures Favor General Jeans (9.9) (11.4) (12.6) Favor Captain Kids (10.1) (12.7) (8.7) Average (9.9) (11.9) (10.7) Difference b Average Common less Unique (9.5) (8.0) (8.3) t = t = t = p = p = p = n = 35 n = 34 n = 34 Panel B: Results of a ANOVA on the Difference in Managers Divisional Performance Evaluations by Experimental Condition (for Treatments with an Asset-Productivity Focused Common Theme) df SS F p-value StrategyInformation OutPerformance StrategyInformation 3 OutPerformance Error 65 Panel C: Results of a ANOVA on the Difference in Managers Divisional Performance Evaluations by Experimental Condition (for Treatments with No Strategy Information) df SS F p-value CommonTheme OutPerformance , CommonTheme 3 OutPerformance Error 65 (continued on next page)

16 96 Humphreys and Trotman TABLE 2 (continued) a All participants made two performance evaluation judgments, one for the divisional manager of General Jeans and the other for the divisional manager of Captain Kids. In half the experimental conditions, performance on common measures favors General Jeans, while performance on unique measures favors Captain Kids. For the remaining experimental conditions, performance on common measures favors Captain Kids, while performance on unique measures favors General Jeans. b A significant positive difference from zero indicates a common measures bias (two-tailed tests used). Variable Definitions: StrategyInformation = whether strategy information is provided (no strategy information; detailed strategy information); CommonTheme = focus of common theme (asset-productivity focused common theme or profit-focused common theme); and OutPerformance = pattern by which the performance of one division exceeds the other division. findings on the common measures bias by examining under what circumstances the provision of strategy information, and the extent to which measures are strategically linked, eliminates the common measures bias on performance evaluation tasks. In previous research, the tasks included some performance measures that were not explicitly linked to the specific strategic objectives for each division (Banker et al. 2004; Libby et al. 2004; Lipe and Salterio 2000). Of particular interest to the present study is the finding of Banker et al. (2004) that the common measures bias persists even when strategy information is provided. In Experiment 1, we manipulate whether all or only some of the scorecard measures are strategically linked. We find that provision of strategy information eliminates the common measures bias, but only when all measures are strategically linked. That is, the Banker et al. (2004) finding that the common measures bias persists when strategy information is provided only replicates in a setting in which some non-linked measures are included in the scorecard. When all measures are strategically linked, the common measures bias is eliminated. This is an important finding, because while all the measures in a balanced scorecard are expected to be linked to strategy (Kaplan and Norton 1996, 2004, 2008), in practice, all measures in multiple performance measurement systems (including the balanced scorecard) are not always strategically linked (e.g., Ittner et al. 2003b). By examining the effects of this variation in practice on the common measures bias, our findings can contribute to the effective development and implementation of balanced scorecards and guide their use for judgment purposes. We conduct Experiment 2 to ensure the results of Experiment 1 are driven by the combination of the provision of strategy information and all measures being fully linked, not just by the latter alone. All participants in Experiment 2 are provided with performance measures that are all strategically linked. Consistent with Experiment 1, the common measures bias is eliminated when strategy information is provided. However, when strategy information is not provided, the common measures bias exists. The benefits of effectively communicating strategy information when using a balanced scorecard have been discussed in the literature (Nørreklit and Mitchell 2007; Kaplan and Norton 2000, 2004; Malina and Selto 2001; Nørreklit 2000; Ittner and Larcker 1998). However, this is the first time that benefits from providing strategy information have been shown empirically with respect to the common measures bias. Experiment 2 also tests the generalizability of the common measures bias finding to the focus of common theme. While all previous research and Experiment 1 provided performance measures based on profit-focused themes, we note that the original Kenyon Stores case, and many major retailers in practice, also use an asset productivity-focused strategy. We find the common measures bias exists in a setting without strategy information, irrespective of whether the common