The determinants of subjective measurement of performance: A longitudinal empirical study. Abstract

Transcription

1 The determinants of subjective measurement of performance: A longitudinal empirical study Abstract The study examines the determinants of the implementation of a branch sales manager incentive system with multiple performance measures. We specifically examine the following question: What are the determinants of subjective measurement of performance in bonus determination? Our research examines these relationships with proprietary data from a Chinese manufacturer with 28 sales branches using a multi period experimental context (5 successive years of changes in the incentive scheme 1998, 1999, 2000, 2001 and 2002). The results using a longitudinal method design provide a unique contribution to the management control literature.

2 The determinants of subjective measurement of performance: A longitudinal empirical study 1. Introduction To overcome the limitations of accounting-based reward systems, many firms include qualitative (subjective) measures of performance in evaluating and rewarding employees. The use of subjective performance measures are aimed at (i) mitigating formula bonus distortions that arise from the incompleteness of quantitative measures or (ii) to reduce risk that is associated with uncertainty such as the need for cooperation in an interdependent environment, or with having difficult performance targets to meet (Gibbs et al., 2004). Yet the jury is still out as to the actual use of and the specific benefits of subjective performance measures. Specifically, in a report of a conference board study (Itnner and Larker, 2003a, p.93) 55% of senior executives surveyed said they were unwilling to measure qualitative results because of a, lack of trust in measures that were unproven and therefore subject to favoritism and bias. Only a few studies have examined the use of subjectivity in incentive contracts (Bushman et al. 1996; Hayes and Schaefer, 2000; Murphy and Oyer, 2003), leading Ittner et al. (2003b) to suggest that more research effort is needed to establish how and why subjective performance measures are used, and what their consequences are. This study helps to fill this gap by providing an examination of the determinants of subjective measurements in incentive contracts of branch office sales managers in a unique dataset of up to 26 branch offices of a large telecom equipment manufacturer over a period of 5 years (1998 to 2002). The firm adopted a formula bonus plan for its branch sales managers and their staff which consisted of four objective and two subjective 1

3 components. We examine the following research question: What are the determinants of subjective measurement of performance in bonus determination? We define subjective measurement of performance as judgments made about qualitative performance measures (Moers, 2005). The findings indicate that both redundant factors (eg. objective measures) and informative factors (eg. branch office employee growth and branch manager experience) influence the subjective measurement of performance. We find that the level of subjective measurement ratings is positively related to: (1) objective measures (cost control) (2) external sources of measurement such as customer satisfaction survey and provincial wage level growth, and (3) informative sources of measurement such as branch office employee growth and branch manager experience. We explain these findings as meaning that for faster growing branch offices subjectivity can play a higher strategic communication and incentive role in the firm. Similarly for newly arrived managers at a branch, subjectivity can play a role of compensating the manager for the short term risk they undertake in the new environment. The performance measurement and compensation literature provide the background for this study. This literature has recently examined the factors that determine the level of subjectivity in compensation contracts (Murphy & Oyer, 2001; Gibbs et al., 2003). Ittner et al. (2003) examined how subjectivity is incorporated into incentive schemes and found that psychology-based explanations are potentially more relevant in understanding measurement practices than economics-based explanations. For example, Ittner et al. (2003) find little evidence for the informativeness hypothesis. Instead they find evidence to support psychology-based explanations, that greater weight 2

4 was placed on quantitative, outcome-oriented financial measures that were a feature of previous plans. By undertaking a longitudinal analysis of a single firm, Ittner et al. (2003) were also one of the first to highlight the evolutionary nature of compensation plans, which they note, is an important issue that is missing in pure cross-sectional studies. Gibbs et al (2004) examined the determinants of subjective measurement in the determination of car sales manager bonuses and found support for economic based explanations such as informativeness (formula bonus incompleteness) and risk reduction (eg, managing interdependencies and difficult performance targets). However, they failed to find support for other explanations such as managing goal congruence and gamming associated with formula bonus distortions, and in managing risk associated with controllability (noise in quantitative measures) of the environment. The contribution of this study is two fold: First, following Ittner et al (2003) this study provides evidence on how subjectivity in performance measurement is incorporated into incentive compensation using a more heterogeneous multi-period context ( ). Unlike Ittner et al (2003) the firm in our study manufactures and sells products in a context, which is more heterogeneous in terms of sales regions (at different levels of growth and competition), manager labor market (different experience and expectations). Thus, there is a greater potential benefit in using subjective measures to enhance goal congruence and unity in the firm compared with the use of objective performance measures. The bank in Ittner et al s case has a more homogeneous context in terms of the different sales regions in the USA and the labor market which can reduce the potential benefits of using subjective measures to differentiate branch manager performance over 3

5 and above that provided by objective performance measures. Nagar (2002) for example note that banks tend to have a homogeneous performance measurement system. Second, in contrast to recent large scale studies of incentive contracts (Gibbs et al 2004; Murphy and Oyer, 2001), our research examines these relationships with time series data from up to 28 sales branches using a multi period experimental context (5 successive years of data on the incentive scheme This helps to overcome the endogeniety problem which has been frequently cited as a weakness of many studies that employ cross-sectional analysis (Luft and Shields, 2003; Ittner and Larker, 2001). The next section describes the firm context and incentive plan. The following section proposes two sets of hypotheses one based on psychological explanations and one based on economic explanations. The research methods and measurement of the variables are then explained, followed by the results. The paper concludes with a discussion of the findings. 2. Research Site and Incentive Plan CTI is a major producer and supplier of telecom equipment in China. The functional branch managers in the 28 branches of CTI expend considerable effort in managing customer service personnel, promoting and obtaining sales and account management. The telecom industry in China is a growth industry with hundreds of different manufacturers and suppliers of components to firms throughout China. Two factors were critical to CTI s profit growth strategy: increasing reliance on sales of higher margin products and cash collection. First, CTI is one of many companies that compete to serve the customer and since all of CTIs customers are State-owned enterprises the 4

6 number of customers does not grow dramatically. Therefore CTI has to rely on the sales of new products with increasing numbers of features. Second, since the state-owned enterprises are a quasi-government entity and have monopoly status as telecom providers in each province, each of the branch offices of CTI needs to do business with the same group of customers year in and year out. The actual number of customers or representatives of the State-owned enterprises numbers around 200 for all of China. As a result, CTI relies heavily on sales to higher quality (paying) customers. With increasing introduction of new products and growth in sales these two factors drove CTI to change the incentive plan to direct management effort towards these objectives. Incentive plan In 1998, CTI replaced a simple financial performance based incentive plan with a more elaborate performance measurement and incentive plan. Before 1998, sales were the main criterion for determining the bonus (up to 40% of fixed salary). However this plan resulted in the run up of bad accounts payable consisting of poor paying customers. In addition, since 1998, more and more products were developed with increasing numbers of features that allowed CTI to gain higher margins from the units sold. The new incentive plan was developed by a team consisting of the financial vice president and the Chief executive officer. The purpose of the more elaborate plan was to direct managerial effort towards three areas of company operations: (i) product sales, (ii) customer development and maintenance, and (iii) leadership development. These three areas are reflected by the three types of performance measures, respectively: financial performance measures, non-financial objective measures and subjective performance 5

7 measures. Table 1 provides the bonus formula that was used in While the bonus formula included a direct sales formula based on objective measures (product sales), the general formula consisted of multiple measures ranging from objective (financial and non-financial) and subjective measures. Figure 1 provides an overview of the direct and indirect linkages of the incentive plan components and financial performance. ********************** Table 1 about here********************** ********************** Figure 1 about here********************** The general formula consists of a bonus of (yuan) 4000 plus a bonus of up to (yuan) 5000 that is determined by the awarding of up to 110 points. 45 points are determined by the financial performance measures, 30 points are awarded based on nonfinancial quantitative measures of service level components and up to 35 points are awarded based on subjective measures of management capability components. The 45 points awarded for financial performance measures are accounts receivable outstanding (40 points) and branch office costs (5 points). Accounts receivable is computed as the amount of days sales outstanding, while branch office costs involved a competitive ranking across branches with only half the branches being eligible for points towards the bonus formula on a decreasing scale (from middle to the least costs incurred). The 30 points awarded for non-financial quantitative performance measures are major incident complaints (15 points) and customer satisfaction (15 points). These two measures where considered an important indicator of the propensity of customers to either repurchase or to pay on time. In the China context, dissatisfied customers are more likely to extend the accounting receivable period out to 12 months or more. There is very little that CTI can 6

8 do to prompt payment and often CTI has to negotiate a settlement of up to 20% reduction of the debt in arrears to gain payment. The 35 points awarded for subjective performance measures are execution of routine maintenance (15 points) and overall integrated assessment (20 points). The inclusion of subjective measures reflects the need to balance sales managers attention between sales and service and the development of the firm as a whole such as branch development initiatives, cross-branch sharing and support, and information sharing with the head office. In particular, there was a need to focus and direct the middle of the range performing managers, those that are not the best performers in terms of objective outcomes, but which engender support from senior management to continue with the company because of other factors such as their ability to contribute to knowledge management and the social culture of the firm. The subjective evaluation is determined once each year by top management (the general manager and deputy general manager of sales). This evaluation could result in an increase or decrease of the bonus by as much as 20%. Table 2 provides the detail of the formula weightings assigned to the various components during each of the five years examined in this study. ******************** Table 2 about here******************** Multi-period changes to the incentive plan The incentive plan has been adjusted to reflect different priorities in terms of sales, cash collection, customer relationship development, information sharing and manager development. During the period new higher margin products were introduced, 7

9 and as a result the firm changed the weights among the financial performance measures to reflect managements desire to provide higher incentives for new (as opposed to old) product sales, since new products were more difficult to sell. Since customer cash collection is a serious problem in China one of the financial measures used was cash collected from new product sales (based on the number of day s accounts receivable). Similarly, senior management also believed that customer quality was critical to gaining cash collections from sales and changed the weights assigned to customer development and maintenance to reflect this priority. Customers frequently used the excuse of inadequate maintenance and support for not paying on time. At the extreme formal complaints about major incidents were recorded and accounted for in the bonus scheme. However, little attention was given to this record in the bonus formula (ie there was little variance in the points awarded to the branches as a result of major incident complaints and in some years no difference was noted across the branches on this factor). By directing the sales managers to spend more effort in customer relationship building (including maintenance and after sales service) and selling to higher quality (in terms of ability to pay) customers, there was a greater chance of being able to collect 100% of the debt from customer within a stipulated time. The direction of sales manager effort towards selling to higher quality paying customers reflected the type of customer (all of the customers are State-owned enterprises) and the little bargaining power CTI had to collect debts. While branch managers found it easier to sell to poorer quality paying customers, senior management believed that effort spent in collecting from poorer quality paying customers was very costly for the firm. Similarly, old products were viewed by managers to be easier to sell, however, senior managements strategy was to sell new 8

10 products, which had more features and brought in higher profit margin to the firm. The introduction of new products was also aimed at improving the company s image as a high tech company as well as a total solution provider. 3. Theory and hypotheses In this section we develop several hypotheses about the various determinants of the weighting given to subjective performance measurement in bonus determination - which formed part of the organization s sales and customer service strategy? Given the unique access to archival data we focus our analysis on several determinants of subjectivity: objective performance measures, external performance measures, branch office employee growth and branch manager experience. In addition to the incorporation of customer satisfaction and major incident complaints into the incentive scheme, several subjective performance measures were introduced (eg. proactivess, suggestions, communication, positive outlook). The element of subjectivity was built into the incentive bonus determination process as part of the firm s communication and manager development strategy. This component can influence as much as 20% of the bonus, while the incentive plan determines at least 80% of the bonus paid. There are two subjectivity components: branch office management and overall integrated assessment. The purpose of the branch office subjectivity component is to promote goal congruence with the head office such that sales, service and development activities of the 28 branch offices are consistent and congruent with head office strategy. Branch office management refers to the deputy general manager s perception of the attitude of the branch manager towards the daily management of the branch and the 9

11 branch manager s ability to work with senior management and with subordinates in the branch office. Typically, the deputy general manager of sales will visit the branch periodically (once every 6 months) unannounced to inspect the branch office operations. While alternative informative measures exist (such as frequency of communication and customer satisfaction) senior management believes that the subjective evaluation component was the best mechanism to promote this congruence. The subjective evaluation is also influenced by the frequency of communication. Maybe this gentleman has a better relationship to communicate with his superior. The subordinate lets his superior know everything he did which is not reflected by the incentive formula, while the other gentleman who tends to be quieter and does not know how to communicate what he did which is not reflected by the formula. This latter gentleman receives a lower subjective evaluation. He should have got more as per the formula but he failed to communicate. The purpose of the integrated subjectivity assessment is two fold. First, to encourage manager communication with the head office about sales, service and development activities that is not fully reflected in the incentive formula. This in turn helps in strategic planning and attention directing as more information is shared upwards from the sales front. The result is that branch managers can receive more or less than the bonus computed as per the incentive formula because of their willingness or unwillingness to communicate about matters that do not directly impact on the objective measures in the incentive plan. The general manager wants an element of control over the relationships with the branch managers and the subjectivity element is the mechanism used for this purpose. Some regional managers have better relationships with the general manager, thus information about the managers actions gets filtered unconsciously by the general manager. Thus, if the general manager thinks that this branch manager is capable then everything he does is good. If the manager makes a mistake then it does not reflect negatively on his evaluation. 10

12 The second purpose of the integrated assessment is to develop the managerial and leadership skills of the manager. At the time of the increased emphasis of subjective measures in 2001, top management felt that the firm had a bright future, thinking that the firm can become a member of the 50 largest firms in China. They felt that in the future they need a lot of people to be a leader, to lead teams and not just be a manager: We want people to be able to see everything. They should not just be able to make sales, but should be able to motivate people, organize things strategically, putting down their thoughts into paper, think logically, and express themselves in a coherent fashion. Thus we observe our subordinates to see who has this potential, who could be a leader in the future. Thus the subjective measures matter to the skill development in addition to motivate sales. The purpose is to broaden the skills of the manager into a well rounded team leader. 3.2 Theoretical determinants of subjectivity in bonus determination Following Ittner et al. (2003), we examine the theoretical determinants of the level of subjectivity component in terms of (i) Psychology and (ii) Economic-based expectations. First, the psychology-based literature predicts that greater weighting will be placed on objective versus subjective performance measures and that greater weighting will be placed on aggregated versus individual measures (Ittner et al., 2003; Lipe and Shapiro, 2003; Banker et al., 2004). This is explained by the satisfying behavior of the individual which means that their attention is likely to be focused more on attributes that are objective and are in aggregate. Ittner et al (2003) found support for this explanation in their study of a balanced scorecard system in a large retail bank. They found that greater attention was given to objective measures and aggregated measures of branch manager performance in the determination of their annual bonus. Based on this psychology-based explanation we should expect that the level of bonus determined by subjective measures 11

13 in the incentive scheme will be a function of objective measures in the performance measurement system such as sales, expenses, cash collections. H 1a : Ceteris paribus, bonuses based on subjective performance measures are positively related to objective performance measures (sales and cash collection). H 1b : Ceteris paribus, bonuses based on subjective performance measures are negatively related to objective performance measures (expenses). The psychology-based literature also suggests that greater weighting will be placed on performance measures that are generated by a score which is external to the firm. External sources which appear to be trustworthy (e.g. customer) can command the credibility that makes such source appear to be free from bias (i.e. objective). Therefore we will expect that the level of the bonus determined by subjective measures will be a function of externally generated measures of performance such as customer survey results. H 1c : Ceteris paribus, bonuses based on subjective performance measures are positively related to external performance measures (customer satisfaction). As part of the intention of the incentive plan is to retain good sales managers it was important that subjective weights play a compensatory role in those provinces which exhibited lower or negative average wage growth. In fast developing economies like China often there is an imbalance in the level of incentives paid and alternative job opportunities. This imbalance arises because of the different growth rates of the different provinces and the threat is that sales managers will move to provinces where wage levels are higher. Thus, in order to retain branch managers an important determinant of the weight of the subjective measure is the level of wages in each province. Thus we expect 12

14 that bonuses based on subjective performance measures will be negatively related to the wages level of each province. H 1d : Ceteris paribus, bonuses based on subjective performance measures are positively related to provincial wages level. From an economics (agency) perspective, the awarding of bonus based on subjective measures is likely to be driven by informativeness and the level of risk aversion of the agent. That is a shift in weight from solely objective measures (eg. financial performance) to subjective measures of performance will reduce the risk faced by the agent if these subjective measures are informative of the agent s actions (Baiman et al., 1995; Baker et al., 1998, 1994). Based on these economics explanations we should expect that the level of bonus determined by subjective measures will be a function of qualitative factors which are informative of agent effort, such as the complexity of the sales process (proxied by branch office employee growth) and the level of sales manager risk aversion (proxied by the amount of manager experience). The informativeness of subjective performance measures is likely to increase according to the growth in branch size. Faster growing branch offices have relatively greater communication and coordination problems compared with their slower growing counterparts.for example, new employees generally decrease the average skill level of the unit and time is needed to bring the average capability of the unit back to the same level through training and socialization. The level of control over the sales and support activities is also affected during the acculturation (staff orientation) process. Therefore greater scope exists for communication about events, factors that are not captured by the objective measures in the incentive system. For those faster growing branch offices, the 13

15 value of the incentive based on subjective evaluation is expected to be greater than their slower growing counterparts. Subjective bonuses play a role of compensating branch managers in the short term for managing employee growth. From this we expect the following. H 2a : Ceteris paribus, bonuses based on subjective performance measures are positively related to sales management process complexity (branch office employee growth). Since there is only one incentive plan structured for all of the branch offices, the impact of an incentive scheme that includes a mix of objective and subjective performance measures on manager effort is likely to be stronger for those branch managers that have less experience (more risk averse). Agency theory predicts that the incentive scheme will induce such managers to exert greater effort towards a range of activities that are not directly sales related, than their more experienced counterparts. Alternatively, there is an implicit expectation that managers with more expertise need to be motivated with rewards that are more tied to the manager s performance (merit-based). The use of more powerful incentives to retain these managers means that greater weight will be given to objective rather than subjective measures in bonus determination. Generally, experienced managers who exhibit lower risk aversion will be more willing to submit to bonus determinations based on objective/formulaic computations rather than subjective evaluations. For those less experienced branch managers the value of the incentive based on subjective evaluation is expected to be greater than their more experienced counterparts. The later branch managers typically view sales as their basic task and prefer incentives that carry a greater weight on their performance outcomes in terms of sales and cash 14

16 collections. Thus, the use of subjective performance measures is unlikely to have as great influence of redirecting the more experienced sales manager effort towards non-sales activities. As a result we expect that the lower performing branch offices to receive on average greater levels of bonuses based on subjective performance measurement. This works for the principal (the firm in this case) in the principal-agent relationship as well. For example, where the firm is more dependent on manager knowledge and skills, then it is more important to motivate the manager to act like an equity holder, than as a fixed claimant. (Deegan, 1997, p. 10). Manager experience has been found to have a positive influence on the adoption of formal incentives (Lambert et al. 1993, p. 445; Fisher & Govindarajan, 1992, p. 208). A similar expectation was empirically substantiated in a study of retail sales managers in a department store chain in USA (Banker, Lee, & Potter, 1996, p.202). From this we expect the following. H 2b : Ceteris paribus, bonuses based on subjective performance measures are negatively related to sales manager experience. 4. Data Collection Annual data were obtained for a period of five years (beginning 1998) for 11 to 28 branch offices of CTI. Sales, expense and bonus payment data were collected from the accounting system maintained at CTI s headquarters. Data on the non-financial measures were collected from the performance management database maintained by CTI. Industry growth and average wage data for each province was obtained from the China statistical markets yearbook ( ). This data was cross-checked with an independent survey of the telecommunications industry in China. Further qualitative data on the company strategy, manager beliefs about the incentive plan was collected from the 15

17 branch managers. Access to the managers was obtained through one of the firm s general managers and the key informant of this study. Numerous interviews were held with a key informant during the period 2001 to 2005 and this provided qualitative data on the strategy of the company and the rationale behind the incentive plan. Financial performance measures - The financial performance measures consist of sales (SALEGWTH) and branch office costs (COSTGWTH). SALEGWTH is computed as the amount of sales revenue obtained by each branch office divided by the number of employees, while COSTGWTH reflects the actual costs per employee of each branch office. Both of these measures were transformed using the natural log. This is consistent with Bryant et al. (2004) who undertook similar transformations of data in order to normalize the distributional properties of the data. Non-financial performance measures - The non-financial quantitative performance measures are accounts receivable collections (COLLECT), and customer satisfaction and incident complaints (CUSTOMER). Accounts receivable collections is computed as the average amount of day s sales collected within each accounting period. Customer satisfaction is obtained annually with a two page survey. The survey is returned directly to the head office. Similarly, although less frequent, major incident complaints by customers are recorded by the head office. These measures were considered an important indicator of the propensity of customers to either repurchase or to pay on time. In the China context, dissatisfied customers are more likely to extend the accounting receivable period out to 12 months or more. There is very little that CTI can do to prompt payment and often CTI has to negotiate a settlement of up to 20% reduction of the debt in arrears to gain payment. 16

18 Subjective performance measures - The subjective performance measures are branch office daily management (MANAGE) and overall integrated assessment (INTEGRATED). MANAGE refers to the steps taken by the branch manager to execute routine maintenance and to keep up to date records of customer service, sales and expense documentation, and other records of branch office activity such as employee attendance. This measure is evaluated by the deputy general manager of sales. INTEGRATED refers to the behavior of the branch office manager in terms of teamwork, cooperativeness, being proactive and innovative (eg. making suggestions, doing things beyond responsibilities and solving difficult problems). The measure is evaluated by the General Manager and Deputy General Manager of Sales The inclusion of subjective measures into the sales incentive scheme reflects the need to balance sales managers attention between sales and service and the development of the firm as a whole such as branch development initiatives, cross-branch sharing and support, and information sharing with the head office. These subjective evaluations are determined once each year by top management (the general manager and deputy general manager of sales). Branch office growth and manager experience - Branch office employee growth (EMPGWTH) refers to the growth in the number of sales employees that worked in each branch in each period. Manager experience (EXPERIENCE) refers to the change in the number of months of experience within a specific branch office. We operationalised this construct in terms of the learning curve that a manager faces when he goes to a new branch office for the first time. Thus, newly arrived branch managers will register higher levels in the change variable in their first year in the office (eg. The denominator equals 17

19 12 or less) compared to their second year (the denominator in the change equation is between 12 and 24 months). Wages The average wage level of workers in each province a branch is located is used as an indicator of the relative cost of the labor market. We calculated the change in this level from year to year (WAGEGWTH). Incentive formula dummy As discussed previously, the formula of the incentive system changed after We include a dummy in the regression model to reflect this. These changes, the focus of the hypotheses was on the relative weighting in bonus determination that was given to subjective performance measures. Over the period of this study the range of the weights given to subjective performance measures was largely unchanged. Thus, the different formula used in 2001 and 2002 did not have a large bearing on the weights given to the subjective measures. We include this dummy to control for the formula change in any case. 5. Estimation Models and Tests Regression model To estimate the model we specify the following change model for the two subjective performance measures MANAGE and INTEGRATE for 108 branch years (pooled time-series data for branches over 5 years). The determinants of the weights given to these two measures are evaluated by estimating the relation between several accounting and non-accounting measures and each subjective measure: MANAGE = Β 0 + Β 1 SALEGWTH it + Β 2 COSTGWTH it + Β 3 COLLECT it + Β 4 CUSTOMER it + Β 5 INTEGRATE it + Β 6 EMPGWTH it + Β 7 EXPERIENCE it + Β 8 WAGEGWTH it + Β 9 YEARDUM it +ε it (1) 18

20 INTEGRATE = Β 0 + Β 1 SALEGWTH it + Β 2 COSTGWTH it + Β 3 COLLECT it + Β 4 CUSTOMER it + Β 5 MANAGE it + Β 6 EMPGWTH it + Β 7 EXPERIENCE it + Β 8 WAGEGWTH it + Β 9 YEARDUM it + ε it (2) where: i = 1-28 represents the indivudal branch offices, t = 1-5 represents the years in the sample period SALEGWTH = the year on year percentage growth of sales revenue per employee in each branch office, COSTGWTH = the year on year percentage growth of expenses per employee in each branch office COLLECT = the year on year percentage change in average days sales of account receivable collected CUSTOMER = the year on year percentage change in customer satisfaction survey result MANAGE = the year on year percentage change in head office assessment of branch office maintenance quality INTEGRATE = the year on year percentage change in head office assessment by general manager sales on teamwork, communications, initiatives etc EMPGWTH = the year on year percentage change in growth in the number of sales employees in each branch EXPERIENCE = the year on year percentage change in number of months of branch manager experience in a specific branch WAGEGWTH = the year on year percentage change in the average level of workers wages within each province YEARDUM = 0 = Years , 1= The above models specify the two subjective performance measures as functions of accounting performance, branch office characteristics and a year dummy. The year dummy that is likely to affect the subjective performance measurement is the change in the incentive system formula which occurred after To control for the possible affect of the change in the incentive formula we included YEARDUM in the models. 19

21 YEARDUM was coded as 0 for the years 1998 to 2000, and 1 for the years 2001 to Results Descriptives analysis Table 3 presents descriptive statistics for the 28 branch offices. The descriptive statistics are given for several branch characteristics (not in the two models) and the levels of the variables in the models. The average branch size was 5 and the average branch manager experience in a particular branch was 24 months. Regarding financial performance of the branch offices, the average sales revenue was more than 2.5 million Yuan per employee and the average branch sales expenses was about 79,000 Yuan per employee. The average day s receivable was 86 with a range of 36 days to 144 days. This compares with the 90 days as being an acceptable receivable target for China (Fiducia, 2005). Referring to the average incentive bonus formula measures, the weight given to sales makes up on average 45% of the bonus, followed by the weight given to cash collection (23.08%) and customer satisfaction and maintenance (21.14%). The average level of bonus that is determined by the subjective measures is less than 10% (MANAGE = 4.88%) and (INTEGRATE = 3.84%). While this average weight associated with subjective measures appears low, it is because there are negative points given in some branches for very poor performance (that is the minimum percentage for one branch was % meaning that the total potential bonus relating to the rest of the formula was reduced by percent for a particular branch in one year. Overall the wide range in 1 To control for other exogenous factors that might influence subjective measurement, we included an industry growth factor that measured the average sales growth of the fixed line phone set industry for each province. When the models were run with the inclusion of this factor the same results were obtained. 20

22 weights given according to the subjective measures is quite significant as indicated by the standard deviations of 7.02 and for MANAGE and INTEGRATE, respectively. Results of the correlations and regression analysis are presented for the percent change model. The use of a percent change model helps to reduce the bias in estimates resulting from spurious correlations and omitted variables (Banker et al., 2000; Ittner and Larcker, 1998). Thus, the regression coefficient indicates the magnitude of the relationship between the percentage change in an independent variable and the percentage change in the weighting given to the subjective measure in bonus determination. Table 4 presents the Pearson coefficient correlations between the variables in the models of this study. Both MANAGE and INTEGRATE are positively related to CUSTOMER (p<.05). In addition, MANAGE is negatively related to COSTGWTH (p<.05) which is consistent with H1. However, MANAGE is also negatively related to SALEGWTH (p<.05) which is opposite to the prediction of H1 that higher sales is related to higher weight given to subjective measures. In contrast, INTEGRATE (but not MANAGE) is positively related to EMPGWTH (p<.05) which is consistent with the expectation of H2a. It is also noted that one of the correlations between the independent variables is negative and quite high (CUSTOMER and SALEGWTH : R = -.575). This is not unexpected in that any positive relationship between customer satisfaction and sales growth is more likely in a lead-lag relationship (see Banker et al., 2000). In order to deal with this potential multicollinearity problem we ran the regressions with CUSTOMER or SALEGWTH excluded and found that the sign and significance of the remaining coefficients to be unchanged. We also report low to moderate variance inflation factors 21

23 (VIF) in the results (see Table 5). Finally, we also found that the change in MANAGE was significantly associated with change in INTEGRATE (R =.860), which indicates that these two subjective weights may not be as independent as planned by the management of the firm. This is also consistent with the psychology explanation that judgments made in the assessment of MANAGE spill over into judgments made in the assessment of INTEGRATE. As such we also present the regression results for the change in the combined weights given to the subjective measures the in the analysis below. ********************** Table 3 about here********************** ********************** Table 4 about here********************** Hypotheses tests Based on the psychology explanations of evaluators relying on objective and aggregate measures in determining the bonus weights, Hypotheses 1 (H1) suggests that the bonus weights based on subjective measures will be a positive function of (a) sales and cash collections. Hypotheses 1 also suggests that the bonus weights determined by subjective assessments will be a negative function of (c) customer satisfaction (d) provincial wage growth. In contrast, based on the need to be informative and to control for risk, Hypotheses 2 (H2) suggests subjective assessments will be a positive function of (a) branch office employee growth and a negative function of (b) branch manager experience. The question addressed was whether such subjective assessments were determined by 22

24 qualitative factors, in which case they were likely to be informative, or quantitative factors, in which case they were redundant for the purpose in the incentive system. Table 5 presents OLS estimates of the relation between two subjective measures (MANAGE and INTEGRATE) and various performance measures. Two types of regressions are presented normal regression which included all of the independent variables and stepwise regression which excluded non-significant variables in the process of improving the adjusted R-Square. The overall normal regressions are significant (ρ<0.05) with adjusted R 2 ranging from to Opposite to our expectations (H1a), we found that SALES was significantly and negatively related to MANAGE in both the normal and stepwise regression models. This suggests that greater weight was given to subjective measures in those branch offices which had negative change in sales during the evaluation period. Consistent with our expectations set out in H1b, the coefficient (η 1 ) of COSTGWTH was positively and significantly associated with MANAGE for both the normal and stepwise regression procedures. We also found that CUSTOMER was not significantly related to MANAGE, but was significantly related to INTEGRATE providing partial support for H1c. Finally, for H1d, WAGEGWTH was found to be negatively and significantly related to INTEGRATE and the COMBINED subjective weights for both the normal and stepwise regression procedures. Thus H1d was partially supported. With respect to H2, the coefficient (η 1 ) of EMPGWTH was one of the qualitative factors was both positively and significantly related to INTEGRATE in both the normal and stepwise regression procedures. However, EMPGWTH was not significantly related to MANAGE. Thus H2a is partially supported. For H2b, we hypothesized that the bonus 23

25 weights placed on subjective measures would be a negative function of manager experience within each branch office. Since we used a change function it is those managers with less experience that should exhibit higher change amounts, thus we expect that the sign of the change coefficient would be positive. Consistent with H2b, EXPERIENCE was positive and significantly related to MANAGE (but not INTEGRATE) suggesting that this element of manager risk was partly taken into account when determining the subjective weight. Overall the results suggest that the subjective weightings are determined partly by accounting and external indicators (COSTGWTH, CUSTOMER and WAGEGWTH) and partly by non-accounting (EMPGWTH and EXPERIENCE) factors. Following the psychology perspective, the result for COSTGWTH, CUSTOMER and WAGEGWTH suggests that subjective weights are determined by accounting and external indicators which are easier to anchor a performance evaluator s judgment on. Following the economic perspective, the result for EMPGWTH and EXPERIENCE indicates that subjectivity in performance measurement is partly used to help manage the employee growth of the branch offices and to manage branch manager risk that is associated with undertaking non-direct sales activities (that is customer servicing and maintenance, vertical and lateral communications throughout the branch network). We argued that there is a lag between the growth of the branch office and sale growth, such that subjective bonus would be required to motivate manager effort in undertaking non-sales activities such as coordination, the sharing of more information and greater demands for sales manager development. If sales growth occurred at the same time as the employee growth then it is possible that the subjective measurement could 24

26 proxy for sales growth and be informative of the manager effort required to develop sales staff. Thus, the results for H2a is particularly relevant in that the coefficient for SALESGWTH was not positively and significantly related to either of MANAGE or INTEGRATE. Indeed, the coefficient for SALES was negative and significant for MANAGE which could explain the presence of non-sales effort by sales mangers who received higher bonuses based on subjective measures. ****************** Table 5 about here****************** 7. Conclusion The purpose of this study is to apply a longitudinal methodology to further understand the determinants of subjective measurement of performance. We argued that the industrial (telecom) context in our study provided a richer environment to capture the variance in the factors that are likely to be informative for the purposes of subjective measurement. We also argued that prior studies did not consider the longitudinal aspects of the influences and effects of subjectivity in performance measurement. With these two factors as a motivation we sought to test several economic and psychology based explanations for what determines the level of subjective measurement of performance. We analyzed field-based data from 26 branch offices of a telecom equipment manufacturer that had use a mix of quantitative and qualitative performance indicators in its incentive plan. The first part of the paper described the process of the incentive system design, its introduction and the subsequent changes that were made to the system over five years. Very few researchers have examined the process of performance measurement 25

27 implementation and the subsequent change (see Malina and Selto, 2004; Ittner et al. (2003) as recent examples). We also documented the nature of the two qualitative performance measures head office subjective assessments of (i) branch office management and (ii) overall integrated involvement of the branch office with the head office in terms of communications, making suggestions and solving problems. We derived expectations for the circumstances when various measures are likely to be related to the change in the subjective weights in the annual bonus determination for the incentive plan based on economic and psychology based explanations. For the economics based explanations, we hypothesized that the weighting given to subjective measures will be a function of sales, branch expenses, customer satisfaction and provincial wage growth. For the psychology based explanations, we hypothesized that the effort encouraged by subjective measures will be greater for faster employee growing branch offices and less experienced branch managers. We found that the subjective assessments of branch office management and integrated assessment to be significantly associated with cost management, customer service and change in wage levels indicating support for psychology based explanations. We also asked the economic question of whether the subjective measures in bonus determination are really informative over and above the objective or externally derived performance indicators in the system. This was important since few studies have documented the determinants of subjective performance measurement (eg. Ittner et al, 2003 and Gibbs, et al. 2003). We found that the overall integrated assessment to be determined by the growth in employees in the branch office, which suggests that the 26

28 subjective assessment may be capturing the more complex nature of managing the branch as its size increases. We also found that the overall branch office management assessment to be determined by the level of manager experience. That is, newer managers in various sales branches were more likely to receive a higher subjective weighting in their bonus determination. Obviously, both psychology and economic based explanations can exist side by side. But what is interesting is that our findings for both theories contrast with Ittner et al (2003) who found results that support predominantly psychology-based explanations. Indeed, several experimental studies have reported the importance of psychology based explanations in a multiple performance evaluation context (Lipe and Shapiro, 2000; Banker et al., 2004). We contend that one of the reasons for our finding results that were equally consistent with economic-based explanations is that our performance evaluation environment was more heterogeneous in terms of the factors that impact on (i) informativeness of the various measures that are used in performance measurement, and (ii) the risk faced by the managers who were being evaluated. Future research could build on this speculation by comparing different organization contexts or industries that are decidedly different in terms of measurement informativeness and in terms of the risk faced by those being evaluated to determine when (and if) one explanation dominates the other. Our results are subject to certain limitations. First, as with the focus of the analysis on a single firm, there is the issue of generalizability of the findings to other contexts. However, there are many instances when the in-depth analysis of a single firm is necessary to fully understand the phenomena being observed. Recent work on the use 27

29 of multiple performance measures has used the single firm setting to understand the more complex linkages between various factors and the use and purpose of various measurement systems (see Ittner et al, 2003; Malina and Selto, 2003 for examples of indepth studies on the balanced scorecard. In our case, we were able to capitalize on the knowledge that CTI specifically set up an incentive system with a particular strategy in order to effectively compete in a complex and heterogeneous environment. This makes our findings even more interesting know that this precondition existed in our firm. Second, several econometric considerations were not able to be fully addressed due to the limited degrees of freedom allows with the sample size. For example, the number of branch offices more than doubled during the time period of this study ( ), thus there was a loss of longitudinal data for the newer branch offices in the sample. Third, we did not consider the performance consequences of the use of subjectivity in performance measurement mainly due to the limited degrees of freedom that limits the estimation of possible lead lag effects that may exist between subjective measures and financial performance (eg, Banker et al, 2000). But also we were not able to measure the change (redirection) in manager effort which the subjective performance measures were partly aimed at achieving. That is the direction of manager effort towards customer relationship and employee development and communications across the branch network and with head office. Even with these limitations, this study has important implications for those companies that use or are considering using multiple performance measurement systems. Our findings imply that even with the best of intentions (strategy), managers still need to 28