The Use of Multi-Attribute Utility Theory to Address Trade-Offs for the Balanced Scorecard

Transcription

1 The Use of Multi-Attribute Utility Theory to Address Trade-Offs for the Balanced Scorecard Alisha D. Youngblood and Terry R. Collins Department of Industrial Engineering University of Arkansas Fayetteville, AR Abstract This project incorporates mu lti-attribute utility theory into a balanced scorecard as a way to quantify results and address trade-offs. It is expected that this combination will provide organizations a better way to evaluate investment alternatives, particularly for organizations that occasionally operate under a different mission statement and different objectives, depending on circumstances. Keywords Performance measurement, Balanced scorecard, Multi-attribute utility theory 1. Introduction This paper is a continuation of an earlier research study that seeks to determine and categorize appropriate performance metrics in a government-sector organization [3]. Since Robert Kaplan s development of the Balanced Scorecard, businesses have focused on more than just the bottom line when evaluating performance. The Balanced Scorecard (BSC) utilizes traditional accounting measures of past performance, but incorporates the ability to measure those factors which drive future performance. Although the Balanced Scorecard provides valuable feedback on a variety of performance metrics, it does not weight the relative importance of these metrics and acknowledge the issue of trade-offs between metrics. When choosing between alternatives where the attributes have some interaction and trade-offs are present, scorecards that incorporate multi-attribute utility theory (MAUT) may prove more appropriate. Combining these two tools poses some interesting issues. A Scorecard must be developed that identifies and addresses the trade-offs, weights the trade-offs appropriately, and assigns scores to various alternatives. For dynamic systems in particular, correctly identifying these trade-offs can either make or break a system. For example, an organization may have different mission statements or purposes depending on the current situation. The purpose of organizations such as the military or those that provide humanitarian services or disaster relief most of the time is to maintain a certain state of readiness in case a situation arises where their services are needed. However, once that situation arises, their mission changes to one of actually moving materials and providing relief. Because the circumstances have changed, emphasis has been shifted to a different set of priorities and other choices usually result from the decision making process. It is expected that developing a balanced scorecard that uses multi-attribute utility theory to address trade-off issues will provide organizations with a better tool to evaluate various alternatives for capital investments, particularly in those cases where the organization might occasionally operate under a different mission statement and different objectives. Using MAUT will allow for the comparison between metrics that Kaplan s Balanced Scorecard lacks for the evaluation of alternatives when facing different, and sometimes conflicting, objectives. 2. Literature Review As noted in BSC literature [5], traditional accounting means give an organization a measure only of past financial performance, or lagging indicators. They do not measure the leading indicators that drive future performance. One advantage of the Balanced Scorecard is that if it is properly implemented it will provide an organization with the ability to see where current performance is likely to lead them in the future. The actual development of the scorecard also opens lines of communication among decision makers to help ensure that they are all focused on the same goals.

2 Unfortunately, BSC lacks comparison metrics, such that a particular value for a metric easily indicates the adequacy of individual performance levels [1]. Enabling users to easily identify the adequacy of a metric within the performance framework enhances the ability of the BSC to lead an organization to better overall performance. The other area that BSC does not address is that of trade-offs. If there are multiple objectives for an organization, then by definition they must be conflicting. Otherwise, if these objectives were not conflicting, they could be consolidated into a single objective [11]. For example, when using Kaplan s dashboard analogy where several aspects of performance are monitored simultaneously [7], BSC does not address how increasing the speed would negatively impact the gas mileage. If one is in a situation where speed is critical, there will likely be little emphasis on mileage; however, if one s fuel is running low, then it may be necessary to reduce speed to improve mileage. As priorities change choices between trade-offs may be affected, but BSC does not address this explicitly. Applying multi-attribute utility theory (MAUT) to the BSC would enable decision makers to not only look at performance metrics, but also to compare these metrics to an established set of standards and better determine the trade-offs between various performance metrics The Balanced Scorecard The Balanced Scorecard (BSC), as developed by Kaplan and Norton, is a unique and innovative tool for measuring performance. Traditional accounting measures look only at past financial performance, known as lagging indicators. These measures are inadequate when it comes to predicting an organization s future performance in a competitive environment, though [8]. The BSC is different in that it looks at the leading indicators of those attributes that drive future performance. It aims to translate mission and strategy into objectives and measures. Much of the benefit of the BSC comes from the actual development of an organization s framework of performance metrics because it forces decision makers to determine, in simple terms, the handful of objectives believed to be important. The Balanced Scorecard looks primarily at four different perspectives when evaluating business performance metrics: The financial perspective How do we look to shareholders? The internal business perspective What must we excel at? The innovation and learning perspective Can we continue to improve and create value? and The customer perspective How do customers see us [7]? The actual performance metrics chosen should reflect the long-term strategies of the business, not only evaluating how successful the strategy has been in the past, but also determining if the current performance is like ly to lead to the goals of the business. It is important when choosing these metrics that a chain of cause and effect relationships be established and that they pervade all four perspectives of a Balanced Scorecard [5]. Another advantage of the BSC is that it can be used as a strategic management system by linking long-term strategic objectives with short-term actions [9]. This is done through four processes: Translating the vision: vision and strategy statements must be expressed as an integrated set of objectives and measure, agreed upon by all senior executives, that describe the long-term drivers of success. Communicating and linking: aligning departmental and individual objectives with the BSC ensures the communication of long-term organizational strategies. Business planning: integrating business and financial plans allows companies to undertake those initiatives that will help them achieve their long-term goals. Feedback and learning: by constantly reviewing short-term goals and their effects on long-term strategies, companies are capable of real-time learning and adjust practices if necessary. In the very few years since the BSC was developed, many major corporations have integrated its philosophies into their organizational structures [6]. These organizations include Rockwater, Apple Computer, and Advanced Micro Devices. An interesting article by Frigo and Krumwiede provides business leaders with ten practical tips for introducing the Balanced Scorecard into an organization [4]. Because various types of organizations have different performance measures, Brewer and Speh have developed a variation of the Balanced Scorecard that is unique to Supply Chain Management (SCM) ventures [2]. The four perspectives that are particularly important to these organizations are (1) Financial goals, (2) SCM Goals, (3) SCM improvement, and (4) Financial benefits.

3 The authors continue to link these perspectives to the typical BSC measures normally used and list goals that an organization might strive to meet Multi-Attribute Utility Theory Unfortunately, BSC lacks comparison metrics, such that a particular value for a metric easily indicates the adequacy of individual performance levels [1]. Enabling users to easily identify the adequacy of a metric within the performance framework enhances the ability of the BSC to lead an organization to better overall performance. The other area that BSC does not address is that of trade-offs. If there are multiple objectives for an organization, then by definition they must be conflicting. Otherwise, if these objectives were not conflicting, they could be consolidated into a single objective [11]. One method for evaluating these trade-offs between various alternatives and their effects on objectives is through multi-attribute utility theory (MAUT). In MAUT, the first step is for a company s decision makers to determine a set of strategic objectives for the organization. Many companies have a mission statement or some broad goals that the company strives for, but frequently these are so general that they are not very useful. In defining a set of objectives, executives open up a line of communication to ensure that they are all on the same page. Keeney describes four basic steps in developing and quantifying these objectives: 1. Identify those objectives that the organization feels are important. 2. Structure these objectives into a hierarchy so that there is a distinction between ends and means and redundancies are eliminated. 3. Define attributes for these objectives so that they are clarified and consequences can be measured. 4. Develop a utility function so that value trade-offs are indicated and the viewpoints of various groups within the organization are reflected [10]. One of the most important things to remember about MAUT is that it should not be used as the sole decision instrument among decision makers, but rather as an aid to facilitating communication among those in decisionmaking roles, and to aid in professional intuition [10]. 3. Methodology For this project, it is anticipated that basic multi-attribute utility theory will be applied to a balanced scorecard to determine value trade-offs between various alternatives. A balanced scorecard will be developed as described by Kaplan and Norton [8], focusing on the four perspectives noted in section 2.1 using those metrics previously determined in earlier research by The Logistics Institute [14]. Using the principles of MAUT, within these perspectives each attribute will be ranked according to a hierarchy and a utility function will be developed for each group of metrics. When developing a utility function, for each attribute a range of values depicting both worst and best case scenarios is determined. In most situations, one end of a range is considered best and the other worst, but occasionally, the best case scenario is a value somewhere in the middle with each end considered equally bad [10]. MAUT allows for situations such as this. The ranges are then converted to a continuous zero-to-one scale, or function, where the worst case is valued at zero and the best case is valued at one. It is then necessary to determine a function joining the two points, depending on whether the decision maker has a risk-taking, risk-avoiding, or risk-neutral philosophy regarding the trade-offs for that objective. If the decision is considered to be risk neutral, then the function will be a straight line. In the case of a risk averse situation, the curve will be convex, while risk taking will produce a concave curve [13]. Once acceptable ranges are determined, a relative scaling factor (k) is chosen for each objective, or factor, so that the magnitude of each factor relative to other factors within its group or subgroup is considered. This is a lengthy process where the decision makers go through and decide how much change they will allow for some attributes before they would be willing to accept a negative impact on another. This process is highly subjective to the decision maker. The next step is to determine a utility function for each group of metrics that combines the scaled value for each metric and the weighting factor (k). Functions can be either additive or multiplicative, depending on the nature of the group of metrics. Additive functions are typically easier to formulate and fit most situations, but in some cases it

4 is necessary to derive a multiplicative function [11]. Using these functions, it is possible to perform numerical analyses on various scenarios in order to make more -informed decisions. Unlike the Balanced Scorecard, MAUT hierarchies can be structured such that it might make it possible to run meaningful sensitivity analysis on those critical assessments that are the hardest to make [12]. This is of tremendous value when there is more than one decision maker involved in the process. Sensitivity analysis allows for the examination of various scenarios and accounts for uncertainty within the model. 4. Case Study Illustration For this example, four metrics are chosen from each of the four categories of metrics featured in Watson et al. [14]. For each metric, minimum (a), maximum (c), and optimal (b) values are chosen. The range for a metric should represent the most likely range in which the measured value will lie. It does not necessarily need to cover values whose occurrence is considered highly unlikely. For many metrics, such as the cost metrics, the minimum value is considered optimum (a=b). In cases like these, anything that might fall below the minimum would automatically receive a score of one, while anything greater than the anticipated maximum would receive a zero (Equation 1). It is acknowledged that the score for all values worse than the anticipated worst case value would receive the same score. However, the importance of a score is to indicate to decision makers the relative performance of a particular measure. The purpose of a score of zero would be to bring awareness to an area that needs immediate attention. 1, x a f ( x) = ( x c) /( a c), a < x < c (1) 0, x c The other two types of functions used in this example are where the maximum value is considered the optimum (b=c), such as with the perfect order (Equation 2), and when the optimum is something in between the minimum and maximum, such as the number of training hours per person (Equation 3). f 0, 1, ( x) = ( c x) /( c a), x a a < x < c x c (2) ( x a) /( b a), ( x c) /( b c), a x b f ( x) = b x c (3) 0, otherwise Once the acceptable ranges are determined, and the appropriate function derived (increasing linear, decreasing linear, or other), the relative weights of the individual metrics within a perspective are determined. For example, within the Financial/Stakeholder Perspective, the cost per operation is weighted at 20%, cost per transaction 30%, inventory carrying cost 35%, and inventory on hand 15%. (It is important to note that these must total to 100%.) After a score is calculated from the current value of a metric, this score is multiplied by the weight of that metric to determine its weighted score. These scores are then totaled to determine the aggregate score for the category as a whole. If desired, a composite score for the organization can be calculated by determining the relative weights of the four perspectives, multiplying these scores be their relative weights, and totaling. In this particular example, the perspectives are equally weighted at 25%, but these need not be the case. The relative weights of the performance metrics can change over time or due to extenuating circumstances. In September 2001, the focus of many American industries shifted to incorporate national security on a level never before experienced. While costs and on-time delivery of goods remained important metrics, for many organizations there were additional concerns of meeting new governmental security regulations and providing a safe and secure working environment for employees. The flexibility of MAUT allows for the relative weights of a group of metrics to change so that a more accurate picture of performance emerges.

5 When interpreting the scores for an individual category or the entire organization, there is nothing magical about a particular number. Rather, scores should be monitored over time to gauge relative performance during the regular course of business. A manager should use the scores for a variety of measures to determine where improvements are most needed. Also, as a particular measure or group of measures reaches high scores and remains at that level, it may be time for a manager to reexamine what an appropriate range would be. For example, as process improvements are made, the range of values for a metric would likely shift such that a new optimum should be strived for. Attaching numerical scores to the performance measures also helps to indicate to a manager how improvement in one area may adversely affect performance in another. An increase in the Shipping Rate (shipments/person-hr) may improve the score in the Internal Business Perspective, but that may come at the expense of Perfect Orders, causing a decrease in the Customer Perspective score. Weighting the relative importance of metrics within and across perspectives would help a decision maker determine if the trade-off between Shipping Rate and Perfect Orders is in the best interest of the overall organization. Table 1. Sample Scorecard Financial/Stakeholder Perspective Category Weight= 0.25 Cost per operation ($) Cost per transaction ($) Inventory carrying cost ($) Inventory on hand ($M) Total: 0.28 Customer Perspective Category Weight= 0.25 Customer complaint rate Perfect Order Repeat Customers 79% 1% 80% 80% Inventory Accuracy 96% 80% 100% 100% Total: 0.89 Learning and Growth Perspective Category Weight= 0.25 Absenteeism 2% 1% 15% 1% Associate retention 80% 25% 85% 85% Supplier Partnership Training hrs per person Total: 0.87 Internal Business Perspective Category Weight= 0.25 Shipping Rate Asset Utilization 80% 70% 100% 100% Fill Rate 85% 75% 100% 100% On-time Delivery 65% 50% 100% 100% Total: 0.37 Total Score for all categories= Current Limitations Two important assumptions need to be addressed. First, for all metrics in the case study, it was assumed that the decision maker in question is risk neutral, therefore a straight-line utility function is used. In actuality, this is not usually the case. Typically, a person is either risk taking or risk averse which generates either a convex or concave

6 curve, respectively. Deriving these kinds of curves is very time intensive, is very subjective to the individual in question, and can change with various states of nature. Therefore, a simplified version is used in this paper. The other assumption that is made in this paper is that the various metrics are independent, when in fact they are not. For example, there is likely a positive correlation between the number of perfect orders and the number of repeat customers and a negative correlation between perfect orders and the customer complaint rate. In a sense, these metrics may measure some of the same processes and functions. Treating them separately is similar to counting the same thing twice. In an actual situation, the correlation coefficient would need to be calculated and an appropriate multiplicative function derived that considers the interaction in a particular situation. 5. Conclusions There are several benefits expected from this research. Current literature repeatedly notes that an initial benefit is the communication between decision makers while establishing metrics and best/worst values. Many executives who thought they were on the same page discovered they did not have the same thoughts about what the organization s true goals were. The more long-term benefit is a decision-making aid that quantifies the expected results, thus providing a more objective picture of the outcomes of various capital investments. Perhaps the most significant advantage of MAUT within the Balanced Scorecard of certain organizations would be that different utility functions could be developed for various states of nature. For example, during a non-crisis situation, the utility function would reflect the need to reduce the time between when an item is needed and the time it is delivered to the end customer, but would also recognize the need to keep costs as low as is feasible. This changes, though, during a crisis situation when material must be shipped out and costs are basically irrelevant. Separate utility functions would allow decision makers to more easily evaluate performance levels keeping the current mission in mind with respect to the particular state of nature. References 1. Bogan, C.E., and English, M.J., 1994, Benchmarking for Best Practices. McGraw-Hill, New York. 2. Brewer, P.C., and Speh, T.W., 2000, Using the Balanced Scorecard to Measure Supply Chain Performance. Journal of Business Logistics, 21, (1), Collins, T.R., Rossetti, M.D., and Watson, J.A., 2001, A Balanced Scorecard Approach to Performance Measurement in the Government Sector. IIE Annual Conference Proceedings, May 20-22, Dallas, TX. 4. Frigo, M.L., and Krumwiede, K.R., 2000, The Balanced Scorecard: A Winning Performance Measurement System. Strategic Finance, 81, (7), Kaplan, R.S., and Norton, D.P., 1996, Linking the Balanced Scorecard to Strategy. California Management Review, 39, (1), Kaplan, R.S., and Norton, D.P., 1993, Putting the Balanced Scorecard to Work. Harvard Business Review, 71, (5), Kaplan, R.S., and Norton, D.P., 1992, The Balanced Scorecard Measures that Drive Performance. Harvard Business Review, 70, (1), Kaplan, R.S., and Norton, D.P., 1996, The Balanced Scorecard: Translating Strategy Into Action. Harvard Business School Press. Boston. 9. Kaplan, R.S., and Norton, D.P., 1996, Using the Balanced Scorecard as a Strategic Management System. Harvard Business Review, 74, (1), Keeney, R.L., 1975, Examining Corporate Policy Using Multiattribute Utility Analysis. Sloan Management Review, 17, (1), Keeney, R.L., and McDaniels, T.L., 1992, Value Focused Thinking About Strategy Decisions at BC Hydro. Interfaces, 22, (6), Keeney, R.L., Raiffa, H., and Meyer, R.F., 1976, Decisions with Multiple Objectives: Preferences and Value Tradeoffs. Wiley, New York. 13. Taha, H.A., 1997, Operations Research: An Introduction. Prentice Hall. Upper Saddle River, New Jersey. 14. Watson, J.A., Malstrom, E.M., Landers, T.L., Dhodapkar, S., Smith, V., and Harris, R., 1999, Best Practices: Logistics Performance Evaluation.