DEVELOPING A SUPPLIER PERFORMANCE MANAGEMENT PROGRAM AT A SOUTHERN CALIFORNIA UTILITY COMPANY. A Project. Presented to the.

Transcription

1 DEVELOPING A SUPPLIER PERFORMANCE MANAGEMENT PROGRAM AT A SOUTHERN CALIFORNIA UTILITY COMPANY A Project Presented to the Faculty of California State Polytechnic University, Pomona In Partial Fulfillment Of the Requirements for the Degree Master of Business Administration By Jennifer M. Doi 2017

2 SIGNATURE PAGE PROJECT: DEVELOPING A SUPPLIER PERFORMANCE MANAGEMENT PROGRAM AT A SOUTHERN CALIFORNIA UTILITY COMPANY AUTHOR: Jennifer M. Doi DATE SUBMITTED: Summer 2017 College of Business Administration Dr. Kevin W. Moncrief Project Committee Chair Adjunct Professor, International Business and Marketing Dr. Cheryl R. Wyrick Project Committee Reader Professor, Management and Human Resources ii

3 ABSTRACT The purpose of this study was to develop, implement, and evaluate a Supplier Performance Management program at a Southern California utility company. The Suppl y Management organization identified the need to effectively manage the performance of the company s IT service providers because they represent a significant amount of cost and resources for the organization. Experience within the organization indicated that without a defined strategy for performance management, it is difficult to calculate and preserve the value expected from negotiated contracts. A Supplier Performance Management (SPM) program provides a method for organizations to track key metrics for cost, quality, and service through the use of scorecards and dashboards. (Rizza, 2006) The industry research performed on companies of similar size and spend indicates that the most successful companies employ SPM programs to maintain and improve the value of their supplier relationships. The study was focused on establishing a SPM program to identify key internal metrics critical to organizational objectives, measure a supplier s performance against such metrics, and design a process to review results and share feedback among the participants. iii

4 TABLE OF CONTENTS Signature Page... ii Abstract... iii List of Tables... vii List of Figures... viii Glossary... ix List of Abbreviations... x Chapter 1: Introduction Chapter Overview Background Problem Statement Research Objectives Significance of Research Research Questions Limitations Chapter Summary... 8 Chapter 2: Literature Review Chapter Overview Industry Research on SPM Programs Industry Review Benefits of SPM Programs Best Practices for Developing SPM Programs iv

5 2.3.1 SPM Best Practices Focus on Scorecard Development Maintaining Effective SPM Programs Challenges of SPM Programs Focus on Continuous Improvement Research Results Chapter 3: Methodology Chapter Overview Research Design Qualitative Survey Quantitative Scorecard Survey and Scorecard Design Sampling Procedures Data Analysis Anticipated Results Chapter Summary Chapter 4: Research Findings Chapter Overview Research Question Research Question Research Question Research Question v

6 4.6 Research Question Research Question Research Question Chapter Summary Chapter 5: Recommendation and Conclusion Chapter Overview Research Summary Project Summary Implications for Management Recommendations for Management Conclusion References Appendix A: Construct and Measurement Table Appendix B: Qualitative Survey Appendix C: Quantitative Scorecard Appendix D: Supplier A Scorecard Appendix E: Supplier B Scorecard Appendix F: Supplier C Scorecard Appendix G: Supplier D Scorecard Appendix H: Overall Ratings from Qualitative Survey Appendix I: Major Issues from Qualitative Survey vi

7 LIST OF TABLES Table 1 Literature Review Table... 9 Table 2 Scorecard Development Best Practices vii

8 LIST OF FIGURES Figure 1 Supplier Segmentation Model... 3 Figure 2 Benefits of Effective Supplier Management... 4 Figure 3 Figure 4 Linking the Supply Management Framework to the Balanced Scorecard.. 18 Scorecard and Quarterly Business Review Process and Guidelines viii

9 GLOSSARY Application Development and Maintenance: A term that refers to custom software development, building of Enterprise Application Integration systems and maintenance of systems and subsystems (MindLance, n.d.). Balanced Scorecard: A set of measures that gives management a fast but comprehensive view of the business (Kaplan and Norton, 1992). Information Technology: The technology involving the development, maintenance, and use of computer systems, software, and networks for the processing and distribution of data (Information Technology, n.d.). Key Performance Indicator: A set of quantifiable measures that a company uses to gauge its performance over time (Key Performance Indicator, n.d.). Return on Investment: A profitability measure that evaluates the performance of a business by dividing net profit by net worth (Return on Investment, n.d.). Service Level Agreement: A document describing the level of service expected by a customer from a supplier, laying out the metrics by which that service is measured, and the remedies or penalties, if any, should the agreed-upon levels not be achieved (Service Level Agreement, n.d.). Strategic Sourcing: A method of managing procurement processes for an organization in which the procedures, methods, and sources are constantly re-evaluated to optimize value to the organization (Strategic Sourcing, n.d.). Supplier Performance Management: The process of measuring, analyzing, and managing supplier performance for the purposes of reducing costs, mitigating risk, and driving continuous improvement (Aberdeen, 2002). ix

10 LIST OF ABBREVIATIONS ADM: Application Development and Maintenance BSC: Balanced Scorecard CEB: Corporate Executive Board CPUC: California Public Utilities Commission EAI: Enterprise Application Integration IT: Information Technology KPI: Key Performance Indicator PSC: Procurement Strategy Council QBR: Quarterly Business Review QL: Qualitative Metric QN: Quantitative Metric ROI: Return on Investment SBR: Supplier Business Review SLA: Service Level Agreement SM: Supply Management SPM: Supplier Performance Management x

11 Chapter 1 Introduction 1.1 Chapter Overview This chapter provides an overview of how a Southern California utility company contracts with Information Technology (IT) service providers and manages related supplier performance. The purpose of this project is to explore the ways that Supplier Performance Management (SPM) can help the utility company to realize the full value of its contracts with IT service providers. Significant loss of contract benefits can occur when the parties governing a relationship fail to develop comprehensive programs to effectively manage supplier performance. By collaborating with suppliers and internal stakeholders to create appropriate performance metrics, Supply Management professionals can measure and improve the operations of IT service providers. 1.2 Background The Supply Management organization of the Southern California utility company under study is tasked with the procurement of materials and services totaling nearly $4 billion in annual spend. Total spend is divided among categories that represent the largest segments of spend across the entire company, and these categories include professional services, construction services, transportation/delivery services, information technology (IT), energy efficiency programs, office supplies, transmission/distribution materials, and so on. The IT Category of Supply Management oversees the direction of approximately 10-12% of the organization s total annual spend, or $ million. The IT Category is responsible for the procurement of goods and services such as hardware (desktops, laptops, servers, computer peripherals), software, 1

12 telecommunications equipment, and related IT services (applications development and maintenance). Although the actual quantity of products and services that the organization acquires is quite large, this study is focused on developing a program for suppliers providing IT services specific to applications development and maintenance. Application Development and Maintenance (ADM) refers to custom software development, construction of Enterprise Application Integration (EAI) systems, and maintenance of systems and sub systems. The IT Category purchases many standard, off-the-shelf software products, but ADM services are required to fulfill the unique requirements of the organization. (MindLance, n.d.) The IT Category manages relationships with many IT service providers delivering a wide range of activities, but this study is focused on developing a SPM program for the four prime suppliers of application development and maintenance services. The IT Category followed Strategic Sourcing procedures to identify the most qualified industry participants for IT services, and these four suppliers were awarded contracts with the utility based on the evaluations of their qualifications. Figure 1 demonstrates an interpretation of various supplier segments presented by the Procurement Strategy Council (PSC), which categorizes suppliers by tiers depending on level of supplier spend and business impact. The suppliers selected for the project have been identified by the organization as Tier 1, or Strategic, suppliers, and according to Figure 1 represent relationships that are critical to long term success and fully integrated activities within the organization. Additionally, these four suppliers were selected for consideration because they represent approximately 20-25% of the category s annual spend, and therefore signify a substantial level of effort for the organization. 2

13 Figure 1. Supplier segmentation model. Adapted from Supplier Segmentation Toolkit by Procurement Strategy Council. The Southern California utility company is regulated by the California Public Utilities Commission (CPUC) and is accountable to its ratepayers to keep the costs of operations low. The Supply Management organization fulfills this obligation by following Strategic Sourcing practices to select suppliers and overseeing existing contracts to maintain competitive pricing and performance. Ongoing economic fluctuations and technological advances have a sizeable impact on the company and 3

14 require new approaches to IT performance management to be explored. (Procurement Strategy Council, n.d.a) Figure 2. Benefits of effective supplier management. Adapted from Supplier relationship management by Procurement Strategy Council. 1.3 Problem Statement The Supply Management organization of the Southern California utility company applies vast resources to discover, qualify, and develop contracts with qualified IT service providers. Figure 2 demonstrates that Supply Management organizations spend approximately 60% of supplier-focused time on Strategic Sourcing and without proper contract management, most of the projected savings will dissolve over the term of the agreement. The company can potentially experience significant performance losses when IT service providers are not managed effectively, as previously there has not been an established formal and uniform Supplier Performance Management (SPM) program. Other negative consequences of overlooking SPM practices are the deterioration of 4

15 supplier relationships, frustration of internal clients, and devaluation of established agreements. 1.4 Research Objectives There are several objectives of the research on Supplier Performance Management programs to be addressed by this project study. One objective is to demonstrate that such a program will help the Southern California utility company to more effectively manage its IT service providers. Another is to identify best practices to develop and implement a SPM program that offers an approach for the IT Category to measure and improve the performance of its Tier 1 IT service providers. Finally, results of the research should associate leading SPM practices and concepts with a program that provides a comprehensive view of selected Tier 1 relationships to the managing parties, allowing them to maximize the value delivered to company. The program development involves participants at various levels throughout the organization to ensure that the needs of all parties are represented. Continuous improvement is an important objective of the project, and steps to achieve the desired progress should be regularly documented and tracked. The IT Category managers are expected to provide consistent and reliable feedback to both internal client and supplier participants, and to higher levels of management in both organizations. An open and collaborative Supplier Performance Management program provides an opportunity for the company to achieve enhanced results from its IT service providers. 1.5 Significance of Research The project objectives are significant for the organization because they present a method to protect valuable company resources such as time and money, and help the 5

16 company identify industry best practices regarding SPM programs. Results of the project should help company resources managing the relationships determine how well IT service providers are performing compared to client expectations and their competitors, and against industry standards. Additionally, the outcomes help detect how satisfied the end users of the IT services are with supplier performance, as well as areas that require immediate attention and remediation. 1.6 Research Questions The research questions for this project are aimed at measuring the satisfaction with the performance and capabilities of IT service providers. To assist in accomplishing this objective, it is critical to develop metrics to measure the satisfaction level of the company s internal clients and end users of the services. Direct comments and feedback from clients are essential to developing performance metrics and recognizing potential areas for growth and improvement. Some possible areas for metrics and performance development include supplier resource quality and capabilities, account management experience, and quality and cycle time of the IT services. The following questions shall represent the ideas and intention of supplier performance management for this project and should be answered by the research. The research questions address the issues that were most critical to the program managers at the time of implementation. The concepts measured to evaluate the variables of each research question make up the related construct, and the constructs specified for each research question are included in the list below. (Zikmund, Babin, Carr, & Griffin, 2010) Additional detail regarding the measurement and operationalization of each construct is 6

17 included in a Construct and Measurement Table in Appendix A and described further in Chapter 3. R1: How is supplier s overall performance measured? Construct: Overall satisfaction with supplier performance R2: How is the quality of the supplier relationship measured? Construct: Attitude toward relationship quality R3: How is the quality of supplier s resources evaluated? Construct: Attitude toward quality of supplier s resources R4: How are supplier s project management capabilities assessed? Construct: Attitude toward supplier s project management capabilities R5: What is the overall satisfaction level with the supplier s account management team? Construct: Overall satisfaction with supplier s account management team R6: How is the quality of the services measured? Construct: Attitude toward service quality R7: How often are services delivered according to schedule? Construct: Overall satisfaction that services are delivered according to schedule 1.7 Limitations There are some limitations in conducting the research on Supplier Performance Management at the Southern California utility company. The results of the survey and scorecard are only as comprehensive and current as provided by the participants in the program. The project is limited to four suppliers and results may not be representative of the performance and attitudes toward other IT service providers. Additionally, depending on the time period covering the development and implementation of the project, there 7

18 may not be enough time to reflect actual performance improvements from the selected suppliers. This project focuses on developing a program in accordance with industry best practices and the intended effect of implementing a Supplier Performance Management program. 1.8 Chapter Summary The IT Category of the Southern California utility company is responsible for managing the performance of the IT service providers that are selected following the process of Strategic Sourcing. The company does not have a formal Supplier Performance Management program, but there is significant potential to increase the value of supplier relationships by measuring performance and detecting areas for growth and improvement. Managing supplier performance will support the effort that related parties commit to the relationship by sharing feedback on past performance and communicating expectations and plans for future progress. 8

19 Chapter 2 Literature Review 2.1 Chapter Overview An initial assessment of the IT Category s practices revealed that there was no centralized method for managing awarded contracts and that many supplier relationships were left unattended by Supply Management after the Strategic Sourcing events had concluded. Industry research provides strong indications that enhanced operations can be achieved through the effective management of supplier performance. Therefore, it is practical and essential to scan the business environment to identify practices of the most successful companies. Table 1 Literature Review Table Subcategory of Literature Review Contribution to Study Section 1: Industry Research on SPM Programs Industry Review Provides information about leading sources of information for the industry Benefits of SPM Programs Presents information about the advantages of implementing SPM programs. Section 2: Best Practices for Developing SPM Programs SPM Best Practices Provides industry leading practices for developing SPM programs Focus on Scorecard Development Presents benefits and best practices for developing a balanced scorecard. Section 3: Maintaining Effective SPM Programs Challenges of SPM Programs Provides information about challenges of managing SPM programs Focus on Continuous Improvement Presents benefits of continuous improvement for maintaining effective SPM programs. 9

20 2.2 Industry Research on SPM Programs Industry Review The Supply Management organization of the Southern California utility company being studied maintains memberships with both the Corporate Executive Board s Procurement Strategy Council and the Ariba Network. The collective knowledge and tools made available from these organizations provided valuable, relevant, and timely material for developing a SPM program. The widespread membership of these two organizations suggests that affiliates have access to the shared knowledge of potentially millions of participating companies which should be expected to represent the best practices across multiple industries. One of the primary sources of industry research used in the development of the SPM program is the Corporate Executive Board s Procurement Strategy Council. The Corporate Executive Board (CEB) was founded in 1979 as the Research Council of Washington. Today, the CEB serves large, mid-sized and government organizations in areas including Innovation and Strategy, Procurement and Operations, and Financial Services and its clients include over 75% of the Fortune (Procurement Strategy Council, n.d.b) The Procurement Strategy Council (PSC) draws on the expertise of procurement professionals from the world's leading companies and provides insights, tools, analysis, and advisory support to help procurement organizations improve performance and drive innovation. The best practices of thousands of PSC member companies are combined with advanced research methodologies to equip organizations with actionable solutions to transform their operations. (Procurement Strategy Council, n.d.b) 10

21 Another source for industry research is Ariba, Inc., an SAP company that has developed the Ariba Network, which is a business commerce network designed to allow companies to collaborate in a global community for buying, selling, and managing cash. The Ariba Network helps organizations of all sizes connect with trading partners using a cloud platform, which provides internet users from over 190 countries globally access to the network s shared application software and databases. Network members can connect and collaborate with nearly 2 million potential partners to lower their costs and risks, enhance revenues, and manage cash and working capital. Its users benefit from information contributed by experienced partners in the areas of compliance, risk, and performance and enhanced market visibility and intelligence allows users to evaluate changing market conditions for greater business decisions and competitive advantage. (Ariba, n.d.) Benefits of SPM Programs In a progressively global economy, the success of most companies is becoming increasingly dependent on a well-managed Supply Management organization. Ariba, Inc. emphasizes that many companies are challenged to measure and enhance the performance of their suppliers, and therefore miss opportunities to increase profits and proactively mitigate risks. Company experts advocate that an appropriately designed SPM solution can help organizations define, measure, and enhance supplier performance to meet business goals and potentially free up resources to perform other critical business functions. (Ariba, 2008) Formal SPM programs are associated with supplier performance improvement results estimated around 20 percent according to The Supplier Performance Measurement 11

22 Benchmark Report, published by the Aberdeen Group in Organizations that solely implement SPM programs by focusing on strategic or critical suppliers can identify and target performance issues that are most impactful and imperative to the success of its operations. The report cautions that organizations that fail to measure and manage their largest suppliers likely expose themselves to quality and service deficiencies, as well as cost overruns that reduce profits and any value obtained from strategic sourcing practices. (Aberdeen, 2002) Supply Management organizations have historically focused SPM practices on suppliers of manufactured goods, even as the services sector has become an increasingly important factor in the global economy. The general perception that service performance cannot be measured as precisely as manufacturing specifications has reinforced the notion that traditional SPM practices cannot be applied effectively to service providers. Due to the growing significance of and spend with service providers, Supply Management experts have concluded that SPM practices are necessary for improvement, cost control and minimization of value leakage in organizations to maintain a competitive advantage. (Ellram, Tate, & Billington, 2004) The process of supplier evaluation is defined as quantifying the efficiency and effectiveness of supplier action and is designed to influence decision-making within a buying organization while encouraging positive behaviors by supplying companies. Results of studies conducted on supplier evaluation practices indicate that if attempts to influence positive behaviors are successful, they can result in improved supplier performance and capabilities and behavior in better alignment with the objectives of the buying organization. Supplier evaluation is a critical element of an effective SPM 12

23 program and can serve as an effective instrument to influence supplier performance. (Hald and Ellegaard, 2011) Supplier performance management is a natural extension of the purchasing function and when implemented carefully, can provide clear value to an organization. However, calculating actual return on investment (ROI) of SPM programs is often difficult. One approach to making a business case for supplier performance management is to estimate the costs associated with poor quality and late delivery that might be avoided. This estimate can provide managers with useful information of the potential value of implementing SPM programs, which includes greater supply quality, lower costs and reduced supply risk. (Hebrand, 2013) 2.3 Best Practices for Developing SPM Programs SPM Best Practices Industry best practices show that a formalized system to evaluate supplier performance is critical to maintaining effective operations. One of the common recommendations for developing a successful and collaborative SPM program is open communication and data sharing among relevant parties. Performance targets and metrics developed jointly with suppliers and internal clients have greater probability for successful application than those imposed by Supply Management organizations. (esourcing Forum, 2006) A common practice is strategic supplier selection, or concentrating significant purchase activities with a few suppliers to enable effective management of contracts and quality improvement initiatives. Upon selection of strategic suppliers, it is important to circulate scorecards to internal clients regularly to ensure that the appropriate metrics are 13

24 being tracked and evaluated. The scorecards should be reviewed often to ensure that the metrics represent the requirements of the relationship and supplier performance issues are resolved quickly. (esourcing Forum, 2006) SPM is a collection of processes that allows organizations to achieve business goals by defining, measuring, and collaboratively increasing the performance of key suppliers. The formation of an effective SPM program will typically vary by organization and should generally include the following steps: 1. Establish a SPM strategy in alignment with corporate objectives 2. Develop uniform supplier performance criteria and expectations 3. Select evaluation tools and process steps to collect, monitor and measure performance and communicate plan to both the stakeholder and supplier communities 4. Collect supplier performance data (qualitative and quantitative) and establish time period associated with data collection 5. Review data against criteria and share results with suppliers and stakeholders 6. Set improvement goals and plans to influence supplier behavior 7. Review and recalibrate goals, strategy and metrics as business grows and changes (Gordon, 2008) Further, Supply Management professionals are encouraged to establish supplier performance measures because: x x Companies can t manage what they don t measure If companies measure suppliers, they will improve 14

25 x Companies can uncover and remove hidden waste and cost drivers in the supply chain x x Companies can facilitate supplier performance improvement Companies can increase competitiveness by shrinking order cycle times and inventory levels x Companies can make informed business decisions that impact the enterprise (Gordon, 2005) Focus on Scorecard Development The Balanced Scorecard (BSC) was first introduced to the business community by Robert Kaplan and David Norton in 1992 to provide companies with a comprehensive structure to align their strategic objectives with a clear set of performance measures. The BSC was designed to be customized to fit an organization s mission, strategy, technology, and culture, and not intended to be viewed as a general template to accommodate all businesses and industries. (Kaplan and Norton, 1993) The Balanced Scorecard is well suited to organizations seeking to create specific measures to express performance, quality, time, and service goals. Additionally, many key organizational initiatives such as customer-supplier partnerships, cross-functional integration, global scale, continuous improvement, and team accountability can be addressed with the BSC approach to performance management. The BSC is intended to facilitate strategy development, implementation, and communication, and should deliver dependable results for management control and performance evaluation when constructed properly. (Kaplan and Norton, 1992) 15

26 Organizations benefit from implementation of the Balanced Scorecard if they can successfully integrate all of their seemingly contrasting and competing objectives with the tool. The BSC forces managers to review and consider the combined results of all operational measures, and protects against the promotion of certain objectives at the detriment of less favored goals. However, the scorecard criteria should be reviewed and updated regularly to avoid representing obsolete objectives. A potential disadvantage of the BSC is that it does not validate the relevance or applicability of an organization s strategy, it only translates the strategy into measurable objectives. (Kaplan and Norton, 1992) The effectiveness of the Balanced Scorecard is examined in one study of a Fortune 500 manufacturing firm s evolution of a BSC over a 15-year period. The results of the study strongly suggest that the BSC has prevailed due to its use as a learning and communication tool and its ability to influence organizational behavior. The following list of considerations is recommended for developing and maintaining a successful scorecard: x Make the development process collaborative among the key stakeholders and establish clear processes for making changes to the scorecard. This creates buyin, makes expectations explicit, and eliminates surprises. x Communication is critical. Whether top-down, bottom-up, or among evaluatees, communication builds trust in the scorecard process and fosters legitimacy and information sharing. x Create a balance of objectivity and subjectivity in scorecard design and use. Objective performance measurement is the unachievable holy grail and 16

27 subjectivity in measurement and evaluation of performance helps overcome its inherent deficiencies. x Embrace the evergreen nature of the scorecard. Stability in its purpose and use, coupled with flexibility in its design may be critical to success. x Delay using the scorecard extensively for compensation. As experience with the scorecard is gained, trust is established and scorecard integration into the compensation system may be seamless. x Periodically review the scorecard for its effectiveness in promoting desired behaviors and financial results. When necessary, make changes to its design or use. (Malina, 2013, pg. 911) The Balanced Scorecard can be applied to support many of the primary goals of a Supply Management organization and can be modified to establish a framework for performance measurement that is consistent with specific organizational objectives. Figure 3 demonstrates an association between a Supply Management framework and the Balanced Scorecard. This view was originally intended to explain how a BSC can be used to measure the performance of a Supply Management organization, but also provides for the study an example of the applicability to various Supply Management performance management goals. The integration of diverse perspectives provides visibility to performance results to managers across an organization s functional borders. (Brewer and Speh, 2000) 17

28 Supply Management SM Goals Waste reduction Time compression Flexible response Unit cost reduction Balanced Scorecard Business Process Perspective Customer Benefits Improved product/service quality Improved timeliness Improved value Unit cost reduction Customer Perspective Financial Benefits Higher profit margins Improved cash flows Revenue growth Higher return on assets Financial Perspective SM Improvement Product/process innovation Partnership management Information flows Threats/substitutes Innovation and Learning Perspective Figure 3. Linking the supply management framework to the balanced scorecard. Adapted from Using the balanced scorecard to measure supply chain performance, by P.C. Brewer and T.W. Speh, 2000, Journal of Business Logistics, 21, pg. 85. A study designed to develop a BSC for a Supply Management organization further investigated the significance of the diverse perspectives represented by an effective scorecard. Following is the description and significance of each of the four perspectives presented in Figure 3: Business Process Perspective: establish core competencies and processes at which companies must excel and that have greatest impact on customer satisfaction 18

29 Customer Perspective: translate customer service strategy into specific measures that matter and reflect most value to customers Financial Perspective: indicate whether the strategy, implementation and execution are contributing to a company s bottom line improvement Innovation and Learning Perspective: determine ability to innovate, improve and learn to create continuous value for company Recommendations of the study indicate that each of the perspectives should reflect strategic objectives through corresponding metrics, and should be regularly reviewed and updated as needed. (Bhagwat and Sharma, 2007) Table 2 presents an overview and recommendations for SPM scorecard development based on best practices reviewed and documented by the PSC. Table 2 Scorecard Development Best Practices Principles Create category-based scorecards Involve internal business partners before launching the scorecard Focus on the critical few KPIs (approximately three to five per metric) Select KPIs that are within the organization s control Use a mix of both quantitative and qualitative KPIs Incorporate existing KPIs and/or SLAs where appropriate Automate data collection, aggregation, and reporting Benefits Enables analysis of similar suppliers in an apples-to-apples fashion Demonstrates how scorecard focuses on elements business partners care about (i.e., metrics beyond cost) Maintains focus on the most important drivers of supplier performance Measures outcomes that can be influenced by the organization and/or the supplier Contributes to a fact-based scorecard while allowing for day-to-day user perception Encourages adherence to existing contract terms and conditions Creates a scalable performance management process Source: Procurement Strategy Council,

30 2.4 Maintaining Effective SPM Programs Challenges of SPM Programs There are many potential hazards for organizations when implementing SPM programs. One of the common problems that program administrators encounter is a shortage of managers to oversee supplier performance. Other drawbacks include a lack of involvement and/or commitment from suppliers and other groups within an organization. (Hebrand, 2013) The failure to implement a joint set of processes across different business units within an organization is another potential difficulty of SPM programs. Program managers can align SPM processes to manage suppliers that work with multiple business units and suppliers within the same segment. Benefits of aligning SPM processes include the ability to measure the performance of one supplier against another, shared training and communication, and process efficiency. (Hebrand, 2013) Focus on Continuous Improvement The drive toward continuous improvement is a strategic initiative of most modern companies, but practical application can vary widely depending on variables such as size, position, or industry of an organization. As continuous improvement measures are motivated by the desire to reduce expenses and enhance profitability, common considerations are improved quality, reduced cycle time, and regular process evaluation. The failure to assess the results of operational improvements can cause poor financial outcomes if excess capacity created by quality or cycle time developments are not used to eliminate expenses or generate revenue. (Kaplan and Norton, 1992) 20

31 The primary objective of SPM is to improve the performance of both the supplier and the purchasing organization. The development of supplier performance metrics can help organizations ensure that suppliers adhere to contract agreements and market standards. The five key areas that should be considered when establishing SPM programs are supplier segmentation, management, scorecard, development, and evaluation. (Hebrand, 2013) Supplier segmentation refers to the level of effort that SPM programs should expend in engaging suppliers of various segments. Supplier management suggests that teams should be established to structure the supplier relationships with meetings and definition of roles and responsibilities. A supplier scorecard should identify critical success factors, define key performance indicators (KPIs), and develop processes for gathering data. Supplier development refers to the process created to foster the growth and continuous improvement of supplier partners. Supplier evaluation describes the method for using supplier performance data in assessing suppliers for potential contracts. (Hebrand, 2013) Continuous improvement practices are in alignment with all primary initiatives of a Supply Management organization, which is constantly seeking solutions to reduce waste, compress time, respond flexibly, and reduce unit costs. Some of the regular practices that successful Supply Management organizations employ to achieve their objectives are: redesign of products or processes to enhance value partner with other organizations to leverage the human knowledge base improve the flow of accurate, timely, and supportive information 21

32 monitor the external marketplace for potential threats and/or substitutes An organization must consistently evaluate its methods and pace for improvement to ensure that its practices remain competitive within the industry. (Brewer and Speh, 2000) 2.5 Research Results After several years of lacking a formal program to manage supplier performance, the IT Category of the Southern California utility company has gathered information to support the efforts to develop, implement and maintain a SPM program. The research on SPM programs suggests that there is significant value to be retained and further achieved through their strategic development and implementation. The Supply Management group is optimistic that SPM practices will help the company retain the value gained by its Strategic Sourcing efforts and deliver increased performance to the company s IT service clients. 22

33 Chapter 3 Methodology 3.1 Chapter Overview After completing the research to determine the Southern California utility company s requirements for a Supplier Performance Management program, the IT Category identified some industry best practices to develop and implement a program to manage the performance of its four largest IT service providers. This chapter includes details regarding the research design, sample size, and procedures used to administer a program survey, as well as an overview of the methods used to analyze the resulting data. This project outlines the required processes and participants of a SPM program designed to deliver an effective tool for suppliers, internal clients and Supply Management to derive added value from the relationships. 3.2 Research Design Research conducted on the design and implementation of successful SPM programs indicated that a thoughtfully developed scorecard allows organizations to measure and report appropriate performance management metrics. Based on the industry best practices identified for scorecard development, the program included both quantitative and qualitative metrics for survey. The required responses were limited to five (5) quantitative and five (5) qualitative metrics to simplify the process for program participants. Two separate groups were selected to participate in the program; one group included the internal client end users of the IT services that received the qualitative survey and the other group included a small group of IT services contract managers responsible for completing the quantitative scorecard. The following queries represent 23

34 the requirements of the qualitative survey and the quantitative scorecard ( QL indicates qualitative metrics and QN indicates quantitative metrics) Qualitative Survey Scoring Scale: 3 = Supplier Exceeds Expectations 2 = Supplier Meets Expectations 1 = Supplier Does Not Meet (or Partially Meets) Expectations N = Not Applicable/Don t Know QL1: Supplier consistently demonstrates a strong commitment to the relationship by being responsive, flexible, and easy to work with. QL2: Supplier provides high caliber resources who consistently demonstrate a solid understanding of the business. QL3: Supplier exhibits strong project management competency and highly effective practices around communications, cost, schedule and scope management, accuracy and timeliness in invoicing/reporting while being fair and transparent in pricing, contracts and agreed upon Service Level Agreements (SLAs). QL4: Supplier s account management team is always responsive, accessible, accountable, and collaborative. QL5: Please rate your overall satisfaction with supplier (1 = Not Satisfied, 2 = Satisfied, 3 = Very Satisfied). Please provide a brief assessment of supplier s strengths and weaknesses, along with any recommendation(s) you may have for the supplier to improve. 24

35 3.2.2 Quantitative Scorecard Related to Application Maintenance: QN1: Percent of implementations completed on time QN2: Percent of defect-free implementations Related to Application Development: QN3: (Number of code defects)/($10k in application developmen t work) QN4: Number of projects 10% behind schedule QN5: Number of projects 10% over budget 3.3 Survey and Scorecard Design The qualitative survey was distributed to internal client end users of the IT services by and included five questions containing a satisfaction rating scale. A 3- point Likert-type scale was used for each qualitative response to determine the level of satisfaction with a supplier and whether or not client expectations were met. The quantitative scorecard distributed to IT service contract managers included five criteria requiring calculation of actual quarterly performance metrics. A ratio scale was used to compare actual scorecard data with established service level requirements to determine supplier s performance against contractual commitments. Appendix B includes a sample of the qualitative survey format used for distribution and response and Appendix C displays a sample subset of the data requested to calculate the quantitative scorecard. 3.4 Sampling For the initial qualitative survey, 178 participants within the IT organization were surveyed and 104 responses were received (58%), but only 84 participants completed the survey for a total response rate of 47%. The quantitative scorecard was distributed to the 25

36 company s IT group responsible for managing the SLAs of IT services contracts, and only one scorecard was required and returned from the group for a total response rate of 100%. The IT group had been tracking the IT service providers performance based on pre-established agreements in the contracts, but was not in the practice of reporting this information back to the suppliers in previous periods. 3.5 Procedures Following each quarter, the Supply Management group compiled completed survey and scorecard response data and calculated results for each IT service provider participating in the SPM program. All scores were populated and totaled in the supplier scorecards, including client comments provided on the qualitative survey, as represented in the examples in Appendices D through G. The completed supplier scorecard and summary information was provided to each client manager for final review and approval prior to quarterly presentation to IT stakeholders and supplier leadership teams. The IT Category designed the SPM program to clearly communicate performance expectations and client feedback to suppliers to facilitate shared understanding of the program s progress. Participants in the program regularly include Category managers, business unit stakeholders, and supplier representatives. Those responsible for conducting the program are required to familiarize suppliers with the performance metrics and reporting system, provide suppliers with visibility into assessment guidelines to foster awareness about the metrics, and communicate results and rankings relative to other suppliers. The goal of the reviews is to provide actionable supplier information and to improve supplier performance. (Procurement Strategy Council, n.d.a) 26

37 Figure 4. Scorecard and quarterly business review process and guidelines. Adapted from Southern California utility company internal process documents. Figure 4 outlines the quarterly business review (QBR) process and guidelines followed by those implementing the SPM program. The IT Category conducted QBRs with its internal stakeholders to review the results of the surveys and scorecards. Conducting the internal QBRs provided opportunities for the IT Category managers to deliver performance and progress updates and present any issues requiring visibility at the executive level. Following the QBRs, the IT Category conducted Supplier Business Reviews (SBR) with each supplier to review quarterly performance, provide feedback, and discuss areas for improvement. 3.6 Data Analysis The results of each quarterly survey and scorecard were entered into Microsoft Excel for analysis and validation. The associated construct and measurement table is included in Appendix A to demonstrate how the quantitative and qualitative responses 27

38 operationalize the construct of each research question. The table specifies the construct definition of each research question and the corresponding survey or scorecard responses referenced to operationalize the constructs. The qualitative survey responses were evaluated using descriptive statistics to determine the percentage of respondents who were either satisfied or very satisfied versus not satisfied with each category of performance. Similarly, the quantitative scorecard responses were also evaluated using descriptive statistics to calculate the rate at which the supplier s overall quarterly performance had met the established contract criteria and SLAs. A preliminary review of the analysis results suggest that the program criteria can be reasonably measured and logically presented in association with the contract requirements and expectations. 3.7 Anticipated Results The IT Category of the Southern California utility company has been able to determine from the responses of its internal clients and contract managers which of the IT service providers are meeting client expectations and/or complying with SLAs. The QBRs and SBRs have provided sufficient opportunity for the IT Category managers to meet with internal stakeholders and supplier representatives to ensure that actual performance is aligned with organizational goals. The SPM program has facilitated an environment of open communication and feedback between internal end users and service providers that is often undermanaged in large organizations. 3.8 Chapter Summary The methods that have been used to implement the SPM program at the Southern California utility company have been developed by combining key internal business 28

39 requirements with industry best practices research. Responses to the qualitative survey and quantitative scorecard have provided the IT Category with relevant data necessary to determine the satisfaction level with IT service providers and the efficiency of supplier relationships. The quarterly results indicate enhanced value for the organization resulting from proactively managing service contracts and encouraging favorable competition among the IT service suppliers. 29

40 Chapter 4 Research Findings 4.1 Chapter Overview This chapter examines the data gathered from the surveys and scorecards distributed to end users of IT service providers at the Southern California utility company. The tracking, evaluation, and analysis of the submitted data was conducted by the IT Category primarily through the use of Microsoft Excel software. The scorecards for the four IT service providers presented in Appendices D through G and the scorecard summary in Appendix H provide the data required to address the objectives of the Supplier Performance Management program. The findings from the initial quarterly review in the first quarter of 2012 relative to each of the research questions are presented in the following sections. 4.2 Research Question 1 The first research question, How is supplier s overall performance measured? is related to the associated construct of overall satisfaction with supplier performance. The construct is addressed in the responses to the fifth qualitative survey question, QL5: Please rate your overall satisfaction with supplier (1 = Not Satisfied, 2 = Satisfied, 3 = Very Satisfied). Please provide a brief assessment of supplier s strengths and weaknesses, along with any recommendation(s) you may have for the supplier to improve. The results from the survey for this first research question show that on average, respondents were generally satisfied or very satisfied with the four suppliers included in in the first round of SPM. Suppliers A and D scored 91%, Supplier C scored 88% and 30

41 Supplier B scored 83% for overall satisfaction level. The information that may be more revealing is the percentage of respondents that were not satisfied with certain suppliers. For example, Supplier B received an overall score of 16% not satisfied, and it will be useful for the IT Category and executive sponsors to understand the reasons for the dissatisfaction. Appendix I provides a summary of the major issues identified from the survey, and while it does not include specific remarks or comments, the table includes the top 3 strengths and weaknesses for each supplier. 4.3 Research Question 2 The second research question, How is the quality of the supplier relationship measured? is related to the associated construct of attitude toward relationship quality. The construct is addressed in the responses to the first qualitative survey question, QL1: Supplier consistently demonstrates a strong commitment to the relationship by being responsive, flexible, and easy to work with. The results from the survey show that the respondents mostly felt that the suppliers met or exceeded their expectations in relation to the quality of the relationship and commitment of the suppliers. Suppliers A and D again scored the highest at 94%, Supplier C had a score of 85% and Supplier B again had the lowest of the four with 84%. 4.4 Research Question 3 The third research question, How is the quality of supplier s resources evaluated? is related to the associated construct of attitude toward quality of supplier s resources. The construct is addressed in the responses to the second qualitative survey question, QL2: Supplier provides high caliber resources who consistently demonstrate a solid understanding of my business. 31

42 The results from the survey show that the respondents generally felt that the suppliers met or exceeded their expectations in regards to the quality of resources provided by the supplier. Supplier C scored the highest at 93%, Suppliers A and D each had a score of 90%, and Supplier D scored the lowest at 85%. 4.5 Research Question 4 The fourth research question, How are supplier s project management capabilities assessed? is related to the associated construct of attitude toward supplier s project management capabilities. The construct is addressed in the responses to the third qualitative survey question, QL3: Supplier exhibits strong project management competency and highly effective practices around communications, cost, schedule and scope management, accuracy and timeliness in invoicing/reporting while being fair and transparent in pricing, contracts and agreed upon Service Level Agreements (SLAs). The results from the survey show that the respondents largely felt that the suppliers met or exceeded their expectations in relation to their project management capabilities. Suppliers A and D each received scores of 92%, while Suppliers B and D both scored 88%. The average score among the four suppliers was relatively high, which indicates that the sampled suppliers tend to perform well in the project management activities. 4.6 Research Question 5 The fifth research question, What is the overall satisfaction level with the supplier s account management team? is related to the associated construct of overall satisfaction with supplier s account management team. The construct is addressed in the 32

43 responses to the fourth qualitative survey question, QL4: Supplier s account management team is always responsive, accessible, accountable, and collaborative. The results from the survey for this question show that the respondents were generally satisfied with the supplier s account management team, although the overall scores for this category were lower on average. Suppliers A and D both received scores of 88% and Suppliers B and C each scored 84%. The lower average scores among the four suppliers indicate that account management is an area where the suppliers tend not to perform well, but implementation of the SPM program should provide some necessary improvement to resolve this issue. 4.7 Research Question 6 The sixth research question, How is the quality of the services measured? is related to the associated construct of attitude toward service quality. The construct is addressed in the responses to the second and third quantitative scorecard questions, QN2: Percent of defect-free implementations and QN3: (Number of code defects)/($10k in application development work). The data presented in the supplier scorecards in Appendices D through G serves to determine whether each of the suppliers is meeting predetermined, contractual goals regarding service quality. All four suppliers scored above the goal of 95% for defect-free implementations, which suggests that this is not currently an area of great concern for the resources managing the IT Service suppliers. Although 3 of the suppliers met the goal for number of code defects per $10K in application development work, Supplier D fell very short of the target and this suggests a focus for the performance review as well as an opportunity for supplier improvement. 33

44 4.8 Research Question 7 The seventh research question How often are the services delivered according to schedule? is related to the associated construct of overall satisfaction that services are delivered according to schedule. The construct is addressed in the responses to the first, fourth, and fifth quantitative survey questions, QN1: Percent of implementations completed on time, QN4: Number of projects 10% behind schedule, and QN5: Number of projects 10% over budget. The data presented in the supplier scorecards regarding the overall satisfaction that suppliers are delivering services according to schedule indicates that performance in this area is very high. All four suppliers exceeded the goal of 95% of implementations completed on time, and all four suppliers met the goals of 0 projects 10% behind schedule or 10% over budget. These results imply that the suppliers are strong in this area of performance and that each works hard to meet the established objectives of the contract. 4.9 Chapter Summary The data offered in the survey and scorecard responses has provided the IT Category with useful preliminary information regarding specific performance measures of the company s prime IT service suppliers. The analysis presented in this chapter suggests that the research questions are directionally aligned with the survey and scorecard responses solicited as part of the SPM program. Additionally, these questions address service provider performance results that are aligned with and most critical to the IT organization s objectives. There appears to be significant material from the initial 34

45 SPM implementation to present management with meaningful results and to effectively plan for future quarters. 35

46 Chapter 5 Recommendation and Conclusion 5.1 Chapter Overview This chapter presents a summary of the research conducted and data reviewed and analyzed in support of the research objectives set forth in Chapter 1. The purpose of this project was to explore the ways that a SPM program can help the Southern California utility company to realize the full value of its contracts with IT service providers and to develop and implement a program in alignment with the best practices identified from the research. The methods applied to design, implement and evaluate the SPM program were determined by research focused on industry analysis and best practices for developing and maintaining effective supplier performance. 5.2 Research Summary The industry research on SPM programs promote the value that companies can realize with thoughtful consideration and implementation. The Southern California utility company studied in this project has invested extensive resources to establish contracts with strategic suppliers, and the benefits of a SPM program promise to help retain some of the value typically lost when supplier relationships are not managed effectively. The implementation of the SPM program is a time-consuming and resourceintensive process, so the aim of this project was to pilot the study with four prime IT service suppliers for a period before extending the program to additional suppliers. 5.3 Project Summary Chapter 1 identified the problem statement as the company s potential to experience a significant decline in service provider performance when suppliers are not 36

47 managed effectively. The lack of a formal SPM program was associated with poor supplier relationships, frustrated business clients, and diminishing value from established agreements. The research objectives to be addressed by the study were to demonstrate that a SPM program would help the company to more effectively manage its IT suppliers, to identify best practices to develop and implement such a program, and to associate leading SPM practices best aligned with the organization s performance management objectives. The literature review in Chapter 2 provided strong evidence that enhanced operations could be achieved through effective management of supplier performance. The first section of the literature review offered leading industry sources of information for Supply Management and presented supporting evidence of the benefits of SPM programs. The second section discussed industry leading methods for developing SPM programs as well as best practices for building a balanced scorecard to support such a program. The third section examined the challenges of maintaining effective SPM programs and presented the benefits of promoting continuous improvement techniques to preserve value within the program. The research conducted suggests that there is significant value to be retained and further achieved through the strategic development and implementation of SPM programs. Chapter 3 presents details regarding the research design, sample size, and procedures used to administer a program survey, as well as an overview of the methods used to analyze the resulting data. Research results indicated that successful SPM programs leveraged thoughtfully developed scorecards to allow organizations to measure and report appropriate performance management metrics, and it was determined that the 37

48 project should include both quantitative and qualitative metrics for survey. Two groups were selected to participate in the program; one group of internal client end users was selected to respond to the qualitative survey and the other group included IT services contract managers responsible for completing the quantitative scorecard. The collection of data was achieved with 84 responses to the qualitative survey and one response to the quantitative scorecard. The qualitative data was analyzed using Microsoft Excel to perform descriptive statistics analysis to validate the satisfaction of respondents with the service providers performance. The quantitative scorecard responses were also evaluated using descriptive statistics to calculate the rate at which the supplier s overall quarterly performance had met the established contract criteria and SLAs. Chapter 4 examined the data gathered from the surveys and scorecards to determine whether the results adequately addressed the objectives of establishing a SPM program. Results of the analysis were generally as expected, and service providers scored high in most capacities including resource caliber, project management, projects on schedule and within budget, and on-time and defect-free implementations. Service providers scored lower overall in areas of relationship quality and account management, but still within acceptable ranges of performance, and implementation of the SPM program should provide some necessary improvement to resolve the deficiencies. The analysis suggests that the research questions are directionally aligned with the survey and scorecard responses and that these questions address service provider performance results that are aligned with and most critical to the IT organization s objectives. 38

49 5.4 Implications for Management The information presented by this project is useful for management because work performed by the suppliers chosen for this study encompasses approximately 20-25% of the annual spend of the IT Category. A combination of a large volume of work and a high percentage of total spend makes this group ideal for introducing the practice of SPM across the organization. The initial quarterly results of SPM indicate areas for improvement and areas of solid performance for each of the four suppliers. The computed scores allow the company s managers to quickly identify suppliers that perform best in each category and provide for future work to be awarded to the most qualified candidates. An important implication is that management can more easily identify areas of concern with a supplier s performance to present in quarterly business reviews. 5.5 Recommendations for Management Although the results seem to indicate that SPM is a valuable practice, it is critical to remember that it must be properly managed to be effective. Based on the lower scores observed in the relationship quality and account management areas, it is recommended that the program participants should focus attention on these areas during initial quarterly review meetings. Another recommendation that should be considered is consolidating suppliers by commodity for more efficient analysis. This allows company resources to evaluate common suppliers using similar efforts and provides a simple and convenient way to compare like suppliers outside of a formal competitive process. The organization should centralize the operation of the SPM program to a concentrated group of resources. SPM programs are costly and time-consuming to establish and implement, and those 39

50 resources best prepared to execute the analysis should assume the work. A final recommendation is to continually evaluate the effectiveness of the SPM programs to identify areas of improvement. If multiple quarters of data demonstrate little to no variation or progress, there is probably a need to re-evaluate the program practices. 5.6 Conclusion The SPM program was eagerly anticipated by the IT Category at its inception following the first quarter of Several IT executives and managers were anxious to establish a program to evaluate the performance of suppliers that were receiving such a significant percentage of the annual budget. The industry research showed that cost savings and value retention were potential benefits that would eventually offset the expense of implementing a SPM program. After four quarters of evaluation, the Chief Information Officer and the Vice President of Supply Management jointly determined that there were diminishing returns provided by the program. While the effort has been put on hold for IT service suppliers, the work was extended to include other categories of suppliers, such as telecommunications providers and IT hardware and infrastructure resellers. The final conclusion as to the effectiveness of the SPM programs is that a company can gain value through the implementation of a carefully planned and executed platform, and continuous improvement is a key component of long-term success. 40

51 REFERENCES Aberdeen Group. (2002). The supplier performance measurement benchmarking report. Boston, MA: Minahan, T.A. and Vigoroso, M.W. Ariba, Inc. (2008). Ariba offers tips for effectively managing suppliers. Retrieved from Ariba, Inc. (n.d.). The Ariba network. Retrieved from Tips-Effectively-Managing-Suppliers Bhagwat, R. and Sharma, M.K. (2007). Performance measurement of supply chain management: A balanced scorecard approach. Computers & Industrial Engineering, 53, Brewer, P.C. and Speh, T.W. (2000). Using the balanced scorecard to measure supply chain performance. Journal of Business Logistics, 21 (1), Brown, C.M. (2010, December 30). 7 tips for rating and evaluating your suppliers and vendors. Retrieved from Ellram, L.M., Tate, W.L. & Billington, C. (2004). Understanding and managing the services supply chain. The Journal of Supply Chain Management, Fall 2004, esourcing Forum. (2006, July 30). Supplier performance management III: Best practices. Retrieved fro m Gordon, S.R. (2005). Seven steps to measure supplier performance. Quality Progress, 38 41

52 (8), Gordon, S.R. (2008). Supplier evaluation and performance excellence: A guide to meaningful metrics and successful results. Fort Lauderdale, FL: J. Ross Publishing. Hald, K.S. and Ellegaard, C. (2011). Supplier evaluation processes: the shaping and reshaping of supplier performance. International Journal of Operations & Production Management, 31 (8), Hebrand, V. (2013, September 6). Utilizing the true potential of supplier performance management. Retrieved from transformation-blog-capgemini/2012/08/utilizing-the-true-potential-of-supplier- performance-management Information Technology. (n.d.). Merriam-Webster.com. Retrieved from Kaplan, R.S. and Norton, D.P. (1992). The balanced scorecard Measures that drive performance. Harvard Business Review, January-February, Kaplan, R.S. and Norton, D.P. (1993). Putting the balanced scorecard to work. Harvard Business Review, September-October, Key Performance Indicator. (n.d.). Investopedia.com. Retrieved from Malina, M.A (2013). The evolution of a balanced scorecard. The Journal of Applied Business Research, 29 (3), MindLance, Inc. (n.d.). Outsourced application development & maintenance. Retrieved from 42

53 Procurement Strategy Council. (n.d.a). Benefits of effective supplier management. Retrieved from s=dd&utm_source=psc.executiveboard.com&utm_medium=home&utm_campaig n=topicslist&utm_content=title_supplier_relationship_management&utm_ter m=cid_ Procurement Strategy Council. (n.d.b). Who we are. Retrieved from Procurement Strategy Council. (2012). Scorecard development best practices (PSC SYN). Retrieved from Procurement Strategy Council. (2016). Compendium of supplier segmentation models (PSC SYN). Retrieved from Return on Investment. (n.d.). Entrepreneur.com. Retrieved from Rizza, M.N. (2006). Supplier performance management can make a difference. Supply Chain Management Review, 10 (7), 13. Service Level Agreement. (n.d.). CIO.com. Retrieved from 43

54 Southern California Edison Company. (2012). Supplier performance management scorecard and QBR process. Unpublished internal document. Strategic Sourcing. (n.d.). BusinessDictionary.com. Retrieved from Zikmund, W.G., Babin, B.J., Carr, J.C., & Griffin, M. (2010). Business research methods (8 th ed.). Mason, OH: South-Western Cengage Learning. 44

55 APPENDIX A: CONSTRUCT AND MEASUREMENT TABLE Construct Definition Operationalization of the Construct Source of the Measure Types of Scale Research Questions Type of Variables/Statistics Overall satisfaction QL5 Made Interval scale; 3 point Likert-type scale R1 Descriptive Statistics with supplier (1 = not satisfied & 3 = very satisfied) Attitude toward QL1 Made Interval scale; 3 point Likert-type scale R2 Descriptive Statistics quality of relationship with not applicable/don t know option (1 = supplier does not meet expectations & 3 = supplier exceeds expectations) 45 Attitude toward quality of supplier s resources QL2 Made Interval scale; 3 point Likert-type scale with not applicable/don t know option (1 = supplier does not meet expectations & 3 = supplier exceeds expectations) R3 Descriptive Statistics Attitude toward QL3 Made Interval scale; 3 point Likert-type scale R4 Descriptive Statistics supplier s project with not applicable/don t know option management (1 = supplier does not meet expectations capabilities & 3 = supplier exceeds expectations) Attitude toward QL4 Made Interval scale; 3 point Likert-type scale R5 Descriptive Statistics supplier s account with not applicable/don t know option management team (1 = supplier does not meet expectations & 3 = supplier exceeds expectations) Quality of application maintenance and development QN2 QN3 Made Ratio scale; actual scorecard data used R6 Descriptive Statistics Cycle time of QN1 Made Ratio scale; actual scorecard data used R7 Descriptive Statistics application QN4 maintenance and development QN5

56 APPENDIX B: QUALITATIVE SURVEY 46 Source: Southern California Edison Company, 2012.

57 APPENDIX C: QUANTITATIVE SCORECARD 47 Source: Southern California Edison Company, 2012.