(Getting & Giving) Superior Service through Measurement Bill Phifer ISMA Conference September 2008 1 / /
Agenda 2 /
The Bait 3 /
Statement(s) of Fact Poor performance by the service provider is the leading cause of organizations bringing services back in-house. This is often the result of misaligned expectations and lack of explicitly determining the expected outcomes of the service being provided. The majority of North American organizations outsourcing today are doing so without a formal sourcing strategy. Gartner, 2006 4 /
An Analogy Trying to determine service provider progress without measurement is like flying without instruments! 5 /
A Joke Tech Support: What s on your screen right now? Customer: A stuffed animal that my boyfriend got me at the grocery store. 6 /
A Story Are you a good customer? or A bad customer? 7 /
The Pitfall 8 /
Outsourcing Failures KPMG Survey of 650 organizations: 47% said service provider brought experience they didn t already have 27% said sourcing has improved competitiveness 87% said the statement that 50% of outsourcing deals fail is an oversimplification and doesn t really reflect reality 72% do not have defined measurement criteria for success 90% didn t understand opportunity costs of sourcing 79% couldn t identify the cost of the sourcing selection process 9 /
The Zero-sum Relationship John Nash Theory of Equilibrium When two or more players are involved in a competitive (noncooperative) situation, with all things being equal, one is going to win and one is going to lose. However, if they agree to collaborate and work together for a shared (cooperative) gain, both can likely win. 10 /
Unanswered Questions? We are all customers and service providers! Client: what will you do for me (or what have you done for me)? Service Provider: what are your needs, and how will I measure results against them? 11 /
Some (Bad) Current Outsourcing Trends As smaller organizations move to outsourcing to reduce costs, lack of sourcing and contract negotiations experience often lead to poor business outcomes from outsourcing Post-contract sourcing advice is often minimal Transparency to outsourced operations is often lacking, as vendor management functions are overloaded Outsourcing vendor sales and delivery are sometimes not aligned, leading to unrealistic expectations that can not be met Source: Forrester, 2006 Sourcing is a tool - it is only as good as those managing it 12 /
Expectations - 1 I want a quality product She needs us to deliver faster Is this a shared understanding? 13 /
Expectations - 2 We need to close Service Desk tickets more quickly Their Service Desk needs to resolve more incidents with the first call Is this a shared understanding? 14 /
What is Value? Value is defined as the customer s overall assessment of the usefulness of a product or service based on the customer s perception of the realized benefit versus the cost. Useful Quality Cost Benefit 15 /
What is Customer Satisfaction? Customer Satisfaction is defined as the difference between the customers expectation for a product or service and the customers perception of the delivered product or service. 16 /
Client/Service Provider Gap Model Perception (Gap 1) (Gap 2) (Gap 3) Customer Expectations Word of Mouth Customer Needs Past experience Perception of Customer Expectations Product or Service Specification External Communications to Customer Service Delivery -People - Environment - Technology - Process Improvement - Systems and Methods Delivery Customer Perception of Product or Service (Gap 4) Performance 17 /
In Search of Client Service Excellence Basic technology metrics alone do not drive the right behavior or results Rigid contracts have been proven not to meet business needs Clients want to look beyond technology to the business drivers for IT IT is not the client 18 / Source: Forrester, 2005
Hard Questions How do you evaluate progress against customer goals for sourcing? Value received? Savings (time, money)? Completion of dirty jobs that we can t or won t do? Cost of capital (NPV)? Ability to focus on core business? 19 /
Measurement s Fault? A 2005 Forrester survey indicated that despite the fact that outsourcers were broadly meeting their clients service level agreements (SLAs), many described outsourcing relationships as difficult and uncomfortable. Instead of falling back on measuring what they know and are comfortable with usually the traditional IT operational metrics both sides should align performance measurement with the key objectives and expectations of the outsourcing initiative and couch metrics in terms of business outcomes. 20 /
So Can we agree that: 1. Clients do not start out as experts in sourcing (or measurement) 2. Measurement is needed to ensure that service provider commitments are achieved 3. Reasonable and achievable service level agreements are a start but not sufficient 4. Zero sum contracts and relationships don't work; both sides must gain from the relationship 5. The relationship ultimately should be evaluated on business value that comes from the synergy of client organization and service provider 21 /
The Rescue 22 /
Two Models for Improving Outsourcing escm-sp: for Service Providers escm-cl: for Clients The esourcing Capability Models from Carnegie Mellon University s ITSqc are designed to share a set of sourcing best practices that benefit both clients and service providers 23 /
Why Another Model? escm is a proven (but still emerging) standard for ITenabled sourcing This best practice model works equally well with insourcing, multi-sourcing and full outsourcing The escm models provide a clear scope of capabilities needed and a framework for evaluation We should measure progress related to a standard rather than simply define a baseline for improvement escm Adapted from the Systems and Software Consortium Quagmap TM 24 /
How to, so what 25 /
Three Dimensions of the escm-sp v2 26 /
Capability Levels of the escm-sp v2 27 /
esourcing Capability Model for Service Providers Measurement-related Capability Areas escm-sp Knowledge Management People Management Relationship Management Performance Management Technology Management Threat Management Contracting Service Design & Deployment Service Delivery Service Transfer 28 / Source: ITSqc, 2006
esourcing Capability Model for Clients Measurement-related Capability Areas escm-cl Sourcing Strategy Mgmt. Governance Management Relationship Management Value Management Org. Change Management People Management Knowledge Management Technology Management Threat Management Sourcing Opportunity Analysis Sourcing Approach Sourcing Planning Service Provider Evaluation Sourcing Agreements Service Transfer Sourced Services Management Sourcing Completion 29 / Source: ITSqc, 2006
Distribution of Practices (escm-sp v2) Capability Level Phase Capability Area 2 3 4 Totals 51 Ongoing Knowledge Management 3 4 1 8 People Management 3 7 1 11 Performance Management 3 3 5 11 Relationship Management 3 4 1 8 Technology Management 4 1 1 6 Threat Management 6 1 7 21 Initiation Contracting 9 2 11 Service Design and Deployment 6 2 8 Service Transfer (in) 2 2 8 Delivery Service Delivery 7 1 8 4 Completion Service Transfer (out) 2 1 1 4 TOTALS 48 26 10 84 30 /
A Simple Sourcing Measurement Roadmap 31 /
Relevant Level 2 Practices (escm-sp) Engagement Objectives (prf01) Define, communicate, and track engagement objectives Verify Service Commitments (del04) Establish and implement procedures to measure and verify that service commitments are being met Verify Processes (prf02) Establish and implement procedures to verify that processes and consistently performed as defined Resource consumption (knw08) - Establish and implement procedures to analyze and use information on resources consumed 32 / Source: Carnegie Mellon University
Engagement Objectives (prf01) Determine engagement objectives Determine measures to track engagement performance Collect data on engagement performance Periodically review performance against the engagement objectives 33 /
Verify Service Commitments (del04) Determine measures to track service performance Collect data on service performance Analyze the data and compare current values and trends against range of acceptable performance Identify gaps between the actual and the planned performance Maintain the data and analysis for future use 34 /
Verify Processes (prf02) Determine measures to track process performance Create a plan for verifying processes Collect data on process performance Periodically review adherence of the process to the specified requirements 35 /
Resource consumption (knw08) Identify the resource data that needs to be collected, aggregated and analyzed Select and document the methods used to collect, aggregate, and analyze the identified resource data. Document resource utilization information for the tasks performed on each client engagement. Aggregate and analyze the data collected Identify and document any ine ciencies in resource utilization by comparing actual to planned utilization levels. 36 /
Relevant Level 3 Practices (escm-sp) Organizational Objectives (prf04) Define, communicate, and track organizational objectives Review Organizational Performance (prf05) Establish and implement procedures to review organizational performance Process Assets (knw04) Establish and maintain a set of process assets for use across the organization 37 / Source: Carnegie Mellon University
Organizational Objectives (prf04) Determine organizational objectives Address the engagement objectives at appropriate levels in the organization Address the organizational objectives at appropriate levels in the organization (functional, department, team, individual) Determine measures to track organizational performance Track status and progress against the plan for organizational objectives 38 /
Review Organizational Performance (prf05) Collect data on organizational performance Periodically review performance against the organizational objectives Determine corrective actions to take when performance trends deviate from achieving the objectives Track the corrective actions, taking action as appropriate 39 /
Process Assets (knw04) Establish a measurement repository to contain the data collected for the process assets 40 /
Relevant Level 4 Practices (escm-sp) Capability Baselines (prf08) Define capability baselines for the organization by analyzing performance data Benchmark (prf09) Benchmark organizational performance to identify opportunities for improvement 41 / Source: Carnegie Mellon University
Capability Baselines (prf08) Select the capabilities to baseline Identify the performance measures to be used for the capability baselines Collect performance data for identi ed measures Analyze performance measurements to derive capability baselines Periodically review data from performance measurements and maintain the capability baselines, as appropriate 42 /
Benchmark (prf09) Identify processes to benchmark based on organizational objectives and client input Identify the best practice processes to use as the basis for comparison and the information required to create the comparison Create the approach used for comparing data between the best practice and the organizational processes Collect internal data on the organizational processes to benchmark, as well as the best practices to benchmark Compare the performance of organizational processes to best practices, identifying gaps in performance Identify the improvement programs that are needed as a result of benchmarking 43 /
Outsourcing Hierarchy of Needs The evolution of client relationships follows a hierarchy of needs: Until the basics are right, you have not earned the right to move up the pyramid of satisfaction Basic Sourcing Measurement True Strategic Partnership Business Transformation Initiatives Business-oriented Metrics Technology Service Levels 44 / Source: Forrester, 2005
Sourcing Relationship Analysis In studies of less successful sourcing relationships, a hard commercial process often left the vendor with no room to deliver and still remain profitable, or there was a complete mismatch of culture or expectations - in spite of growing recognition of the importance of cultural fit For both sides, best practices that promote successful outsourcing contracts include understanding how to measure and manage soft issues like culture and innovation and building a mutually beneficial relationship that doesn t stand or fall by the letter of the contract. (or)? 45 / Source: Forrester, 2005
Context for Sourcing Measurement What to measure??? 46 /
Why Companies Outsource Support change in overall management philosophy Allow companies to focus on core competencies Convert relatively fixed allocated cost to variable direct costs Take the IT function to the next level of capability Improve time to market Financial reasons Facilitate best of breed implementation Leverage business critical infrastructure Change the culture and re-skill IT Extend global reach to international locations Reduce costs Shift costs to later years Fund development Dispose of overvalued assets Form preferred supplier relationships Reduce procurement costs Improve service levels Secure access to best resources Convert legacy system resources Focus resources on new development 47 / Source: TPI
Information Needed Productivity Quality IT Infrastructure Service Delivery Level Attainment Business Value OBJECTIVES Finance/ Budget Process Conformance Customer Satisfaction/ Loyalty Human Resources Time/ Schedule 48 /
Sourcing Services and Measurement Example Sourced Service Call center operations Attributes Measures Data Time to respond Number of minutes on hold before being answered 2.5 minutes on hold Indicators Less than 2 minutes 49 /
Measurement Activities by Phase & Level escm-sp Level/ Phase Level 1 Level 2 Level 3 Level 4 Level 5 Initiation Baseline current performance; estimate costs, staffing and technology performance. Build estimation model, tracking model, and technology performance & capacity model. Implement knowledge sharing & optimization. Optimize models. Delivery Contract specific measures Track & control engagement and supplier/ partner performance. Measure & improve organizational performance. Implement performance engineering. Maintain continual improvement. Completion Perform root-cause analysis; account for resources transferred back. Institute organizational measurement; maintain service continuity. Transfer knowledge. Ongoing Profit & performance Maintain engagement-byengagement performance & status. Perform engagement resources and conformance measurements Maintain consistency. Reduce variation from engagement to engagement. Achieve internal business objectives. 50 / Benchmark internal capability baselines and external businesses. Seek worldclass innovation. Paulk, et al. 2005 Maintain continual improvement.
The Role of Function Points (For application outsourcing) Size the application portfolio (establish a baseline to measure productivity over time) Measure new functionality provided for application development or changes "Maintenance scope" (# of FPs maintained per resource) Must provide context such as complexity and maintainability 51 /
Service Level Agreements Experience has taught companies how to develop a robust set of SLAs for the contract. But without context and further levels of control, there will never be a mutually beneficial relationship. Both sides should align performance measurement with the key objectives and expectations of the outsourcing initiative and couch metrics in terms of business outcomes. 52 /
A Word on Innovation If it's not in the contract or cost model, clients should not expect it (innovation costs money) However, this is mattering less to clients, who believe they are paying for continuous improvement and then not getting it. How do you measure innovation? 53 /
The Call to Action 54 /
To Do s Download the escm models and review them Map your current processes to the model practices at various levels (engagement, industry, enterprise) if you are a client, consider using escm- CL as a model for sourcing effectiveness; apply it together with your service provider If you are a service provider, consider adopting the model and perform a baseline appraisal and bring your clients along 55 /
Recommendations (1) Establish clear rules of engagement, governance and transparency (program status, issues, and risks) between parties Then focus on mutually establishing fair measures of performance for service providers that tie to the business objectives of the client; make exceptional performance financially rewarding Develop a community of trust by participating in mutually beneficial collaborations that advance value creation for the client and also provide a fair return for the service provider 56 /
Recommendations (2) Align cultures your common attitude to people, processes, performance, values, outlook, and expectations of each other. Commit the resources you ll need to build and actively manage your sourcing relationships Measure performance as quantitatively as possible to supplement the trust you ve built with your service providers Tie performance to profitability (and penalties) 57 /
Begin with the End in Mind Be a good customer - tell your service provider exactly what you want and then measure to your expectations Be a good service provider - identify a baseline of performance and then manage to improve it Both: negotiate a contract that is fair to both and that provides synergies and winwin opportunities (rather than zero-sum) 58 /
Summary Consider the value of the client & service provider relationship as pivotal to the success of the sourcing initiative Look at the Carnegie Mellon University IT Services Qualification Center esourcing Capability Models to enhance client & service provider relationships Measure success! 59 /
Thank You! Questions? 60 /
Bill Phifer EDS Quality Management 1550 Liberty Ridge Suite 120 Wayne, PA 19087 +1 610 232 5203 bill.phifer@eds.com or eds.com EDS and the EDS logo are registered trademarks of Electronic Data Systems Corporation. EDS is an equal opportunity employer and values the diversity of its people. 61 /
References Harris, I., Parker, A. (2005). The IT outsourcing satisfaction paradox - delivering to SLAs is not enough. Forrester Best Practices. December 15, 2005. Hefley, W., Loesche, E. (2006). The esourcing Capability Model for Client Organizations: Practice Details, V1.1. Pittsburgh: Carnegie Mellon University IT Services Qualification Center. Hyder, E., Heston, K., and Paulk, M. (2004). The esourcing Capability Model for Service Providers V2. Pittsburgh: Carnegie Mellon University Institute for Software Research International. Kern, T., Willcocks, L. (2001). The Relationship Advantage: Information Technologies, Sourcing, and Management. Oxford: Oxford University Press. Paulk, M., Dove, S., Guha, S., Hyder, E., Iqbal, M., Jacoby, K., Northcutt, D., and Stark, G. (2005). Measurement and the esourcing Capability Model for Service Providers v2. Pittsburgh: Carnegie Mellon University Institute for Software Research International. Roehrig, P. (2005). Outsourcing success requires collaborative program management. Forrester Trends. November 18, 2005. Willcocks, L., Feeny, D. (2006) IT outsourcing and core is capabilities: challenges and lessons at Dupont. ISM Journal, Winter 2006 62 /