building a performance system for it White paper

Transcription

1 building a performance system for it White paper

2 Table of contents Eecutive summary 3 Key requirements for an IT performance system 4 The anatomy of a performance system 4 Governance 4 Delivery 4 Measurement 4 Building a performance-oriented culture 5 The HP IT Performance Suite 5 The HP IT Eecutive Scorecard: a system of measures 6 Establishing a common data model 7 Building and using your system of measures 8 Key takeaways 8 Suggested net steps 8 Appendi: key performance indicators 9 Customers 9 IT value 10 Future orientation 11 Operational ecellence 12

3 Eecutive summary Every leader in today s modern enterprise should have a performance system a means to transparently measure and manage progress against the goals they are given. With $5 trillion per year invested in enterprise IT, it is essential that each IT leader has a digitized system that enables them to manage and improve the key performance indicators (KPIs) that matter to the organization. This white paper documents the requirements of an effective IT performance system and introduces a set of associated measures and best practices that are both aspirational and practical. In addition, the paper outlines a framework of key performance indicators to help IT perform better and presents specific KPIs to consider for leaders within different IT domains.

4 Figure 1. Elements of an IT performance system. Governance and alignment Customer Sat. Future Orientation IT Value Operational Ecellence The anatomy of a performance system An IT performance system consists of three key functional layers: governance, delivery, and measurement, as indicated in Figure 1. Delivery - people, processes and tools Narrative Governance The process of direction and evaluation of IT from a business outcomes perspective. This is also where performance objectives are agreed and documented, establishing a balanced scorecard comprised of key performance indicators which are set and tracked for variance from plan. IT staff Backup Testing Measurement and relationships Server Automaton Etc... Processes By providing a written as well as visual history of progress against goals, a scorecard enables your stakeholders to maintain the story behind the numbers, or narrative. This is especially important when performance goals are not being met. In these situations, a documented narrative serves as a valuable record that can help isolate rootcause information and/or historical contet that would otherwise not show up in a purely data-driven dashboard. Performance Data Data Model Key requirements for an IT performance system To be effective, an IT performance system must meet three requirements: 1. Comprehensive. Your CIO and IT leadership team collectively require oversight and control across all of the people, processes, and systems in the IT domain, from infrastructure to information, applications, and security, and from planning to development and operations. You cannot manage what you cannot measure, so comprehensive visibility into the performance of the entire IT investment portfolio is required for success. 2. Connected. To make informed decisions, IT leaders need a dynamic performance system comprised of IT management and security tools that not only automate their discrete functions, but also deliver insight by feeding digital updates to their KPIs in a timely manner. IT leaders should be able to view and share their KPIs in a series of connected and cascading scorecards so that each leader s KPIs support their manager s up to the CIO level. 3. Fleible. An IT performance system needs to provide the fleibility to enable your IT leaders to set and evolve the KPIs that map to their unique priorities, create scorecards that align with their organizational structures, and select the underlying IT management tools that automate and run inside their diverse and heterogeneous operating environments, without creating vendor lock-in. The Balanced Scorecard: Translating Strategy into Action. Robert S Kaplan, David P. Norton, ISBN Delivery This is where performance is improved, sustained, or reduced. The delivery aspect of a performance system encompasses the people, processes, and technology involved in the day-to-day delivery of new and eisting IT services. There are three critical subsets inside every organization: People including employees, contractors, and vendor partners. Processes typically a hybrid of organization-specific best practices, industry-standard frameworks (such as COBIT, ITIL, TOGAF, ISSA, EDRM), and enterprisespecific processes and cultural norms, which can be documented and/or formalized within an IT performance system. Technology the software systems used by IT leaders and their teams to automate, monitor, and measure specific IT functions. Typically tools vary by role or function. A chief information officer (CIO) will likely focus on high-level tools for strategy, planning, and governance, including tools for financial planning and analysis, portfolio and asset management, and workforce management. A vice president (VP) of applications and his or her organization will likely use tools for application lifecycle management, including quality assurance, project and portfolio management, and business service management. The VP of operations, chief information security officer (CISO), and VP of information and their teams will similarly use specialized tools for ensuring the success of their functions. Measurement This is the functional layer where performance metrics are created and performance data is stored. Continual improvement also requires that IT establish a common data model which enables the consolidation of facts from disparate sources. To allow for eception-based management and continual improvement, these measurements should be made consistent such that they can be compared across the processes, systems, and functional teams outlined above. 4

5 Figure 2. The HP IT Performance Suite Eecute systematically Strategy, planning and governance Build faster Application lifecycle management Secure proactively Security intelligence and risk management Operate simply management Store efficiently lifecycle management IT Performance Suite foundation Collaboration, orchestration and analytics Building a performance-oriented culture If you are establishing a performance system, fostering a performance-oriented culture is a critical success factor. A performance system typically brings with it a level of transparency of results that, unless suitably prepared, can intimidate employees and in some cases create behaviors that lead to poor outcomes. For eample, your organization might seek to measure percentage (%) first call resolved rates, a common indicator of operational ecellence within a help desk environment. However this goal, if applied without establishing a percentage (%) incidents reopened or customer satisfaction KPI, may lead to help desk staff closing incidents prematurely in order to meet their target at the epense of the desired result in this case, a quality outcome and a satisfied customer. As a result of both of these factors, you should plan for goals to be implemented as part of a balanced system of measures and accompanied by a structured organizational change management program. The HP IT Performance Suite Based on years of eperience working with enterprises around the world, HP has created the HP IT Performance Suite, (see Figure 2), a comprehensive, connected, and fleible portfolio of software and best practices that enables your IT leaders to run IT as a business. The suite is a complete IT performance system that provides you with the confidence and insight to always perform better. Enabling the systematic eecution of cascaded goals across the HP IT Performance Suite is the HP IT Eecutive Scorecard, a single pane of glass into the performance of your entire IT portfolio. 5

6 Figure 3. Four key reporting areas of an IT Eecutive Scorecard. Operational ecellence Customers Common data model Future orientation The HP IT Eecutive Scorecard: a system of measures IT value Critical to delivering better performance is the ability to set, cascade, and measure KPIs such that they become part of the day-to-day behaviors of your IT organization. HP research has identified 170 KPIs associated with world-class IT service spanning all key IT processes and organizations. Once set, each KPI can then be cascaded through your organization to drive alignment to the performance initiatives of the entire team, effectively establishing a common language to describe IT performance. The IT Eecutive Scorecard automates the collection, presentation, and tracking of a growing number of KPIs that populate a system of scorecards that can cascade up and down the IT leadership team. You can compose your own performance system from a KPI library or start with one of the persona-specific editions HP has created for the CIO and key IT function leaders, and then etend it as needed. Each HP IT Performance Suite edition features the HP IT Eecutive Scorecard software bundled with a set of software tools that improve domain-specific performance and automatically populate the KPIs with up-to-date status. This approach is designed to provide rapid benefits realization in terms of measuring and improving your performance along dimensions of cost, quality, innovation, and your organization s readiness to address future demands. The data is organized into the four categories: IT value, customers, operational ecellence (including risk and security), and future orientation, all of which are driven from HP s common data model based on the HP Universal Service Model architecture (Figure 3). The IT value element of an IT balanced scorecard seeks to capture and categorize business and IT alignment at the highest level, primarily around areas such as alignment with business strategy, the stewardship of the overall IT spend, and improved return on investment as a function of value versus cost. The customers element of the IT balanced scorecard tracks improvement or declines in metrics that impact customer satisfaction, such as service-level attainment, as well as qualitative metrics, such as customer survey results. Highly correlated to customer satisfaction are service availability and the duration, mean-time-to-repair (MTTR), and number of open incidents associated with a service. The operational ecellence element of the IT balanced score card seeks to track and maimize the throughput of planned work, such as new projects. This element focuses on streamlining and automating processes for unplanned or reactive work, such as help desk incidents, as well as metrics that reflect the quality and effectiveness of the broader delivery process, such as measure of rework, escalations, and work backlog. Here also, you track security, risk, and compliance metrics. Finally, to enable your IT leaders to better respond to change, the IT balanced scorecard tracks the future orientation of the IT function. In addition to tracking information about the age and compleity of major IT systems, the CIO edition also tracks measures related to human capital, such as the relative contribution of the eternal versus internal labor required to respond to enterprise IT demands, which provides a measurable proy of internal readiness. 6

7 Figure 4. Delivering KPI-based Scorecards on top of an Open Data Model Eecutive scorecard Scorecards, KPIs/metrics, dashboard pages KPI library KPI engine Collaboration Administration Open data model Cost Incident SLA Service Deployment Project Asset Customer Vendor Change Contract Configuration Security Data integration, consolidation, stewardship XLS Qualitative Finance Configuration Business services Application Lifecycle Lifecycle Risk and Security Establishing a common data model A comprehensive and connected IT performance system requires a consistent data model that enables consolidation, sharing and comparison of trusted metrics which form a basis for continuous improvement. When establishing a common data model, it is important to use a comprehensive model that is independent of any particular source system and open to any and all source systems, so that the entirety of IT can be covered, including today s public Cloud service providers, and so that source changes do not affect performance history as seen by the governance user. The primary consideration when importing measurement data is enabling consistency across data sources. In order to support both quantitative- and qualitative-based data, the HP IT Performance Suite ETL architecture allows for data to be imported and normalized across both traditional SQL databases and flat files, such as spreadsheets or tabular data. HP, along with a number of Fortune 500 companies, has also led the creation and publication of an open, industry-standards-based model for configuration meta-data (refer: open data model). This data model allows you to formalize, specify, and query the relationship between business processes, applications, infrastructure, and service providers in a consistent manner using industry standards, such as TQL and XML. Further information on the open data model can be found in the references section of this document. 7

8 Figure 5. An IT Performance System provides scorecards that cascade to multiple levels of IT leadership. Office of the CIO Project Office VP Application Delivery Business Analyst Quality Assurance CIO Development VP Infrastructure & Network Manager Building and using your system of measures Chief Security Officer Service Desk Manager Having established a functioning data model and structured a balanced scorecard, the net step is to determine what KPIs to track at what level of the IT organization. While there are no hard rules, there are a number of best practices to consider as you structure your own system of measures and scorecards: 1. Less is more. A few, well selected KPIs on each leader and individual contributor s scorecard is more likely to be successfully eecuted than a comprehensive but overwhelming one. 2. IT is a team sport. Scorecards and associated KPIs should cascade clearly across the IT leadership team (see Figure 5). Many scorecards reinforce silos within IT by creating isolated KPIs for each functional team in the delivery chain. While only one functional team should be accountable for attaining specific KPIs, IT leaders should ensure at least one of the cascaded goals is heavily weighted toward the end-to-end attainment of enterprise outcomes. 3. You get what you inspect, not what you epect. A transparent rewards and measurement system can have a powerful effect on driving behavior. Where possible, ensure that metrics are balanced such that behaviors reflect the broader objectives of the organization. 4. Focus on benchmarks and continuous improvement. Benchmark and continuously improve equally important to defining what KPIs to measure is establishing the benchmarks and goals. KPIs that are unrealistic reduce the IT team s confidence of achieving them equally, KPIs that are too easy to achieve may translate into sufficient effort put into improvement. By constantly testing and measuring, comparing inside the organization and outside against industry peers, you can set benchmarks to drive continuous improvement in a meaningful manner. Key takeaways In order to always perform better, CIOs and IT eecutives need a digitized and automated system that continuously measures, communicates, and improves IT performance by instrumenting the entire IT function in a comprehensive manner. To gain timely insights and provide fact-based evidence of how IT supports the business, it is essential that such a performance system dynamically updates KPIs and is connected to every element of the IT landscape. The IT performance system should provide the fleibility for IT to move fast, leverage eisting capabilities, operate in a heterogeneous landscape, and eploit new technology and opportunities to drive better business outcomes. The HP IT Performance Suite not only meets these core requirements comprehensive, connected, and fleible but also offers your IT leaders rapid deployment editions for KPIs and scorecards to accelerate their time to insight and promote greater confidence. Suggested net steps If you are interested in establishing an IT performance system, you can select from a range of HP workshops, including: IT Performance Workshop that accelerates alignment around an IT end-state vision required to deliver the desired enterprise outcomes in the contet of the current shifts in IT production and consumption models (four-hour workshop onsite, with follow-up) Value Discovery Workshop that develops a roadmap for key transformational IT initiatives, including people, process, technology, and related KPIs (approimately four days onsite) Readiness Workshop that assists in building out process-and-tool specific implementation plans for better business outcomes (one to two days onsite) If you are interested in sharing and improving best-practices with your peers, join the Discover Performance Community: 8

9 Table 1. Customers Appendi: key performance indicators HP research has identified a number of KPIs frequently associated with measuring performance at leading IT organizations across the domains of customers, IT value, operational ecellence, and future orientation. The actual number and specific KPIs implemented by customers will differ based on the performance framework they design as well as their unique culture and performance initiatives. Note: We have not provided targets and best practice levels for KPIs, as they vary by industry and need to be tailored to each customer s circumstances. Also, in some cases, there are multiple ways to measure a KPI (such as customer satisfaction) and this paper does not prescribe a single formula for measuring a KPI. HP and our partners, as well as third parties, offer workshops and other programs to help customers select KPIs, determine appropriate measurement criteria, and apply best practice benchmarks. Personas Solution Domain Optimization Opportunity/KPI CIO VP of VP of Applications Applications % deviation from planned hours worked Applications % of healthy projects Applications % of project tasks on time Applications % of projects on time Applications % of projects with unresolved urgent issues VP of Chief Security Officer Applications Customer satisfaction via survey (sometimes in lieu of timeliness and/or budget KPIs) Improve user access to information Time to respond to e-discovery # unplanned outages per month % of IT budget spent on maintaining eisting systems % of met SLAs % of met SLOs for IT process Activities % of met SLAs % of met SLOs for IT process Activities % of outages/total SLA uptime over time % of satisfied customers % of service availability % of service performance not met Availability & performance SLAs of mission critical applications Avg. outage duration per incident Customer satisfaction for help desk Hours monthly downtime Mean time to implement changes Mean time to repair incidents Mean time to resolve incidents & alerts MTBF of business services MTTR of business service incidents 9

10 Table 2. IT value Personas Solution Domain Optimization Opportunity/KPI CIO VP of VP of Applications Applications % of change in projects cost Applications % of projects - cost reduction VP of Chief Security Officer Applications % of projects associated with business objectives Applications % of projects budget at risk Applications Innovation - hours worked on new vs. maintenance Applications On budget variance Financials % of actual vs planned projects cost Financials % of cape vs ope spending Financials % of change in business service cost Financials % of IT POR vs. total revenue Financials % variance of actual vs. planned costs Financials Avg cost of IT delivery per customer Financials Business service cost reduction Financials Innovation delivery % of data/information stored with 3rd parties % of information management processes that are automated Customer satisfaction with information management processes Past budget vs. current budget for data/information stored in cloud Fully loaded cost per terabyte ROI analysis % of assets cost reduction % of software license in use Availability & performance SLAs of mission critical applications, especially cloud providers (How close am I to breaching a contract?) Cost of monitoring per application % of IT budget spending on ope Revenue loss per outage The revenue impact of a business service outage 10

11 Table 3. Future orientation Solution Domain Optimization Opportunity/KPI CIO VP of VP of Applications Applications % of employee utilization rate Personas VP of Chief Security Officer Applications % of FTE Applications % of project effort done by eternal resources Applications % of project effort done by eternal staff Applications Employee retention/morale Litigation readiness to support in-house, automated, repeatable e-discovery process % of employees eceeding leadership competency model % of satisfied employees Employee turnover 11

12 Table 4. Operational ecellence Personas Solution Domain Optimization Opportunity/KPI CIO VP of VP of Applications Applications Avg delivery time for new products or services Applications Avg project initiation time Applications Effort variance VP of Chief Security Officer Applications Nimbleness Mean duration from change request to production Applications On time delivery variance Applications Quality (defect leakage, normalized by % per function point or hours worked) Applications Resource utilization Applications Time-to-market of new products or services # of data leakages/losses # of past platforms vs. current # of platforms # of sanctions or penalties for non-compliance with discovery or regulatory request % improvement of staff productivity and process improvements % of information assets governed by proper access controls % of legacy data classified as epired and disposed of % of records classified with retention policy % of storage mapped to tiered storage profiles % of tests/simulations passed measuring impact on business services through policy non-compliance % of user information/data backed up under retention compliance policies % virtual server/storage implementation Active user processes % of business records with retention policy Compliance audit Compliance/regulatory mandates supported Control storage capacity with deduplication, compression and storage tiering Dispositions scheduled, completed by number and storage size Hours worked/time spent searching for information Implement policy based information archiving strategy Ingestion volumes 12

13 Table 4. Operational ecellence continued Personas Solution Domain Optimization Opportunity/KPI CIO VP of VP of Applications VP of Chief Security Officer Number of Data sources Past storage cost vs current storage cost Pursue/implement cloud services strategy Current and tested backup and recovery architecture Retention schedules, by number and storage size Tier 1 storage growth Total records/documents/objects stored Upgrade disaster recovery and business continuity capabilities Headcount ratio storage capacity to one admin Mean time to identify storage bottleneck Storage capacity reclaimed % events processed without human intervention % of assets in maintenance % of assets returned to supplier % of changes resulting in outage % of devices compliant with corp policies % of devices compliant with regulatory policy % of emergency changes % of escalated Incidents % of FCR % of incident aging % of interactions in backlog % of network service availability % of outages due to changes % of problems resolved within the required time period % of reopened Incidents % of SLA epirations % of SLAs coverage % of unauthorized implemented changes % of unplanned changes % of urgent changes % problems with RC % servers under management % storage utilization Application (custom app) provisioning (hrs.) 13

14 Table 4. Operational ecellence continued Personas Solution Domain Optimization Opportunity/KPI CIO VP of VP of Applications Availability of network devices Average age of hardware assets Average time to procure Time needed to achieve, percentage correct Change, configuration management Accuracy Closure duration Database provisioning (hrs.) Ensuring optimal connectivity Headcount ratio client devices to one admin Headcount ratio databases to one DBA VP of Chief Security Officer Headcount ratio network devices to one admin Headcount ratio servers to one admin Infrastructure utilization (server, storage) Mean time between failures Mean time to resolution for network incidents Mean time to restore services Mean time to resolution for network incidents Network efficiency/latency Network provisioning (hrs.) Number of decommissioned servers % of IT budget spending on ope Physical server provisioning (hrs.) Problem queue rate Storage provisioning (hrs). Virtual server provisioning (hrs.) VM to server ratio Changes to backlog size Security Security Security Security Number of security and compliance (corporate, government, industry) violations Security and compliance (corporate, government, industry) Simulation/change & configuration impact on security and compliance Status of organizational controls for each compliance area Status of compliance to internal policies Device coverage (%) devices protected by security controls Number of security incidents open/closed, response time Patch latency (avg missing patches/machine, days old) Status of on-going posture improvement efforts as aligned to priorities and objectives Platform compliance (0-10) ports open, default passwords/policies, etc. mapped to CIS controls 14

15 Table 4. Operational ecellence continued Personas Solution Domain Optimization Opportunity/KPI CIO VP of VP of Applications VP of Chief Security Officer Regulatory controls top out-of-compliance controls Level of risk by asset class (quarterly) Accepted loss epectancy (annual, by asset class, $ revenue) Safeguard implementation status Total accepted cumulative risk (dynamic measure) Data loss prevention controls Status of security project activities and remediation/cases (time/budget) 15

16 Get connected Get the insider view on tech trends, alerts, and HP solutions for better business outcomes Share with colleagues Copyright 2011 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. The only warranties for HP products and services are set forth in the epress warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein. 4AA3-8565ENW, Created November 2011