Chapter 13: Performance Monitoring

Transcription

1 Chapter 13: Performance Monitoring What is performance monitoring? Performance monitoring is the process of monitoring and reviewing the performance of asset management and the performance and/or condition of assets. The performance framework discussed in Chapter 5 creates a top-down connective thread from organizational direction and goals down to individual, day-to-day management activities. Similarly, the bottom-up monitoring of performance (asset characteristics, management systems problems, risks, and opportunities) should provide the factual basis for adjusting and refining realistic asset management strategies and plans through a process of continual improvement. Therefore, the organization should establish, implement, and maintain processes and/or procedures for the following: 1. To monitor and measure the performance of asset management and the performance and/or condition of assets and/or asset systems 2. To handle and investigate failure, incidents, and non-conformities associated with assets, asset systems, and asset management 3. To evaluate the organization s compliance with applicable legal and other regulatory or absolute requirements 4. To conduct audits of the asset management approach 5. To investigate corrective and/or preventive actions 6. To assess, prioritize and implement actions 7. To review the organization s asset management system in order to ensure its continuity, suitability, adequacy, and effectiveness This process is not intended to replace audit processes that may already be in place. The performance monitoring process should cover the following areas: Strategic monitoring: To verify that outcomes are being met and asset management has been implemented and documented Performance measures and targets: To assess the effectiveness and efficiency of asset management System audits: To review and evaluate the effectiveness of asset management retrospectively (e.g., on an annual basis) 100

2 Compliance monitoring: To evaluate the organization s compliance with applicable legal and absolute or regulatory requirements Performance monitoring is the process and procedure of monitoring and reviewing the asset management framework. Frequency of monitoring should find the right balance between the cost of collecting the monitoring data and information and the risks of not having the information available. Finding the right balance is particularly important when considering compliance with statutory obligations and demonstrating value for money. It is also important that the benefits from implementing asset management are captured and measured against those identified in the case for investment, or to support value-for-money initiatives or greater efficiency in delivery of the service. Recording and demonstrating the benefits may provide essential evidence for further investment. Demonstrating benefits is therefore a key success factor in the implementation of asset management and should form part of the monitoring process. Why should performance monitoring be conducted? Monitoring and reporting performance is an important part of demonstrating whether the organization is delivering the agreed levels of service (NAMS Group ). The measurement of organizational performance provides the following: Measurement of actual against desired levels of customer service Information to demonstrate the achievement of the organization s strategic goals Accountability to customers Identification of areas for improvement Benchmarking among different organizations A means of monitoring performance of service delivery providers (NAMS Group 2011) Information and data arising from the implementation and delivery of asset management provide the following: Quantifiable measures of progress in the delivery of the asset management strategy, performance requirements, and works programs A basis for identifying actions aimed at the continual improvement of the approach, including delivery of the overall service Implementing asset management will also enable critical issues regarding performance to be identified and improvement plans to be developed. 101

3 Maturity levels In terms of performance monitoring, asset management maturity levels can be defined as shown in Table Table 13.1 Maturity levels for performance monitoring Level Maturity Description Examples Basic Proficient Advanced Not applicable Asset performance is monitored through data collected to estimate the level of the various performance measures. Some indicators of management performance, such as progress in program delivery, are also calculated. This information is periodically reported to relevant stakeholders. Benchmarking is used to determine how the organization s monitoring and reporting procedures compare to those of other organizations. In addition to the Basic and Proficient characteristics: A comprehensive approach to review the progress in delivering the asset management strategy periodically is in place. This review allows for the identification of any gaps between the actual and desired performance, which serve as a basis for developing and implementing improvement plans. In this process, benchmarking of organizational performance is also taken into account. Asset condition and performance is reported publicly, together with measures of management performance, using applications such as dashboards. The agency is capable of issuing periodic reports on the achievement of strategic objectives. Evidence can be found of the implementation of a continuous improvement process. The performance monitoring process During the implementation of the asset management plan, the performance of the asset management plan and the performance and/or condition of assets and/or asset systems are assessed through collecting information and data (see Chapter 6) and evaluating the levels of service defined in the performance framework (see Chapter 5). Measuring current performance, which should be reported through various means, allows the organization to compare actual and expected performance by identifying any existing gaps. Gap analysis, in conjunction with performance reviews and benchmarking, are used in determining actions to be implemented if the current performance falls below the organization s requirements or if ways to improve the efficacy and efficiency of the asset management approach are identified. The performance monitoring process is depicted in Figure

4 Performance framework System data Others Performance reporting Dashboard Balanced scorecard Gap analysis Performance reviews Performance benchmarking Performance of other authorities No Action required Yes Corrective actions Improvement plan development Preventive actions Improvement plan implementation Performance reporting Figure 13.1 Performance monitoring process There are a number of ways in which performance measures can be summarized and reported. In deciding upon a reporting format, consideration should be given to the following: Showing the alignment of measures with outcomes/strategic goals Reflecting a balance among competing demands Presenting only the information that the audience requires 103

5 Keeping the number of measures manageable At present, there is an increasing demand for organizations to report not only financial and service outcomes, but also the social and environmental effects, and to demonstrate to stakeholders and political authorities that the organization is managing its social and environmental responsibilities. In this case, the balance scorecard and the dashboard approaches, described below, are appropriate as a tool for demonstrating the level of attention given to each outcome area among competing demands. The balance scorecard and dashboard techniques use technical, non-technical, economic, and financial performance measures, normally less than 25, spread across two or more themes or perspectives (see examples in Figures 13.2 and 13.3). Virginia Department of Transportation Figure 13.2 Example of interactive dashboard 104

6 Minnesota Department of Transportation Results Scorecard Figure 13.3 Example of balanced scorecard At the proficient level of maturity, performance measures related to the strategic objectives of the organization are normally reported at all organizational levels and to customers in an appropriate form, often via an annual report (NAMS Group 2011). 105

7 Gap analysis Gap analysis refers to the process of comparing the measured performance with targets to ascertain whether gaps exist and whether asset management policies, strategies, and objectives are met; the asset management plans are implemented; and the processes, procedures, or other arrangements to control asset lifecycle activities are effective. A performance gap is the difference between the current performance and the desired performance of an asset. Performance gaps can initially be evaluated by simply identifying those performance measures for which the target measure has not been met. It should be noted, however, that a performance gap could exist for a number of reasons, such as the following (Figure 13.4): Gap 1: Customer expectations versus management perception: The customer s expectations of the service provided do not match the service provider s management perception of what is to be provided. Gap 2: Management perception versus actual performance: The management perception of the service quality does not match the actual quality of service being provided. Gap 3: Actual performance versus specified performance: The service is not being delivered to the quality specified in the relevant standards and/or contracts. Gap 4: Actual performance versus communication to customers: There has been inadequate communication with the customers, resulting in customers having a skewed perception of the service delivered. 106

8 Performance reviews From CSS/TAG 2004 Figure 13.4 Performance gap model] As previously noted, performance monitoring is an element in a cycle structured to support a process of continuous audit and review of the plan and the processes used. Therefore, information from the performance monitoring and reporting processes should be used to review the asset management approach. Review activities may include the following: Performance review Management review The performance review involves a consideration of results, the factors contributing to performance, and options for dealing with substandard outcomes; it is carried out at regular intervals, usually on annual basis. The management review involves an assessment of the need for changes to the asset management framework, including asset management policies, strategies, and objectives. The outputs from the management review may include changes in policies, strategies, and objectives; performance requirements; resources; or other elements of the asset management approach. Some of these outputs may also generate changes in the organization s strategic plan. Plans are normally reviewed and updated every three years because this time is considered long enough for the action indicated by the original plan to have taken effect. Benchmarking performance The purpose of benchmarking is to compare the organization s own asset management practices against those of a selection of similar organizations, but it is important that the comparison takes 107

9 into account the organization s overarching context and circumstances. Benchmarking should be seen as a positive and proactive process whereby an organization assesses how it performs activities in comparison to other organizations. Four approaches to benchmarking may be considered, each of which provides a different perspective: Strategic benchmarking compares outcome performance in the implementation of strategic or policy objectives across organizations. Functional benchmarking compares the performance and structure of an entire service area or function within an organization. Process benchmarking compares and measures processes, sequences, or activities with those of other organizations to identify how existing methods can be improved. Data benchmarking involves the use of objective data for comparing performance, very often cost- or measurement-related data. Improvements actions and plans As a result of any of the reviews, organizations are likely to identify a series of desirable improvements they wish to put in place in order to advance their asset management practice. These improvements may be formally documented in an improvement plan. The plan should not only detail the specific actions to be taken, but also outline which levels of service the actions are intended to benefit. Furthermore, in developing the improvement plan, it is important that organizations be pragmatic about what can be delivered given the likely staff resources and budget. Improvement actions should be prioritized and placed into timeframes, and they can be classified as corrective or preventive, as follows: Corrective actions are required to eliminate the reasons for poor performance as a result of using the asset management approach. Preventive actions are required to eliminate potential reasons for poor performance. References 1) Performance Measures and Targets for Transportation Asset Management, National Cooperative Highway Research Program - Report 551, Transportation Research Board, Washington, DC ) Asset Management Overview, FHWA-IF , US Department of Transportation Federal Highway Administration Office of Asset Management December ) Asset Management Primer, 1999, Federal Highway Administration, Washington, DC. ( 108

10 4) County Surveyor s Society (CSS) and Transport Analysis Guidance (TAG) Framework for Highway Asset Management. ( 5) Highway Asset Management Quick Start Guidance Note Getting Started, CSS/TAG and Department of Transport, 4 august 2009 ( 6) BSI PAS 55 (2008): Asset Management Parts 1 and 2 ( 7) United Kingdom Roads Liaison Group (UKRLG) and Highways Maintenance Efficiency Programme (HMEP) Highway Infrastructure Asset Management Guidance Document. Department for Transport, London. ( 1CE0-477F-933ACBFA169AF8CB). 8) Transportation Asset Management Guide, National Cooperative Highway Research Program (NCHRP) Project 20-24(11), American Association of State Highway and Transportation Officials, Washington DC, November 2002 ( 9) Transportation Asset Management Assessment, Department of Transportation - Federal highway Administration, AASHTO Transportation Asset Management Guide Executive Summary, Federal Highway Administration (FHWA) and AASHTO, June 2013 ( 10) CSS/TAG, Preparation of a Model Framework for Asset Management (Plans) for a Highway Network in the UK: Stage One Report: Review of International and Cross- Sector Experience, May ) PIARC Technical Committee D.1 - Management of road infrastructure assets / Comité technique AIPCR D.1 - Gestion du patrimoine routier High Level Management Indicators/. World Road Association (PIARC). Paris, France. ISBN ( High%20level%20management%20indicators.htm?catalog&catalog-topic=33). 12) Annual Transportation Performance Report 2012 ( 13) New Zealand National Asset Management Support (NAMS) Group International Infrastructure Management Manual. Wellington, New Zealand. 109