The WMO Monitoring and Evaluation (M&E) System

Transcription

1 WMO The WMO Monitoring and Evaluation (M&E) System Dr Assia Alexieva, M&E Officer, Strategic Planning Office (SPO), WMO

2 The Results-Based Management Framework Planning Documents Mission, Vision Statement Global Societal Needs Strategic Thrusts Expected Results Key Outcomes Deliverables/ Outputs WMO Strategic Plan WMO Operating Plan Defines who we are, our desired future, how to realize that future, results to be achieved and required resources. Programme Activities and Projects WMO Results based Budget

3 The WMO M&E System: Forms part of the WMO Results Based Management System: Strategic plan (SP) Operating plan (OP) Results-based budget (RBB) Monitoring and evaluation (M&E) Measures progress made in: Achieving the Expected Results (ERs) defined in the SP Aims to: Enhance performance and impact; Identify areas where corrective action is required; Demonstrate organizational effectiveness, efficiency, and accountability; Facilitate dialogue and discussions.

4 Functions of the M&E System Addresses the following questions: How can we measure performance? What data do we need? What is the best type of method(s) to collect data for each KPI? And when, where and who should be responsible for this? What types of resource support is needed to set up such a system? Who will collect, collate, enter and analyse the data collected by this system? What format should the feedback be in for WMO and its stakeholders? How can we support the use of this information to enhance organizational performance?

5 The Results Chain Resources Activities Deliverables Key Outcomes ERs (inputs) (outputs) (outcomes) (impact) For example: Activity: Training on seasonal forecasts Deliverable/Output: X number of meteorologists trained in seasonal forecasting in Country Y; A 20% increase in the number of trained forecasters in Region Z; Outcome: Increased user satisfaction with the accuracy of seasonal forecasts in Country Y/Region Z (as a result of user surveys)

6 Structure of an Expected Result an example Expected Result 3: Enhanced capabilities of Members to produce better weather, climate, water and related environmental information, predictions and warnings to support in particular disaster risk reduction and climate impact and adaptation tion strategies KO 3.1: Improved climate monitoring, long range forecasts and long-term projections KO 3.2: Climate information and prediction products for climate adaptation and risk management are improved KO 3.3: Hydrological information and products, including water resources, are improved KO 3.4: Drought information and prediction for risk management is improved KPI 3.1.1: Number of Members issuing (a) monthly predictions, (b) seasonal predictions, (c) climate watch bulletins and (d) long-term projections KPI 3.1.2: Perceived quality of the issued (a) monthly predictions, (b) seasonal predictions, (c) climate watch bulletins and (d) long-term projections KPI 3.2.1: Perceived quality of the products of WMO Regional Climate Centres used at the national level KPI 3.2.2: Number of Members operationally developing and disseminating climate products and information for national needs KPI 3.2.3: Perceived quality of the national climate information and products available in Member countries KPI 3.3.1: Number of Members using a Quality Management Framework for Hydrology based on current guidance materials KPI 3.3.2: Number of regional hydrological databases developed in transboundary river basins KPI 3.4.1: NMHSs and Regional Centres issuing drought information and prediction KPI 3.1.3: Perceived timeliness of the issued (a) monthly predictions, (b) seasonal predictions, (c) climate watch bulletins and (d) long-term projections KPI 3.2.4: Number of Members providing targeted/tailored climate information, products and services, through formal mechanisms including National Climate Outlook Forums, to support user requirements in their countries for adaptation and climate risk management in key socio- economic sectors

7 Key Performance Indicators (KPIs): Definition: A quantitative or qualitative metric used to measure the achievement of, or the progress made towards achieving, an expected result. Use: Observe progress and measure actual vs. expected results Make benchmarking comparisons Prompt questions Characteristics of a good KPI (SMART): Specific Measureable Attainable Relevant Time-bound Baselines, targets, performance results

8 Tracking progress WMO

9 Data Collection and Analysis At the outcome level Survey on Impacts of Achieved Results on Members o Every 2 years o Across WMO (all Members) o 2012 Survey: established baselines o 2013 Survey: measure progress in achieving targets Mid-Term Monitoring and Performance Evaluation Report (to be produced in 2014) At the output level Secretariat Progress on Deliverables Reports o Every 6 months o By department o Timeliness, cost and effectiveness in delivering outputs of activities

10 Examples of Climate Services Related Data Collected (2012) Observations: Number of Members w/ climate monitoring and/or watch systems in use Perceived quality of climate observations provided by NMHSs Number of Members carrying out data rescue projects WIS: Number of Members implementing functions of WIS Number of Members whose data processing and management capabilities enhanced as a result of WIS Distribution systems used to deliver products and services to users Service Delivery Number of Members providing climate services to various sectors Number of NMHSs who report improvements in service delivery (e.g. in the surface- and upper-air observing network, equipment for satellite data, data processing, etc. Perceived level of skill of climate predictions and projections

11 Additional Climate-Related Monitoring Data To Be Collected (2013): Climate products and information: Perceived timeliness and quality of national-scale products (e.g. monthly and seasonal predictions, climate watch bulletins, long-term projections) Number of Members operationally developing and disseminating climate products and information for national needs (e.g. basic climate products, climate diagnostics, hazards and extreme value analysis, monthly and longer climate predictions, climate advisories and bulletins, downscaled long-term climate projections, adaptation- and policy-oriented climate products, regional and global scale climate models, etc.) Perceived quality of the above national climate information and products Number of Members participating in RCOFs Number of Members with technical training in development of climate products Sectors to which Members provide targeted climate information Existing mechanisms linking climate providers and users Climate data management systems: Level of adequacy of climate data management systems in use at NMHSs

12 Resources: M&E webpage

13 Resources: guides

14 Concluding Remarks No standard approach or model for building an M&E system; No static M&E system always work in progress; WMO M&E system as a building block: established structure, processes, mechanisms and tools; M&E for climate services: design, implementation, contributions?

15 Thank you