Organizing Framework for a Functional National HIV Monitoring & Evaluation System

Transcription

1 Regional Training on Strategic and Operational Planning in HIV & AIDS Organizing Framework for a Functional National HIV Monitoring & Evaluation System Final Draft for submission to the HIV Monitoring and Evaluation Reference Group (MERG), September 2007 Noddy Jinabhai School of Public Health Nelson Mandela School of Medicine University of KwaZulu-Natal

2 Approach The course will include lectures, case studies, applied group work & small group Exercise sessions, with a detailed course notes folder and a CD reference library with supporting M&E materials and tools. AIM Utilize M&E data to improve decision making and implementation performance

3 OBJECTIVES 1. Integrate the M&E System in the Management and Planning of HIV AIDS Programmes 2. Understand and articulate the rationale for effective M&E based on the UNAIDS & WB Results Based M&E System 3. Explain the elements of a comprehensive M&E system, encompassing biological and behavioral surveillance, research and program monitoring 4. Understand elements of an M&E logical framework, with key indicators, targets and data sources

4 OBJECTIVES 5. Contribute to an M&E operational plan and budget, with milestones and responsibilities 6. Contribute to a tailored, functional M&E system for their own institution / programme / Country 7. Stress the epidemiological and economic foundations of M&E 8. Identify and disseminate proven, evidence-based HIV/AIDS practices

5 M&E for Program Improvement Program Improvement Share Data with Partners Reporting/ Accountability

6 STRATEGIC PLANNING Policy implementation High level Resource mobilisation Political accountability M & E Data for Decision Making TOOLS OPERATIONAL PLANNING Focus IMPLEMENATION Human Resource Development Resource Allocation Budgeting Health Services Level Technical Skills Data Intensive Epidemiology EPP Epidemiology SPECTRUM NASA (Expenditure) Resource Needs Model GOALS

7 A Public Health Approach HIV M&E Determining Collective Effectiveness OUTCOMES & IMPACTS MONITORING Are collective efforts being implemented on a large enough scale to impact the epidemic (coverage; impact)? Surveys & Surveillance Monitoring & Evaluating National Programs OUTCOMES OUTPUTS Are interventions working/making a difference? Outcome Evaluation Studies Are we implementing the program as planned? Outputs Monitoring ACTIVITIES What are we doing? Are we doing it right? Process Monitoring & Evaluation, Quality Assessments Understanding Potential Responses INPUTS What interventions and resources are needed? Needs, Resource, and Response Analysis & Input Monitoring What interventions can work (efficacy & effectiveness)? Are we doing the right things? Special studies, Operations research, Formative research & Research synthesis What are the contributing factors? Determinants Research Problem Identification What is the problem? Situation Analysis and Surveillance

8 Regional Training on Strategic and Operational Planning in HIV & AIDS

9 NATIONAL HIV M&E SYSTEM Indicators A that are informed by NATIONAL HIV RESPONSE MEASURE LEVEL OF ACHIEVEMENT OF GOALS AT THESE LEVELS Data sources B impact outcomes WITH WITH Population-based surveillance: 1. Biological surveillance 2. Behavioural surveillance Other surveys (health facility surveys, etc) outputs inputs WITH WITH 1. NAC Activity report system 2. Other programme reports from public and private sector that are provided at regular intervals to the NAC D who provide funding for and who implement Stakeholders E that are disseminated to Management of the NATIONAL HIV AIDS M &E Operational Plan The NAC analyses, interprets and summarises the data sources in order to prepare C Information Products

10 ORGANIZING FRAMEWORK FOR NATIONAL HIV M&E SYSTEM The outer ring of components represents the planning, human resources, and partnerships that support data collection and data use. The middle ring focuses on the functions of the system through which it collects, captures, and verifies data and transforms data into useful information. The center of the diagram represents the central purpose of the M&E system: using data for decision-making. The main purpose of M&E is to provide the data needed to : guide the planning, coordination, and implementation of the HIV response; assess the effectiveness of the HIV response; identify areas for programme improvement ensure accountability to those infected / affected by HIV and AIDS & those providing financial resources for the HIV response.

11 People, partnerships and planning 1. Organisational structures with HIV M&E functions 2. Human capacity for HIV M&E 3. Partnerships to plan, coordinate, and manage the HIV M&E system 4. National, multi-sectoral HIV M&E plan 5. Annual, costed, national HIV M&E work plan 6. Advocacy, communication and culture for HIV M&E Collect, capture and verify 7. Routine HIV programme monitoring 8. Surveys and surveillance 9. National and sub-national HIV databases 10. Supportive supervision and data auditing 11. HIV evaluation and research agenda Use data for decision-making 12. Data dissemination and use

12 Twelve Components of a Functional National HIV M&E System Each of the twelve components of a national HIV M&E system has a performance goal and associated performance elements; representing the essential building blocks that should be in place to achieve a fully functioning M&E system.

13 COMPONENT 1:ORGANISATIONAL STRUCTURES WITH HIV M&E FUNCTIONS Performance Goal: Establish and maintain a network of organizations responsible for HIV M&E at national, sub-national and service delivery levels Performance Elements: Leadership: effective leadership for M&E in key organisations Human resources: job descriptions for M&E staff; adequate number of skilled M&E staff; defined career path in M&E Organisational culture: national commitment to ensuring M&E system performance Organisational roles and functions: well-defined organizational structure including a national HIV M&E unit; M&E units or M&E focal points in other public, private and civil society organisations; written mandates for planning, coordinating, and managing the HIV M&E system; well-defined M&E roles and responsibilities for key individuals and organisations at all levels Organisational mechanisms: routine mechanisms for M&E planning and management, for stakeholder coordination and consensus building, and for monitoring the performance of the M&E system; incentives for M&E system performance Organisational performance: key organisations achieve their annual work plan objectives for M&E

14 COMPONENT 2: HUMAN CAPACITY FOR HIV M&E Performance Goal: Ensure adequate skilled human resources at all levels of the M&E system in order to complete all tasks defined in the costed, national HIV M&E work plan Performance Elements: Defined skill set for individuals and organizations at all levels Work force development plan, including career paths for M&E Standard curricula for organizational and technical capacity building Local and/or regional training capacity, including links to training institutions Supervision, in-service training, mentoring

15 COMPONENT 3: PARTNERSHIPS TO PLAN, COORDINATE & MANAGE THE HIV M&E SYSTEM Performance Goal: Establish and maintain partnerships among in-country and international stakeholders who are involved in planning and managing the National HIV M&E system Performance Elements: National M&E Technical Working Group Mechanism to coordinate all stakeholders Local leadership and capacity for stakeholder coordination Routine communication channel to inform all stakeholders

16 COMPONENT 4: NATIONAL, MULTI-SECTORAL HIV M&E PLAN Performance Goal: Develop and maintain a national M&E plan including identified data needs, national standardized indicators, data collection tools and procedures, and roles and responsibilities in order to implement a functional national HIV M&E system Performance Elements: Multi-sectoral participation of stakeholders in developing the national M&E plan The national HIV M&E plan is explicitly linked to the National Strategic Plan The national HIV M&E plan describes the implementation of all twelve components of a national HIV M&E system The national HIV M&E plan adheres to international and national technical standards for HIV/AIDS M&E National M&E system assessment completed and findings addressed in a revised national M&E plan

17 COMPONENT 5: ANNUAL, COSTED, NATIONAL HIV M&E WORK PLAN Performance Goal: Develop an annual, costed, national M&E work plan including specified and costed HIV M&E activities of all relevant stakeholders and identified sources of funding. Performance Elements: The M&E work plan contains activities, responsible implementers, timeframe, activity costs, and identified funding The M&E work plan links to work plans & government Medium Term Expenditure Framework (MTEF) budgets of NAC Resources (human, physical, and financial) committed to implement the M&E work plan Commitment of all relevant stakeholders to the national M&E work plan M&E work plan is developed / updated annually based on performance monitoring Workplan used for planning, coordination & prioritization of activities

18 COMPONENT 6: ADVOCACY, COMMUNICATION &CULTURE FOR HIV M&E Performance Goal: Ensure knowledge of and commitment to HIV M&E and the HIV M&E system among ppolicy-makers, programme managers, programme staff, and other stakeholders Performance Elements: HIV M&E communication and advocacy plan, as part of the national HIV communication strategy M&E reflected in national HIV policies and the National Strategic Plan M&E champions among high level officials for endorsement of M&E actions M&E advocacy activities M&E materials targeted at different audiences to support data sharing and use

19 COMPONENT 7: ROUTINE HIV PROGRAMME MONITORING Performance Goal: Produce timely and high quality routine programme monitoring data Performance Elements: Data collection strategy that is explicitly linked to data use Clearly defined data collection, transmission, and reporting mechanisms, including collaboration and coordination between different actors involved Essential tools and equipment for data management (collection, analysis, transmission, and storage) Routine procedures for data transfer from sub-national to national levels Well-defined and managed national HIV database to capture, verify, clean, analyse and present programme monitoring data from all levels and sectors including prevention, treatment, care and support, finances

20 COMPONENT 8: SURVEYS AND SURVEILLANCE Performance Goal: Produce timely and high quality data from surveys and surveillance Performance Elements: Protocols for all surveys and surveillance based on international standards Specified schedule for data collection linked to stakeholders needs, including identification of resources for implementation Inventory of HIV-related surveys conducted Well-functioning biological surveillance system Well-functioning behavioural surveillance system, including cultural practices and second generation surveillance

21 COMPONENT 9: NATIONAL AND SUB-NATIONAL HIV DATABASES Performance Goal: Develop and maintain national and sub-national HIV databases that enable stakeholders to access relevant data for policy formulation, programme management and improvement Performance Elements: Data base s designed to respond to the decision-making and reporting needs of stakeholders Linkages among different relevant databases to ensure data consistency and avoid duplication of effort Well-defined and managed national HIV database for collating, verifying, cleaning, analysing and presenting programme monitoring data from all levels and sectors

22 COMPONENT 10: SUPPORTIVE SUPERVISION AND DATA AUDITING Performance Goal: Monitor data quality periodically and address obstacles to producing high quality data (i.e., valid, reliable, comprehensive, timely) Performance Elements: Guidelines for supervising routine data collection at facility and community-based HIV service delivery levels Routine supervision visits including data assessments and feedback Periodic data quality audits Supervision reports and audit reports

23 COMPONENT 11: HIV EVALUATION AND RESEARCH AGENDA Performance Goal: Identify key evaluation and research questions and coordinate studies to meet the identified needs Performance Elements: Inventory of completed and ongoing country-specific HIV evaluation and research studies Inventory of local HIV evaluation and research capacity, including major research institutions and their areas of work National HIV evaluation and research agenda Ethical approval procedures and standards Guidelines on evaluation and research standards and methods National conference or forum for dissemination and discussion of HIV research and evaluation findings Evidence of use of evaluation / research findings (e.g., referenced in planning documents)

24 COMPONENT 12: DATA DISSEMINATION AND USE Performance Goal: Disseminate and use data from the M&E system to guide policy formulation and programme planning and improvement Performance Elements: Data use plan as part of the National M&E Plan Analysis of data needs and data users Data use calendar to guide timetable for major data collection & reporting requirements Standard format for reporting and data tabulation Timetable for national reporting Information products tailored to different audiences and dissemination schedule Evidence of information use (e.g. data referenced in funding proposals, planning documents)

25 Global HIV/AIDS M&E Framework Assessment & Planning Input (Resources) Activities ( Interventions, Services) Output (Immediate Effects) Outcomes (Intermediate Effects) Impact (Long-term Effects) Situation Analysis Response Analysis Stakeholder Needs Resource Analysis Collaboration plans Staff Funds Materials Facilities Supplies Trainings Services Education Treatments Interventions # Staff Trained # Condoms Distributed # Clients Served # Tests Conducted Provider Behavior Risk Behavior Service Use Clinical Outcomes Quality of Life Social Norms HIV prevalence STI Incidence AIDS Morbidity AIDS Mortality Economic Impact Program Development Data Program-based Data Population-based Biological, Behavioral & Social Data In addition to monitoring these data types, select programs conduct process and outcome evaluations

26 Management of the M&E system The Framework should clearly describe the M&E roles and responsibilities of various stakeholders; the need and plans for M&E capacity building, the establishment / maintenance of the national M&E database, the national M&E TWG and/or other mechanisms for M&E partnerships and coordination, the need and plans for M&E advocacy and communications, the mechanisms for M&E resource mobilization, and the timeline for reviewing and updating the M&E Framework.

27 Strategic Planning for M&E: Setting Realistic Expectations Monitoring and Evaluation Pipeline # of Projects All Input/Output Monitoring Most Some Few* Process Outcome Impact Monitoring/ Evaluation Monitoring/ Evaluation Evaluation Levels of Monitoring & Evaluation Effort *Disease impact monitoring is synonymous with disease surveillance and should be part of all national-level efforts, but cannot be easily linked to specific projects. Thus, few should try to do this due to the complex methodology required.

28 M2 S5: CHALLENGES USING M&E DATA FOR STRATEGIC PLANNING Instruction: Participants work in country teams to complete the following table. CHALLENGES FACED UTILIZING M&E DATA RECOMMENDATIONS FOR OVERCOMING CHALLENGES TWELVE (12) COMPONENTS OF A FUNCTIONAL M & E SYSTEM