Principles, frameworks and indicators for a monitoring system for home. Micronutrient Powders

Transcription

1 Workshop on Home Fortification of foods with Multiple Micronutrient Powders in Brazil Principles, frameworks and indicators for a monitoring system for home fortification of foods with Multiple Micronutrient Powders Rafael Flores-Ayala International ti Micronutrient i t Malnutrition Prevention and Control Program (IMMPaCt) Nutrition Branch Division of Nutrition, Physical Activity and Obesity

2 The findings and conclusions in this presentation are those of the author and do not necessarily represent the official position of the Centers for Disease Control and Prevention.

3 Overarching Issues Affecting Micronutrient Program Implementation: ti Weak program monitoring, evaluation, and documentation of performance and impact hinders efforts to strengthen programs, advocacy, accountability, and guidance to country-level managers Limited nutrition research funding for implementation research restricts our understanding of how best to strengthen the design, management, implementation, evaluation, and financing of micronutrient programs at scale. Kl R & C l Mi t i t P Wh t W k d Wh t N d M W k? A Klemm, R. & Col. Micronutrient Programs: What Works and What Needs More Work? A Report of the 2008 Innocenti Process. 2009, Micronutrient Forum, Washington, DC.

4 Simplified project management cycle Project Planning Project Implementation M&E Feedback to Project and Policy process Impact & Efficiency Evaluation Adapted from Valadez (1991)

5 Program/project evaluation Assessment, as systematic and objective as possible, of a planned, ongoing, or completed program/project p that covers its need, design, implementation (M), impact (E), efficiency and sustainability, so as to incorporate lessons learned into the decision-making process about the program and inform public policy

6 Program/project Monitoring (implementation ti evaluation = performance monitoring = process evaluation) An ongoing process of collecting, analyzing, interpreting ti and reporting indicators, on program/project p inputs, activities, outputs and outcomes, to assess how the program/project is performing against predefined criteria and benchmarks

7 Types of monitoring Type Objective Timing Performance Diagnostic Financial Track inputs, activities and outputs Why problems occurred & solutions Check use of funds Monthly or quarterly Follow-up on Performance results Monthly, quarterly & annually

8 Monitoring and impact Evaluation Monitoring Translates objectives into performance indicators and set benchmarks Collects data on these indicators & compares with benchmarks Reports progress to managers and alerts them to problems, in order to implement timely remedial actions Impact Evaluation Documents impacts Analyzes why intended impacts were or were not achieved Explores unintended d impacts Informs practice, decision-making and policy

9 What is a framework? A series of inter-related units that demonstrate causal pathways between an outcome and several underlying determinants A potential ti mess if not used correctly

10 Why do we have frameworks? Clarify y the end results - program objectives / strategic outcomes / response variables Interrelate units, levels and direction of expected actions Allow for consensus building around a common paradigm Use as a planning and management tool

11

12 INPUTS ACTIVITIES OUTPUTS OUTCOMES Policies, Production, Delivery, Quality, Knowledge, Demand, Access Impact on Intake, & Behaviour Change Communication Coverage & Appropriate Use Status and Function in Participants es, ers l resource om partne, financial utions fro coalition, al contribu national er materia ent, staff, ture, othe Manageme i nfrastruc POLICIES PRODU UCTION & SU UPPLY DELIVERY ALITY BEH HAVIOR CHANGE QU CO OMMUNICATION Development & implementation of policies, legislation regulations & registrations Development & implementation of provision, production, procurement & training strategies Development of delivery system Development & implementation of strategy for management, training & maintaining motivation among providers & distributors Development & implementation of an external & internal quality control system Engagement of stakeholders & advocacy Development & implementation of intervention strategy for information, education & communication for behaviour change Implementation of industry marketing Availability of product in country Importation, production & distribution of products meeting quality standards & specifications Providers/ distributors have knowledge & motivation to adequately distribute, inform & problem solve with participants Access to or presence of intervention in communities or facilities Participants know, demand, & have ability to appropriately use intervention Receipt (coverage) of intervention by participants Presence (coverage) of intervention in households or facilities Regular appropriate use of intervention among participants Improved Intake of vitamins & minerals Diminished loss of vitamins & minerals Effective Project Management, Monitoring & Evaluation Decreased mortality & Morbidity Improved development, performance & productivity Improved nutritional status Non vitamin & mineral interventions Achieved Millenium Development Goals industry marketing WHO/CDC 2010

13 The Logical Framework The Logframe is a powerful tool to summarize the scope & key building blocks of a proposed Project in an easyto- understand, structurally-interrelated format The Logframe is used initially to Plan & Design Projects subsequently, it provides a baseline reference for Monitoring, Evaluation, Progress Reporting, and Briefings

14 Benefits of Logframe Integrates planning, implementation & evaluation Prevents ents mismatch between een activities ities & effects Leverages power of partnerships p and produces consensus Keeps stakeholders focused on outcomes Helps planners set priorities for resource allocation Reveals data needs & provides framework for M&E

15 What Does Logframe Matrix Look Like? Narrative Summary Performance Indicators M&E Assumptions/ Risks Goal Purpose Output Activities iti * PSI Logframe Handbook, 2000

16 Logframe Matrix cont. Narrative Summary Goal higher objective to which project will contribute t Purposes project impacts Outputs t the project deliverables for which project pojec can be held ed accountable Activities a summary of the main activities to generate outputs Performance Indicators Indicators to measure achievement of the project Goal Indicators to measure achievement of project Purposes Indicators to measure project Outputs Inputs Budget by activities: monetary, physical and human resources required to carry out project Activities * PSI Logframe Handbook, 2000

17 The Logic of the Framework DESIGN SUMMARY HOW? Column GOAL CAUSE & EFFECT Hypothesis Attain Goal through Purpose PURPOSES OUTPUTS ACTIVITIES Attain Purposes through Outputs Attain Outputs through Activities iti Attain Activities through Inputs

18 Goal Logframe example Narrative Performance Monitoring & Assumptions & summary Indicators Evaluation Risks Improved nutritional status among children 6-23 months of age by Prevalence of anemia in children 6-23 months reduced from 60% to 20% Goal 1. Baseline source: Nutrition Survey 2012 ( Chamada Nutricional with biomarkers?) 1. Follow up source: Nutrition Survey Funding commitment remains stable

19 Purposes Logframe example Narrative Performance Indicators Monitoring & Assumption summary Evaluation s & Risks 1. Increased coverage of IYCF strategies & use of MNP among caretakers Purposes 1. Coverage targets of MNP distribution for children 6-23 months Targets: - 90% received > 1 package of 60 MNP - 80% received >2 packages of 60 MNP - 70% received all 3 packages of 60 MNP Baseline source: Nutrition Survey MoH MIS? & annual external monitoring survey Intervention continues to focus on priority IYCF indicators

20 Output Logframe example Narrative Performance Monitoring & summary Indicators Evaluation Output 1 Increased access to behavior change communication (BCC), IYCF supportive strategies, & MNP in communities 1. Government agrees to include MNP in MoH delivery system 2. Minimum of 1 trained provider in each community level clinic 3. Minimum of 1 trained volunteer in each zone (lowest administrative level) 1. MoH letter directing addition of MNP to MoH delivery system product list 2. MoH documentation of staff training 3. MoH documentation of volunteer training

21 Monitoring For whom the indicator will be collected (stakeholders)? How the indicators will be collected (source of verification, methodology)? How often the indicator will be collected (frequency)? Who will report the indicator? Who does what based on the information?

22 Data collection methods Existing data (secondary data collection) New data (primary data collection). cross-sectional surveys ( Chamadas Nutricionais ) Lot quality-assurance assurance sampling Sentinel e surveillance Focus groups Observation Document review

23 Awareness Output Example Project Performance M&E Description Indicators Output: Increased % of mothers For whom? of children 6-23 Manager & IEC awareness months of age How? of MNPs among that report Chamada mothers of hearing about Who? C-Team children 6-23 MNPs (85% by When? Yearly months of age 2013) Who reports? M&E responsible Who does? IEC responsible Assumptions Successful IEC activities leads to awareness

24 Thank you Questions?