FEDERAL DEMOCRATIC REPUBLIC OF ETHIOPIA

Transcription

1 FEDERAL DEMOCRATIC REPUBLIC OF ETHIOPIA MINISTRY OF AGRICULTURE AGRICULTURAL GROWTH PROJECT II MONITORING AND EVALUATION MANUAL

2 Table of Contents ACRONYMS 4 OVERVIEW OF AGP2 M&E SYSTEM 6 AGP2 M&E SYSTEM 6 AGP1 M&E LESSONS 6 PURPOSE OF THE M&E MANUAL 7 INSTITUTIONAL ROLES AND RESPONSIBILITIES FOR M&E 8 MANAGEMENT INFORMATION SYSTEM 11 RESULTS FRAMEWORK AND OTHER OUTCOME INDICATORS: INDICATORS INDEX CARDS 11 PROJECT DEVELOPMENT OBJECTIVE INDICATORS 12 COMPONENT 1: AGRICULTURAL PUBLIC SUPPORT SERVICES INDICATORS 21 COMPONENT 2: AGRICULTURAL RESEARCH INDICATORS 27 COMPONENT 3: SMALL SCALE IRRIGATION INDICATORS 34 COMPONENT 4: AGRICULTURE MARKETING AND VALUE CHAINS INDICATORS 42 COMPONENT 5: PROJECT MANAGEMENT, CAPACITY BUILDING AND MONITORING AND EVALUATION INDICATORS 45 MONITORING OF INPUTS, OUTPUTS, SELECTED OUTCOMES AND PROCESSES 46 MONITORING ANNUAL AND QUARTERLY WORK PLANSAND BUDGET PROGRESS 46 TRACKING FEASIBILITY STUDIES/NEEDS ASSESSMENTS PROGRESS 47 MONITORING AND REPORTING OF SELECTED INDICATORS TO TRACK PROGRESS 47 COMPONENT 1: AGRICULTURAL PUBLIC SUPPORT SERVICES 48 COMPONENT 2: AGRICULTURAL RESEARCH 51 COMPONENT 3: SMALL-SCALE IRRIGATION 53 COMPONENT 4: AGRICULTURAL MARKETING AND VALUE CHAINS 54 COMPONENT 5: PROGRAM MANAGEMENT, CAPACITY BUILDING AND MONITORING AND EVALUATION 57 SETTING, ASSESSINGAND USING INDICATORS PERFORMANCE TARGETS 58 SAFEGUARDS MONITORING 59 PROJECT EVALUATION 59 EVALUATION OF OUTCOME AND IMPACT 59 EVALUATION OF CAPACITY BUILDING ACTIVITIES AND IMPACT 61 GENDER IMPACT EVALUATION 61 LEARNING WORKSHOPS AND FORA 61 IMPLEMENTATION COMPLETION REPORT 62 MAINSTREAMING GENDER, NUTRITION AND CLIMATE SMART AGRICULTURE IN PLANNING AND M&E 62 CAPACITY BUILDING FOR M&E 63 ANNEX 1: FEASIBILITY STUDIES/NEEDS ASSESSMENT AND IMPLEMENTATION PLANFOR FINANCING SPECIFIC INVESTMENTS 64 2

3 ANNEX 2: REPORTING FORMATS 74 REPORTING ON ARDPLACS 74 REPORTING FORMAT FOR DEVELOPMENT AGENTS ON FARMING TRAINING CENTER STATUS AND FARMING TRAINING CENTER ACTIVITIES 75 REPORTING ON DEMONSTRATIONS OF IMPROVED TECHNOLOGIES AT FARMER TRAINING CENTER OR FARMERS PLOT OR FARMER FIELD DAYS BY DEVELOPMENT AGENT 78 REPORTING FORMAT FOR QUEEN REARING CENTERS 80 ANIMAL PRODUCTION AND FEED RESOURCES DEVELOPMENT DIRECTORATE 80 REPORTING FORMAT FOR FISH HATCHERY CENTER AMHARA REGION 81 ANIMAL PRODUCTION AND FEED RESOURCES DEVELOPMENT DIRECTORATE 81 REPORTING FORMAT FOR HERD PERFORMANCE RECORDING AND ELECTRONIC DATA MANAGEMENT AT SEMEN PRODUCTION CENTER 82 REPORTING FORMAT FOR ANIMAL HEALTH CLINICS 83 ANIMAL HEALTH WORKER 83 REPORTING FORMAT FOR ANIMAL HEALTH POSTS 85 ANIMAL HEALTH WORKER 85 REPORTING FORMAT FOR ANIMAL HEALTH LABORATORIES 87 REPORTING FORMAT FOR VEGETABLES AND TREE NURSERIES 88 DEVELOPMENT AGENTS/FARMERS 88 REPORTING ON BEST PRACTICES 89 MOA TECHNICAL DEPARTMENT 89 REPORTING ON SMALL-SCALE IRRIGATION INFRASTRUCTURE 90 REPORTING ON WATER USERS ASSOCIATION 94 REPORTING ON COMMUNITY WATERSHED DEVELOPMENT 97 REPORTING ON ROAD INFRASTRUCTURE 100 REPORTING ON MARKET INFRASTRUCTURE 103 ANNEX 3: QUARTERLY/ANNUAL REPORTS FORMAT 106 WOREDA REPORT (TO BE DEVELOPED BY PCU) 106 REGION AND FEDERAL REPORT (TO BE DEVELOPED BY PCU) 106 ANNEX 4: FEDERAL AND REGIONAL M&E OFFICER TERMS OF REFERENCE 107 FEDERAL M&E OFFICER 107 REGIONAL M&E OFFICER 109 ANNEX 5: SELECTED INDICATORS TO TRACK PROGRESS MATRIX 111 ANNEX 6: RESULTS FRAMEWORK 112 ANNEX 7: FREGS PARTICIPATORY M&E AND LEARNING 113 ANNEX 8: CROSS FARMER GROUP MONITORING 118 ANNEX 9 ASSESSING QUALITY AND TIMELINESS OF WOREDA REPORTS 120 3

4 ACRONYMS ADPLAC AGP1 AGP2 BoA BoW CASCAPE CBSP CDSF CIG CLPP CSA CU DA DFATD DSM EIAR ESMF ESMP ET FAR-CU FCA FCU FHH FREG FSC FTC ha HHI IA IWUA KDC M&E MHH MHIS MoA MoFED MTR PDO PIM MIS RAR-CU RCU RARI Agriculture Development Partners Linkage Advisory Council First Agricultural Growth Project Second Agricultural Growth Project Bureau of Agriculture Bureau of Water Capacity Building for scaling up of evidence based best practices in agricultural production in Ethiopia Community Based Seeds and forage Production groups Capacity Development Support Facility Common Interest Group Community Level Planning Process Climate Smart Agriculture Coordination Unit Development Agent Department of Foreign Affairs, Trade and Development of Canada Direct Seed Marketing Ethiopian Institute of Agriculture Research Environmental and Social Management Framework Environmental and Social Management Plan Ethiopian Birr Federal Agricultural Research Coordination Unit Federal Cooperative Agency Federal Coordination Unit Female Headed Household Farmer Research and Extension Group Federal Steering Committee Farmer Training Center Hectare Household Irrigation Implementation Agency Irrigation Water User Association Kebele Development Committee Monitoring and Evaluation Male Headed Household Micro-irrigation and Household Irrigation Systems Ministry of Agriculture Ministry of Finance and Economic Development Mid Term Review Project Development Objective Project Implementation Manual Management Information System Regional Agricultural Research Coordination Unit Regional Coordination Unit Regional Agricultural Research Institute

5 RUSACCO SC SMS SNNPR SSI TC THH USAID VC WCU WDC WOA WoFED WSC ZOA Rural Savings and Credit Cooperative Steering Committee Subject Matter Specialist Southern Nations, Nationalities and People s Region Small Scale Irrigation Technical Committee Total Households United States Agency for International Development Value Chain Woreda Coordination Unit Woreda Development Committee Woreda Office of Agriculture Woreda Office of Finance and Economic Development Woreda Steering Committee Zonal Office of Agriculture 5

6 OVERVIEW OF AGP2 M&E SYSTEM AGP2 M&E SYSTEM A robust M&E system is critical for effective management of the Agricultural Growth program 2. The AGP2 M&E system will build on the existing AGP1 system and will have the following characteristics: i. It generates continuously information on progress, some processes, and performance through the project cycle from design to end of project; ii. The continuous flow of information generated by the M&E system is useful for project management at all levels of the project (national, region, woreda, kebele) and external partners; iii. It validates, analyzes and aggregates data generated at various levels to track progress, assess if specific targets are met and monitor process quality, outputs, outcomes, program impacts, and sustainability: iv. It identifies implementation gaps and challenges for proactive corrective actions as well as documents and incorporates lessons learned into program implementation. v. It includes effective channels of communication from the field to the national level and back to: (i) inform management decisions and ensure that corrective action are taken if necessary; (ii) disseminates key lessons from evaluations to project implementers, beneficiaries and external partners; and (iii) provides access to and displays project key information at all levels to enhance transparency and accountability. vi. It supports continuous M&E capacity building of implementers and PCU staff. vii. It pays particular attention to the crosscutting themes (gender, nutrition and CSA). AGP1 M&E LESSONS Despite AGP1 Federal and Regional Coordination Unit strong commitment to M&E, there are still deficiencies and inconsistencies in the AGP1 M&E system in terms of quality of data and timeliness. The efficiency of the AGP1 M&E system is being particularly affected by (i) the lack of M&E capacity of implementers and CU staff; (ii) the lack of tailored guidelines and reporting formats for implementers; and (iii) the lack of commitment of certain IAs to reporting. What will be different from AGP1 M&E system? The AGP2 M&E system gives particular attention to: (i) developing tailored made simple M&E guidelines for IAs at all level (kebele, woreda, region and Federal)based on the content of this M&E manual; (ii) capacity building support to all actors/ias involved in M&E activities (with a learning by doing approach with DFATD capacity development support facility); (iii) more robust triangulation of data through diverse sources and methodologies (more contracted evaluations); (iv) enhanced monitoring and evaluation of processes and quality of implementation of the project; (v) tailored monitoring and evaluation of cross-sectoral issues (gender, nutrition and climate smart) from annual plan of activities to final project evaluation and (vi) clear reporting of additional indicators outside of annual plans.

7 PURPOSE OF THE M&E MANUAL This manual has been developed considering AGP1 lessons and the new requirements and activities under AGP2. It is the guiding document for component leaders and M&E specialists at FCU and RCU. This M&E manual will: Identifies M&E roles and responsibilities for AGP2 implementers. Provides common references for results framework indicators. Defines the characteristics of AGP2 monitoring and tools Identifies additional indicators to track AGP2 progress. Guides on setting, assessing and using indicators performance targets. Defines the purpose and type of AGP2 evaluations and studies. Guides on mainstreaming gender, nutrition and climate smart agriculture in planning and M&E. This manual will be the foundation to develop user-friendly guidelines for each AGP2 implementers. AGP2 implementers requiring M&E guidelines and reporting formats tailored to their responsibilities under the project are: DAs. Woreda Coordinators Federal/Regional Agriculture Research Coordination Unit Soil Laboratories Bureau of Water/Focal person at woreda Federal/Regional Cooperative Agencies Etc. This M&E manual is a living document and will be updated as the Project Implementation Manual and project activities evolve. AGP2 Monitoring, Evaluation and Learning Tools Activities Main Tools Evaluation Evaluation of project outcome and impact (household survey) Gender impact evaluation Qualitative studies of extension services and adoption of technologies. Qualitative studies of the FREGs. Qualitative evaluation of efficiency use of water combined with technical audit and cost benefit analysis of infrastructures. 7

8 Monitoring of inputs, outputs, selected outcomes and processes Internal Learning Information dissemination Various qualitative evaluation of trainings. Other evaluations to be determined Quarterly and Annual Progress reports Work and Budget Plan Matrix to report on indicators Indicators Index Cards Feasibility studies/needs assessments progress matrix IAs reporting formats CASCAPE learning case studies Federal and Regional workshops Community learning fora Cross-farmer monitoring Selected case studies Lessons learned Story telling To be determined with project knowledge management strategy FTCs display board Leaflets To be determined with project communication strategy INSTITUTIONAL ROLES AND RESPONSIBILITIES FOR M&E The responsibility for M&E of AGP2 will take place at four levels: federal, regional, woreda, and kebele/sub-kebele. All IAs will report on performance. Federal level. Overall M&E will be coordinated at the federal level from the CU by AGP2 M&E Officers in collaboration with M&E Officers based at the RCU. The coordination of all M&E activities of the AGP2 comprises overseeing data collection, analysis and reporting on the implementation and progress of each component, sub-component, cross cutting issues, and regions of the AGP2. It also includes managing occasional evaluations and impact evaluations carried out by external firms and supporting M&E staff in the regions, zones, and woredas as well as relevant directorates/departments of MoA/BoA (WAD for instance) with regard to M&E requirements, capacity development, and M&E tools (manuals, etc). The FCU will submit quarterly reports for review by the FSC, the World Bank, and other DPs. In addition, the content of the quarterly M&E reports will be summarized in annual reports whose release will be timed to coincide with the completion of the financial year. Tasks also include conducting staff trainings (on basic M&E, reporting formats, M&E manual, etc.) and providing feedback on performance to regional M&E Officers, woreda coordinators, etc. Regional level. The M&E Officer at each RCU will coordinate M&E for their respective woredas, ensuring that M&E data are collected regularly in accordance with agreed upon procedures. The M&E Officers will compile and cross-check reports submitted by woreda. They will also provide M&E TA and training including on how to collect and analyse data on cross cutting issues. They will advise woredas based on inputs and outputs reported and will provide feedback on performance to regional M&E Officers, woreda coordinators, etc. In addition, the M&E Officers will undertake timely qualitative case studies of activities (such as beneficiary interviews, lessons learned, and pictures of sub-projects) and a yearly qualitative analyses of M&E data. The regional M&E Officers will receive in depth training in M&E and computer skills. The RCU will submit quarterly reports to the FCU M&E Officer. 8

9 Implementation Agencies (IAs). EIAR, BoWs, Soil Fertility Laboratories, Bureaus for Cooperative Promotion, and others IAs will report quarterly on their activities following the required formats to the regional M&E Officers The format will be developed in a way that progress on integration of cross cutting issues are captured. Woreda level. The woreda AGP2 Coordinator will coordinate reporting, compiling and crosschecking data collected by the DAs and IAs. The coordinator will also provide TA, training, and advice based on inputs and outputs reported by DAs and IAs. Woreda AGP2 Coordinator will work closely with SMSs and DAs and conduct regular field visits to kebeles and sub-kebeles to engage directly with DAs and verify that M&E activities are being implemented properly. The woreda coordinators will receive training in M&E (including reporting), facilitation, reporting, use of reporting formats, and computer skills. The woreda coordinators will submit monthly reports to the regional M&E Officers copied to the zone for review. Kebele/sub-kebele level. At the kebele level, DAs will be responsible for submitting the data collected by DAs to woreda coordinators. SMSs will ensure that information from project implementation activities and progress is posted on FTCs boards. The DAs will receive training on reporting, use of reporting formats, documentation and facilitation.das will collect project information from beneficiaries using the required reporting format and submit the reports to SMSs. In addition, DAs will create information from project activities and progress available at FTCs. The community learning facilitator (a member of the kebele) with the support of DAs will organize community-learning fora. Farmers. Farmers will actively participate in community learning fora. It will be mandatory for community learning facilitators to facilitate and ensure female farmers active engagement. This could mean for example providing them transport and childcare. Institutional roles and responsibilities for each results framework indicators are further defined in each results framework indicator index card. 9

10 10

11 MANAGEMENT INFORMATION SYSTEM AGP2 will continue building/strengthening the MIS system launched under AGP1. It will include assessing the equipment and staffing need in terms of MIS Officers at regional and federal level. Significant capacity building of the MIS system will be conducted of MIS Officers, M&E Officers and other PCU staff. Support to undertake these activities, will be provided by the Capacity Building Facility. The updating of the MIS system for AGP2 in terms of content will also require updating the indicators handbook once the M&E manual is finalized and indicators to be tracked have been defined. Once the MIS is fully functional between Region CU and Federal CU, the MIS will be piloted at the woreda level (woreda will be selected based on access to connectivity). The MIS system will be supported by an MIS Officer at the Federal level and at the Regional level (as need arise for the Regions). RESULTS FRAMEWORK AND OTHER OUTCOME INDICATORS: INDICATORS INDEX CARDS The RF design builds on the strength and weaknesses of AGP1. As such, the RF includes tailored indicators to capture progress and results in terms of gender equality, quality of capacity development and M&E, which have been weak under AGP1. It provides high-level attention to these issues, quality of implementation and processes. In addition, gender equality, nutrition and climate smart agriculture results are tracked through a consistent disaggregation across most indicators as well as some specific indicators. The results framework is only one small part of the overall M&E system. Significant project progress and results will be tracked through additional outcome indicators outside the results framework (identified in section X Monitoring of inputs, outputs, selected outcomes and processes) and studies/evaluations (quantitative, qualitative, and quality of implementation and processes) targeting specific results. The Results Framework is used to monitor progress towards achieving the PDO. The current targets of the RF indicators are not final, they will be updated once a full fledge baseline is conducted the 1st year of the program. The indicators of the results framework under the PCU responsibility will be reported regularly and frequency of reporting on the indicators is defined in each indicator index card. Results framework indicators index cards contribute to increasing data quality. It ensures that all stakeholders/implementers have a common understanding of each indicator of the RF in terms of: - Definition - Justification - Unit - Frequency of data collection - Data source - Calculation and data collection Methodology. - Baseline value, target value and current value of the indicator. 11

12 PROJECT DEVELOPMENT OBJECTIVE INDICATORS The Project Development Objective (PDO) is to increase agricultural productivity and commercialization of smallholder farmers targeted by the project contributes to dietary diversity and consumption at HH level. Five indicators in the RF measure the PDO (see index cards below) and some additional indicators relating to nutrition linkages with production and consumption have been included. Progress on indicators will be tracked through contracted evaluation, except for the indicator relating to the number of beneficiaries, which is under the PCU responsibility (see index card 5). Some of the indicators definition will be updated based on the methodology discussion for collecting the baseline data. Indicator 1 PDO Indicator (RF) Definition Justification Unit Frequency Data Source Calculation and Collect Methodology Year Baseline Total MHH FHH Target Value Total: MHH FHH Current Value Total Total: PERCENTAGE INCREASE IN YIELD FOR SELECTED CROPS IN TARGETED HOUSEHOLDS BENEFITING DIRECTLY FROM THE PROJECT (Disaggregated in 2 indexes: cereals/pulses (MHH and FHH) and vegetables/fruits (MHH and FHH)) Assess agricultural productivity with yield for selected key crops in targeted households (MHH and FHH). The proposed key crops are the following: cereals (teff, wheat, maize, barley and sorghum), pulses (chickpea, fava bean), oil crops (sesame), stimulants (coffee), vegetables (tomato, potato and onion) and fruits (banana and mango).two indexes will be developed based on the selected crops ((i) cereals and pulses and (ii) vegetables/fruits). The results are disaggregated by male headed household (MHH) and female head of household (FHH). This indicator allows the tracking of the productivity of male headed household (MHH) and female head of household (FHH) in AGP2 selected crops value chains. Percentage The indicator will be reported on at mid-term review and end of project. The value of the indicator at baseline will be collected in August 2016 and indicator targets will be defined based on baseline results. The baseline value is expressed in quintals/hectare/hh. Data will be collected by CSA with the support of a consulting firm to be contracted by AGP PCU. Two indexes will be developed based on the selected crops ((i) cereals and pulses and (ii) vegetables/fruits). A full fledge baseline will be conducted and based on results targets will be set for the indicator. Data will be collected through a household survey. FY16 FY17 FY18 FY19 FY20 FY21 12

13 MHH FHH Comments: Project will close October 10 th of FY21. Indicator 2 PDO Indicator (RF) Definition Justification Unit Frequency Data Source Calculation and Collect Methodology Year Baseline Total MHH FHH Target Value Total: MHH FHH Current Value Total MHH FHH Comments: PERCENTAGE INCREASE IN YIELD FOR SELECTED ANIMAL PRODUCTS IN TARGETED HOUSEHOLDS BENEFITING DIRECTLY FROM THE PROJECT (disaggregated in MHH and FHH) Assess agricultural productivity by a proxy with yield for selected key livestock products in targeted households (MHH and FHH). The proposed selected key livestock products at this stage are the following: honey and cattle milk. An index will be developed based on the selected livestock products. This indicator definition has been updated since the PAD, as it will be too difficult to measure cattle and shoat meat. Therefore, the project will track the number of animal sold disaggregated per type (chicken, cattle and shoat), as it is easier to measure. The project will also track animal mortality per household. This indicator allows the tracking of the productivity of male-headed household (MHH) and female head of household (FHH) in AGP2 selected livestock value chains. The baseline will be expressed in quintal/animal/hh. Percentage The indicator will be reported on at mid-term review and end of project. The value of the indicator at baseline will be collected in August 2016 and indicator targets will be defined based on baseline results. Data will be collected by CSA with the support of a consulting firm to be contracted by AGP PCU. An index will be developed based on the selected livestock products. A full fledge baseline will be conducted and based on results targets will be set for the indicator. Data will be collected through a household survey. To be further defined. FY16 FY17 FY18 FY19 FY20 FY21 Project will close October 10 th of FY21. Indicator 3 PDO Indicator (RF) PROPORTION PRODUCTION SOLD BY TARGETED HOUSEHOLDS FOR SELECTED CROPS (Disaggregated in 2 indexes: cereals/pulses (MHH and FHH) and vegetables/fruits (MHH and FHH)) 13

14 Definition Justification Unit Frequency Data Source Calculation and Collect Methodology Year Baseline Total MHH FHH Target Value Total: MHH FHH Current Value Total Total: MHH FHH Comments: Indicator 4 PDO Indicator (RF) Definition Justification Unit Assess the level of commercialization of the crop production by targeted households (MHH and FHH) for selected key crops. The proposed key crops are the following: cereals (teff, wheat, maize, barley and sorghum), pulses (chickpea, fava bean), oil crops (sesame), stimulants (coffee), vegetables (tomato, potato and onion) and fruits (banana and mango). The indicator is disaggregated by cereals/pulses and vegetables/fruits. This indicator allows the tracking of the commercialization of male headed household (MHH) and female head of household (FHH) in AGP2 selected crops value chains. Percentage The indicator will be reported on at mid-term review and end of project. The value of the indicator at baseline will be collected in August 2016 and indicator targets will be defined based on baseline results. The baseline value is expressed in quintals. Data will be collected by CSA with the support of a consulting firm to be contracted by AGP PCU. An index will be developed based on the selected crop products. A full fledge baseline will be conducted and based on results targets will be set for the indicator. Data will be collected through a household survey. FY16 FY17 FY18 FY19 FY20 FY21 Project will close October 10 th of FY21. PROPORTION OF ANIMAL PRODUCTION SOLD BY TARGETED BENEFICIARIES FOR SELECTED PRODUCTS (disaggregated by male and female) Assess the level of commercialization of the livestock production by targeted beneficiaries for selected key livestock products. At this stage, the livestock products are the following: poultry (meat), honey, cattle milk (cow), dairy products (yogurt, butter, cheese, etc.), cattle and shoats (meat) and hide and skins. The current list of livestock products will be further defined. Female refers to Female Headed Household and married females. This indicator allows the tracking of the commercialization of male and female in AGP2 selected livestock value chains. Percentage 14

15 Frequency Data Source Calculation and Collect Methodology Year Baseline Total MHH FHH Target Value Total: MHH FHH Current Value Total Total: MHH FHH Comments: Indicator 5 PDO Indicator Definition Justification Unit Frequency The indicator will be reported on at mid-term review and end of project. The value of the indicator at baseline will be collected in August 2016 and indicator targets will be defined based on baseline results. The baseline value is expressed in kilograms Data will be collected by CSA with the support of a consulting firm to be contracted by AGP PCU. An index will be developed based on the selected crop products. A full fledge baseline will be conducted and based on results targets will be set for the indicator. Data will be collected through a household survey. It might be useful to identify products that would be produced and commercialized by women to develop the methodology to collect this indicator for women (women responsibilities in MHH). FY16 FY17 FY18 FY19 FY20 FY21 Project will close October 10 th of FY21. PERCENTAGE INCREASE IN ENERGY PRODUCED FROM NONE- STAPLE CROP BY TARGETED HOUSEHOLD Percentage Data Source Calculation and Collect Methodology Year Baseline Total MHH FHH Target Value FY16 FY17 FY18 FY19 FY20 FY21 15

16 Total: MHH FHH Current Value Total Total: MHH FHH Comments: Project will close October 10 th of FY21. If there is a drought, it also helps assessing the resilience (when there is food crisis it drops). It also provides a nutrition point view. It looks at diversification from staple. Indicator 6 PDO Indicator Definition Justification Unit Frequency PERCENTAGE INCREASE IN NUTRITIONAL YIELD BY TARGETED HOUSEHOLD Percentage Data Source Calculation and Collect Methodology Year Baseline Total MHH FHH Target Value Total: MHH FHH Current Value Total Total: MHH FHH Comments: FY16 FY17 FY18 FY19 FY20 FY21 Project will close October 10 th of FY21. Ethiopian Institute has identified a gap in Vitamin A and Zinc per hectare. Indicator 5 PDO Indicator (RF) Definition NUMBER OF DIRECT PROJECT BENEFICIARIES PERCENTAGE OF FEMALE BENEFICIARIES Direct beneficiaries are people or groups who directly derive benefits from an 16

17 Justification Unit intervention: Men and women farmers benefiting from specific trainings at FTCs Men and women farmers in CIGs Men and women farmers in IWUAs Men and women farmers in FREGs Men and women farmers being linked to the market by the project Men and women farmers using animal health services Men and women farmers members of cooperatives supported by the project Men and women farmers members directly benefiting from USAID component Female beneficiaries directly derive benefits from an intervention (as above) Female refers to Female Headed Household and married females. This results might include some level of double counting of beneficiaries. This indicator allows the tracking of the number of direct beneficiaries and specifically women farmers. Number Frequency Bi-Annually This indicator will be reported on twice a year (2 nd quarter and 4 th quarter) starting FY17 of the project (August 30rd, 2016). DAs/SMSs will report bi-weekly to woreda coordinator. Woredacoordinator will report on January 5 th and July 5 th of each year to Region Coordination Unit and RMoA (5 days after end of quarter). RARIs will report on January 5 th and July 5 th of each year to EIAR (5 days after end of quarter). Region Coordination Unit will report (submit to FCU and RMoA) January 25 th and July 25 th each year to Federal Coordination Unit (25 days after end of quarter). EIAR will report (submit to FCU) January 25 th and July 25 th each year to Federal Coordination Unit (25 days after end of quarter). Federal Coordination Unit will report on February 29 th and August 30 th of each year submit to World Bank and MoA (60 days after end of quarter) Data Source Data will be collected by DAs/SMSs, Regional Research Centers (with the Crop, Livestock/Forage, Soil and Water and Agriculture Implements Research Directorates) project research component staff, Irrigation specialists from BoW. Calculation Data will be collected using the project reporting formats and transmitted to the and coordination unit with quarterly reports. The Woreda coordinator, Region and Collect Federal component specialists and M&E specialists at the RPCU and FCU will Methodology be responsible for cross-checking the quality of data and aggregating data. The Federal M&E specialist will be calculating the indicator. The targets are cumulative targets. Year FY16 FY17 FY18 FY19 FY20 FY21 17

18 Baseline Total Percentage female Target Value Total: Percentage female Current Value Total Percentage female Comments: 0 NA Project will close October 10 th of FY21. Need to be defined Need to be defined Need to be defined 40% 40% 40% 1,597,730 40% Indicator 7 PDO Indicator PERCENTAGE OF HOUSEHOLDS IN PROJECT AREA REPORTING THAT CHILDREN HAVE A MINIMALLY ACCEPTABLE DIET Definition This indicator measures the proportion of households with a children 6-23 months of age who receive a minimum acceptable diet (MAD), apart from breast milk. The minimum acceptable diet indicator measures both the minimum feeding frequency and minimum dietary diversity, as appropriate for various age groups. If a child meets the minimum feeding frequency and minimum dietary diversity for their age group and breastfeeding status, then they are considered to receive a minimum acceptable diet. Justification Consumption of foods from at least 4 food groups out of 7 on the previous day would mean that in most populations the child had a high likelihood of consuming at least one animal-source food and at least one fruit or vegetable that day, in addition to a staple food (grain, root or tuber). CUT-OFF (Available) The cut-off of at least 4 of the above 7 food groups above was selected because it is associated with better quality diets for both breastfed and non-breastfed children. METHODOLOGY (Standardized): This indicator is published by a large technical stakeholder group (WHO, UNICEF, USAID, AED, FANDA, UC Davis, IFPRI)*. Unit Frequency Data Source Calculation and Collect Methodology Percentage Baseline, midterm, end line HH survey (DHS, MICS, etc) Tabulation of the indicator requires that data on breastfeeding, dietary diversity, number of semi-solid/solid feeds and number of milk feeds be collected for children 6-23 months the day preceding the survey. The indicator is calculated from the following two fractions: 18

19 1. Breastfed children 6-23 months of age in the sample who had at least the minimum dietary diversity and the minimum meal frequency during the previous day divided by Breastfed children 6-23 months of age in the sample with MAD component data AND 2. Non-breastfed children 6-23 months of age who received at least 2 milk feedings and had at least the minimum dietary diversity not including milk feeds and the minimum meal frequency during the previous day divided by Nonbreastfed children 6-23 months of age in the sample with MAD component data Minimum dietary diversity for breastfed children 6-23 months is defined as four or more food groups out of the following 7 food groups (refer to the WHO IYCF operational guidance document cited below): 1. Grains, roots and tubers 2. Legumes and nuts 3. Dairy products (milk, yogurt, cheese) 4. Flesh foods (meat, fish, poultry and liver/organ meats) 5. Eggs 6. Vitamin-A rich fruits and vegetables 7. Other fruits and vegetables Minimum meal frequency for breastfed children is defined as two or more feedings of solid, semi-solid, or soft food for children 6-8 months and three or more feedings of solid, semi-solid or soft food for children 9-23 months. For the MAD indicator, minimum dietary diversity for non breastfed children is defined as four or more food groups out of the following six food groups: 1. Grains, roots and tubers 2. Legumes and nuts 3. Flesh foods (meat, fish, poultry and liver/organ meats) 4. Eggs 5. Vitamin-A rich fruits and vegetables 6. Other fruits and vegetables Minimum meal frequency for non breastfed children is defined as four or more feedings of solid, semi-solid, soft food, or milk feeds for children 6-23 months. For non-breastfed children to receive a minimum adequate diet, at least two of these feedings must be milk feeds. Year Baseline Total MHH FHH Target Value Total: MHH FHH Current Value Total Total: MHH FHH FY16 FY17 FY18 FY19 FY20 FY21 4.3% 4%? 4.5% 4.7% 5.2% 5.6% 19

20 Comments: Indicator 8 PDO Indicator Definition Justification Unit Frequency Data Source Calculation and Collect Methodology Project will close October 10 th of FY21. AGP2 contributes to improvement in consumption, which is a higher outcome. Improvement in consumption is the combination of AGP2 activities with the ministry of health particularly health DAs and the ENGINE project. PERCENTAGE OF HOUSEHOLDS IN PROJECT AREA REPORTING THAT WOMEN HAVE A MINIMALLY ACCEPTABLY DIVERSE DIET This indicator reflects consumption of at least five of ten food groups (1. All starchy staple foods, 2. Beans and peas, 3. Nuts and seeds, 4. Dairy, 5. Flesh foods, 6. Eggs, 7. Vitamin A rich dark green leafy vegetables, 8. Other Vitamin A rich vegetables, 9. Other vegetables, 10. Other fruits), and can be generated from surveys. Women of reproductive age (15-49 years) are at risk for multiple micronutrient deficiencies, which can jeopardize their health and ability to care for their children and participate in income generating activities. Maternal micronutrient deficiencies during lactation can directly impact child growth and development but the potential consequences of maternal micronutrient deficiencies are especially severe during pregnancy, when there is the greatest opportunity for nutrient deficiencies to cause long term, irreversible development consequences for the child in-utero. Dietary diversity (assessed here as the number of food groups consumed) is a key dimension of a high quality diet with adequate micronutrient content; and thus, important to ensuring the health and nutrition of both women and their children. This indicator has been validated as an indicator of likelihood of micronutrient adequacy among women of reproductive age. There is a recent global consensus on this indicator as the best, most valid measure of women s dietary diversity; it replaces the WDDS (women s dietary diversity score) that had been previously developed by FAO and FANTA. Unlike former measurements, it offers a threshold for women s micronutrient needs. CGIAR and USAID Feed the Future have mainstreamed the use of this indicator in their evaluations. EU is scaling up its use. Percentage or Number (for mean dietary diversity score) Baseline, midterm, endline HH survey Data are collected on the foods and beverages consumed in the previous 24 hours which are aggregated into 10 distinct food groups. CUT-OFF (Available)Women who consume foods from at least 5 out of 10 food groups have a higher likelihood of micronutrient adequacy. Several indicators can be derived from the basic data, including (i) proportion of HH reporting that women in the HH have minimal dietary diversity (5/10 food groups) (ii) proportion of women who consume 5 or more food groups out of ten, (iii) mean dietary diversity score, (iv) proportion of women consuming any specific food group such as animal source foods. 20

21 Year Baseline Total MHH FHH Target Value Total: MHH FHH Current Value Total Total: MHH FHH Comments: A simple questionnaire could be added to the Module 9 of the AGP MTR survey, similar to the HH dietary diversity section. FY16 FY17 FY18 FY19 FY20 FY % 12% 12.3% 12.9% 14.0% 15.2% Project will close October 10 th of FY21. AGP2 contributes to improvement in consumption, which is a higher outcome. Improvement in consumption is the combination of AGP2 activities with the ministry of health particularly health DAs and the ENGIN project. COMPONENT 1: AGRICULTURAL PUBLIC SUPPORT SERVICES INDICATORS Component Objective: To increase access to public agricultural services for smallholder farmers. Indicator 6 (RF) Definition Justification Unit Frequency Data Source Calculation and PERCENTAGE OF FARMERS USING PUBLIC AGRICULTURAL SERVICES (male and female farmers) The percentage of farmers using public agricultural services (male and female) will be analyzed per type of services: (i) extension services (through (a) farmers training and demonstration at FTCs by DAs; (b) farmer field days; (c) advice/demonstrations by DAs (crops, livestock, NRM) on farmers plots and other site); (ii) animal health services (farmers using animal health clinics and animal health posts); and (iii) farmers benefiting from insemination services for their livestock. Female: refers to Female Headed Household and married females. It informs on progress in terms of access and usage of public agricultural services by male and female farmers. Percentage The indicator will be reported on at mid-term review and end of project. The value of the indicator at baseline will be collected in August 2016 and indicator targets will be defined based on baseline results. Data will be collected by CSA with the support of a consulting firm to be contracted by AGP PCU. Data will be collected through household survey/ evaluation. In addition, a qualitative study of extension services will be conducted to assess the quality of 21

22 Collect Methodology Year Baseline Total Male Female Target Value Male Female Current Value Total Male Female Comments: access to extension for male and female farmers. It will be conducted by Consulting firm/university under responsibility and supervision of CU M&E Officer and TC. FY16 FY17 FY18 FY19 FY20 FY21 Project will close October 10 th of FY21. Indicator 7 (RF) Definition Justification NUMBER OF GENDER SENSITIVE TECHNOLOGIES DEMONSTRATED IN THE PROJECT AREA The term technology includes a change in practices compared to currently used practices or technologies (seed preparation, planting time, feeding schedule, feeding ingredients, post-harvest, storage, processing, etc). If one specific technology is demonstrated in more than one location in the project area, it will count as one technology. If the project introduced or promotes a technology package in which the benefit depends on the application of the entire package (e.g., a combination of inputs such as a new variety and advice on agronomic practices such as soil preparation, changes in seeding time, fertilizer schedule, plant protection, etc) this will count as one technology. This indicator assesses the number of gender sensitive technologies demonstrated by the project. Gender sensitive technologies are defined as: (i) technologies based on needs and interest of female farmers; (ii) technologies that reduce time and labor for women farmers; and (iii) technologies that are accessible and affordable by women farmers. Technologies for nutrition refer to technologies: (i) increasing production and consumption for a range of diverse nutrient dense food; and (ii) improving postharvest handling, preservation and processing to improve availability of good nutritional quality and safe food. CSA technologies under the project refer to technologies that increase productivity and resilience (adaptation). Not all technologies are gender sensitive or contributing to improved nutrition or climate smart, but it is still critical to know for the project how many are being promoted to public extension services. Demonstrated:Includes advice given or demonstrated by producer organizations, cooperatives, extension service, etc. Technologies can be demonstrated during field days, at FTCs training, on farmers plot, etc. Demonstrating gender sensitive technologies increase of women adopting these technologies. In addition, technologies reducing women farm labor and the time that women farmers need to perform household duties could enable them to 22

23 Unit Frequency Data Source Calculation and Collect Methodology Year Baseline Total Target Value Total Current Value Total Comments: devote more time to productive farm activities. Number Annually This indicator will be reported on once a year (4 th quarter) starting FY17 of the project (August 30rd, 2016). DAs/SMSs will report bi-weekly to woreda coordinator. Woredacoordinator will report on January 5 th and July 5 th of each year to Region Coordination Unit and RMoA (5 days after end of quarter). Region Coordination Unit will report (submit to FCU and RBoA) July 25 th each year to Federal Coordination Unit (25 days after end of quarter). Federal Coordination Unit will report on August 30 th of each year submit to World Bank and MoA (60 days after end of quarter) Data will be collected using the project reporting formats and transmitted to the coordination unit with quarterly reports. The Woreda coordinator, Region and Federal component specialists and M&E specialists at the RPCU and FCU will be responsible for cross-checking the quality of data and aggregating data. The Federal M&E specialist will be calculating the indicator. In addition, a qualitative study of gender sensitive technologies demonstrations will be conducted to assess the quality of these technologies and quality of demonstrations for female farmers. It will be conducted by Consulting firm/university/capacity Building Facility under responsibility and supervision of CU M&E Officer and TC. The targets are cumulative targets. FY16 FY17 FY18 FY19 FY20 FY Indicator 8 (RF) Definition Justification PERCENTAGE INCREASE IN CROP DIVERSITY (number of HH cultivating 3 or more crops) IN TARGETED HOUSEHOLDS BENEFITING DIRECTLY FROM THE PROJECT Disaggregated by female head of household. The increase in crop diversity refers to the increase in production crop diversity of households benefiting directly from AGP2. Crop diversity is based on the Minimum Dietary Diversity Women s food groups. AGP2 production crop diversity will be considered achieved if the household produces crops in at least 3 of the following food groups: (i) all starchy staple food; (ii) beans and peas; (iii) nuts and seeds; (iv) vitamin A-rich dark green leafy vegetables; (v) other vitamin A-rich vegetables and fruits; (vi) other vegetables and (vii) other fruits. This indicator assesses nutritional diversity of AGP2 households cropping system. This indicator provides an intermediate indication of AGP2 contribution to improve production diversity and diet diversity of households within AGP2 woredas. 23

24 Unit Frequency Data Source Calculation and Collect Methodology Year Baseline Total MHH FHH Percentage The indicator will be reported on at mid-term review and end of project. The value of the indicator at baseline will be collected in August 2016 and indicator targets will be defined based on baseline results. The baseline value is in number of households and number FHH. Data will be collected by CSA with the support of a consulting firm to be contracted by AGP PCU. For each household data on crop cultivated will be collected. The analysis of the crops cultivated by HH will be compared to AGP2 standards (based on Minimum Dietary Diversity Women s food groups) and each household meeting these standard will be accounted as targeted households cultivating crops reflecting crop diversity. Percentage HH/FHH cultivating crops reflecting crop diversity = Total households cultivating crops reflecting crop diversity / Total households interviewed in the sample survey. FY16 FY17 FY18 FY19 FY20 FY21 Target Value Total: MHH FHH Current Value Total Total: MHH FHH Comments: Project will close October 10th of FY21 (2020). Indicator 8 Definition Justification PERCENTAGE INCREASE IN AGRICULTURAL PRODUCTION DIVERSITY IN TARGETED HOUSEHOLDS BENEFITING DIRECTLY FROM THE PROJECT Disaggregated by female head of household. The increase in agricultural production diversity refers to the increase in crop and animal production diversity of households benefiting directly from AGP2. Production diversity is based on the Minimum Dietary Diversity Women s food groups. AGP2 agricultural production diversity will be considered achieved if the household produces in at least 5 of the following food groups: (i) all starchy staple food; (ii) beans and peas; (iii) nuts and seeds; (iv) dairy; (v) flesh foods; (vi) eggs; (vii) vitamin A-rich dark green leafy vegetables; (viii) other vitamin A-rich vegetables and fruits; (ix) other vegetables and (x) other fruits. It will require the production of at least 3 crops in the above food groups and at least 2 animal products in the above food groups. This indicator assesses nutritional diversity of AGP2 households agricultural production system. This indicator provides an intermediate indication of AGP2 contribution to improve production diversity and diet diversity of households within AGP2 woredas. 24

25 Unit Frequency Data Source Calculation and Collect Methodology Year Baseline Total MHH FHH Percentage The indicator will be reported on at mid-term review and end of project. The value of the indicator at baseline will be collected in August 2016 and indicator targets will be defined based on baseline results. The baseline value is in number of households and number FHH. Data will be collected by CSA with the support of a consulting firm to be contracted by AGP PCU. For each household data on crop and animal production will be collected. The analysis of the production by HH will be compared to AGP2 standards (based on Minimum Dietary Diversity Women s food groups) and each household meeting these standard will be accounted as targeted households meeting production diversity. Percentage HH/FHH with crop and animal production reflecting agricultural production diversity = Total households/fhh reflecting crop and animal production diversity / Total households interviewed in the sample survey. FY16 FY17 FY18 FY19 FY20 FY21 Target Value Total: MHH FHH Current Value Total Total: MHH FHH Comments: Project will close October 10th of FY21 (2020). Indicator 9 (RF) CLIENTS WHO HAVE ADOPTED AN IMPROVED AGRICULTURAL TECHNOLOGY PROMOTED BY THE PROJECT (TOTAL) CLIENTS WHO HAVE ADOPTED AN IMPROVED AGRICULTURAL TECHNOLOGY PROMOTED BY THE PROJECT (FEMALE) Definition The term technology includes a change in practices compared to currently used practices or technologies (seed preparation, planting time, feeding schedule, feeding ingredients, post-harvest, storage, processing, etc). If one specific technology is demonstrated in more than one location in the project area, it will count as one technology. If the project introduced or promotes a technology package in which the benefit depends on the application of the entire package (e.g., a combination of inputs such as a new variety and advice on agronomic practices such as soil preparation, changes in seeding time, fertilizer schedule, plant protection, etc) this will count as one technology. Adoption refers to a change of practice or change in use of a technology that was 25

26 Justification Unit Frequency Data Source Calculation and Collect Methodology Year Baseline Total: Female: Target Value Total: Female: Current Value Total: Female: Comments: introduced/promoted by the project. The term technology includes a change in practices compared to currently used practices or technologies. It will assess the proportion of technologies adopted per type: (i) gender sensitive; (ii) nutrition; and (iii) CSA. Female: refers to Female Headed Household and married females. It tracks the level of adoption of technologies by male and female farmers demonstrated by AGP2. Number Annually This indicator will be reported on once a year (4 th quarter) starting FY19 of the project (August 30rd, 2018). DAs/SMSs will report bi-weekly to woreda coordinator. Woredacoordinator will report on January 5 th and July 5 th of each year to Region Coordination Unit and RBoA (5 days after end of quarter). Region Coordination Unit will report (submit to FCU and RBoA) July 25 th each year to Federal Coordination Unit (25 days after end of quarter). Federal Coordination Unit will report on August 30 th of each year submit to World Bank and MoA (60 days after end of quarter) This indicator will be reported on once a year (4 th quarter) starting FY18 of the project (August 30rd, 2017). DAs/SMSs will report bi-weekly to woreda coordinator. Woredacoordinator will report on January 5 th and July 5 th of each year to Region Coordination Unit and RMoA (5 days after end of quarter). Region Coordination Unit will report (submit to FCU and RMoA) July 25 th each year to Federal Coordination Unit (25 days after end of quarter). Federal Coordination Unit will report on August 30 th of each year submit to World Bank and MoA (60 days after end of quarter) Data will be collected using the project reporting formats and transmitted to the coordination unit with quarterly reports. The Woreda coordinator, Region and Federal component specialists and M&E specialists at the RPCU and FCU will be responsible for cross-checking the quality of data and aggregating data. The Federal M&E specialist will be calculating the indicator.in addition, a qualitative study of technologies adopted will be conducted to assess the quality of these technologies in general and specifically for gender (quality of technologies for female farmers), nutrition and CSA. This evaluation will also look at the benefits for farmers having adopted these technologies and the number of years the technologies adopted are implemented by farmers. It will be conducted by Consulting firm/university under responsibility and supervision of CU M&E Officer and TC. The targets are cumulative targets. FY16 FY17 FY18 FY19 FY20 FY Project will close October 10 th of FY , ,000 1,400, ,000 1,530, ,800 26