Agriculture Sector Dialogue Phase II Lecture 1 Introduction & Overview of the Training Why Evaluate Agricultural Projects Challenges in Evaluating Agricultural Projects
Overview Goal: To provide an orientation on methods for analyzing the impact of agricultural interventions Day 1: Methods Day 2 (morning): Critical Implementation Questions Day 2 (afternoon): Applying Methods to Cases
This Session An Example: NERICA Rice Dissemination Project Benefits of Evaluation Evaluation Defined Internal and External Validity Challenges in Evaluating Agricultural Projects
Context Agriculture is an Important Component of GDP in Sub Saharan Africa Percentage of GDP
Kilogram per Hectare Context Agricultural Productivity is Lagging in Sub Saharan Africa 6000 Cereal Yield Per Hectare 5000 4000 3000 East Asia & Pacific Europe & Central Asia Latin America & Caribbean Least developed countries Middle East & North Africa 2000 1000 Sub-Saharan Africa World 0 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 Year Source: World Bank (2000-2012)
What we Know, but How? Plenty of theories as to where to intervene: Fertilizer Use Improved crops Cultivation methods Agricultural extension Finance for agriculturalists
Example NERICA Rice Project New Rice for Africa (NERICA) rice varieties were first introduced in 1996 in Cote d Ivoire and have since spread rapidly across many Sub-Saharan African (SSA) countries. By 2006, 200,000 acres of land were devoted to NERICA varieties in the whole SSA region.
Example NERICA Rice Project Benin, Burkina Faso, Burundi, Cameroon, Chad, Democratic Republic of the Congo, Cote d'ivoire, Ethiopia, The Gambia, Ghana, Guinea, Guinea-Bissau, Kenya, Madagascar, Malawi, Mali, Mauritania, Mozambique, Namibia, Niger, Republic of Congo Brazzaville, Rwanda, Senegal, Sierra Leone, Sudan, Tanzania, Togo, Uganda, Zimbabwe Key: Reduce Food Insecurity Reduce Dependency on Imports Increase Farmer Incomes
Example NERICA Rice Project NERICA introduced to Africa in 1996 and spread from Ivory Coast, to Uganda, Benin, Burkina Faso and Mali etc. Along the way, program was re-designed and re-evaluated in order to achieve the best outcome possible. Studies (Kijima, 2006) have shown that the impact of NERICA adoption on average yields in Uganda are twice the average rice yield in Africa NERICA adoption has also shown to increase per capita income and decrease the rate of poverty incidence The success of this program is partly due to the extensive effort put in to evaluate its impact.
Rice Yields 2000 10
Rice Yields 2005 11
Rice Yields 2010 12
Rice Yields 2012 13
Benefits of Evaluation Following an Impact Evaluation one can: Estimate the impact of the program on selected outcomes Take steps to improve effectiveness, efficiency and quality of the program during various stages of the implementation process Calculate its expected benefits and do a cost-benefit analysis Use successful programs as a guideline for implementation in other settings
Benefits of Evaluation Growth in agriculture not only leads to GDP growth but also to the alleviation of poverty. Benefits to evaluation include: Know where best to invest (Value for Money) Can better advocate for funds (both internally and externally) Can move the policy conversation forward when it s stuck evidence-based policy making
Evaluation Defined Monitoring is a continuous process where inputs, activities and outputs are tracked on a daily basis Evaluation is a periodic review of a planned, ongoing, or completed program. An impact evaluation is an evaluation with the aim of asking and answering cause-and-effect type of questions.
Internal & External Validity 1 2 3 Internal Validity: Is the Study Accurate? External Validity: How Far Can we Carry What we ve Learned? These broad concerns apply to all impact evaluations 17
Challenges in Evaluating Agricultural Projects 1. External validity: How Far Will the Impact Travel? Other outcomes Other settings RESULTS Other conditions Larger Population
Challenges in Evaluating Agricultural Projects 2. Internal Validity: What does the Data Tell Us? Rigour of methods: What are the right questions to ask? How do these methods work? Proper unit of analysis: Crop? Household? Community? Choosing good indicators How to measure accurately
Challenges in Evaluating Agricultural Projects 3. Agricultural projects are likely to create externalities (spillovers) Participants Non Participants Those who were actually treated
Spillovers Can be Good but they make it harder to find an impact. People learn from their neighbors, so after a program the non- participants may get some of the benefits. This is great (sometimes) for value-for-money, but bad for measuring impact.
Spillover Effects 25 20 15 10 5 0 Treatment Control Programme Impact: 20 Differences- indifferences: Difference between 14 15 Treatment and Control post intervention less the difference between Treatment and Control preintervention (20-15) (17-14) = 2 17 T=0 T=1 No Spillovers 22
Spillover Effects 30 25 20 15 10 5 Treatment 17 14 Control 24 21 Programme Impact: Differences- indifferences: Difference between Treatment and Control post intervention less the difference between Treatment and Control preintervention (24-21) (17-14) = 0 0 T=0 T=1 Spillovers 23
Spillovers: Across Individuals and Crops Learning from neighbors via social learning In the Akwapim South district of Ghana, technology diffusion happened within farmers information neighbourhoods. (Conley and Udry 2005) Credit programs for fertilizer benefit multiple crops, not just the targeted one (Jayne, Yamano and Nyoro 2003)
What Comes Next Today Ex-post Quantitative Evaluation Methods Diving into Quantitative Methods Qualitative Methods and Ex-Ante Evaluation 25