DIME Impact Evaluation Workshop

Transcription

1 DIME Impact Evaluation Workshop DIME Impact Evaluation Workshop Innovations for Agriculture Innovations for Agriculture June 2014, Kigali, Rwanda June 2014, Kigali, Rwanda

2 make extension services more effective florence kondylis maria jones daniel stein

3 extension services can do good things Maybe I should go to the trainings There is another extension training today Sigh Sigh mmm Are these trainings worth my time? Depends: how much time do you have?

4 feedback

5 a winning collaboration

6 do farmers actually want extension trainings? In Rwanda, farmers purchase a private extension service bundled with inputs

7 do farmers actually want extension trainings? In Rwanda, farmers purchase a private extension service bundled with inputs Do you go to the extension trainings?

8 do farmers actually want extension trainings? Not very much! Do you go to the extension trainings?

9 do farmers actually want extension trainings? Not very much! Do you go to the extension trainings? 37% of men 18% of women

10 do farmers actually want extension trainings? Not very much! Do you go to the extension trainings? 88% of groups lose at least one member every season

11 do farmers actually want extension trainings? Not very much! Do you go to the extension trainings? A representative group of 12 farmers loses about 4 members

12 why aren t farmers going?

13 plausible reasons Opportunity costs Poor content of trainings Ability to determine content/quality Not aware of the benefits one can draw from training Extension providers are not responsive to farmers needs

14 hypotheses Providing farmers with a way to affect content and quality and keeping service providers accountable, can enhance farmers willingness to participate The way is provided matters

15 we test possible solutions logbook

16 ADD SUBTITLES logbook

17 logbook

18 we test possible solutions Demand-side mechanisms logbook scorecard

19 scorecard

20 types of scorecards

21 frequency of feedback : trainings : scorecards SEP FEB JUL Planting Harvest Planting Harvest logbook

22 satisfaction & perceptions

23 perceptions Index of perception (1) (2) Treatments Male Female Scorecards [0.09] [0.11] Logbooks 0.11* 0.15* [0.06] [0.08] Observations Mean of control Robust standard errors in brackets, clustered at the farmers group level *** p<0.01, ** p<0.05, * p<0.1

24 perceptions Index of perception (1) (2) Treatments Male Female Scorecards Logbooks Scorecards did not change the way farmers perceive their extension providers Logbooks improved farmers perception of their extension providers by % Observations Mean of control Robust standard errors in brackets, clustered at the farmers group level *** p<0.01, ** p<0.05, * p<0.1

25 What feedback?

26 what type of logbook? Index of perception (1) Treatments Men and Women Paper logbook, Manager collects 0.16*** [0.05] Paper logbook, M&E collects 0.10** [0.05] Hotline 0.13** [0.05] # observations 1010 Mean of control 0.62 Robust standard errors in brackets, clustered at the farmers group level *** p<0.01, ** p<0.05, * p<0.1

27 what type of logbook? Index of perception (1) Treatments Men and Women Paper logbook, Manager collects Paper logbook, M&E collects All effects are similar Hotline # observations 1010 Mean of control 0.62 Robust standard errors in brackets, clustered at the farmers group level *** p<0.01, ** p<0.05, * p<0.1

28 participation

29 participation in trainings Participation in trainings (1) (2) Treatments Male Female Scorecards ** [0.08] [0.10] Logbooks ** [0.06] [0.07] # observations Mean of control Robust standard errors in brackets, clustered at the farmers group level *** p<0.01, ** p<0.05, * p<0.1

30 participation in trainings Participation in trainings (1) (2) Treatments Male Female Scorecards Logbooks women s attendance increased by % no effect on men # observations Mean of control Robust standard errors in brackets, clustered at the farmers group level *** p<0.01, ** p<0.05, * p<0.1

31 what feedback?

32 what type of logbook? Participation in trainings (1) Treatments Women Paper logbook, Manager collects 0.15** [0.07] Paper logbook, M&E collects 0.08 [0.07] Hotline 0.16** [0.07] # observations 573 Mean of control 0.18 Robust standard errors in brackets, clustered at the farmers group level *** p<0.01, ** p<0.05, * p<0.1

33 what type of logbook? Participation in trainings (1) Treatments Paper logbook, Manager collects Women x2 Paper logbook, M&E collects 0 Hotline x2 # observations 573 Mean of control 0.18 Robust standard errors in brackets, clustered at the farmers group level *** p<0.01, ** p<0.05, * p<0.1

34 what type of scorecard? Participation in trainings (1) Treatments Women scorecard individual 0.32*** [0.10] scorecard phone 0.25** [0.10] scorecard group, visual 0.18* [0.10] # observations 573 Mean of control 0.18 Robust standard errors in brackets, clustered at the farmers group level *** p<0.01, ** p<0.05, * p<0.1

35 what type of scorecard? Participation in trainings (1) Treatments Women scorecard individual x2.5 scorecard phone x2 scorecard group, visual x2 # observations 573 Mean of control 0.18 Robust standard errors in brackets, clustered at the farmers group level *** p<0.01, ** p<0.05, * p<0.1

36 sign up

37 sign up (1) (2) new members dropouts Treatments Scorecards 0.28** -0.44*** [0.13] [0.14] Logbooks 0.26*** -0.29*** [0.10] [0.11] # observations Mean of the control Robust standard errors in brackets, clustered at the farmers group level *** p<0.01, ** p<0.05, * p<0.1

38 sign up (1) (2) new members dropouts Treatments Scorecards x4 2 Logbooks x4-30% # observations Mean of the control Robust standard errors in brackets, clustered at the farmers group level *** p<0.01, ** p<0.05, * p<0.1

39 is it empowerment or is it monitoring? Check that this is a true demand-side story Competing hypothesis is that supply improves due to added monitoring We (falsely) announce the scorecards in a random set of groups x

40 It s empowerment x + + x

41 It s empowerment Extension workers can only tell imperfectly where the scorecards take place >> Scorecards are truly empowering!

42 the power of feedback ability to affect quality

43 the power of feedback ability to affect quality awareness of the content

44 simple feedback tools boost demand

45 the power of feedback ability to affect quality awareness of the content

46 the power of feedback ability to affect quality awareness of the content

47 simple feedback tools boost demand

48 make extension services more effective

49 make extension services more effective

50 thank you! florence kondylis maria jones daniel stein

51 DIME Impact Evaluation Workshop Innovations for Agriculture June 2014, Kigali, Rwanda

52 Information Constraints: Price Information Tara Mitchell Trinity College Dublin 17/06/2014 Tara Mitchell 52

53 Why is information valuable? Useful when we have the ability to take action based on the information. Information can help us to make better decisions. 17/06/2014 Tara Mitchell 53

54 Why is price information valuable for farmers? Where to sell? When to sell? Which crops to produce? How much to produce? What price to accept from trader? 17/06/2014 Tara Mitchell 54

55 Direct Access to Farmers, and Rural Market Performance in Central India Aparajita Goyal (2010) Madhya Pradesh, India: soybean is a major crop Usually sold through mandis: government regulated wholesale markets Small number of traders in mandi, therefore they may collude and keep prices low for farmers. 17/06/2014 Tara Mitchell 55

56 e-choupal Intervention ITC Limited: large buyer and processor of soybean ITC wanted: Better quality soybean Lower prices Aim of intervention: eliminate intermediaries and buy directly from farmers 17/06/2014 Tara Mitchell 56

57 e-choupal Intervention Provided farmers with: Information on prices in other markets. Information on quality. Other options for selling their produce. 17/06/2014 Tara Mitchell 57

58 Internet Kiosks Information: min. and max. price from 60 local mandis ITC offer price for high-quality soybean. 17/06/2014 Tara Mitchell 58

59 Hubs (Warehouses) Tests for quality. Opportunity to sell directly to ICT. 17/06/2014 Tara Mitchell 59

60 Data Panel data: monthly prices and volume of crops sold in main mandis from April September Dates of installation and location of kiosks and hubs. 1,704 kiosks and 45 hubs established in 23 districts from Oct 2000 to January /06/2014 Tara Mitchell 60

61 Identifying Impact Different districts receive kiosks and hubs at different times. Observe prices and volumes of crops in all districts before and after kiosks and hubs introduced. 17/06/2014 Tara Mitchell 61

62 Identifying Impact 17/06/2014 Tara Mitchell 62

63 Market prices increased Price information had an effect on market prices: the average monthly price of soy increased by 1.7% after a kiosk was introduced in that district 17/06/2014 Tara Mitchell 63

64 Market prices increased Why is there no additional effect of a hub? Competition effect: + Composition effect: - 17/06/2014 Tara Mitchell 64

65 Market prices increased Increase in minimum price received No change in maximum price received 17/06/2014 Tara Mitchell 65

66 Soy production increased Total area cultivated under all crops stayed the same Presence of kiosks 19% increase in area under soy Presence of kiosks decrease in area under rice 17/06/2014 Tara Mitchell 66

67 Summary of Results Increase in market price of soybean Increase in the production of soybean Good for farmers: increased prices Bad for traders? Probably (but maybe not that bad!) 17/06/2014 Tara Mitchell 67

68 Summary of Results Good for ITC: ITC calculated that it saved Rs 12.9 million in the first year of operation through better quality oil and DOC obtained from processing soybeans procured through the e-choupal intervention. Good for consumers? Maybe 17/06/2014 Tara Mitchell 68

69 DIME Impact Evaluation Workshop Innovations for Agriculture June 2014, Kigali, Rwanda

70 Information Constraints in Agriculture: Potential and Pitfalls of ICTs Jenny C. Aker, Tufts University DIME Impact Evaluation Workshop Innovations for Agriculture Kigali, Rwanda June 16-20, 2014

71 Es mates of Cereal Grain Average Yield by Region (MT/ha) Sub-Saharan Africa LAC Rest of World East and Southeast Asia South Asia Source: Masters (2010). Sub-Saharan Africa

72 Source: R.E. Evenson and D. Gollin, 2003.

73 Source: Ghandi, Mittel and Tripathi,

74 What affects agricultural production, profits and well-being? Information (social networks, trial and error) Access to markets (inputs, outputs) Market structure Credit and savings Risk and shocks (informal and formal insurance) Farmer-level characteristics (tastes, preferences, education, wealth, gender) 74

75 Why is Information a Constraint?

76 Why is Information a Constraint? 65 km~3 hours Zinder (Thursday) Bakin Birgi (Monday) 20 km~1 hour Tanout (Friday) Cost: $4-$10 76

77 Why is Information a Constraint? Zinder (Thursday) 65 km~2 min Bakin Birgi (Monday) 20 km~1 hour Tanout (Friday) 750 km~2 min Niamey (Sunday)

78 % Network Coverage 477 million people covered by mobile 70% This represents 477 million people 60% Coverage (by area) Coverage (by population) This represents 11.2 million square kilometres 59.99% 50% 42.20% 43.97% 40% 30% 30.95% 20% 10% 10.14% 2.98% 0% Source: GSMA 2010

79 Mobile Phone Adoption on Less than US1$ per day Number of Subscribers (Millions) 450 Total Number of Subscribers Number of unique individual mobile phone subscribers in 2012 was 356 million (WI 2012) Source: Wireless Intelligence

80 (How) can information and communication technologies improve poor farmers access to information and agricultural outcomes?

81 How can ICTs Affect Agriculture? The Potential Increase access to private information (social network) Increase access to public information (government, NGOs) Improve farmers /pastoralists management of input and output supply chains Improve accountability via data collection Strengthen links with research systems Facilitate service delivery (insurance, credit) via mobile money 81

82 Increase Access to Private Information Increase farmers and pastoralists contact with their social network and facilitate learning about agricultural technologies, prices or risks Allow farmers and pastoralists to engage in more optimal decisionmaking (arbitrage) $1.20 $1.00 $0.80 $0.60 $0.40 $0.20 $0.00 Per Search Cost by Mechanism, $USD Landline Radio Mobile Phone Personal Visit Newspaper 82

83 Increase Access to Public Information Increase scale and sustainability of those services Improve information quality by providing more timely and contextspecific information Allow farmers and pastoralists to engage in more optimal decisionmaking (arbitrage) $2.00 $1.80 $1.60 $1.40 $1.20 $1.00 $0.80 $0.60 $0.40 $0.20 $0.00 Cost of extension in USD$ per person Extension visits by agent (2) Extension visit plus radio Extension visit Extension visit plus 2 SMS plus hotline (4-minute call) 83

84 Mobile Development Projects by Country and Topic 12 A Agriculture + MIS Health Financial Agriculture 10 MIS Governance and Conflict Preven on 8 Emergency Response Educa on Governance Nigeria Hai Afghanistan Ethiopia Rwanda Mozambique Senegal Cote d'ivoire DRC Benin Zimbabwe Niger Papua New Guinea Somalia Yemen Laos Libya 84

85 What is the evidence? Jensen (2007) India Aker (2010) Niger Aker and Fafchamps (2014) Niger Goyal (2010) India Fafchamps and Minten (2012) India Aker and Ksoll (2013) Niger Cole and Fernando (2013) India Casaburi et al (2014) Kenya Camacho and Conover (2012) Colombia Private, Markets Public, Behavior 85

86 What is the evidence? Private Information and Markets Mobile phone coverage makes markets more efficient (price differences across markets go down), but the effect depends upon: o Type of product (perishable or not) o Market structure (competitive or monopoly) o Other market failures (access to credit, information) Not everyone wins o India: Increase in farmers /fishermen s profits and less waste (Jensen 2007, Goyal 2010) o Niger: Lower consumer prices (Aker 2010), little effect on farm-gate prices (Aker and Fafchamps 2014)

87 What is the evidence? Public Information and Behavior Mobile phone information systems (price, weather, crop advisory, extension) increase farmers knowledge, despite low take-up (Niger, India) Other impacts are mixed: o More likely to produce certain cash crops (Niger, India) o No impact on marketing behavior or prices farmers receive (India, Niger, Colombia) o Mixed impact on input use (India) o Increased (sugar) yields (Kenya) 87

88 Pitfalls Failing to measure impact (ICT is great, of course it s better!) All information is created equal ICT means high tech Mobile Phones are the Silver Bullet 88

89 Failing to Measure the Impact Wellbeing BEFORE AFTER Extension + ICT Extension Control (no extension) Impact evaluations should not only compare the ICT intervention with no intervention, but also with the traditional (non-ict) approach Time INTERVENTION 89

90 All Information is Created Equal ICT can be successful in increasing knowledge, changing behavior and improving agricultural outcomes if: Information is a constraint The missing information is simple and easy to share via mobile phones (ie, SMS reminders) The information provided is relevant and timely

91 ICT means High Tech What mobile phone technology are farmers and pastoralists using? o Smart phones are not widespread SMS holds limited information and requires some literacy Voice platforms can be very costly Smart phones offer new opportunities but add new challenges (and costs) 91

92 Mobile Phones are the Silver Bullet Even if the information provided via ICT is good, farmers and pastoralists need access to markets, credit and other public goods

93 Conclusion ICT can reduce search costs and increase access to information and services via the private and public sector o Often easier for simple, objective information Information alone may not improve farmers and pastoralists welfare need other markets and public goods Evaluate, evaluate, evaluate!! 93

94 Thank you Merci 94

95 DIME Impact Evaluation Workshop Innovations for Agriculture June 2014, Kigali, Rwanda

96 Who knows? Education, Information and Behavior Change Prashant Bharadwaj CEGA

97 Who has information? Everyone has some information! Who has the correct information? Do people who have the correct information do things aligned with that information? If not, then why? Or if yes, then is it just information?

98 A simple hypothesis Do the more educated know more? Will provide evidence from the health sector Similar issues in agriculture Takeaway: information might be good, but understanding how it interacts with broader market constraints is fundamental

99 Common issues in studying education and health Education rarely randomly assigned Need to use non-rct methods Schooling expansion programs are a great natural experiment Zimbabwe saw rapid school expansion after independence

100 Zimbabwe School Expansion

101 Zimbabwe School Expansion

102 Do the more educated know more?

103 How did they know more?

104 Less likely to be HIV positive?

105 Are the more educated better mothers? Not much change in knowledge But large impacts on infant mortality!

106 Are the more educated better mothers?

107 What can this teach us about information constraints in general? Information is important Knowledge alone might not translate into action Misery loves company: constraints do not come alone!

108 DIME Impact Evaluation Workshop Innovations for Agriculture June 2014, Kigali, Rwanda