Impact Evaluation and the Project Cycle

Transcription

1 Aidan Coville Florence Kondylis (DIME) Impact Evaluation the Project Cycle Development Impact Evaluation Initiative innovations & solutions in infrastructure, agriculture & environment naivasha, april 23-27, 2011 in collaboration with Africa region, SD network, GAFSP AGRA

2 A tale of two tablets apple blackberry

3 Roadmap: profit maximization Product Development Initial Operation Maintenance Adoption

4 Development Share price Impact Changing the Y-Axis

5 Roadmap: development impact Product Development Initial Operation Maintenance Adoption

6 PART 1: BEGIN

7 Roadmap: development Product Development (i) procurement, (ii) supply chain management, (iii) monitoring Initial impact Operation Maintenance Adoption

8 Key steps in product development Procurement Managing Supply Chain Monitoring Get the right team Get the right stuff at the right time Make sure its done right

9 (i) Procurement/ Contracting Should we release cost estimates for tenders? Oklahoma: (i) Average bid price decreases; (ii) larger effect on projects with more cost uncertainty [De Silva et al, 2008] Should preferred suppliers be pre-selected? Japan: 3% decrease in cost of procurement by removing selection restrictions on bids [Ohashi, 2006]

10 (ii) Supply Chain Management Should we centralize to improve supply chain efficiency? Zambia: stock out rate of essential drugs ~ 44% 2 supply chains romly tested (i) drugs sealed at central office (ii) drugs sealed at district office delivers to facilities on request Option (i) drastically reduces stock outs Scaled up can avert over under 5 deaths per year [Vledder et al., 2011]

11 (iii) Monitoring

12 (iii) Monitoring Top-down audits or local monitoring to reduce corruption? Indonesia: Top-down auditing reduces missing expenditures in road construction by 8 percentage points Community monitoring has little effect [Olken, 2008]

13 Roadmap: development impact Product Development Initial (i) information/training, (ii) credit constraints, (iii) risk aversion Operation Maintenance Adoption

14 Build it they will come: barriers to Information Credit constraints Risk aversion Can people if the want to? Do they want to?

15 (i) Skills/Training Not as simple as it seems: new technology may need training to maximize benefits

16 (i) Skills/Training How can we increase training delivery without reducing quality? Training comminicators in Malawi: Extension workers increase farmer knowledge by XXX. Local peer farmers have the same impact. [Ben Yishay et al., 2012] Leveraging spillovers in Rwa: Gives coffee growing training to different amounts of people in different villages Goal is to see what is necessary to have the knowledge disseminate through the rest of the village [Duflo et al, forthcoming]

17 (ii) Credit Constraints If people are given credit, will they adopt a new service? Morocco: Loans increase of water connections by 69 percentage points [Devoto et al., 2011] Are there viable alternatives to credit? Philippines: 28% of precommitment savings accounts > 81 percentage point increase in savings [Ahsraf et al., 2006]

18 (iii) Risk Aversion Take up of a new product/technology may require an initial leap of faith

19 Utility (iii) Risk Aversion s Partner in Crime LOSS AVERSION Returns to Investment -5 Losses weighted twice as heavily as gains

20 (iii) Risk Aversion Can financial instruments (eg. insurance) overcome risk aversion? India: low rate of rainfall insurance. Information credit are important determinants of [Cole et al., 2010] Credit + insurance provided on improved seeds: insurance made people less likely to adopt Limited liability means that farmers can default when things go wrong [Gine Yang, 2008] Ghana: multiple interventions to promote ag investment: (i) grant; (ii) free insurance; (iii) insurance at different prices; (iv) grant insurance [Karlan Udry, forthcoming]

21 Overview [Part 1] Developing a quality product requires a number of steps that can be tested: (i) Procurement, (ii) Supply chain management, (iii) Monitoring, But a build it they will come philosophy may not cut it To get people to use it, a number of constraints to need to be overcome (i) Information, (ii) Credit, (iii) Risk aversion, etc

22 Cliffhanger Developing a product getting people to use it is great, but How can we ensure that our project will continue to change people s lives once we have laid the foundations? Stay tuned for more

23 PART 1: END

24 PART 2: BEGIN

25 Roadmap: development impact Product Development Initial Operation Maintenance Adoption

26 General thoughts individual behavior is what we need to think about to ensure our projects will continue to change peoples lives once we have laid the foundations 1. How to align individual decisions with group welfare? 2. How to get people to make choices now that will have large positive impacts a little later?

27 Roadmap: development impact Product Development Initial Operation accountability, governance, property rights, monitoring, group size Adoption

28 Key stages in O&M Property rights Governance Monitoring Payment Schemes

29 Operations Maintenance Once a product/service is developed, it needs to be sustained this means maintaining the physical quality of the road, power lines, transport services this means ensuring that an irrigation scheme remains functional through regular cleaning of drains, planting of riparian weeds, etc How to?

30 a bit of theory (i) how do we reconcile private group incentives? Tragedy of the Commons in the absence of private property rights collective action problems will lead a common resource to be overused / destructed over time can common property regimes solve commons problems? [Oström] Coase Theorem in the presence of clear property rights agents are able to reach an efficient allocation through bargaining

31 a bit of theory (ii) What does this mean for our projects, what should we try out? property rights make contracts enforceable governance structure monitoring arrangements payment systems

32 property rights Providing individual property rights is shown to increase investment in the property [Field, 2012; Galiani, 2010; Kondylis et al, 2012; Deininger & Jin, 2005] shift perceptions, e.g. trust [Di Tella et al, 2007; Kondylis et al, 2012] Can a combination of private property rights common property regime sustain collective action for adequate? e.g.: Irrigation scheme Individual property rights over l Common property over the irrigation infrastructure Common property still needs to be governed

33 governance Governance: who should be in charge? Gender matters: Beaman et al, 2009 Tail-end users households most in need of / excluded from the infrastructure existing local leaders Elections vs. Designation vs. Reservation

34 monitoring Monitoring schemes can help keep service providers accountable do we want the service provider itself to do the monitoring? Problem: it is hard to predict which level / type of monitoring will achieve best results In Indonesia, they compared different methods of accountability at community level, finds different results [Olken, 2007]

35 Payment Systems We need funds to enable how to get people to contribute to the of the infrastructure? Salience of information: do we know how much we pay? Toll roads, tax on goods [Finkelstein, 2007; Chetty et al, 2010] Poor price information can lower usage [Shin, 1985; Coppejans et al, 2006] Get people to contribute labor use monitoring system to set up labor contributions

36 Roadmap: development impact Product Development Initial Operation Maintenance Adoption

37 Roadmap: development impact Product Development Initial Operation Maintenance Adoption - Ensure sustained use to secure high returns

38 Continued Many people will try *new things* but then will stop using them after the first use Will people keep using electricity if they have to pay for it? Will they plant improved seeds year after year? Will a community keep disseminating useful technology? Will rural workers keep migrating to the urban centers during the off season?

39 durable behavioral change Often individuals do not choose to do now what would increase their wellbeing later Problem of self control Can precommitment devices help?

40 precommitment Why don t people keep purchasing fertilizer? In Kenya, some people were given the opportunity to purchase fertilizer for the next season after harvest (Free delivery included) [Duflo et al 2011] Precommitment option is popular: 40% of farmers who are offered the service take it The option to precommit causes the same increase in fertilizer as a 50% price reduction at planting time The increase in goes down once the intervention is phased out

41 social networks for sustained Do social networks influence individual choices? Farmers in Ghana are more likely to adopt a technology (pineapples) when their social network have had recent good experiences [Conley Udry, 2008] In Mozambique, patterns of technology show an initial increase with size of network, then a decrease [Biera Rasul, 2005] >> some individuals rely on others to learn make mistakes before adopting

42 social networks for sustained social networks matter, but how do we use them? Malawi used impact evaluation to find a better way to train farmers [first half] Malawi is now testing ways to capitalize on network effects to further increase technology diffusion [Beaman et al, ongoing] Training peer farmers with various degrees types of social connections

43 network returns to infrastructure eventually, returns to infrastructure happen as people to markets networks are key in determining the extent of these connections well connected migrants to the US find it easier to get jobs higher wages when their networks are strong [Beaman, 2012] less connected individuals are less likely to chose to migrate in the low season despite high returns to migration [Mobarak et al, 2011] how to increase network connections increase returns to transport investments?

44 payment for environmental services Maintaining a natural resource comes at a cost for the individual although it clearly benefits the larger group How do we make individuals do the *right thing*? Espirito Santo, Brazil: run-offs from cultivated l cause sedimentation in the river [Kondylis Sy, ongoing] Increase costs of water treatment plant How do we design individual incentive schemes that can help overcome this source of inefficiency?

45 payment for environmental services Two other ongoing projects: In Bolivia, price incentives information are being compared for promoting sustainable cattle management practices [Ashraf] In Zambia, similar methods are being used to promote tree planting reduce erosion [Walker et al ]

46 Conclusion {PLACEHOLDER}

47 END