Fragile Settings. Lasting impact. 12 th Evaluation Report on Projects and Programmes in Developing Countries 2011-2012 Frankfurt, 13 November 2013 Bank aus Verantwortung
Financial Cooperation: evaluation is a long-standing tradition Providing transparency for over 20 years 1991 1994 1995 1997 1999 2001 2002 2004 2006 2009 2011 2
Scope of the evaluation: A representative random sample 3
Evaluation method: Each sample project is evaluated by independent internal or external experts Based on facts and figures Usually includes an on-site inspection Roughly 3 years after end of technical project Based on OECD Development Assistance Committee criteria: relevance, effectiveness, efficiency, impact and sustainability Outcome expressed as a rating 4
Evaluation case studies Bangladesh and Bolivia Bangladesh: rural access roads in partnership with TC and the Asian Development Bank Rating: 1 very good Bolivia: Oruro waste water disposal Rating: 5 highly unsatisfactory Access roads to markets, schools and hospitals can now be used all year round. Maintaining roads allows women in extreme poverty to earn a living. Household income in the beneficiary villages has increased by around 40% over 3 years. Income in comparable villages that did not receive assistance down approximately 10%. The waste water disposal programme was intended to improve living standards in Oruro and reduce pollution in the Uru Uru lake. Reasons why project to separate waste water and clean rain water failed: institutional fragmentation; users did not grasp the importance of separating clean and waste water, poor decisions about scale of certain parts of the plant. Roads still flooded by waste water after heavy rain. Approximately 75% of waste water still runs into the lake without being treated. 5
All projects and programmes use one effectiveness rating Estimates are based on random samples covering over 50% of projects More than 100 projects and programmes evaluated over the two-year period Average success rate 80%, with variations linked to total and funding volume of projects Not all FC projects are a success: they are high-risk investments that cannot be considered by other investors. 6
... and a wealth of information about successes and failures by sector and region 7 The evaluation reviewed 129 projects and programmes in receipt of budget funds totalling over EUR 1 billion
In-depth evaluations on priority issues INR (billions) 2.50 2.25 2.00 1.75 1.50 1.25 1.00 0.75 0.50 0.25 2005 2006 2007 2008 a 2008 2009 2009 2010 2010 2011 b 2011 2012 (provisional) Fiscal Year (1 April to 31 March) 10.0 9.0 8.0 7.0 6.0 5.0 4.0 3.0 2.0 1.0 Polio Eradication : Routine Immunisation Routine Immunisation Polio Eradication Ratio Polio Eradication : Routine Immunisation (RHS) Source: National Rural Health Mission (NRHM), 2008 & 2011 8
Not all natural parks generate significant revenue flows Comparison of eight projects to promote natural parks in seven countries Nature conservation and biodiversity outcomes usually rated satisfactory and above As a rule, no sustained increase in income from tourism, forestry, etc. among residents and people living in neighbouring areas Park income often only just covers park running costs Conservation needs continued global support 9
Financial cooperation in fragile countries: Evaluation conclusions Many development cooperation partner countries do not have a stable institutional framework Political and social fragility cannot be eliminated overnight 10
Fragile settings increase risk Effectiveness ratings are lower, but still fairly high An analysis of all 628 evaluated FC projects and programmes classified in the State Fragility Index (SFI) throughout the project (rating scheme in place since 1995). Lower scores in fragile countries are statistically significant (6 to 7 percentage points lower, significance level 10%). Gratifyingly high number of successful projects given the risks involved. 11
Setting targets for fragile countries Improving stability is always a factor Case study: building schools in Palestine Infrastructure: classroom utilisation rates were below usual levels achieved by projects. Improving stability: Schools were also built in locations designed to avoid children having to go through a checkpoint. Case study: Kosovo water supply Infrastructure: Low bill collection rates because one ethnic enclave refused to pay. Improving stability & conflict prevention: Villages containing different ethnic groups were supplied from the same source. Recommendation: when defining objectives, always consider and measure infrastructure improvements but also enhanced stability and conflict prevention. 12
Fragile settings and executing structures Involve legitimate state structures as much as possible Case study: Mali In order to speed up progress, the North Mali programme for reconstruction in the wake of the 1990-94 Touareg rebellion began work as a separate parallel structure. The evaluation report published in 2011 concluded that the rural irrigation element of the North Mali programme even in its ninth phase had failed to create a sense of ownership in government, but had succeeded in winning over users. Case study: Burundi The government supported the reintegration programme for former combatants from the outset. The ministry in Burundi was the official project agency, with German technical cooperation advice on implementation, in the absence of elected local government structures. The local authorities were also brought on board once local elections had been held. Recommendation: Try to avoid separate parallel structures. Involve and strengthen legitimate state implementers. 13
Fragile settings and target groups The need to be sensitive to existing conflicts when choosing target groups Case study: Indonesia In the province of Aceh, support was provided both for victims of the 2001 tsunami and for those affected by the civil war for independence in the mountainous region. This avoided potential tensions that could have arisen if we had focused support on tsunami victims and neglected the mountainous regions. In the evaluation, project partners from coastal and mountainous regions consistently rated stability and security as high. Case study: Burundi A programme to reintegrate former combatants provided assistance for former soldiers, displaced persons and host village communities. The result was peaceful village community life. Virtually all conflicts were resolved within the villages. Different groups were equally represented in local administrations. The programme allowed everyone to earn a living. Recommendation: If possible, always opt for a broad target group, even if that means less funding is available for the individual households. 14
Thank you for your attention. Bank aus Verantwortung