Do Public Works Programmes Work? A systematic review of the evidence in Africa and the MENA region

Transcription

1 Do Public Works Programmes Work? A systematic review of the evidence in Africa and the MENA region

2 Map of countries with public works programmes and evaluations Public works programme with robust evaluation Public works programme without robust evaluation No public works programmes Upper middle or high income countries (excluded from review)

3 Do Public Works Programmes Work? A systematic review of the evidence in Africa and the MENA region The trend for conducting rigorous impact evaluations of development interventions has prompted many researchers to start looking more closely at public works programmes (PWPs). This policy brief summarises the main lessons that can be drawn from these studies for Africa and the Middle East and North Africa (MENA) region. It is based on a comprehensive systematic review (Beierl and Grimm 2018), which was commissioned by the Deutsche Gesellschaft für Internationale Zusammenarbeit (GIZ) GmbH on behalf of the German Federal Ministry for Economic Cooperation and Development (BMZ). The systematic review is an attempt to reveal factors and design features that are likely to increase the probability of interventions being successful. First and foremost, the review underlines how little we know about the effectiveness of PWPs and especially about the impact of the assets that are created through these programmes. Yet, as long as strong evidence on the effectiveness of PWPs remains absent, there is little reason to favour them over simple cash transfer programmes. 3

4 The rationale of public works Four types of public works programme Public works programmes, often also known as cash-forwork programmes, include a wide range of interventions which share the common objective of income stabilisation through the provision of temporary paid work and asset creation, but which differ in terms of their prioritisation, exact programme design, and mode of implementation. In practical terms, they all entail the payment of a wage (in cash or in kind) by the state, or by an agent acting on its behalf, in return for the provision of labour (McCord 2012). In a nutshell, PWPs are expected to yield positive impacts through three main channels: first, through the wage that is paid to those working on a public works site and that may have a more or less effective insurance function; second, through the productive assets created, which are intended to benefit the wider community or a more specific group, depending on the type of asset in question; and third, through the skills learned by participants that improve their employability or their capabilities to boost income from self-employment Four types of PWP can be distinguished. The first two types to mention are programmes with a short-term focus (Type 1) and programmes with a medium- to long-term focus (Type 2). The key difference between these two is the duration, continuity and predictability of the employment offered to individual beneficiaries. The remaining two types result when the employment offered is accompanied with complementary measures, such as a fertiliser subsidy programme or training, in which case the programmes are classified as Type 1 Plus or Type 2 Plus respectively. The evidence summarised in this policy brief is based on a detailed analysis of 28 rigorous impact evaluations covering activities in seven different countries. However, most of these studies assess public works in Ethiopia, with only six covering other African countries (Côte d Ivoire, Ghana, Malawi, Sierra Leone and Rwanda) and just one analysing a MENA-region country (Yemen). As the overview below shows, with the exception of the studies on Ethiopia s Productive Safety Net Programme (PSNP), all the reviewed studies assess Type 1 (Plus) programmes. 4 Do Public Works Programmes Work?

5 Author(s) Country Programme acronym PWP type Bertrand et al. (2016) Côte d Ivoire PEJEDEC-THIMO (Emergency Youth Employment and Skills Development - Labour Intensive Public Works Sub-Component) 1 (Plus) Bertrand et al. (2017) Côte d Ivoire PEJEDEC-THIMO 1 (Plus) Gilligan & Hoddinott (2007) Ethiopia EGS (Employment Generation Schemes) 1 Bezu & Holden (2008) Ethiopia FFW (Food-for-Work) 1 Quisumbing (2003) Ethiopia FFW 1 Andersson et al. (2011) Ethiopia PSNP (Productive Safety Net Programme) 2 Béné et al. (2012) Ethiopia PSNP 2 Berhane et al. (2011) Ethiopia PSNP 2 (Plus) Berhane et al. (2014) Ethiopia PSNP 2 (Plus) Berhane et al. (2015) Ethiopia PSNP 2 Berhane et al. (2016) Ethiopia PSNP 2 Debela et al. (2014) Ethiopia PSNP 2 Favara et al. (2016) Ethiopia PSNP 2 Gebrehiwot & Castilla (2016) Ethiopia PSNP 2 Gilligan et al. (2009a) Ethiopia PSNP 2 Gilligan et al. (2009b) Ethiopia PSNP 2 Hoddinott et al. (2009) Ethiopia PSNP 2 Hoddinott et al. (2012) Ethiopia PSNP 2 Porter & Goyal (2016) Ethiopia PSNP 2 Sabates-Wheeler & Devereux (2010) Ethiopia PSNP 2 Tafere & Woldehanna (2012) Ethiopia PSNP 2 Weldegebriel & Prowse (2013) Ethiopia PSNP 2 Woldehanna (2009) Ethiopia PSNP; EGS 2; 1 Osei-Akoto et al. (2014) Ghana GSOP-LIPW (Ghana Social Opportunity Project - Labour Intensive Public Works Programme) Beegle et al. (2017) Malawi MASAF III (Malawi Social Action Fund Public Works Programme - Phase 3) 1 (Plus) Hartwig (2013) Rwanda VUP (Vision 2020 Umurenge Programme) 1 Rosas & Sabarwal (2016) Sierra Leone YESP/CfW (Youth Employment Social Support Project/Cash for Work) 1 Christian et al. (2015) Yemen LIWP (Labour Intensive Work Programme) 1 Source: Beierl & Grimm, Do Public Works Programmes Work? 5

6 What have these programmes achieved? When it comes to the impacts PWPs have on income and consumption, the striking finding is that only a handful of studies find positive impacts, while the majority detect no or no robust impacts. Half of the studies that do find positive effects assess the short-term effects of Type 1 PWPs (Côte d Ivoire, Ethiopia and Sierra Leone). They therefore mainly capture the direct income-effect of the wages received rather than the post-programme impacts. Interestingly, the two studies that do also cover longerterm effects (Type 2) are from Ethiopia and involve situations where payments for work were not solely made in the form of cash but instead were made in the form of food only or a mixture of cash and food. In the few cases where beneficiaries experienced an increase in income from self-employment all of which are set in a semi-urban context no increase in the share of households engaged in self-employment or in hours worked in self-employment could be found. The increase in self-employment income is therefore due to an increase in the profitability of existing activities (i.e. growth at the intensive but not extensive margin). Consistently, none of the evaluations provides robust empirical evidence that a PWP of any type generates sustainable extra employment over and above the public works employment, neither in the medium nor in the long term. There is also no indication that there are crowding-out effects i.e. that offering public works employment replaces other economic activities of beneficiary households. Of the 25 studies that investigate impacts on food consumption and food security, 19 cover Ethiopia. Of these, all but one assess the PSNP and its variants. The overall picture is positive for Ethiopia s PSNP (Type 2 [Plus]), whereas it is inconclusive for the Type 1 PWPs run in other countries (Ghana, Malawi, Rwanda and Sierra Leone). However, in the case of Ethiopia it has not been investigated whether the benefits gained by participating households extend beyond the latter s time on the programme. Almost all the evidence on the topic of nutrition comes from Ethiopia s PSNP. While the Ethiopian studies mostly focus on anthropometric outcomes, the two other studies (from Malawi and Yemen) addressing this topic only investigate dietary diversity outcomes. Overall, the findings from Ethiopia on anthropometric outcomes are inconclusive, irrespective of whether one looks at acute undernutrition (measured through WHZ and wasting), chronic undernutrition or both. An evaluation of an Ethiopian Type 1 PWP found no strong indications of reductions in acute or chronic undernutrition. So far, there is also no robust evidence for improvements in education. The Type 2 Plus variant in Ethiopia seems to outperform the other variants with respect to asset accumulation, especially of livestock. However, no robust evidence has yet been produced that sheds light on the question of whether asset accumulation persists beyond the point after which households no longer benefit from the programme. Regarding other design features, the Type 2 variant that pays wages in the form of food performed better in terms of asset accumulation than the mixed payment modality which, in turn, performed better than the cash-only variant. Ultimately, too few studies are available to draw any significant conclusions, especially beyond the case of Ethiopia. 6 Do Public Works Programmes Work?

7 Figure 1: Evidence patterns for all outcome areas at a glance Ethiopia Other countries Inc., consump. & expend. Labour supply Food security Nutrition Asset holdings Agricultural techniques Agricultural production Education Inc., consump. & expend. Labour supply Food security Nutrition Asset holdings Agricultural techniques Agricultural production Education Studies with no significant effects Studies with significant positive effects Studies with inconclusive or significant negative effects Do Public Works Programmes Work? 7

8 Regarding agricultural technology adoption, the PSNP s regular Type 2 Plus variant on the whole performs well and outperforms the regular Type 2 variant (especially with respect to fertiliser use and the adoption of stone terracing and fencing). However, this does not in all cases translate into tangible increases in agricultural production. Agricultural technology adoption has been rarely investigated in the context of Type 1 programmes. In short, the findings both overall and in most outcome areas are quite heterogeneous. This implies that the assumed benefits of PWPs can by no means be taken for granted, even with respect to the limited objective of enabling consumption smoothing. In fact, for all the outcome areas investigated in the systematic review that are expected to be positively influenced by PWPs, there are in each case some studies that support this expectation and some that do not. However, since for each outcome area, we found at least some programmes that meet their objectives, we conclude not that PWPs are ineffective per se, but rather that they can be effective under certain conditions. These conditions include in particular the PWP s specific design and implementation features. Another striking feature of the literature reviewed is that it contains almost no evidence on impacts stemming from the assets created through the public works and from the skills inculcated through training or on-the-job practice. Almost all the evaluations limit their analysis to the wage paid or the total net effect without disentangling from this what the direct wage-effect is and what is due to the two other channels of assets and skills. What are the lessons learned from cash-for-work programmes in countries neighbouring Syria? In response to the situation in Syria, BMZ launched the Partnership for Prospects initiative that provides refugees and local people from host communities with cash-for-work activities that offer readily available opportunities for income generation. These cash-for-work measures include (i) local services (simple tasks like collecting waste), (ii) labour-intensive infrastructure projects (building homes, schools and roads), (iii) the financing of wages (additional teachers and classroom assistants as well as health personnel), and (iv) in the long run, the rebuilding and repair of municipal infrastructure (buildings and roads) in liberated areas. So far, no rigorous evaluation has been undertaken to assess how effective these programmes have been in achieving their objectives. However, information drawn from project monitoring activities suggests that, by mid 2017, more than 110,000 jobs had been offered under the initiative. If the families of those holding these jobs are included, then both directly and indirectly the initiatives will have reached around 300,000 people. The monitoring also suggests that a large proportion of these jobs also served to alleviate the stress on public infrastructure in the regions that are hosting many of the refugees. At a later stage, impact evaluations will have to be conducted that compare the measures and their benefits against a hypothetical situation in which the measures were not implemented. It will also be necessary to investigate the cost-effectiveness of these measures in comparison to alternative safety nets. 8 Do Public Works Programmes Work?

9 Why do some programmes work and others not? Which PWP model is appropriate in what context? Regarding implementation, it should be noted that, compared to regular cash transfers, PWPs are much more demanding administration-wise, which introduces a number of additional potential pipeline breaks. While some of the studies analysed in the systematic review contain information on whether and where implementation fell short, they offer limited rigorous evidence of how this affected impacts. In addition, differences in the transfer value (i.e. wage rate times number of workdays) may also partly explain differences in the observed outcomes. The key lesson that can be drawn from the literature regarding the transfer value is that it is often too low, not paid regularly and not adjusted in line with inflation. That said, implementers do, of course, need to keep an eye on market wage rates in order to prevent crowding-out effects. For this reason, extending the employment duration (i.e. shifting from Type 1 towards Type 2) may be a more sustainable option to increase the transfer value for beneficiaries. If recommendations on programme design were to be formulated based on the most positive available evidence for each PWP type, they might be as follows (note that more rigorous testing is needed to confirm these): Programmes offering short-term employment at low wages (Type 1) only seem suitable in contexts of acute poverty and to achieve a few basic objectives, such as enabling short-term consumption smoothing. In contexts where chronic poverty and underemployment are widespread and persistent throughout the year, having PWPs that pay adequate wages over an extended period (Type 2) may enable beneficiaries to (a) accumulate enough savings and assets to build some resilience against minor shocks and (b) accumulate assets and make productive investments that are at least sufficient to marginally boost post-pwp income. However, such PWPs are unlikely to reduce poverty on any significant scale and are not a complete substitute for responses to severe (especially covariate) shocks. In order to reach the poorest and enable them to draw tangible benefits from employment that improve their livelihoods, the targeting mechanism needs to be more sophisticated than a system that relies solely on self-targeting based on low wages. Do Public Works Programmes Work? 9

10 Conclusion References As the rigorous evidence that is available largely fails to cover the role played by the created assets and skills in achieving the observed outcomes, for the time being the case for PWPs vis-à-vis alternative social protection instruments such as cash transfers mainly rests on assumed benefits and theoretical deliberations. Furthermore, it has been insufficiently investigated whether combined effects realised through the three potential channels wages, skills and assets persist beyond the programme duration. More research and thorough evaluations are needed to find out whether public works programmes can work and what conditions and design features are likely to enable them to realise their full potential. This review is a starting point for this endeavour. Looking ahead, the Deutsche Gesellschaft für Internationale Zusammenarbeit (GIZ) GmbH, on behalf of the German Federal Ministry for Economic Cooperation and Development (BMZ), and the University of Passau are planning a collaborative research project on PWP experiences and experiments in Malawi, the aim of which is to close many of the remaining knowledge gaps. Studies included in systematic review Andersson C., Mekkonnen A., & Stage J. (2011). Impacts of the Productive Safety Net Programme in Ethiopia on livestock and tree holdings of rural households. Journal of Development Economics, 94(1), Beegle, K., Galasso, E., & Goldberg, J. (2017). Direct and indirect effects of Malawi s public works program on food security. Journal of Development Economics, 128, Béné, C., Devereux, S., & Sabates-Wheeler, R. (2012). Shocks and Social Protection in the Horn of Africa: Analysis from the Productive Safety Net Programme in Ethiopia. IDS Working Paper, 2012 (395). Brighton, UK: Centre for Social Protection/Institute of Development Studies. Berhane, G., Hoddinott, J. Kumar, N., & Margolies, A. (2016). The Productive Safety Net Programme in Ethiopia: impacts on children s schooling, labour and nutritional status. 3ie Impact Evaluation Report, 55. New Delhi: International Initiative for Impact Evaluation (3ie). Berhane, G., Abay, M., & Woldehanna, T. (2015). Childhood shocks, safety nets and cognitive skills: Panel data evidence from rural Ethiopia. ESSP II Working Paper 73. Washington, D.C. and Addis Ababa, Ethiopia: International Food Policy Research Institute (IFPRI) and Ethiopian Development Research Institute (EDRI). Berhane, G., Gilligan, D. O., Hoddinott, J., Kumar, N., & Taffesse, A. S. (2014). Can social protection work in Africa? The impact of Ethiopia s Productive Safety Net Programme. Economic Development and Cultural Change, Do Public Works Programmes Work?

11 Berhane, G., Hoddinott, J, Kumar, N., & Tafesse, A.S. (2011). Evaluation of Ethiopia s Food Security Program: Documenting Progress in the Implementation of the Productive Safety Nets Programme and the Household Asset Building Programme. Brighton, UK: Institute of Development Studies. Bertrand, M., Crépon, B., Marguerie, A., & Premand, P. (2017). Contemporaneous and Post-Program Impacts of a Public Works Program: Evidence from Côte d Ivoire. Unpublished paper. Bertrand, M., Crépon, B., Marguerie, A., & Premand, P. (2016). Impacts à Court et Moyen Terme sur les Jeunes des Travaux à Haute Intensité de Main d oeuvre (Thimo). Washington, D.C.: World Bank Group. Bezu, S., & Holden S. (2008). Can food-for-work encourage agricultural production? Food Policy, 33(6), Christian, S., Janvry, A. de, Egel, D., & Sadoulet, E. (2015). Quantitative evaluation of the Social Fund for Development Labour-intensive Work Program (LIWP). CUDARE Working Paper. Berkeley: University of California at Berkeley. Debela, B., Shively, G., & Holden, S. (2014). Does Ethiopia s Productive Safety Net Program Improve Child Nutrition? Ås, Norway: Norwegian University of Life Sciences (NMBU). Favara, M., Porter, C., & Woldehanna, T. (2017). Smarter through Social Protection? Evaluating the Impact of Ethiopia s Safety-Net on Child Cognitive Abilities. IZA Discussion Paper Series, Gebrehiwot, T. & Castilla, C. (2017). Impact of Ethiopia s Productive Safety Net Program on Household Dietary Diversity and Child Nutrition in Rural Ethiopia. Unpublished paper. Gilligan, D. O., Hoddinott, J., & Taffesse, A. S. (2009a). The Impact of Ethiopia s Productive Safety Net Programme and its Linkages. Journal of Development Studies, 45(10), Gilligan, D. O:., Hoddinott, J., Kumar, N., &, Tafesse A.S. (2009b). An Impact Evaluation of Ethiopia s Productive Safety Nets Programme. Washington, DC: International Food Policy Research Institute. Gilligan, D., & Hoddinott, J. (2007). Is There Persistence in the Impact of Emergency Food Aid? Evidence on Consumption, Food Security, and Assets in Rural Ethiopia. American Journal of Agricultural Economics, 89(2), Hartwig, R. (2013). The short-term impact of public works on household welfare in rural Rwanda: A mixed methods approach. Unpublished paper. Hoddinott, J., Berhane, G., Gilligan, D., Kumar, N., & Taffesse, A.S. (2012). The Impact of Ethiopia s Productive Safety Net Programme and Related Transfers on Agricultural Productivity. Journal of African Economies, 21(5), Hoddinott, J., Gilligan, D., & Taffesse, A.S. (2009). The Impact of Ethiopia s Productive Safety Net Program on Schooling and Child Labour. Holden, S., Barrett, C.B., & Hagos, F. (2003). Food-for-Work for Poverty Reduction and the Promotion of Sustainable Land Use: Can it Work? Working Paper, Ithaca: Department of Applied Economics and Management, Cornell University. Do Public Works Programmes Work? 11

12 Osei-Akoto, I., Bawakyillenuo, S., Owusu, G., & Offei, E.L. (2014). The impact of public works on poverty and welfare: evidence from Ghana. Institute of Statistical, Social and Economic Research (ISSER), University of Ghana. Unpublished paper. Porter, C., & Goyal, R. (2016). Social protection for all ages? Impacts of Ethiopia s Productive Safety Net Program on child nutrition. Social Science & Medicine, 159, Quisumbing, A.R. (2003). Food Aid and Child Nutrition in Rural Ethiopia. World Development, 31(7), Rosas, N. & Sabarwal, S. (2016). Public Works as a Productive Safety Net in a Post-Conflict Setting: Evidence from a Randomized Evaluation in Sierra Leone. Policy Research Working Papers, Washington, DC: World Bank. Rosas, N. & Sabarwal, S. (2014). How productive are productive safety nets? Evidence from Public Works in Sierra Leone. Washington, DC: World Bank. Rosas, N., & Sabarwal, S. (2014). Opportunity and resilience: Do public works have it all? Evidence from a randomized evaluation in Sierra Leone. Mimeo. Washington, DC: World Bank. Sabates-Wheeler, R., & Devereux, S. (2010). Cash transfers and high food prices: Explaining outcomes on Ethiopia s Productive Safety Net Programme. Food Policy, 35(4), Tafere, Y. & Woldehanna, T. (2012). Beyond Food Security: Transforming the Productive Safety Net Programme in Ethiopia for the Well-being of Children. Young Lives Working Paper, 83. Oxford: Young Lives. Weldegebriel, Z., Prowse, M.P. (2013). Climate change adaptation in Ethiopia: to what extent does social protection influence livelihood diversification? Development Policy Review, 31 (S2), Woldehanna, T. (2009). Productive Safety Net Programme and Children s Time Use Between Work and Schooling in Ethiopia. Young Lives Working Paper, 40. Oxford: Young Lives. Other references Beierl, S. & Grimm, M. (2018). Do public works programmes work? A systematic review of the evidence from programmes in low and lower-middle income countries in Africa and the MENA region. Bonn and Eschborn: Deutsche Gesellschaft für internationale Zusammenarbeit (GIZ) GmbH. BMZ. (2018). Cash for Work: Job-Offensive schafft neue Perspektiven. Retrieved from BMZ. (2017). Bildung, Beschäftigung, Infrastruktur: Entwicklungsministerium erreicht mit Sonderinitiative Flucht zehn Millionen Menschen vor Ort. Retrieved from McCord, A. (2012a). Public Works and Social Protection in Sub-Saharan Africa: Do Public Works work for the Poor? Tokyo: United Nations University Press. World Bank. (2015). The State of Social Safety Nets Washington, DC: World Bank. 12 Do Public Works Programmes Work?

13 Published by Deutsche Gesellschaft für Internationale Zusammenarbeit (GIZ) GmbH Registered offices Bonn and Eschborn / Germany GIZ Bonn Friedrich-Ebert-Allee Bonn Germany T F GIZ Eschborn Dag-Hammarskjöld-Weg Eschborn Germany T F Sector Initiative Social Protection E social-protection@giz.de I As at November 2018 Authors Stefan Beierl and Michael Grimm Chair of Development Economics University of Passau Cover Design and Layout Bettina Riedel / briedel64@gmx.de Photo credits Stefan Beierl On behalf of Federal Ministry for Economic Cooperation and Development (BMZ) Division Health, Population Policy and Social Protection Address of the BMZ offices Bonn and Berlin / Germany BMZ Bonn Dahlmannstraße Bonn Germany T F BMZ Berlin Stresemannstraße Berlin Germany T F poststelle@bmz.bund.de GIZ is responsible for the content of this publication. 13

14 Deutsche Gesellschaft für Internationale Zusammenarbeit (GIZ) GmbH Registered offices Bonn and Eschborn On behalf of the Friedrich-Ebert-Allee Bonn, Germany T F Dag-Hammarskjöld-Weg Eschborn, Germany T F E info@giz.de I