Validation Report North East Community Restoration and Development Project (Sri Lanka) (Loan 1846-SRI)

Transcription

1 Board of Directors IN September 2010 Validation Report North East Community Restoration and Development Project (Sri Lanka) (Loan 1846-SRI) The attached Report is circulated at the request of the Director General, Independent Evaluation Department. The report is also being made publicly available. For Inquiries: R. B. Adhikari, Independent Evaluation Department (Ext. 4119) S. Shrestha, Independent Evaluation Department (Ext. 5375)

2 Validation Report Reference Number: PCV: SRI Project Number: Loan Number: 1846-SRI(SF) September 2010 Sri Lanka: North East Community Restoration and Development Project Independent Evaluation Department

3 ABBREVIATIONS ADB Asian Development Bank CBO community-based organization IDP internally displaced person M&E monitoring and evaluation NECORD North East Community Restoration and Development (project) NGO nongovernment organization O&M operation and maintenance PCR project completion report PMU project management unit RRP report and recommendation of the President NOTES (i) (ii) In this report, $ refers to US dollars. For an explanation of rating descriptions used in ADB evaluation reports, see Asian Development Bank Guidelines for Preparing Performance Evaluation Reports for Public Sector Operations. Manila. Key Words adb, agriculture, asian development bank, community restoration, conflict, education, finland, fisheries, germany, health, housing, income generation, internally displaced person, irrigation, livelihood, netherlands, north east sri lanka, norway, opec fund, roads, shelter, water supply and sanitation Director Team leader Team members R.B. Adhikari, Independent Evaluation Division 1, Independent Evaluation Department (IED) S. Shrestha, Evaluation Specialist, Independent Evaluation Division 1, IED A. Morales, Evaluation Officer, Independent Evaluation Division 1, IED V. Melo, Operations Evaluation Assistant, Independent Evaluation Division 1, IED In preparing any evaluation report, or by making any designation of or reference to a particular territory or geographic area in this document, the Independent Evaluation Department does not intend to make any judgments as to the legal or other status of any territory or area.

4 PROJECT COMPLETION REPORT VALIDATION FORM A. Basic Project Data PCR Validation Date: September 2010 Project and Loan Number: 34442; 1846-SRI(SF) Approved ($million) Actual ($million) Project Name: North East Community Total Project Costs: Restoration and Development Project Country: Sri Lanka Loan: a Sector: Agriculture and Natural Total Cofinancing: Resources ADB Financing: ADF: $25.00 million Borrower: OCR: 0.00 Beneficiaries: Cofinanciers: OPEC Fund Others: Government of Germany Government of the Netherlands Government of Norway Government of Finland OPEC Fund Government of Germany Government of the Netherlands Government of Norway Government of Finland Approval Date: 16 Oct 2001 Effectiveness Date: 09 Feb Jan 2002 Signing Date: 07 Nov 2001 Closing Date: 30 Jun Feb 2009 Project Officers: Name: K. M. Tilakaratne M. Thiruchelvam Location: Resident Mission Resident Mission From To Validator: Quality Control Reviewer/Peer Reviewer: J. Cook Consultant S. Shrestha Evaluation Specialist Director: R.B. Adhikari, IED1 A. Morales Evaluation Officer, IED1 ADB = Asian Development Bank, ADF = Asian Development Fund, IED1 = Independent Evaluation Division 1, OCR = ordinary capital resources, OPEC = Organization of the Petroleum Exporting Countries, PCR = program completion report, SF = special fund, SRI = Sri Lanka. a Based on the ADB Loan Financial Information System as of 15 September B. Project Description (summarized from report and recommendation of the President 1 [RRP]) (i) Rationale. The armed conflict between the Government of Sri Lanka and the Liberation Tigers of Tamil Eelam commenced in At the time of design, about 60,000 people had been killed, and a further 700, ,000 displaced, often repeatedly. The conflict also destroyed or damaged houses, infrastructure, and livelihoods, and led to redistribution of ethnic populations, loss of access to markets and social services, and minimal functioning of the civil administration in the project area. The disruption of economic activity was estimated to have decreased the contribution of the Northern and Eastern provinces to national gross domestic product from 15% in the 1980s to 4% in Support was required to restore infrastructure, services, and livelihoods. 1 ADB Report and Recommendation of the President to the Board of Directors: Proposed Loan to Sri Lanka for the North East Community Restoration and Development Project. Manila.

5 2 (ii) Impact. The North East Community Restoration and Development Project (NECORD) was intended to contribute to the government s overall relief and rehabilitation program for the North East. It would contribute to a reduced level of poverty and improved living standards for a substantial proportion of the population of the North East conflict region, particularly communities with significant proportions of internally displaced people (IDPs). As part of the government s overall relief and rehabilitation program for the North East, it would coordinate closely with projects and programs financed by the government and other international and domestic agencies. The project would implement a range of complementary activities in the project area to ensure that the inhabitants (be they the original occupants or resettled or relocated people) would experience broad improvements in their livelihoods and in their access to services, utilities, and income generating opportunities (footnote 1, para. 49). (iii) Objectives or expected outcomes. The project purpose was to improve the living conditions and wellbeing of communities in the project area that have been affected by the conflict, particularly communities that contain significant proportions of internally displaced people. (footnote 1, Appendix 1). The following sectoral outcomes were defined (at times incorrectly) as outputs in the project s design and monitoring framework (DMF): (a) Health. Improved health conditions in the North East by providing health facilities and training, and upgrading of district hospitals; decreased incidence of malaria, typhoid, and dysentery. (b) Education. Educational facilities and opportunities for internally displaced people provided by upgrading selected schools and providing infrastructure facilities for the selected schools. Improved quality of education in the North East. (c) Water supply and sanitation. Drinking water and sanitary facilities provided for IDPs and poor people in rural villages in the North East. (d) Shelter. Resettled and relocated IDPs provided with land, housing, basic infrastructure and community facilities. (e) Agriculture irrigation and fisheries. Improved agricultural production and opportunities. (f) Roads. Improved access roads to town centers. (g) Income generation. No specific outcomes identified. (iv) Components and outputs. The project had two broad components: (a) larger-scale anchor subprojects, which typically consisted of investments that would benefit a broad cross-section of each district s population, and were to be implemented largely through conventional government contracting arrangements; and (b) smaller-scale community-level subprojects, which were typically investments for individuals or specific communities, and which would be implemented by local organizations and communities as implementing partners assisted by service providers, including government agencies at the province, district, and division level; project field staff; and nongovernment organizations (NGOs). C. Evaluation of Design and Implementation (Project completion report [PCR] 2 assessment and Validation) (i) Relevance of design and formulation. The design and formulation of the project, including related cofinancing, were in line with the needs and requirements of the four (later eight) participating districts, and the villages and communities in their divisions. It was generally consistent with (a) the Asian Development Bank's (ADB's) operational strategy for Sri Lanka (to promote economic efficiency and higher sustainable growth to improve employment and reduce poverty); (b) ADB's Disaster and Emergency Assistance Policy (1987); (c) Sri Lanka s 2000 Framework for Relief, Reconstruction, and Rehabilitation; and (d) the purpose (if not all details) of the national 2002 Poverty Reduction Strategy. The project rehabilitated damaged infrastructure and supported livelihood restoration. The rehabilitated facilities would enable beneficiaries to resume a normal life by helping to restore basic services and income-generating opportunities; this was particularly important for villages with new resettled people and for communities with large numbers of returning IDPs. The project was implemented as part of the government's overall program of restoring reasonable living conditions for people affected by the conflict. 2 ADB Completion Report: North East Community Restoration and Development Project in Sri Lanka. Manila.

6 3 The small-scale technical assistance that prepared the project appears to have been well-conducted, although this is not mentioned by the PCR, which refers to the technical assistance in discussing the performance of ADB, rather than in Chapter 2A as indicated by PAI The project was expected to work in four districts in the Eastern and Northern provinces: Ampara, Batticaloa, Mannar, and Vavuniya. However, once the February 2002 ceasefire was declared, the government and ADB agreed to extend the project to all districts in the two provinces, including Jaffna, Kilinochchi, Mullaittivu, and Trincomalee. This change does not appear to have been covered by a formal change in scope. The project attempted to ensure that there was fair distribution of funds among the eight districts and 74 divisions, and also balanced support to all cultural and religious communities. This change in scope, together with the recorded change in scope in 2007 to include the Kilinochchi hospital, are discussed elsewhere in the PCR, but not in this section, as required under PAI 6.07 (footnote 3, Appendix 2, para. 3). (ii) Project Outputs. Outputs are summarized below by sector summarized from the PCR (footnote 2, Appendix 2). (a) Health. The project completed 102 contracts for 62 hospitals, including one general hospital in Kilinochchi, seven district hospitals (four in Ampara, two in Jaffna, and one in Mannar), and one divisional hospital in Trincomalee, as well as six rural hospitals. To support these hospitals, 1 primary care unit, 2 peripheral units, 16 dispensaries, 17 health centers, and 26 blocks of staff quarters were constructed. The project also provided 20 ambulances, 26 trucks, and 15 tractors for operation and maintenance (O&M), and 3 emergency generator sets. A total of 640 midwives were trained. (b) Education. Outputs in this sector included constructing teachers' quarters in remote and rural areas to improve recruitment and retention of teachers. Works included water supply, sanitation, and toilets. Furniture and educational materials, including textbooks, were provided. The project constructed 1,486 new classrooms, 50 offices, 20 activity rooms, 1 student hostel, 11 sets of teachers' quarters, and 30 fully equipped science laboratories. Desks and chairs (19,410 for students and 1,950 for teachers), water supply, and 357 female and 237 male latrines and washrooms were provided. The Education Department was supported in recruiting 1,950 Tamil-medium teachers, about a third of the vacancies in the project areas. (c) Water supply and sanitation. This component was substantially smaller than envisaged at appraisal. Works were constructed through 12 subprojects covering 4 town and 4 village piped systems, 117 community systems, and 827 latrines. The component directly benefited an estimated 18,700 families. (d) Shelter. The budget for this component in RRP was $4.0 million. Actual disbursement was $2.5 million. The allocation per dwelling increased substantially over the project period to respond to cost inflation, and the number of dwellings constructed was substantially below the target. Output under the shelter component included 2,377 houses (out of 7,000 planned), and 317 kilometers of village roads. (e) Community development. The component aligned closely with the shelter component. The main aspects were (1) providing basic community and village infrastructure, roads, water, sanitation, and power; (2) short-term income restoration through labor to construct the basic utilities and services attached to the shelter, schools, community development, and agricultural irrigation works; and (3) longer-term income activities including vocational training. Outputs included (1) 58 community subprojects for basic services and facilities, 21 subprojects for preparation work for new resettlement villages, land clearing, roads and paths, and electrification for 74 villages; and (2) vocational training for 502 men and 93 women in skilled trades such as plumbing, carpentry, electrical installation and repair, masonry, and auto mechanics. In addition, 58 women's rural development societies with over 5,000 women members were supported. 3 ADB Project Completion Report. Project Administration Instructions. PAI No. 6.07A. Manila.

7 4 (f) Agriculture and fisheries. This component comprised 44 subprojects. In the agriculture sector 13 major irrigation systems, including 14 storage reservoirs, were rehabilitated and restored, as were 13 veterinary facilities (4 with equipment), 21 agrarian outreach centers, and 1 agricultural research station. Equipment for the agrarian centers included 4 tractors with plows and wagons, 6 extension offices, and 7 blocks of staff quarters. The fisheries subcomponent constructed 5 anchor points and 1 fish market in Jaffna, 2 fishery facilities in Mannar (including 30 quarters for the poorest fishers), and provided 144 new outboard motors, 81 fishing boats, and 310 sets of fishing nets. An estimated 44,000 families benefited from the various agricultural works and 780 fishing families from the fisheries subcomponent. (g) Roads. Provincial and district roads, and access roads to the existing communities, had deteriorated due to neglect during the conflict and needed rehabilitation. A total of 77 kilometers of rural roads including 15 bridges were rehabilitated, improved, and widened. Two ferries were supplied. In the absence of specific targets, it is not possible to assess whether these outputs were sufficient to meet the basic needs of the local population. However, with the reported close involvement of local government and the communities in subproject selection, it is likely that the subproject mix represented a reasonable reflection of local priorities (iii) Project Cost, Disbursements, Borrower Contribution, and Conformance to Schedule. Project cost totaled $44.6 million compared to $40 million budgeted. Of the increase, the government contributed $2.7 million and $0.85 million was provided by the governments of Norway and Finland to pay for civil works and equipment at the Kilinochchi general hospital. ADB contributed loan funds of $25.8 million, the government $9.7 million, the Organization of the Petroleum Exporting Countries Fund $3.9 million, and the government of Germany $2.5 million and the government of the Netherlands $0.8 million. Loan effectiveness was in January 2002, final disbursement in June 2008 and loan closing in February 2009, 19 months behind schedule. No extension was granted. The PCR (footnote 2, para. 15) states that project physical completion was scheduled for 31 December 2006, with loan closure by 30 June 2007, and that this schedule was achieved; this conflicts with the information included in the project basic data. Approval for the change in scope was not granted until 2007, suggesting that physical completion occurred at some point in Why loan closing was so delayed is not discussed. (iv) Implementation Arrangements, Conditions and Covenants, related Technical Assistance, and Procurement and Consultant Performance. The executing agency was the Ministry of Local Government and Provincial Councils which provided liaison with the central ministries in Colombo. The implementing agency was the Northern and Eastern Provincial Council. In 2007, the province was divided into Northern and Eastern provinces, resulting in bifurcation of the implementing agency, under the direction of the chief secretaries of each province. The Provincial Project Coordinating Committees met every 2 months to review subproject proposals from district governments for inclusion under the project. Eight district project management units (PMUs) were responsible for day-to-day implementation; facilitating the preparation of all plans, designs, and bidding documents; and undertaking construction supervision on behalf of the departments and line agencies. Unusually, all project covenants were reported by PCR to have been fully complied with on a timely basis. Consultant and contractor performance is reported by PCR as satisfactory. The resumption of hostilities in 2006 meant that nine contracts could not be completed and had to be finished under the ongoing NECORD II project. Five contracts were also dropped due to tsunami damage in December 2004, reducing the total number of completed contracts to 596, covering 451 subprojects. (v) Performance of the Borrower and Executing Agency. The PCR rates the performance of the borrower and/or executing agency as satisfactory on the following basis: (a) counterpart funding increased by 30% over the appraisal estimate, (b) full-time staff were allocated to key positions, (c) frequent meetings of the coordinating committee took place (the PCR reports 23 meetings in 60 months), (d) sound links were established between the various levels of government, and (e) covenants were fully complied with.

8 5 (vi) Performance of ADB. Responsibility for project supervision was delegated to the Sri Lanka resident mission when the loan became effective. ADB accepted a rapid subproject approval process under its Disaster and Emergency Assistance Policy (1987), with post-facto approval (after the first three contracts in each sector). Resident mission staff participated in the provincial project coordination committee meetings and conducted several reviews, although the PCR indicates there were long gaps between missions during the middle period of the project 14 months from December 2002 to the mid-term review, and then 21 months to December 2005, after which only one review mission occurred prior to completion. The low frequency of review missions does not appear to have had a negative impact on overall performance. This validation rates ADB s performance satisfactory. The PCR does not explicitly rate ADB s performance, as indicated in PAI 6.07 (footnote 3, para. 21). D. Evaluation of Performance (PCR assessment and Validation) (i) Relevance. This validation finds that the project was highly relevant, compared to the PCR rating of very relevant, which is not a term used by the Independent Evaluation Department (IED) to rate relevance. The project was the first major intervention by ADB in the much-neglected conflict zone. The state of the basic infrastructure, communities, and housing was very poor. Some areas had not had any new development or maintenance for many years, some since the hostilities broke out in This, combined with the physical damage caused by the conflict, left many residents without proper basic services and infrastructure. Many residents had relocated, either because of the conflict itself or because of the lack of economic activity, and were living with relatives or in welfare centers in the area, or in India. The project focused on the most urgent requirements as determined in consultation with the local communities and their local and divisional governments. (footnote 2, para. 33) NECORD aligned fully with ADB s Country Strategy and Program ( ) 4 and contributed to several of the program s planks that aimed at sustainable reduction in poverty through (a) broad-based pro-poor growth to generate employment opportunities and increase rural income; (b) social development focused on enhancing the quality of and access to education and increasing nutrition awareness; (c) promotion of community-based natural resources management; (d) good governance to make the public service more accountable and responsive to the needs of the poor; and (e) reduction of conflictrelated poverty. The country program was to focus on seven sectors: (a) agriculture and rural development, (b) roads (and ports), (c) energy, (d) finance, (e) education, (f) water supply and sanitation and urban development with a strategic regional focus, and (g) environment and natural resources management. Although the support provided was relevant, the validation notes that it is an agriculture and natural resource sector project. It would have been appropriately classified as a multisectoral support project. In terms of ADB s Country Partnership Strategy , 5 published in October 2008, NECORD was fully responsive to several of the planks supporting Pillar 2 (social development). Its outputs or approaches were closely linked to (a) assistance to the poor to connect to power and water supply systems; (b) provision of rehabilitated basic economic and social infrastructure in lagging regions and conflict-affected areas; (c) greater community involvement in project decision making and implementation; and (d) targeted pro-poor programs in education, bringing centers of excellence into the poorest districts. NECORD also corresponded well with Sri Lanka s development priorities expressed in the December 2000 Framework for Relief, Reconstruction, and Rehabilitation (the 3R Framework) to provide policies, directions and a basis for effective assistance to conflict-affected communities in the country, especially in the North East (footnote 1, para. 44). The 2002 Poverty Reduction Strategy, however, took a somewhat different approach to poverty reduction, focusing on removal of barriers and distortions, which largely focused on non-conflict areas and only had limited consonance with NECORD. 4 ADB Country Strategy and Program Update: Sri Lanka, Manila. 5 ADB Country Partnership Strategy: Sri Lanka, Manila.

9 6 Overall, this validation considers the highly relevant rating to be warranted, based on the following relationships with the subcriteria check list suggested by the project performance evaluation report guidelines. 6 (a) Adequacy of assessment of problems, opportunities, and lessons at the time of approval: (b) Consistency of the project s impact, outcome, and outputs with the government s development strategy (stated and real), ADB s strategy and program for the country, and ADB s strategic objectives: (c) Extent to which stakeholders saw the project as their own. (d) Choice of modality and instrument: (e) Extent to which project formulation (design) adopted the correct solution for the identified problem: (f) Degree of coordination and/or complementarity with development partners. Fully adequate in the context of a conflict and post-conflict environment. Fully consistent with ADB s policies and strategies. In the context of the peace agreement, the expansion of the project to cover eight districts was appropriate, and rapidly followed by design of a follow-on project (NECORD II). According to the PCR, beneficiary and local government ownership were strong Appropriate. The small-scale technical assistance proved to be well suited to project needs, and allowed rapid startup. While the project could be criticized for taking a broad approach to rural development and not fully designing subcomponents at the outset, this validation considers that this was the optimal approach under the (post-conflict) circumstances with a high proportion of IDPs, and a consequently mobile population. Linkage with cofinancing partners appears to have been good. The PCR reports good coordination with local NGOs and/or community-based organizations (CBOs). The situation following the 2004 tsunami is likely to have been confused, particularly in coastal towns and villages, with numerous aid organizations seeking to assist the population. The project documents do not provide a basis for assessing this issue. However, it is evident that NECORD was well advanced by the date of the tsunami, and had little potential to vary its contracts or otherwise respond to particular tsunami-related problems. (ii) Effectiveness in Achieving Outcomes. This validation concurs with PCR rating of effective. However, the PCR bases this on two main factors: (a) that processes followed the government's existing requirements and procedures, and (b) that 464 subprojects and 596 contracts under the project were completed within the original schedule and within the total loan amount. (footnote 2, para. 35) Since effectiveness should rate the extent to which the project achieved its planned outcomes, the PCR approach is inadequate. It would be expected that the investments made under the project should lead to the outcome objectives listed in B(iii), e.g., (a) improved living conditions and well being of communities affected by the conflict; (b) decreased incidence of malaria, typhoid, and dysentery; (c) improved quality of education; and (d) improved agricultural production and opportunities. However, the PCR does not 6 ADB Guidelines for Preparing Performance Evaluation Reports for Public Sector Operations. Manila.

10 7 provide information that allows such judgments to be made, or evidence that any data were collected by project benefit monitoring and evaluation. Consequently, this validation assumes the reported outputs led to the planned outcomes. (iii) Efficiency in Achieving Outcome and Outputs. The loan closing was 19 months later than planned, but the actual completion of works appears to have been achieved close to the target date. Process efficiency through normal government systems was high. The PCR (footnote 2, para, 37) quotes several examples of the project being able to bring assets back into use cheaply. While it can be presumed that these works were cost effective, no data are provided to support this. This validation rates the project efficient. (iv) Preliminary Assessment of Sustainability. This validation considers project outcomes likely sustainable. PCR rated the project sustainable to date. The PCR s failure to follow the IED guidelines is unfortunate, as the terminology used gives no indication of how sustainable the project outputs or outcomes will be in the future, as required for an assessment of sustainability. However, it does give strong support to a likely sustainable rating due to (a) O&M costs of the various works and programs being part of existing budgets, (b) the posting of doctors to rural hospitals and clinics, (c) the full utilization of all schools with sufficient teachers for the student population, (e) ownership and operation of the village and town piped water supply facilities constructed under the project by the National Water Supply and Drainage Board (the restored systems have house meters and consumers are paying a reasonable tariff helping offset the cost of O&M), (f) the flourishing of women's rural development societies, and (g) the (implied) operation of irrigation systems. Fisheries investments suffered a setback following the resumption of hostilities in However, boats are now being used and markets operated. (v) Impact (both intended and unintended). It is evident that the project has had a significant positive socioeconomic impact. As noted by PCR (footnote 2,para. 47) The NECORD project had a major beneficiary [sic] impact by helping to restore basic services, shelter, and livelihoods for 500,000 beneficiaries and benefiting a further 500,000 people indirectly. This helped to stabilize the economies of the Northern and Eastern provinces and helped people displaced by the hostilities return home to more productive and better quality lives by improving their standards of living and economic opportunity. It would have been helpful to this assessment if the PCR had included information on changes in poverty levels in target communities over the course of the project. Despite the rapid changes in the communities over the period, some data could probably have been collected and presented. However, it is recognized that poverty data in Sri Lanka are not always reliable because they are used to define the recipients of direct subsidies. The PCR indicates positive gender outcomes. Importantly, the project applied the national do no harm strategy, which tried to prevent exacerbation of ethnic tensions and worsening of the conflict through unbalanced or inappropriate development. The PCR makes no comment on environmental or resettlement impacts, but the selection criteria applied tried to eliminate adverse impacts in these areas. E. Overall Assessment, Lessons, and Recommendations (Validation of PCR assessment) (i) Overall Assessment. This validation rates the performance of NECORD successful based on ratings against the core evaluation criteria of highly relevant, effective, efficient, and likely sustainable. The PCR rating of highly successful is not supported it appears to have been based on subjective assessment and not on the ratings basis required under IED s guidelines. (ii) Lessons. The PCR identifies a number of useful lessons: (a) Working with and supporting existing government structures and procedures, works, and programs can be done quickly and efficiently. Implementation through provincial councils supported the principles of devolution and improved coordination between central and provincial governments. (b) The bottom-up approach to select the most urgent and locally prioritized works led to better ownership of the various works, thus contributing to continued sustainability.

11 8 (c) By determining specific requirements and works as implementation progressed, project authorities were able to respond to the most urgent needs, even in a fluid conflict situation. (d) Working with CBOs that were already established in each village helped in selecting the highestpriority works and improving local income-earning activities. CBOs also remained to support the various activities, thereby benefiting sustainability. (e) The involvement of multiple development partners helped the overall implementation and impact of the project through the funding of cash grants, consulting services, and NGO support, which the government hesitates to finance from loan funds. (f) It is important to ensure equity between projects, (e.g., in terms of funding per dwelling), as imbalances can lead to jealousy; this is particularly true in conflict and post-conflict situations. In addition, it is noted by this validation that the project worked well with both sides in the conflict, and was well accepted in conflict-affected areas. Developing this type of rapport is important, and reflects well on the project implementers. The lessons should be relevant in other conflict or post-conflict situations. (iii) Recommendations. PCR does not make any project-specific recommendations. General (and useful) recommendations made by PCR (footnote 2, para. 58 onwards) include: (a) ADB should work through existing government departments and agencies following their standard procedures. (b) The establishment of a dedicated PMU to facilitate implementation and coordination with different government agencies is required, especially when the project involves multiple sectors, multiple layers of governments, wide geographical areas, and multiple development partners. The PMU project director should be a senior respected government officer who reports directly to the chief secretary or other senior administrative officer in the local government. (c) Suitable management in a conflict situation requires extensive coordination with the groups involved in the conflict. (d) ADB should allow adequate time for planning, design, tendering, and construction of projects, including emergency projects. The NECORD project was allowed 5 years for implementation rather than the usual 3 years under ADB emergency loans. The additional time enabled the government to properly plan and implement the project works and livelihood programs, and was one of the reasons for the project's success. (e) ADB and the borrower should plan for adequate incremental expenditure and consulting support to plan, design, and implement such a project. In the case of NECORD, about 10% was allowed for this under ADB and participating development partner funding. In addition the German grant of $2.5 million provided extensive additional support to the project's implementation through experts provided by German development cooperation through GTZ. (f) The bottom-up approach to the selection of subprojects should be followed in all projects, to ensure there is both a need and demand for the new works and ownership of completed facilities. (g) ADB should ensure that emergency assistance projects are administered by resident missions (rather than headquarters). (h) ADB's policy on emergency assistance should look at (1) the time period for implementation, given that there are often physical and human resource constraints; (2) flexibility in interpretation of the safeguard policies in these special circumstances; (3) procurement methods; and (4) an increased role for the resident mission in preparing and implementing projects. This validation notes that monitoring and evaluation (M&E) appears to have focused on planning, design and implementation of subprojects. This has also been evident in other ADB projects with major components involving small-scale rural infrastructure construction or rehabilitation, where the number of subprojects and contracts is large. Project management in this situation can view benefit monitoring as being an unnecessary encumbrance to implementation and burden on the limited number of monitoring staff. However, the collection of basic information on a number of key indicators should not be impossible, even under a complex project such as NECORD. It is consequently recommended by IED that ADB project officers encourage implementing agencies to establish adequate project monitoring mechanisms so that information is available during implementation and post-completion to allow performance against outcome indicators to be assessed. The collection and analysis of such information should also assist in improving implementation performance by providing early feedback on the success

12 9 of particular interventions. F. M&E Design, Implementation, and Utilization (PCR assessment and Validation) While it was planned that project consultants (mainly from GTZ) would establish an M&E system, it appears that the focus was on planning, design, and implementation of the subprojects. The PCR does not report on the collection of information relating to the indicators included in the DMF. According to PAI 6.07 (footnote 3), reporting on M&E should be discussed in the overall evaluation section of the PCR. G. Other (e.g., Safeguards, including governance and anticorruption; Fiduciary aspects; Government assessment of the project, as applicable) (PCR assessment and Validation) Other projects in ex-conflict zones have experienced issues relating to pressures on contractors to employ ex-combatants or particular suppliers. There is no report on whether this was an issue under the project; however, it could have been useful for the PCR to mention issues identified during implementation, if any. No information is available on the application of ADB s safeguard policies. Independent Evaluation Department H. Ratings Project Completion Report Review Reason for Disagreement/Comments Relevance Very relevant Highly relevant It is presumed that PCR s very relevant rating is the same as the guideline s highly relevant. Effectiveness in Achieving Outcome Efficiency in Achieving Outcome and Outputs Preliminary Assessment of Sustainability Borrower and Executing Agency Performance of the Asian Development Bank Impact Overall Assessment Quality of Project Completion Report Effective Effective PCR does not make any specific assessment of outcomes, but only talks about outputs in this section. Good (Efficient) Sustainable to date Satisfactory Efficient Likely Sustainable Satisfactory Presumably PCR means efficient. In terms of efficiency of process, it is likely that the project was efficient. While PCR states that rehabilitation gave high benefit for low cost, there is no assessment of cost-effectiveness. Not rated Satisfactory Responsibility for project supervision was appropriately delegated to Sri Lanka resident mission when the loan became effective. ADB accepted a rapid subproject approval process under its Disaster and Emergency Assistance Policy (1987). Major Significant beneficiary impact Highly successful Successful Partly Satisfactory In the absence of clearly defined ratings against the core evaluation criteria, PCR does not adequately justify its rating of highly successful.

13 10 I. Comments on PCR Quality IED assesses the quality of the PCR partly satisfactory. (i) The PCR is well written and adequately describes project implementation. (ii) Unfortunately, the PCR does not fully follow PAI guidelines (footnote 3). Few of its ratings are consistent with PAI 6.07 para. 5, which requires that PCR ratings conform to IED s PPER guidelines (footnote 6). (iii) The PCR s ratings against the core evaluation criteria do not appear to support a rating of highly successful. (iv) Lessons and recommendations are considered appropriate and relate well to the PCR s analysis. However, the PCR made no project-specific recommendations. (v) The PCR contains almost no assessment of outcomes, or of progress against the indicators identified in the DMF. This is probably due to an inadequate M&E system under the project, a project failing which itself merited some discussion in the PCR. J. Recommendation for IED follow up The project has useful lessons relating to the development of small-scale rural infrastructure using a participatory approach, and to project design and implementation in a conflict zone. It is recommended that post-evaluation be considered for 2012, when livelihood investment outcomes and the longer-term sustainability of infrastructure and other investments should be clear. K. Data Sources for Validation Key data sources for validation were back-to-office reports and Aide Memoires of ADB missions, the RRP, and the PCRs of ADB and the Borrower.

14 REGIONAL DEPARTMENT S RESPONSE TO THE PROJECT COMPLETION REPORT VALIDATION REPORT On 28 May 2010, the Independent Evaluation Department (IED) circulated the draft validation report for interdepartmental comments. IED received comments from the Sri Lanka Resident Mission and these were duly incorporated in the final draft. The resident mission reviewed the final report, supports the changes made by IED, and has no further comments.