Bangladesh: Emergency Flood Damage Rehabilitation Project

Transcription

1 Validation Report Reference Number: PCV: BAN Project Number: Loan Number: 2156-BAN (SF) December 2009 Bangladesh: Emergency Flood Damage Rehabilitation Project Independent Evaluation Department

2 ABBREVIATIONS ADB Asian Development Bank BWDB Bangladesh Water Development Board EA executing agency EIRR economic internal rate of return IED Independent Evaluation Department LGED Local Government Engineering Department PCR project completion report RRP report and recommendation of the President TA technical assistance TCR technical assistance completion report NOTES (i) (ii) In this report, $ refers to US dollars. For an explanation of rating descriptions used in ADB evaluation reports, see: ADB Guidelines for Preparing Performance Evaluation Reports for Public Sector Operations. Manila. Key Words adb, asian development bank, bangladesh, emergency flood damage rehabilitation, ied, independent evaluation department, oed, project completion report validation Director R. B. Adhikari, Independent Evaluation Division 1, Independent Evaluation Department (IED) Team leader W. Kolkma, Senior Evaluation Specialist, Independent Evaluation Division 1, IED (until 5 January 2009) Team members A. Anabo, Senior Evaluation Officer, Independent Evaluation Division 1, IED V. Melo, Operations Evaluation Assistant, Independent Evaluation Division 1, IED In preparing any evaluation report, or by making any designation of or reference to a particular territory or geographic area in this document, the Independent Evaluation Department does not intend to make any judgments as to the legal or other status of any territory or area.

3 PROJECT COMPLETION REPORT VALIDATION FORM A. Basic Project Data PCR Validation Date: December 2009 Project and Loan Number: BAN(SF) Approved Actual Project Name: Emergency Flood Total Project Costs Damage Rehabilitation Project ($M): Country: Bangladesh Loan ($M): Sector: Multisector Total Cofinancing ADB Financing ($M): Cofinanciers: ($M): Borrower ($M): Government of the Netherlands - SIDA Beneficiaries ($M): Others ($M): 0.00 a Approval Date: 20 Jan 2005 Effectiveness Date: 2 May May 2005 Signing Date: 2 Feb 2005 Closing Date: 31 Jul Dec 2007 Project Officer Name H. Carlsson A. Faruque Location (HQ or RM): BRM BRM From To Validator: Quality Control Reviewer/Peer Reviewer: M. A. Quassem W. Kolkma Senior Evaluation Specialist, IED1 Director: R. B. Adhikari, IED1 ADB = Asian Development Bank, BAN = Bangladesh, BRM = Bangladesh Resident Mission, HQ = headquarters, IED1 = Independent Evaluation Division 1, M = million, PCR = project completion report, RM = resident mission, SF = special fund, SIDA = Swedish International Development Cooperation Agency. a The governments of the Netherlands and Sweden, through SIDA, expressed interest in providing $25 million of cofinancing for the Project, subject to clearance from their management in late In absence of a firm commitment for cofinancing SIDA (Part A) and the Netherlands (Part D), ADB provided a loan under the umbrella or standby financing mechanism, whereby ADB confirmed for a $180 million loan but canceled a portion of its own loan when the grant financing became available. B. Project Description (summarized from the report and recommendation of the President 1 [RRP]) (i) Rationale. The devastating 2004 flooding caused severe damage to infrastructure and other assets, and disrupted economic activities, inflicting heavy losses to agricultural and industrial output, and slowing the expansion of services. The flooding also had a negative impact on poverty reduction, with the poor and the vulnerable suffering most in terms of losses to crops, livestock, property and housing, and reduced income opportunities. The flood-related damage to small-scale enterprises, the export-oriented knitwear industry, and jute, leather, and frozen foods was likely to affect export growth and place increasing pressure on imports due to the sharp rise in post-flood imports of food grains, medicines, and construction materials. By providing financing for the most urgently needed 1 ADB Report and Recommendation of the President to the Board of Directors on a Proposed Loan and Technical Assistance Grants to Bangladesh for the Emergency Flood Damage Rehabilitation Project. Manila.

4 2 components and works, complementing assistance by other financiers, the Project was designed to enable the Government to finance a portion of the overall recovery program, as well as help restore the economic activity essential for recovery of affected areas. (ii) Impact. Project impacts included (a) enabling people to resume their normal lives by restoring access to basic services; (b) benefiting those people, especially the poor, in the flood-affected areas by restoring rural infrastructure, and community infrastructure facilities; (c) helping to restore essential economic and social activities disrupted by the flood by rehabilitating vital public and community infrastructure; (d) helping to restore infrastructure and basic access to health and educational facilities and services by repairing roads, bridges, and railway tracks, and improving transport; (e) protecting people exposed to risk of inundation from future flooding, and raising agricultural output through flood control and irrigation measures; (f) improving health conditions, particularly of the slum dwellers, with improved water supply and sanitation, and creating greater access to employment opportunities through improvement of municipal infrastructure; and (g) providing significant employment opportunities for skilled and unskilled labor during implementation. The Project was also expected to have several environmental benefits, including limiting land erosion and failure of embankments along damaged infrastructure; and reducing health risks by rehabilitating damaged water supply and sanitation facilities, and cleaning drainage systems. (iii) Objectives or Expected Outcomes. The objective was to contribute to sustainable economic growth by minimizing the devastating impact of severe floods. This was viewed as essential to enable early restoration of economic and social activity in the affected areas, especially for the poor. The Project was to focus on priority areas identified through consultation with the Government and civil society, including public and private sectors and nongovernment organizations, and involve community participation. The Project included (a) capacity building and training to strengthen the Government s disaster preparedness by adopting flood-resistant infrastructure design standards, and (b) analytical input and capacity building to enhance early warning systems. (iv) Components and/or Outputs. The Project comprised five parts: (a) rural infrastructure, (b) roads, (c) railways, (d) water resources, and (e) municipal infrastructure. Part A: Rural Infrastructure. Rehabilitation of rural infrastructure, including rural roads (damaged paved upazila [subdistricts] and union roads), bridges, and culverts; restoration of existing flood and cyclone shelters; and provision of latrines to help communities, especially the poor, during future floods. The appraisal target was to rehabilitate about 5,500 kilometers (km) of rural roads and 13,200 meters of bridges and culverts. Part B: Roads. Rehabilitation of flood-damaged national, regional, and zila (district) roads, bridges, and culverts. The estimate at appraisal for road repair and rehabilitation was 2,968 km of pavement, 306 bridges and culverts, and 26 ferry landings. Part C: Railways. Rehabilitation of flood-damaged rail infrastructure and facilities on the core rail network, including civil works to restore embankments, tracks, bridges, essential buildings, station roads, and signaling and telecommunications facilities. Provision of emergency equipment and materials required to undertake expeditious repair work in future emergency situations. Initial assessment by Bangladesh Railway showed physical damage to about 320 km of track in 117 locations and 110 bridges in the east zone, and 96 km of track in 40 locations and 18 bridges. Part D: Water Resources. Rehabilitation of flood control, drainage, and irrigation facilities; repair of embankment breaches; and repair or replacement of water control structures, protective works, and canals. The total damage in flood control and irrigation schemes was estimated as follows: (a) embankment: full km, partial km; (b) irrigation and drainage: full 3.9 km, partial 25.4 km; (c) water control structures: full 33.0 km, partial km; and (d) protective work: full 15.5 km, partial 14.1 km.

5 3 Part E: Municipal Infrastructure. Rehabilitation of (a) roads, drains, bridges, and culverts; and (b) footpaths and drains in the slums in about 55 pourashavas (municipalities) in six divisions. This component was supposed to restore some 1,028 km roads, 47 km drains, 393 meters of bridge and culverts, and 63 km footpaths of slums and low-income areas. C. Evaluation of Design and Implementation (Project completion report 2 [PCR] assessment and validation) (i) Relevance of design and formulation. The PCR noted that the Project was designed and formulated to reduce poverty and contribute to Bangladesh s economic recovery, according to the country strategy and overall goal of the Asian Development Bank (ADB). The Project was relevant to ADB s country strategy, and its components were designed and implemented according to Government policy in the relevant sector. The design of part A was in line with the Government s sixth 5-year plan ( ); part B was consistent with the recommendations of the road master plan; the design for part C supported the Government s policy of promoting the railway subsector as the pro-poor mode of transport, facilitating inexpensive yet environmentally friendly transportation of long-haul freight; part D was designed and implemented according to the Government s national water policy and flood action plan; and part E conformed with the objectives of the Government s sixth 5-year plan to improve municipal infrastructure. The project s objective of poverty reduction can be linked directly to the focus of the design on productive infrastructure investment. The Independent Evaluation Department 3 (IED) endorses the assessment of the PCR. (ii) Project Outputs. The PCR briefly described the major outputs achieved under each of the five components in the main text (paras ), and in detail in its Appendix 1. The last column of Appendix 1, which deals with key issues and recommendations, makes confusing observations of the project completion review mission about achievement of targets. IED notes that the PCR has not addressed the assumptions and risks in the project framework appended to the RRP (Appendix 1). Some information is missing. The PCR did not record the achievement of the target for the repair of 26 ferry landings, nor the total number of bridges and culverts built. The PCR reported only that 2,115 meters of bridges and culverts were repaired. The PCR reported railway outputs, but did not relate it to the appraisal estimates. (iii) Project Cost, Disbursements, Borrower Contribution, and Conformance to Schedule. Project costs were adequately reported in paras of the PCR, with further details being reflected in Appendixes 3 and 4. The actual costs were almost 15% lower than the appraisal estimates, and the PCR has stated the reasons for this. IED analyzed the savings per component against target achievements and provides comments in the following table. Summary of analysis of savings and comments Component/Part Savings ($ million) Target achievement PCR s statement vis-à-vis Independent Evaluation Department s Comment A. Rural Infrastructure 5.53 Shortfall The PCR states that budget limitations dictated the cutbacks in rehabilitation targets (para. 13), but IED notes budget savings occurred. B. Roads Shortfall The PCR states that savings were due to lower contractors bid prices compared with the appraisal estimates, and devaluation of the taka relative to the US dollar (para. 20). IED assumes that failure to achieve some targets set at appraisal also contributed to lower expenditures. 2 ADB Project Completion Report on the Emergency Flood Damage Rehabilitation Project in Bangladesh. Manila. 3 IED was named the Operations Evaluation Department, or OED, until December 2008.

6 4 C. Railways Shortfall IED notes that $10 million in savings occurred even after enhancing the expenditure for railways emergency stock from $1.60 million to $8.78 million (PCR, Appendix 3). The PCR did not comment on this. D. Water Resources 8.98 Shortfall The PCR states that financial constraints hampered the achievement of the targets set for embankments and protective works (para. 17), but IED notes that $8.98 million was nevertheless saved. E. Municipal Infrastructure 1.10 Exceeded Loan savings and fluctuations of exchange rates financed the increase in actual works over the targets (PCR, para. 18). Even with the increase, there was still a savings. IED = Independent Evaluation Department, para. = paragraph, PCR = project completion report In the absence of an implementation schedule at the time of appraisal, the PCR prepared a projected disbursement schedule (in Appendix 5) along with the actual disbursements during implementation. The PCR could have stated in the main text what is clear in Appendix 6: that all the components suffered about 14 months delay in bidding and awarding of contracts. This factor alone may largely explain why there are both savings and shortfalls. However, the PCR describes some reasons for the delays and related issues in para. 25. IED agrees with the PCR that timely action by the executing agencies (EAs), and especially Bangladesh Railway, could have minimized the delays and brought the project benefits to the affected people earlier (PCR, para. 25). (iv) Implementation Arrangements, Conditions and Covenants, related Technical Assistance, and Procurement and Consultant Performance. The PCR has reported that the implementation arrangements followed the proposals that were envisaged at appraisal, and assessed them as satisfactory and effective (para. 26). The PCR has assessed that the Government and the EAs generally complied with the covenants, and substantially met the reporting requirements (para. 28). The related TA 4 has been evaluated by the Technical Assistance Completion Report (TCR) as successful and the PCR also has the same evaluation. It would have been useful if the PCR had reflected on whether the follow up actions recommended in the TCR were implemented. The PCR has reported (para. 32) that the recruitment of the consultant was as envisaged and conformed to ADB s Guidelines on the Use of Consultants (2007, as amended from time to time). The PCR has found that The performance of the consultants under all five components was generally satisfactory (para. 36). It agrees with the TCR evaluation of the consultants performance. The PCR reported that the procurement of project goods and services financed by ADB was done according to ADB s Procurement Guidelines (2007, as amended from time to time). It described the problems, complaints and relevant inquiries in the procurement process by different EAs, but did not mention whether action was ever taken. IED notes that the available documents and back-to-office reports support the PCR and TCR evaluation. (v) Performance of the Borrower and Executing Agency. The PCR has rated the performance of the Borrower and the EAs satisfactory (para. 39), in spite of substantial delays in approving the award of contracts and release of government counterpart funds, which delayed the progress of work in early stages and ultimately caused some shortfalls in the anticipated outputs. IED agrees with this assessment, in view of the complexity of the Project, the quick establishment of the Project Steering Committee, and the committee s fulfillment of most of its obligations in monitoring project implementation. IED would have agreed with the highly satisfactory rating given by the PCR to the Local Government Engineering Department (LGED) (Parts A and E), Roads and Highways 4 ADB Technical Assistance Completion Report for the Early Warning Systems Study (attached to the PCR for the Emergency Flood Damage Rehabilitation Project). Manila (TA 4562-BAN, approved on 20 January 2005, for $250,000).

7 5 Department (Part B), and Bangladesh Water Development Board (BWDB) (Part D) if they had utilized more than 90% of their expenditures estimated at appraisal, and if there had been more information on the actions taken against staff and contractors involved in the corruption cases reported. Instead, IED rates them satisfactory. IED agrees with the PCR s partly satisfactory performance rating of Bangladesh Railway. (vi) Performance of the Asian Development Bank. The PCR has rated the overall performance of ADB satisfactory (para. 41). IED appreciates that the PCR noted the flexibility shown by ADB in authorizing advance procurement action, recommending simplified approval procedures for contract awards, and retroactive financing to facilitate faster implementation of the sub-projects. As mentioned, the PCR could have contained better reporting on actions taken with respect to corruption. D. Evaluation of Performance (PCR assessment and Validation) (i) Relevance. The PCR considers the Project highly relevant to ADB s present country operational strategy and the Government s latest development strategy, and finds it remains relevant (para. 42). IED agrees with the PCR rating. The IED rating applies to the project both before and after implementation. (ii) Effectiveness in Achieving Outcome. The PCR has rated the Project highly effective, as the completed subprojects in all components achieved their principal objective of restoring damaged infrastructure to pre-flood levels, thus enabling the country to return to the normal economic and social activities that had been disrupted by the floods. Nevertheless, IED notes that many of the appraisal targets were not achieved in component 1, the Project s largest component. IED noted that the achievements fell short of the appraisal targets by 56% for rehabilitation of upazila and union roads into bitumen-surfaced roads, 16% for bridges and/or culverts, 70% for tree planting, and 10% for replacement of flood shelters (para. 12). IED could not compare some of project outputs with the appraisal targets due to a lack of information, and the use of different units of measurement in the PCR. The PCR reports slow progress by the Roads and Highways Department with the bridge component, and problems with disbursement by Bangladesh Railways, but the PCR contains no quantitative information on the achievement of bridge and railway targets. The project completion review mission observed (para. 50) that in some urban areas; the drains were blocked by waste materials from local residents. Some of the works were also reported to be damaged during the 2007 flood. The previous BWDB project director informed the validation team that during the 2007 flood, two spurs in Sirajganj district were damaged, which were repaired using the government fund. The Mayor of Shajadpur observed that around 40% of the roads reconstructed after the 2004 flood were damaged again by the 2007 disaster. The project completion review mission observed damage to roads constructed by the 2005 project in Sirajganj. The validation team was also told that some damage occurred in LGED works, which were later repaired. More could have been achieved if the project had not lost 14 months at the outset. This validation exercise acknowledges the very good work done by the project, and the many achievements made in a short time. Without the project, the 2007 disaster would have caused far more damage; the effective repairs done by the Project saved many roads from being severely damaged in Based on the above, IED considers that a rating of effective is more justified for the project than a rating of highly effective. (iii) Efficiency in Achieving Outcome and Outputs. The PCR rated the Project highly efficient. IED looks at efficiency from two aspects efficiency of process and efficiency of investment. IED notes that the loan was closed as envisaged at appraisal (31 July 2007). However, in view of the long delays in signing the loan agreement signing and the delay in procurement and contract award, as well as cancellation of some contracts (PCR, para. 25), IED rates efficiency of process less satisfactory.

8 6 Regarding the efficiency of the investment, the Project did not undergo financial evaluation because it was not revenue-generating (para. 45). As the Project was of an emergency nature, no economic analysis was undertaken to justify the investments at appraisal. Nevertheless, economic efficiency as measured by the economic internal rate of return (EIRR), was assessed by the PCR for two sample road sections in Part B (a regional road and a district road). The calculated EIRRs were 39.9% for the regional road, and 23.9% for the district road (PCR, para. 47). IED concurs with these calculations and also assesses the EIRRs of the completed works as high. At appraisal, the EIRRs on past flood damage rehabilitation projects were in the range of 18% to 55% (PCR, para. 46). In view of the high EIRRs, and in spite of the lower process efficiency, IED therefore concurs with the PCR rating of highly efficient. (iv) Preliminary Assessment of Sustainability. The PCR has assessed the project likely sustainable. The PCR has explained the issue in paras , and explained the Government s financing plans in detail. IED noted that the Government immediately repaired the rehabilitated and repaired project works that were damaged during the 2007 flood. IED did not find any reports that conflict with the PCR statements, and validates the PCR rating. (v) Impact (both intended and unintended). The PCR reports that an initial environmental examination of the entire project area indicated that no significant adverse environmental impact was associated with the Project. As the subprojects already existed, no additional environmental impact assessment was considered necessary. No environmental problems were observed by the project completion mission except that in some urban areas (Part E), drains were blocked by waste materials from local residents. The project completion mission suggested that the EA undertake a public awareness campaign and discussions with local residents. The Project has had significant direct and indirect socioeconomic impact. All works were undertaken in existing alignments or original sites, and no land acquisition or resettlement was undertaken. Although the PCR has not made an overall assessment of the impacts, IED assesses the impacts of the project to be positive. E. Overall Assessment, Lessons, and Recommendations (Validation of PCR assessment) (i) Overall Assessment. The PCR has rated the Project highly successful. While IED sided with the PCR ratings on relevance, efficiency, and sustainability, IED considers that rating the project effective is more justified than the PCR rating of highly effective. The basis of the downgrading is that many of the appraisal targets were not achieved. Although 17.4% of the estimated budget for civil works at appraisal remained unutilized, the PCR did not reflect the Project s success in meeting the appraisal targets in some cases (e.g., drains were already blocked by waste materials in some urban areas, and some of the rehabilitation and repair works were damaged during the 2007 flood). The downgrading of the effectiveness rating from highly effective to effective lowers the weighted score of the Project to 2.5. This is below the 2.7 switching value between successful and highly successful. 5 (ii) Lessons. The PCR lessons flow well from the text and are fully endorsed by IED. IED has the following lessons to add: (a) Small contractors often lack financial as well as technical and project management capacity, which causes delay in mobilization and completion of the works. (b) Legal action against the responsible contractor(s) and agency staff is a necessary instrument to control corruption. (iii) Recommendations. The recommendations of the PCR flow well from the discussion in the main text, and are supported by IED. The PCR could have provided some lessons or recommendations on how to prevent risks of irregularities and corruption in emergency projects. The PCR should specifically address whether any action has been taken against those involved (contractors and staff) in corruption or failures. No project performance evaluation is recommended at the moment. 5 The overall assessment is based on four criteria: relevance, effectiveness, efficiency, and sustainability. A project or program is rated highly successful if the overall weighted average is greater than or equal to 2.7; successful, if greater than or equal to 0.8 and less than 1.6; or unsuccessful, if less than 0.8.

9 7 F. Monitoring and Evaluation Design, Implementation, and Utilization (PCR assessment and Validation) The RRP discusses project performance monitoring and evaluation in para. 57; and, in para. 68 (xiii), under Specific Assurances: The Government shall ensure that each project EA shall conduct annual and random performance audits during project implementation to determine the degree to which project funds have been effectively and efficiently utilized to implement the project and achieve its objectives, outputs and performance indicators. The PCR mentions in status of compliance with major loan covenants that it has been complied with and that the monitoring has been done through periodic ministerial review meetings. As this is an important issue for IED evaluation, the PCR could have focused more on this aspect. G. Other (e.g., Safeguards, including governance and anticorruption; Fiduciary aspects; Government assessment of the Project, as applicable) (PCR assessment and Validation) The PCR reported that although no land acquisition or involuntary resettlement was anticipated, a resettlement framework was prepared at appraisal as required under ADB s safeguard policy for sector loans. No fiduciary case has been reported. The PCR has mentioned corruption cases and relevant inquiries in the procurement process by different EAs. It has also reported the investigation by ADB s Integrity Division of the Office of the Auditor General, which strongly indicated the occurrence of coercive and restrictive practices by some contractors to bar others from submitting bids. IED considers such incidents as a serious problem in Bangladesh, and ADB projects should have an effective policy to deal with such cases. Independent Evaluation Department H. Ratings Project Completion Report Review Relevance: Highly relevant Highly relevant Effectiveness in Achieving Outcome: Efficiency in Achieving Outcome and Outputs: Preliminary Assessment of Sustainability: Borrower and Executing Agency: Performance of Asian Development Bank: Impact: Reason for Disagreement/Comments Highly effective Effective Some appraisal targets were not met, and some achievements were not reported in quantitative form Highly efficient Likely to be sustainable Satisfactory Satisfactory No overall assessment appears positive. Highly efficient Likely to be sustainable Satisfactory Satisfactory Positive Overall Assessment: Highly successful Successful Downgrading in effectiveness rating leads to a lower rating 6 Satisfactory Quality of Project Completion Report: 6 Based on IED's ratings for (i) relevance: highly relevant (0.6); (ii) effectiveness: effective (0.6); (iii) efficiency: highly efficient (0.9); and (iv) sustainability: likely (0.4), the Project is rated successful (2.5). Had the Project retained its

10 8 I. Comments on PCR Quality The PCR is clear in most cases and is constructive. The issues and the evidence are generally well presented. However, the PCR missed a few questions that have been highlighted by this validation. Areas that could have been improved include reporting on the appraisal target achievements, the description of the outputs, the presentation of Appendix 1, and addressing the corruption issue. The PCR could also have focused on the role of the beneficiaries at various stages of the Project instead of mentioning this only in the status of compliance of major covenants, as this is an important area of focus for ADB. The lessons and recommendations are, nevertheless, broadly consistent with the narrative. The PCR is generally consistent with the ADB Project Administration Instructions on PCRs 7 and relevant IED guidelines for PCRs using the project performance evaluation reports methodology. Overall, the quality of the PCR is satisfactory. J. Recommendation for IED follow up There is no recommendation for IED follow up. K. Data Sources for Validation RRP; PCRs of both ADB and the Government (BWDB and the LGED); back-to-office reports and Aide Memoires of missions; some contacts made with stakeholders in Bangladesh. highly effective rating (0.9), the Project would have been rated highly successful (2.8). The switching value for a highly successful rating is ADB Project Administration Instructions. Project Completion Report. PAI No. 6.07A. Manila.

11 REGIONAL DEPARTMENT S RESPONSE TO THE PROJECT COMPLETION REPORT VALIDATION REPORT On 29 March 2009, the Independent Evaluation Department 1 (IED) received the following comments from the Bangladesh Resident Mission (BRM), South Asia Department: We would like to inform you that our comments still prevail from our perspective. However, we accept your downgrading based on the justification that you have provided. BRM's earlier comments, which were received on 15 February 2009, read as follows: "As there were two consecutive floods, one in 2004 and the other one in 2007, which were both severe in nature, some extra damages occurred due to this in 2007, even repairing after 2004 flood. However, had there not been Emergency Flood Damage Rehabilitation Project after 2004 flood, the devastations and damages after 2007 flood could have been much more intense. This point has not been highlighted adequately in the project completion report (PCR). However, IED's Validation Team should not have taken these two events in conjunction. At the same time, it recorded 14 months delay, whereas actually there was only 3 months delay in bidding and contract awards from June to September As it was monsoon period, the delay did not affect much the physical implementation. IED's Validation Team noted that the achievements for rehabilitation of upazila and union roads into bitumen-surfaced roads, bridges/culverts, tree plantation, and replacement of flood shelters fell short of the appraisal targets by 56%, 16%, 70% and 10% respectively; which actually were not the core activities of the Project, and as such did not affect the overall progress. On this basis, the highly effective rating could still have been restored, resulting in highly successful rating, as was rated in the PCR. Based on these facts, BRM still believes that the PCR validation on the "Effectiveness in Achieving Outcomes" should rather be highly effective, which may be reflected in final version of the PCR Validation Document." IED addressed some of BRM's comments in this version of the validation report for the PCR. However, the overall project rating was not upgraded as suggested by BRM, since IED was not convinced that this was justified. 1 The Independent Evaluation Department was named the Operations Evaluation Department, or OED, until December 2008.