L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) TF Dec ,830, Original Commitment ,000,000.

Transcription

1 Public Disclosure Authorized Independent Evaluation Group (IEG) 1. Project Data Report Number : ICRR Public Disclosure Authorized Project ID P Country China Project Name CN-GEF-IF Huai River Basin Marine Pollut Practice Area(Lead) Environment & Natural Resources L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) TF Dec ,830, Bank Approval Date 23-Feb-2012 Closing Date (Actual) 31-Dec-2015 IBRD/IDA (USD) Grants (USD) Public Disclosure Authorized Original Commitment ,000, Revised Commitment ,944, Actual ,944, Sector(s) Wastewater Treatment and Disposal(45%):Irrigation and drainage(26%):agricultural extension and research(14%):solid waste management(9%):public administration- Water, sanitation and flood protection(6%) Theme(s) Pollution management and environmental health(42%):environmental policies and institutions(28%):water resource management(20%):rural services and infrastructure(10%) Public Disclosure Authorized Prepared by Reviewed by ICR Review Coordinator Group John Redwood John R. Eriksson Christopher David Nelson IEGSD (Unit 4) 2. Project Objectives and Components a. Objectives Project Appraisal Document (PAD) and Financial Agreement Development Objective: To demonstrate innovative and cost-effective water pollution control practices in the Guangli River catchment of Dongying Municipality, contributing to pollution reduction in the Bohai Sea.

2 b. Were the project objectives/key associated outcome targets revised during implementation? No c. Components 1. Wetland Construction and Sluice Gate Operation Optimization (Appraisal Cost: US$ million; Actual Cost: US$ million), comprising the following subcomponents: (i) construction of the wetlands at Dongbalu consisting of free-surface flow wetlands, an ecological retention pool, an entrances gate, a gated overflow weir, and a pumping station together with provision of related equipment; (ii) upgrading the automatic gate control system covering three sluice gates on the Guangli River and the gates at the entrance and exit of the Dongbalu wetlands; and (iii) provision of cash transfers to affected persons of the wetlands construction. 2. Agricultural Pollution Control and Rural Waste Management (Appraisal Cost: US$ million; Actual Cost: US$ million) consisting of: (i) wastewater, human and livestock waste collection and treatment in participating villages; (ii) (a) introduction of agricultural pollution reducing technologies and management practices in participating villages through comprehensive and balanced fertilizer applications, provision and use of insect luring lamps, and construction of eco-trenches and buffer strips in crop fields and (b) monitoring of the results of implementation of these technologies and practices; and (iii) establishment, equipping and operation of Farmer Environmental Protection Associations (FEPAs) in participating villages. 3. Capacity Building and Policy Studies (Appraisal Cost: US$ million; Actual Cost: US$ 355,000) comprising: (i) establishment and operation of an environmental protection education and training center to be located in Dongying Municipality for training and dissemination of technologies and good practices in environmental protection, nutrient management, and pollution reduction; (ii) capacity building activities to provide technical and project management training for staff involved in project implementation and management; and (iii) (a) evaluation study of the effectiveness of constructed wetlands in the treatment of polluted water based on the analysis of project monitoring results; (b) development of an agricultural pollution reduction and rural waste management strategy and plan for the Guangli River Watershed in Dongying Municipality, including an evaluation study of the related project interventions for this purpose; and (c) development of a Huai River Basin-wide replication strategy for cost-effective water pollution control, including dissemination and training and workshops as required for this purpose. 4. Project Management and Implementation Support (Appraisal Cost: US$ million; Actual Cost: US$ 826,000) consisting of: (i) provision of technical assistance for the review of technical designs and tendering documents, construction quality of project facilities, and for project reporting; (ii) project monitoring and evaluation; and (iii) support for project management by the Project Management Office (PMO) and Project Implementation Units (PIUs). d. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost: Actual project costs were 119% of those estimated at appraisal -- US$ million compared with US$ million. However, this increase was entirely due to a 53% increase in the costs of Component 1 (Wetland Construction and Sluice Gate Operation Optimization) over the appraisal figure (US$ million as opposed to US$ million), while actual expenditures for each of the other three components was substantially lower than originally anticipated, particularly those for capacity building, implementation support, sluice gate operation optimization, monitoring and evaluation, agricultural pollution control, and the education and training center -- all of which consumed less than 20% of the appraisal allocations -- but also for the establishment and operation of FEPAs and policy studies. The main overruns were for the constructed wetlands at Dongbalu, 75% more than anticipated, and project management, 42% above the appraisal estimate. Financing: Only 78.9% (US$ million) of the US$ 5 million Global Environmental Facility (GEF) was disbursed,. Borrower Contribution: This was 25% higher than originally anticipated -- US$ million compared with US$ 32,828 million. Dates: The project closed as scheduled on December 31, Relevance of Objectives & Design a. Relevance of Objectives The project objectives continue to be highly relevant both for the Chinese Government and the Bank. China's 13th Five Year Plan (for ) includes among its highlights implementation of "the strictest management of water resources." together with other strengthened environmental measures. The current (FY ) Bank Group Country Partnership Strategy, in turn, identified persisting water pollution as a significant part of the country's "environmental deficit" (para. 15, pg. 5) and put forward "supporting greener growth," including "demonstrating pollution management," (Figure 1, pg. 13) as one of its three "strategic themes" and main areas of engagement. In this context, specific attention was to be given to non-point source water pollution (Table 2, pg. 160), among other sources, including with GEF

3 funding. In addition, the CPS pledged further Bank support for "demonstrating innovative ways to manage wetlands and lakes, including the increased use of wetlands to filter urban and industrial run-off before excess nutrients cause eutrophication in lakes, rivers, and bays (including the Bohai and South China Seas)" (para. 25, pg. 23). However, in retrospect, they were too ambitious with respect to the project's ability to contribute to pollution reduction in the Bohai Sea (see section 4 below). Thus, the overall relevance of objectives rating is Substantial. Rating Substantial b. Relevance of Design The project suffered from a number of design shortcomings in addition to the overly ambitious objective with respect to its ability to contribute to pollution reduction in the Bohai Sea. These included an overly short implementation period (three and a half years), a lack of full alignment between project objectives and key performance indicators (KPIs) in the Results Framework, and the oversight or underestimation of certain significant implementation risks, including adverse shifts in local government priorities and increased urbanization in parts of the project intervention area, as well as inadequate demand estimates with regard to proposed rural wastewater treatment facilities and unrealistic assumptions regarding the willingness of farmers to participate in the purchase of balanced fertilizer (to increase its effectiveness) and other project activities. In hindsight, the project also attempted to do too much by combining wetland construction with rural pollution management to be undertaken by two distinct executing agencies. Rating Modest 4. Achievement of Objectives (Efficacy) PHREVISEDTBL Objective 1 Objective To demonstrate innovative and cost-effective water pollution control practices in the Guangli River catchment of Dongying Municipality. Rating: Modest. Rationale Outputs: Wetland area that possesses water treatment capacity constructed vs an appraisal target of 2,485 mu, with one mu equivalent to 15% of a hectare); according to the ICR (pg. 26), the actual effectiveness of this area was even lower because the completed vegetation coverage was only 64% of the water covered area (i..e, 561 mu covered by emerged aquatic plants). Proportion of farmers adopting integrated and balanced fertilizer application technology in participating villages -- 55% participation achieved; technology applied on 1,852 ha vs. appraisal target of 2,278 hectares of land; fewer farmers participated due to decreasing land area and increased migration as the result of urbanization. Number of Farmers Environmental Protection Associations (FEPAs) established as opposed to a target of 22 (45%); however, most of those established are reportedly not operational. Number of people trained -- largely achieved (4,464 against an appraisal target of 4,500). Study of the impact of pollution reduction through constructed wetlands in the Guangli River Watershed -- partially achieved; the study has been completed, but was based on simulated rather than actual monitoring data because the wetland had not become operational by the time the project closed. Evaluation study and management planning for agricultural and rural pollution reduction in the Guangli River catchment -- partially achieved;

4 report completed but based on simulated data because most of the planned rural wastewater treatment facilities were either not completed or put into operation. Development and dissemination of Huai River basin-wide replication strategy -- partially achieved; report completed but largely based on simulated data because wetland and most rural wastewater facilities had not been put into operation. Outcomes: Annual volume of treated wastewater at wetlands -- not achieved, as the wetlands were not yet operational at the time the project closed. Rural wastewater pollution load reduction in participating villages -- not achieved, as only one rural wastewater facility is in intermittent operation. Livestock waste pollution reduction in participating villages -- not achieved mainly due to low demand from farmers. Agricultural pollution/nutrient load reduction in participating villages -- targets were overachieved because, according to the ICR (pg. 8), they were set "too low". Even on the assumption that they were "innovative," there was no specific indicator to assess the extent to which the water pollution control practices implemented under the project were "cost-effective." Rating Modest PHREVISEDTBL Objective 2 Objective To contribute to pollution reduction in the Bohai Sea. Rating: Negligible. Rationale Outputs: There were no specific outputs (other than those) listed in connection with Objective 1 with regard to this objective. Outcomes: Project-induced reduction in pollution/nutrient loads entering the Bohai Sea from the Guangli River Watershed -- not achieved; while the wetland and rural wastewater treatment facilities actually implemented by the project, which were substantially less than expected at appraisal, had reportedly been completed, they were not yet operational at the time the ICR was issued. As a result none of the target water quality indicators had been met at the time the project closed. Reduction in pollutants/nutrients through constructed wetlands at Donbalu -- not achieved; again the (smaller than expected) wetlands have been constructed, but are not operational, so actual project outcomes in this regard were zero at the time the project closed. Rating Negligible 5. Efficiency

5 While no cost-effectiveness analysis was carried out at appraisal, one of the project's explicit objectives was to demonstrate "cost-effective water pollution control practices." Project appraisal did include a cost-benefit analysis, which suggested a positive rate of return, but the ICR (pg. 28) correctly states that this approach was "problematic as it is practically impossible to attribute the land value increase (accounting for 80% of the quantifiable benefits as an assumption at appraisal) to the construction of the wetland." It also indicates (pg. 29) that, while the wetland treatment area constructed under the project was considerably smaller than expected at appraisal (see section 4, Objective 1 above), its actual cost was 54% higher than the original estimate, indicating low cost-effectiveness. This was due in part to the need to redesign the wetland which was already under construction at the time the project started but was smaller than and differed considerably from the approved design. This also led to substantial delays in project implementation which adversely affected all of its intended outcomes. The rural wastewater treatment plants also appear to demonstrate low cost-effectiveness because their design excluded toilet water and fewer farmers remain in the project villages due to urbanization. In addition, there are insufficient data to support the assumption that farmers have increased their productivity and/or reduced their input costs as a result of the project. Efficiency Rating Negligible a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) Appraisal ICR Estimate Not Applicable 0 Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome The relevance of project objectives is Substantial, but the relevance of project design is Modest as is the achievement of one project objective while Efficacy with respect to the second objectives and Efficiency are rated Negligible. Thus, the overall project outcome rating is Unsatisfactory. a. Outcome Rating Unsatisfactory 7. Rationale for Risk to Development Outcome Rating As the wetland was not yet operational at the time the project closed, it is uncertain how effectively it will function in practice. In a discussion between the ICR team and IEG, it was indicated that the pumping station built under the project will not become operational before December 2016 as the government is waiting to synchronize its start-up with that of both the GEF-financed wetland and another local wetland that will not be completed before that time. Its operation and maintenance (O&M) costs are likewise presently unknown, but could prove to be substantial, while local farmers appear to lack sufficient incentive to pay for the O&M of the wastewater treatment facilities, which, in addition, only address part of the problem. The same lack of incentives seems likely to doom the effectiveness of the Farmers Environmental Protection Associations (FEPA), many of which had already ceased to function prior to project closure, while a number of the other anti-pollution measures (e.g., lower application of fertilizers, construction of buffer strips and "eco-trenches") also appear to be of doubtful sustainability given lack of farmer commitment to their use. Farmer refusal to adopt the no fertilizer application approach, according to the ICR team, was due to the fact that they believed this would have an adverse impact on their productivity and because associated training to be provided by the project was rushed and

6 came very late due to other project implementation delays. a. Risk to Development Outcome Rating High 8. Assessment of Bank Performance a. Quality-at-Entry Despite Bank efforts to ensure adequate project preparation and an innovative design with respect to no-point source water pollution control, there were several significant quality-at-entry shortcomings as outlined in section 3b (relevance of project design above). These included overly ambitious objectives and overly complex project design together with inadequate oversight of critical risks such as increasing urbanization and shifting priorities on the part of the municipal government, which undermined local commitment and led to the need for wetland redesign, substantial implementation delays, and deferred and uncertain project outcomes. Quality-at-Entry Rating Moderately Unsatisfactory b. Quality of supervision The Bank team provided strong technical support for efforts to redesign the wetland (which had been altered by local authorities without Bank approval) so that it would better contribute to the project's original intended pollution reduction objectives, and sought to accelerate project implementation once a more appropriate new design was finally agreed. It also reportedly provided valuable assistance with respect to procurement, financial management, and safeguards compliance. However partly as a result, insufficient attention was given during supervision to the other project components, such as wastewater management, and to persisting institutional and coordination problems that were also adversely affecting project performance. According to the ICR (pg. 31), possible opportunity to restructure and extend the project, which could have altered the outcome in a positive way and reduced the risk to development effectiveness, appears to have been missed, and ISR ratings for both implementation progress and progress toward achievement of development objectives should have been downgraded much sooner given the initial unauthorized change in wetland design. Quality of Supervision Rating Moderately Unsatisfactory Overall Bank Performance Rating Moderately Unsatisfactory 9. Assessment of Borrower Performance a. Government Performance During the early stages of project implementation, the Dongying municipal government's commitment to the original project objectives and wetland design lapsed and an alternative design not consistent with the project's pollution management objectives was adopted and started to be implemented. The need to redesign the wetland led to significant implementation delays, as well as a reduction in the size and pollution control capacity of the wetland, thereby adversely affecting the project's outcome and development effectiveness. In the ICR's view (pg. 32), this was the "main reason" for the project's inability to achieve its development objectives. IEG agrees with this assessment. Occasional delays in the provision of counterpart funding were also reported. Government Performance Rating Unsatisfactory

7 b. Implementing Agency Performance The provincial Project Management Office (PMO) was unable to properly coordinate implementation in part because it was located in a different city than the project site but also because of the cross-sectoral nature of the operation, which also limited the effectiveness of the municipal PMO. The Dongying Municipal Urban Construction Bureau, which was responsible for wetland construction, possessed strong technical capacity, but failed to seek Bank approval for the initial change in wetland design and proceeded to start its implementation without obtaining the Bank's approval. But this decision later had to be reversed, leading to the significant delays that negatively affected the project's outcome. The Dongying Municipal Agricultural Bureau, which was responsible for rural waste management and agricultural pollution control together with establishment of the Farmers Environmental Protection Associations (FEPAs) and farmer training, also performed poorly likewise contributing to implementation delays and the reduced scope of some project activities, while the training activities were not assessed so their effectiveness is unknown. Implementing Agency Performance Rating Unsatisfactory Overall Borrower Performance Rating Unsatisfactory 10. M&E Design, Implementation, & Utilization a. M&E Design The M&E design was only partially satisfactory as no indicators were included in the Results Framework to measure the key objective regarding the extent to which the project's pollution control interventions were, in fact, innovative and cost-effective. The focus instead was entirely on pollution reduction due to project interventions. b. M&E Implementation M&E activities started late due to the delays resulting from the need to redesign the wetland after project implementation started. Water quality monitoring for the constructed wetland had still not begun at the time the project closed because it had not yet become operational, and rural wastewater treatment indicators were similarly not available at the time of project closing for the same reason. Water quality monitoring, more generally, which is the responsibility of the Dongying Environmental Protection Bureau, reportedly began in September 2015 and is expected to continue to June It is not clear from the ICR what will happen after that. Monitoring of safeguard measures, including resettlement, was carried out independently by the Shandong Construction Development Research Institute and was deemed satisfactory. c. M&E Utilization As a result of the delays and absence of actual monitoring data described above, few data were available to guide project resource allocation and decision making. Several studies, which were intended to rely on project water quality monitoring results, had to use simulated data based on similar operations elsewhere instead and, thus, were only partly satisfactory (see section 3 above), as their accuracy and applicability to the specific environmental conditions in the project area is uncertain. M&E Quality Rating Negligible 11. Other Issues a. Safeguards

8 The project was classified as Category B and triggered the Environmental Assessment (OP 4.01), Pest Management (OP 4.09), and Involuntary Resettlement (OP 4.12) safeguard policies. An Environmental Management Plan (EMP) was developed to mitigate the relatively minor and localized impacts associated with project construction activities, as was a Pest Management Plan (PMP) to help farmers in the participating villages reduce the use of agro-chemicals. Similarly, a Resettlement Action Plan (RAP) was prepared establishing necessary compensation and relocation measures, as the land use change brought about by the project required the displacement of a number of small enterprises and other facilities. Cash transfers under this Plan were reportedly completed in 2012 involving a total cost of US$ 7.4 million, which was fully financed out of counterpart resources. b. Fiduciary Compliance Procurement was undertaken by the municipal Project Management Office with support from the provincial PMO and the Bank but was delayed as a result of the overall implementation delays caused primarily by the need to redesign the wetland and lack of counterpart funding at times. Financial Management was managed by the Shandong Provincial Finance Bureau (SPFB), which was also supported by training provided by the Bank. Implementation of the withdrawal application and reimbursement processes were nonetheless slow due to the Project Management Unit's lack of familiarity with Bank requirements and poor communication between it and the Municipal Finance Bureau. However, financial reports were submitted in a timely manner and there were no qualified audited financial statements. c. Unintended impacts (Positive or Negative) --- d. Other The project reportedly raised awareness among participating villagers with respect to the importance of environmental protection, while also improving local sanitation conditions due to wastewater treatment and animal waste management improvements. It also reportedly benefited local institutions by exposing them to innovative water pollution control practices, including international experience in wetland design and more sustainable farming practices. 12. Ratings Ratings ICR IEG Outcome Unsatisfactory Unsatisfactory --- Risk to Development Outcome High High --- Bank Performance Moderately Unsatisfactory Moderately Unsatisfactory --- Borrower Performance Unsatisfactory Unsatisfactory --- Quality of ICR High --- Reason for Disagreements/Comment Note When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons Among the principal lessons highlighted by the ICR (pp ) are the by now generally well-understood needs for strong and persisting

9 government commitment to project objectives and design and for local associations, such as the FEPAs, to be based on firm beneficiary ownership. It also pointed to the needs for appropriate institutional arrangements -- in the present case to permit adequate cross-sectoral coordination -- an adequate implementation period, and for the restructuring of poorly implementing and overly complex projects in relation to borrower institutional capacity. To these lessons should be added the needs: (i) to ensure that key project outcome indicators are fully and clearly aligned with project development objectives; and (ii) for the economic analysis methodology applied at appraisal to be appropriate, in the present case costeffectiveness rather than cost-benefit analysis. 14. Assessment Recommended? No 15. Comments on Quality of ICR The ICR is complete, of good quality, and very forthcoming with regard to the significant implementation problems faced, the associated disappointing results, and the project's highly uncertain sustainability. Its ratings are appropriate and its lessons of general applicability to other GEF and Bank-financed operations in China and elsewhere. In addition, it provides a useful critique of the project's quality-at-entry, including the cumbersome project institutional arrangements, the inappropriateness of the initial economic analysis, and failure to properly align its key performance indicators with its development objectives, which are also important lessons. a. Quality of ICR Rating High