Document of The World Bank IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA-33690) ON A CREDIT

Transcription

1 Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Document of The World Bank Report No: ICR IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA-33690) Sustainable Development Department Central Asia Country Unit Europe and Central Asia Region ON A CREDIT IN THE AMOUNT OF SDR 14.9 MILLION (US$20.0 MILLION EQUIVALENT) TO THE KYRGYZ REPUBLIC FOR AN ON-FARM IRRIGATION PROJECT November 26, 2008

2 CURRENCY EQUIVALENTS (Exchange Rate Effective 09/01/2008) Currency Unit = KGS KGS 1.00 = US$0.028 US$1.00 = KGS 36.0 SDR 1.00 = US$0.64 FISCAL YEAR JANUARY 1 DECEMBER 31 ABBREVIATIONS AND ACRONYMS APL ADB CAS CDS CSU DWR ERR FAO I&D IDA IRP ISF MOAWRPI M&E MOF MOM NIRSAP O&M OIP ORT OSU PAD PDO PIU RSU SDR SU USAID WB WMIP WUA Adaptable Program Lending Asian Development Bank Country Assistance Strategy Country Development Strategy Central Support Unit Department of Water Resources Economic Rate of Return Food and Agriculture Organization Irrigation and Drainage International Development Association Irrigation Rehabilitation Project Irrigation Service Fee Ministry of Agriculture, Water Resources, and Processing Industry Monitoring and Evaluation Ministry of Finance Management, Operation and Maintenance National Irrigation Rehabilitation Strategy and Action Plan Operation and Maintenance On-Farm Irrigation Project Oblast Rehabilitation Team Oblast Support Unit Project Appraisal Document Project Development Objective Project Implementation Unit Raion Support Unit Special Drawing Rights Support Unit US Agency for International Development World Bank Water Management Improvement Project Water Users Association Vice President: Shigeo Katsu Country Director: Annette Dixon Sector Manager (acting): Holger Kray Project Team Leader: Joop Stoutjesdijk ICR Team Leader: Benoist Veillerette (FAO)

3 KYRGYZ REPUBLIC On-Farm Irrigation Project CONTENTS Data Sheet A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring I. Disbursement Graph 1. Project Context, Development Objectives and Design Key Factors Affecting Implementation and Outcomes Assessment of Outcomes Assessment of Risk to Development Outcome Assessment of Bank and Borrower Performance Lessons Learned Comments on Issues Raised by Borrower/Implementing Agencies/Partners Annex 1. Project Costs and Financing Annex 2. Outputs by Component Annex 3. Economic and Financial Analysis Annex 4. Bank Lending and Implementation Support/Supervision Processes Annex 5. Beneficiary Survey Results Annex 6. Stakeholder Workshop Report and Results Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders Annex 9. List of Supporting Documents MAP

4 A. Basic Information Country: Kyrgyz Republic Project Name: On-Farm Irrigation Project Project ID: P L/C/TF Number(s): IDA ICR Date: 11/27/2008 ICR Type: Core ICR Lending Instrument: SIL Borrower: KYRGYZ REPUBLIC Original Total Commitment: XDR 14.9M Disbursed Amount: XDR 14.9M Environmental Category: B Implementing Agencies: Department of Water Resources Cofinanciers and Other External Partners: B. Key Dates Process Date Process Original Date Revised / Actual Date(s) Concept Review: 01/24/2002 Effectiveness: 04/08/ /08/2003 Appraisal: 03/25/2002 Restructuring(s): Approval: 05/30/2002 Mid-term Review: 05/03/2004 Closing: 03/31/ /31/2008 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Risk to Development Outcome: Bank Performance: Borrower Performance: Satisfactory Substantial Satisfactory Satisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Quality at Entry: Satisfactory Government: Satisfactory Quality of Supervision: Satisfactory Implementing Agency/Agencies: Satisfactory Overall Bank Performance: Satisfactory Overall Borrower Performance: C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Indicators Performance (if any) Potential Problem Project No at any time (Yes/No): Quality at Entry (QEA): Satisfactory Satisfactory Rating i

5 Problem Project at any time (Yes/No): DO rating before Closing/Inactive status: Yes Satisfactory Quality of Supervision (QSA): Unsatisfactory D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Central government administration Irrigation and drainage Theme Code (Primary/Secondary) Other rural development Secondary Secondary Rural services and infrastructure Primary Primary E. Bank Staff Positions At ICR At Approval Vice President: Shigeo Katsu Johannes F. Linn Country Director: Annette Dixon Kiyoshi Kodera Sector Manager: Holger A. Kray Joseph R. Goldberg Project Team Leader: Joop Stoutjesdijk Joop Stoutjesdijk ICR Team Leader: ICR Primary Author: Joop Stoutjesdijk Benoist Veillerette F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The development objective of the On-farm Irrigation Project is to achieve increased crop production through reliable and sustainable water distribution on about 160,000 ha of irrigated land across the country's seven oblasts. Revised Project Development Objectives (as approved by original approving authority) ii

6 (a) PDO Indicator(s) Indicator Indicator 1 : Value quantitative or Qualitative) Baseline Value Original Target Values (from approval documents) Formally Revised Target Values Actual Value Achieved at Completion or Target Years Increase crop yields in the project farms by ten percent within two years after rehabilitation works have been completed. No conclusive evidence as yet, as collected data not Yields of crops depressed 10% yield increase sufficiently reliable by at least 10 percent, two years after to fully assess because of inadequate and completion of changes in crop untimely water rehabilitation yields and distribution within farms. works. insufficient time between completion of rehabilitation and end-of-project. Date achieved 12/31/ /31/ /31/2008 A comparison between groups that received different assistance showed that Comments yields tend to be better (almost 12% on average) in WUAs that received (incl. % assistance for training and rehabilitation, compared to WUAs that did not receive achievement) any assistance. On-farm irrigation systems properly operated and maintained after completion of Indicator 2 : rehabilitation work Value quantitative or Qualitative) No formal operational regime of systems and negligible maintenance. WUAs working towards sustainable O&M (as per annual O&M plan). Operation of systems well organized. ISF increasing and substantial in-kind maintenance. More than 75% of farmers satisfied with O&M. Date achieved 12/31/ /31/ /31/2008 Encouraging signs found in relation to system O&M in rehabilitation schemes. Comments Water management staff, processes and procedures in place and functioning; (incl. % well established institutional base. Equity of water distribution found to be achievement) satisfactory. Distribution within project farms of adequate irrigation water in a timely manner, Indicator 3 : in line with irrigation water requirements. Value quantitative or Qualitative) Distribution Distribution efficiency efficiency depressed by 10 percent, improved, with 75 and water distribution not percent of farmers in line with irrigation receiving required water requirements. water. Significant improvements for rehabilitated WUAs (121,000 ha) in water users' assessment of the reliability, iii

7 adequacy, timeliness and equity of water delivery. Date achieved 12/31/ /31/ /31/2008 Comments (incl. % achievement) Difficult so far to measure actual efficiency percentages. Majority of farmers pleased with WUA performance and the improvement in water distribution (see Annex 5). (b) Intermediate Outcome Indicator(s) Indicator Indicator 1 : Value (quantitative or Qualitative) Baseline Value Original Target Values (from approval documents) Formally Revised Target Values Actual Value Achieved at Completion or Target Years Number of O&M plans developed in detail, fully costed and implemented each year after completion of rehabilitation works. 375 WUAs have achieved milestone All rehabilitated 3, well in excess of Zero. WUAs (about 80). the target set, i.e. O&M plans have been prepared. Date achieved 12/31/ /31/ /31/2008 Comments Milestone 3 -- O&M plan prepared taking into account ISF requirements to cover (incl. % O&M. Not all plans are related to optimum O&M, but trend is towards this. achievement) At least 160 WUAs established and legally registered, training program started Indicator 2 : that will lead to sustainable WUAs. Value (quantitative or Qualitative) Zero WUAs established according to the new WUA law. A total of 160 WUAs established and registered and 20,000 individuals have participated in awareness seminars and training courses. 454 WUAs have been established and legally registered, over 50,000 individuals have participated in awareness seminars and training courses. Date achieved 12/31/ /31/ /31/2008 Comments (incl. % achievement) Achievement much higher than expected. WUA support without rehabilitation is also showing positive impact, which shows the importance of WUAs for O&M, even without having a fully operational irrigation system in place. Indicator 3 : Collection rates at least 90 percent of estimated tariff within two years after completion of rehabilitated works. Value (quantitative or Qualitative) Very low collection rates, At least 90% on with nothing collected for rehabilitated use by WUAs. Whatever WUAs 2 years was collected was for after rehabilitation DWR. (approx. 80) Average ISR collection ratio for rehabilitated WUAs is 73% (16 WUAs > 90%, 22 WUAs > iv

8 80%). Date achieved 12/31/ /31/ /31/2008 Comments (incl. % achievement) Rehabilitation works have been completed over 2 years ago on 17 WUAs, which have among the highest collection rates. G. Ratings of Project Performance in ISRs No. Date ISR Archived DO IP Actual Disbursements (USD millions) 1 06/29/2000 Satisfactory Satisfactory /27/2000 Satisfactory Satisfactory /26/2001 Satisfactory Satisfactory /30/2001 Satisfactory Satisfactory /16/2002 Satisfactory Satisfactory /16/2002 Satisfactory Satisfactory /21/2003 Satisfactory Satisfactory /16/2003 Satisfactory Satisfactory /29/2003 Satisfactory Unsatisfactory /16/2004 Satisfactory Unsatisfactory /16/2004 Unsatisfactory Unsatisfactory /18/2005 Moderately Moderately Unsatisfactory Unsatisfactory /27/2005 Satisfactory Satisfactory /21/2005 Satisfactory Satisfactory /19/2006 Satisfactory Satisfactory /22/2006 Satisfactory Satisfactory /23/2007 Satisfactory Satisfactory /04/2007 Satisfactory Satisfactory /29/2008 Satisfactory Satisfactory H. Restructuring (if any) Not Applicable v

9 I. Disbursement Profile vi

10 1. Project Context, Development Objectives and Design 1.1 Context at Appraisal The agricultural sector in the Kyrgyz Republic went through a sharp recession during the 1990s as a result of major changes in the macro-economic environment, including changes in terms of trade and large reductions of subsidies, as well as the rapid deterioration of the former production system based on collective and state farms. Considering that agriculture was (and still is) the mainstay of the Kyrgyz economy, the government placed high priority to the agriculture sector in its strategy for economic recovery. The World Bank (WB) Country Assistance Strategy (CAS) prepared in 1998 was also in line with this priority, considering the attainment of long-term sustainable growth in agriculture as one of the pillars to solve many of the social and economic issues in the rural areas. One of the major reforms during the second half of the 1990s was a comprehensive and largely equitable land distribution, resulting in multiple small individual farms. These newly established private farms faced a number of issues, including lack of access to inputs, markets, credit, machinery and technical services. The transition process during the 1990s led to a vacuum in the management, operation and maintenance (MOM) of on-farm systems and the rapid deterioration of irrigation and drainage (I&D) infrastructure covering about 1.07 million ha, equivalent to 80 percent of arable land in the country. 1 Irrigation is a critical input into agricultural production and government realized that it could not allow I&D systems to collapse. In 1995, government introduced an irrigation service fees (ISF) to charge farmers for water received from the Department of Water Resources (DWR). The ISF was set by Parliament at a rate of KGS 15 per 1,000 m 3, but was doubled in A 1997 Government Resolution allowed the legal establishment of Water Users Associations (WUA) and set the basis for the transfer of on-farm irrigation infrastructure to WUAs. The On-farm Irrigation Project (OIP) was designed to assist government with the reversal of the deterioration of on-farm I&D infrastructure and the development of WUAs. 1.2 Original Project Development Objectives (PDO) and Key Indicators (as approved) The development objective of the On-farm Irrigation Project was to achieve increased crop production through reliable and sustainable water distribution on about 160,000 ha of irrigated land across the country's seven oblasts. The Project Appraisal Document (PAD) set three main impact indicators: percentage of operation and maintenance (O&M) needs of on-farm irrigation systems covered; distribution of adequate irrigation water in a timely manner, in line with crop water requirements; and increase in crop yields by 10 percent. 1 On-farm systems refer to the irrigation and drainage infrastructure that used to be part of the state or collective farms. 1

11 In addition, three output indicators were provided: number of well-functioning WUAs, with users representation and participation; number of O&M plans developed, fully costed, and implemented; and level and collection rate of irrigation services fees. 1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification There was no formal revision of the PDO, but in late 2006 it became clear that the expected project command area could not be achieved for various reasons as explained in Section 1.7. The total command area covered was about 122,000 ha. The PDO was formulated specifically around increased crop production on a certain project area. It is now clear that this was of too high an order, as achievement of agricultural production is not only impacted by irrigation inputs, while it often takes time for farmers to adapt to changing circumstances. The PDO will be measured again a few years from now, as part of the monitoring and evaluation (M&E) of the IDA-funded Second On-farm Irrigation Project (OIP-2). 1.4 Main Beneficiaries The primary beneficiaries of the project were expected to be farm families (water users) in all seven oblasts of the country through strengthening of as many as 160 WUAs. Out of these, 80 WUAs were also expected to benefit from rehabilitation and O&M improvement of their onfarm systems servicing 150,000 farm families on 160,000 ha. The project was also expected to strengthen the capacity of DWR at central and field level (oblasts and raions), in particular to promote WUAs both through the development of Support Units (SU) as well as by introducing a client-driven approach in the dealing of public authorities with WUAs. 1.5 Original Components The project consisted of three components: (i) (ii) WUA Strengthening Services. This component included support for the establishment and training of WUA Support Units at the central DWR office, in the seven oblasts, and in an estimated 19 raions. This new support function and the SUs were expected to progressively be integrated in DWR. Technical assistance was financed to provide training and knowledge transfer to the staff of the SUs. These, in turn, would provide promotion and awareness seminars to a minimum of 160 WUAs and strengthen them through an estimated 100 training courses in such topics as governance, administration, financial management, and irrigation system O&M. On a demand basis, small equipment and motor cycles would also be provided to WUAs on a 100 percent cost recovery basis; Infrastructure Rehabilitation and Operation and Maintenance. This component included: (i) rehabilitation of on-farm irrigation systems commanding up to 160,000 ha 2

12 under the management of about 80 WUAs where there was strong interest in WUA development and readiness to fulfill agreed selection criteria and milestones, as well as limited rehabilitation of selective off-farm irrigation infrastructure where the supply of water to WUA-managed systems was a constraint; (ii) design and construction supervision services; and (iii) implementation of O&M plans to be financed by WUAs; and (iii) Project Implementation Support. A Project Implementation Unit (PIU) that had been established for the implementation of the Irrigation Rehabilitation Project (IRP) was strengthened with staff and equipment to implement OIP as well. 1.6 Revised Components Project components were not formally revised. 1.7 Other significant changes In order to overcome the persistent lack of counterpart funding while ensuring a satisfactory implementation of the second project component, the financing rule of civil works was modified in March 2005 with the share of IDA increasing from 75 to 85 percent. The project followed a programmatic approach and therefore could easily adapt its scope to changing needs and constraints arising during implementation. At appraisal, the first component (capacity building of WUAs) was expected to benefit an estimated 160 WUAs while the second component (rehabilitation) was expected to cover half of these, i.e. 80 of those 160 WUAs after reaching a sufficient level of institutional development. In view of the high demand for supporting the establishment of WUAs, the number of WUAs established during the project life eventually reached about 455 WUAs which is much higher than the target. This required an increase in the number of SUs from 19 originally planned to a total of 40, covering all main irrigation raions of the country. Nevertheless, the budget actually spent for Component 1 at project completion was lower than foreseen: US$4.56 million versus US$6.2 million planned. The number of WUAs which benefited from rehabilitation had to be reduced from the original 80 to 63. The irrigated area benefiting from rehabilitation was about 122,000 ha, which represents about 76 percent of the initial target of 160,000 ha. This was due to a combination of factors: (i) unforeseen high price increases of civil works during implementation; (ii) the abovementioned change in disbursement percentage for civil works; (iii) the appreciation of the KGS compared to the US Dollar, which was only partly compensated by the appreciation of the SDR compared to the US Dollar; and (iv) increased costs due to the replacement of some contractors who failed. The funding for the third component was also substantially increased during implementation. The project was originally implemented by the PIU of the IRP which was strengthened. However, during implementation, the Bank and the government jointly agreed to establish a separate PIU in view of the workload of both projects and the need to increase implementation efficiency, although fiduciary tasks (procurement and financial management) continued to be shared among the IRP and OIP. Also, contrary to what was originally planned and in order to 3

13 compensate for the lack of capacity of engineers in the rural areas, oblast rehabilitation teams were established to assist local engineers in designing on-farm rehabilitation works in collaboration with the concerned WUAs. Finally the project completion date was extended from March 31, 2007 to May 31, 2008, in order to enable the completion of a number of contracts for on-farm system rehabilitation. 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry The quality at entry is considered satisfactory, with a good analysis of the sectoral issues and with the project responding to priorities identified. The project had a simple design, addressing immediate needs while aiming to ensure long-term sustainability. It also adopted a demanddriven programmatic approach, rejecting to select project WUAs upfront. The project was designed to complement the IRP (satisfactorily implemented from September 21, 1998 to May 31, 2006), which addressed the rapid deterioration of a number of existing main and secondary irrigation systems by rehabilitation of their most critical sections. The project was logically articulated around two components addressing institutional development and irrigation system rehabilitation and modernization. The project proposed an innovative approach to support the institutional strengthening of WUAs responsible for O&M of I&D systems within the former state and collective farms as a critical pre-requisite for rehabilitation of the on-farm infrastructure (second component). This reflected a worldwide lesson that in order to achieve long-term sustainability, institutional aspects are often more important than physical rehabilitation. The institutional development component emphasized the importance of developing an internal capacity within the DWR in order to sustain the support to WUA development in the long run, i.e. after project completion. This was done through the establishment of WUA Support Units at different levels (central, oblast, raion) and the appropriate training of their staff. Given the limited experience in WUA development, technical assistance was provided by international experts during the first years of the project. The institutional development progress of each WUA was measured through seven well-defined milestones 2 and WUAs had to reach milestone 4 before being eligible for infrastructure rehabilitation under the second component. Based on demand from the qualified WUAs, the second component was designed to focus on priority works, rather than comprehensive rehabilitation and modernization, so that the project s impact could be maximized over as many WUAs as possible. Two risks were underestimated during project preparation. The full O&M for the command areas managed by the WUAs that have received support from the OIP is not adequately financed 2 Milestone 1 (Formally established); Milestone 2 (Staff hired and training started); Milestone 3 (O&M plan prepared taking into account ISF increase to cover O&M); Milestone 4 (ISF paid); Milestone 5 (Rehabilitation alternatives developed); Milestone 6 (Rehabilitation alternative selected, 25 percent Repayment Agreement agreed); Milestone 7 (WUA ready for cooperation with the rehabilitation of the on-farm irrigation system). 4

14 by the water users was considered moderate at project preparation, while this risk was repeatedly highlighted as a substantial one during implementation and remains a concern at the time of completion despite major achievements (see Section 4 below). Although positive trends are observed, it became clear during project implementation that the achievement of sustainable O&M is a process that takes longer than the duration of a typical project. The risk of cost overruns during project implementation was rated negligible because OIP followed a program approach with flexibility in the number of WUAs to be covered. This is correct, but rapid cost increases and other reasons as explained later led to a reduction in project area covered from 160,000 ha initially planned to be rehabilitated to about 122,000 ha at project completion. Finally, the risk that DWR would not be able to absorb WUA Support Units and its structure was rightfully rated as substantial in the PAD and therefore given full attention during implementation. 2.2 Implementation Throughout implementation and especially as a result of the mid-term review, the project went through important changes both on the institutional and physical rehabilitation fronts. These changes aimed to adapt the project to changing needs (e.g. with design capacity and number of WUAs to be supported) and to substantial difficulties resulting from the lack of experience with the implementation of projects such as the OIP in the country. On the institutional side, while the establishment of WUAs was faster than foreseen, the institutional process aiming to strengthen them and bring them to a sufficient level of maturity was more difficult and time-consuming than foreseen. The project spent considerable efforts to strengthen this process, including training, technical assistance, and more intensive coaching activities by the SUs which eventually resulted in the satisfactory achievement of the institutional component. Moreover, in view of the widespread demand throughout the country for the development of WUAs, government and IDA decided to increase the number of SUs at raion level from 19 originally planned to a total of 27, complemented by 13 individuals in other raions, thus covering all 40 raions where irrigation is a major activity. This resulted in the establishment and training of 455 WUAs by the project against an initial target of 160 WUAs in the PAD, which has been a major achievement. The project team made efforts and showed creativity in trying to establish real participatory decision-making mechanisms for O&M of the on-farm systems. In terms of physical construction, the project faced a number of difficult conditions: (i) harsh weather conditions and the obligation to ensure continuous water supply during the irrigation season, which left only a short period available for construction; (ii) most of the local contractors at the initial stage of project implementation lacked experience with work planning and implementation of this type of extensive rehabilitation and faced some difficulty with applying international standards and specifications in bidding documents; (iii) lack of experienced design companies resulting in inadequate design work at the beginning of the component implementation, which resulted in delays during the initial years as well as the cancellation of some of the 63 civil works contracts; and (iv) high inflation during implementation. Between December 2003 and June 2005, both the PDO and Implementation Performance (IP) ratings were (moderately) unsatisfactory, mainly as a result of slow project implementation and very low disbursements. However, considerable efforts have been made by both PIU and IDA supervision 5

15 teams during project implementation to adapt the project in order to ensure acceptable quality of both design and civil works. These included: (i) the establishment of oblast rehabilitation teams (not foreseen at appraisal) to compensate for the low capacity at raion level; (ii) various amendments and improvement in the contracting arrangement (e.g. price adjustment clause); (iii) a change in disbursement percentage for works; (iv) training of contractors; and (v) the recruitment of full-time field-based supervision engineers. 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization An M&E system was put in place that was well structured and maintained, and which contains a wealth of information largely based on a comprehensive WUA database (in MS Access) by aggregating information from a systematic reporting system of WUAs to the Central Support Unit (CSU). It allows numerous analyses and monitoring of the development process of the WUAs. Different Bank review missions and consultants reports developed in-depth analysis of a series of indicators based on the system. The M&E system was instrumental in monitoring the institutional development of WUAs thereby constantly feeding project management and staff in the CSU and in selecting those WUAs eligible to benefit from the second project component. A more systematic and accurate collection of crop yields to better monitor the project impact on agricultural performance would have been desirable, but this would have involved much additional cost and time spent at field level, not just by project staff but also by other agencies that are involved in the agricultural sector. The M&E system will continue to operate under OIP-2. It could be further improved by adding the following features: (i) monitor more systematically crop yields and cropping patterns between various groups of WUAs tracked by the system in order to evaluate the impact on crop performances of various project activities; (ii) undertake comparative analyses of the rehabilitated WUAs with other WUAs in order to better assess the impact of rehabilitation on ISF, recovery rate, O&M expenses, etc.; (iii) automatically generate key indicators of project performances and impact such as the annual WUA expenses on maintenance, the recovery rate of the ISF, the total cropped area; (iv) include in the M&E a more systematic means to monitor the implementation of the O&M plans of the WUAs; and (v) qualitative monitoring mechanisms could be added such as rapid water user surveys to collect their assessment of changes in their farm performances, and their level of satisfaction with the delivery of irrigation water (reliability, timeliness, adequacy of ISF, etc.). 2.4 Safeguard and Fiduciary Compliance Procurement was generally compliant with rules and procedures. The works were executed through a single civil works contract for each WUA selected for inclusion in the rehabilitation program. A total of eight contracts were cancelled, especially for reasons of non-performance by contractors. The affected contractors had typically quoted a low bid price and because of rapidly rising construction costs were not able to complete the works. The early contracts did not have price adjustment included, which was another reason why several of the contracts failed. When rapidly increasing prices for materials and construction became an issue, IDA agreed to incorporate price adjustment for each contract, independent of the contract duration. 6

16 The project faced some financial management issues, especially related to contractual obligations, including lack of committing expected price adjustment. During the second part of the project, the PIU set up a full complement of financial management staff serving both OIP and the IDA-funded Water Management Improvement Project (WMIP), including an experienced financial manager. Moreover, the accounting system was upgraded and all former data was entered into the new system. The financial manager introduced a separate schedule for each contract detailing all financial transactions relating to that contract, including advance payments, invoice payments for works accomplished, advance recovery, and two-stage repayment of retention funds. The staff, aided by the new financial management system, was also able to properly reconcile all the payments made to contractors, including advance payments, price adjustment, and retention payments. On the environmental side, OIP was classified as category B. Each design report of a sub-project included an environmental impact assessment. There was no negative environmental impact observed from the project activities as the implemented works were only rehabilitative in nature and did not change the general layout and operation of the existing irrigation systems. Since October 2005, water management specialists of raion WUA Support Units (RSU) gathered environmental data from 189 sampling points throughout the project area using monitoring equipment supplied by the project. The sampling points are located at sites where some impact from rehabilitation activities could be expected, and data on soils, surface water, ground water levels, and mineralization were collected and submitted to the PIU on a regular basis. Water monitoring parameters included acidity, mineral content (including salinity), temperature and turbidity of water flows from water sources selected by the environmental specialist. The frequency of measurements was between 3-9 times annually. Samples were sent for chemical analysis to a national laboratory, contracted by the project. A short-term environmental specialist was periodically engaged to analyze and report on the environmental monitoring program, recommending appropriate mitigation measures as needed. Very few pollution problems were found, except some unrelated to project activities, such as the presence of heavy metals in some areas. 2.5 Post-completion Operation/Next Phase The progressive integration of the WUA SUs in the DWR was agreed upon during the design of the project in order to ensure sustainability of the WUA support process. This involved a gradual financing by government budget of the costs of operating them, complemented by a formal integration of their staff as civil servants of the DWR. At project completion, the SUs were 100 percent financed by the Government (from counterpart funds) and the integration of staff is now complete after a Government Decree that waived Government s hiring freeze of civil servants in the country for the Ministry of Agriculture, Water Resources and Processing Industry (MOAWRPI) so that DWR could increase its staffing numbers in order to integrate the WUA Support Units. At the beginning of 2009, with the start of the new budget cycle, the SUs will be financed from DWR s budget. IDA is financing a second phase of the project (Second On-farm Irrigation Project; OIP-2). This five-year project was approved in June 2007 for a total cost of about US$20.5 million and aims at sustaining and further expanding the activities of OIP. It will further strengthen the WUAs, with a focus on financial and technical sustainability through support in water management, 7

17 O&M, and agricultural production. OIP-2 has the same components as the first project, i.e. besides the WUA strengthening component, an infrastructure rehabilitation and modernization component targeting those mature WUAs that could not benefit from the first project although they have reached a sufficient level of institutional development. Finally an IDA-funded Water Management Improvement Project, partly conceived as a follow up of the IRP, is inter alia working on the transfer of off-farm works to federation of WUAs and the financial sustainability of DWR. 3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation At the time of completion, the project objective and design remain highly relevant. For the past 10 years, agriculture has remained the main contributor to the country s economic development, consistently representing percent of the GDP and providing employment for around 50 percent of the population. Therefore, the Country Development Strategy (CDS) for considers that agriculture will continue to be among the main sectors of the economy. Irrigated crops are a major source of farm income and therefore a powerful tool for reducing poverty. As recognized by the CDS, one of the most effective ways of increasing labor productivity in agriculture is a reliable and effective irrigation of agricultural lands. The CDS sees a major role for WUAs, and a need to increase ISF and to transfer on-farm systems to WUAs, all activities that have been promoted by the project. During implementation, while the above circumstances and priorities did not change, the project was able to adapt to the fact that the development of WUAs became a national priority promoted by the government so that the project had to extend its coverage to the entire territory while it initially intended to only cover a fraction of on-farm schemes. 3.2 Achievement of Project Development Objectives The PDO was to achieve increased crop production through reliable and sustainable water distribution on about 160,000 ha of irrigated land across the country's seven oblasts. The project had set as a target to increase crop yields by 10 percent (in comparison with the project area before investments) and to support the development of 160 WUAs, including up to 80 of them that would benefit from rehabilitation services (on 160,000 ha). In addition to yield increase, the PAD had set two other impact indicators: (i) percentage of O&M needs of on-farm irrigation systems covered; and (ii) distribution of adequate irrigation water in a timely manner, in line with crop water requirements. At the time of project completion, it still appears difficult to obtain solid data to demonstrate the full impact of the project on crop performance for the following reasons: (i) most rehabilitation took place during the last years of project implementation, i.e. between 2005 and 2007 only 5 WUAs benefited from rehabilitation in or before 2005; (ii) it is too early to witness full benefits from rehabilitation which are expected to take a few years to materialize; (iii) at the time of ICR field work, the latest full set of data available in the WUA database was from 2006; and 8

18 (iv) there is no systematic comparative analysis in the WUA database between WUAs having benefited from rehabilitation and those that did not. Difficulties with the method of measurement of crop yields - relying much on average scheme data rather than data broken down by zones that are performing differently - have meant that it has not been easy to quantify the anticipated yield and crop production improvement in the rehabilitated WUAs. A set of data collected from 33 WUAs in April 2007 ( Case Study on Onfarm Rehabilitation Impact under OIP by the local NGO M-Vector see Section 3.6 below for more details) allowed an impression of yield data and an analysis of the changes in average crop yields between 2005 and 2006 in WUAs interviewed during the survey. These WUAs were grouped in three categories: (i) 13 WUAs having benefited from support from SUs and from rehabilitation (Group 1); (ii) 10 WUAs only having received support from the SUs (Group 2); and (iii) 10 recently established WUAs had not yet benefited from the project (Group 3). In addition, an assessment of the 2005/06 seasons in 13 WUAs where rehabilitation works had been completed (not the same as the Group 1 WUAs) showed that yield levels have fluctuated, with yields of many crops increasing often more that 10 percent, but other crops had reduced yields. It was found that water users are still conservative in selecting crops, and cropping patterns have not changed much until now. Data collected found that the yields of the main crops in irrigation systems that have benefited from rehabilitation (Group 1) were higher than the yields in the WUAs that have received little or no project support (Group 3). The average yield for winter wheat was 2.8 ton/ha for Group 1 and 1.6 ton/ha for Group 3. Similarly, for maize the yields were 3.0 ton/ha and 1.7 ton/ha, for kidney beans 1.7 ton/ha and 1.2 ton/ha, for cotton 2.5 ton/ha and 1.7 ton/ha, and for permanent grasses 3.1 ton/ha and 2.8 ton/ha. The comparison of crop yields in 2005 and 2006 showed that on average crop yields have remained the same in rehabilitated WUAs (-0.4 percent difference), they decreased by 12.2 percent in the control group, and decreased by 4.9 percent in the WUAs which only benefited from the SUs. In other terms, rehabilitation appears to have maintained yields while these have decreased in other WUAs: the net yield impact from the project would be 11.8 percent (which is in line with the assumptions made at appraisal; two years of agricultural data is of course not conclusive for trends, but at least a comparison can be made for WUAs that received different level of assistance) from the combination of the two project components and 7.3 percent from the first component. More agricultural benefits are expected to materialize in the future beyond these preliminary trends. In addition, in several WUAs some land could be reclaimed as a result of the project, but this could not be captured by the M&E system as the changing WUA area in the database does not distinguish between what resulted from improved irrigation and what simply resulted from an increased number of WUA members. Yet, it can be assumed that in a number of WUAs the actual irrigated area increased by a small percentage, which is an important net benefit from the project. While the PDO in the PAD only considered the agricultural impact on the rehabilitated WUAs, the project is also likely having an impact on agricultural performances of WUAs which have only benefited from support and training. The project has exceeded its institutional development target. By project completion, 455 WUAs (managing about 70 percent of the country s irrigated land) had been established with the support 9

19 of the project, versus 160 envisaged by the PAD. However, these 455 WUAs have reached different levels of development. The M&E database shows that 305 of them can be considered as having reached a level which ensures their development towards sustainability. They have reached milestone 4 (out of 7), i.e. they are operational with trained staff, they have prepared O&M plans taking into account needed ISF increases to cover O&M cost, and they are already collecting increasing amounts of ISF. This figure (305) represents 190 percent of the target in the PAD. Field investigations show that these 305 WUAs: (i) have an adequate institutional framework for management (management board, functional meeting schedules, minutes recorded for meetings, etc.); (ii) allow adequate representation for water users through Representative Assemblies and Representative Zones; (iii) have at least basic office facilities (often in Ayil Okmotu offices); (iv) are adequately staffed with both administrative and technical personnel; (v) have accounting and financial control procedures; and (vi) have developed operation procedures, including those for water management. Other achievements of the project include: (i) effectively filling the gap left in on-farm irrigation management after the dissolution of the collective and state farms. In the absence of the project, the deterioration of on-farm schemes would have certainly worsened considerably with serious negative impacts on crop performances; and (ii) establishing a sustainable process to support the development and future strengthening of WUAs which is now embodied in the DWR (see Section 3.5 (b) on institutional strengthening). Extensive field visits during Bank review missions and the ICR mission showed that the reliability and timeliness of water delivery has greatly improved, leading to more efficient water use by crops. This is confirmed by the earlier mentioned case study which found significant improvements for rehabilitated WUAs in water users assessment of the reliability, adequacy, timeliness, and equity of water delivery (see Section 3.6 below). The project has also led to a better organization in water distribution and a sharp decrease in conflicts over water and in time spent by farmers on irrigation. Even in WUAs where the actual rehabilitation did not cover the entire scheme, the improvement in water efficiency in the area concerned by the rehabilitation indirectly impacts on areas that did not benefit from rehabilitation, since more water will be available in those areas as well. Finally, a sufficient percentage of operation and maintenance (O&M) needs of on-farm irrigation system covered has not yet been achieved, which is a concern in relation to sustainability (see Section 4 below). While the M&E system accurately captures the actual expenses on O&M per WUA, O&M plans are not yet fully based on actual assessment of the scheme requirements. The PAD mentions the figure of US$10 per ha as a typical on-farm ISF, about 70 percent of which would have to be spent on actual O&M. The ISF that was collected by the 63 WUAs between 2003 and 2007 was KGS 105 million (about US$3 million) or about US$5 per ha on average. In 2007, the average amount of the ISF spent on O&M was KGS 109 (US$3) per ha. This figure does not take into account the considerable amount of maintenance carried out by the farmers through self-help. Considering the positive trends, it is expected that during the next years many WUAs, with the assistance of OIP-2, can reach the needed level of ISF to provide adequate O&M of their managed systems. 10

20 3.3 Efficiency The PAD included a cost-benefit analysis based on quantifying projected agricultural benefits by assuming: (i) average yield increases by 10 percent at full development in rehabilitated WUAs; (ii) average yield decreases by 1 percent per year in the without-project scenario. In April 2007 and February 2008, respectively, the PIU undertook two field surveys to complement the data available in the M&E unit in order to quantify the impact of the project on WUAs and crop performances. One of the purposes was to be able to perform a cost-benefit economic analysis. However, neither during these two attempts nor during the ICR mission was it possible to formally re-run the economic analysis included in the PAD, with the main reasons listed above. A recalculation of the economic rate of return (ERR) could be performed by assuming that: (i) the above-mentioned results from the survey with 33 WUAs are extrapolated to the entire project, i.e. an average increase by 11.8 percent in rehabilitated WUAs compared to nonbeneficiary WUAs; and (ii) compared to the PAD, benefits can be discounted by 24 percent as the WUAs that eventually benefited from rehabilitation was only 76 percent of what was planned at appraisal (121,000 ha instead of 160,000 ha). Based on these assumptions, the Economic Rate of Return of the project would be reduced to about 20 percent. However, this is very conservative considering that: (i) the above yield increases were measured for one year only while full benefit would materialize after a few years; and (ii) other substantial benefits were not incorporated in the calculation as explained below. Cost Effectiveness. The project is considered cost effective. Despite sharp increases in construction materials during the project implementation period, the average cost per hectare of on-farm I&D rehabilitation was contained at US$166/ha, which was only 20 percent higher than what was planned at appraisal. The WUA development component was even more cost-effective as 455 WUAs were strengthened by the project against 160 foreseen at appraisal for a similar cost. This means that the actual cost of strengthening one WUA was slightly over US$10,000, which is only 26 percent of the about US$39,000 projected at appraisal. Finally, other benefits have not been quantified in the economic analysis which might be substantial. They include: (i) some changes in cropping patterns, in particular an increase in areas in vegetables, potato, and peas at the expense of cotton and cereals; (ii) the potential downstream economic use of water saved as a result of better scheme efficiency; (iii) the benefits from institutional strengthening as the economic analysis focused on the benefits from physical rehabilitation on 160,000 ha; and (iv) social benefits such as a reduction in conflicts and in time spent by users on irrigation. As a conclusion, although it is too early to fully document project impact on agricultural performances, the above partial analyses show that the project is likely to reach its development objectives and therefore an economic performance close to the one projected at appraisal. 3.4 Justification of Overall Outcome Rating Rating: Satisfactory In light of the above and of the definition of rating criteria ( the extent to which the project s objective were achieved, or are expected to be achieved, efficiently ), the overall outcome rating 11

21 is satisfactory. Even if agricultural improvements have yet to fully materialize and be adequately documented, the PDO is likely to be achieved as demonstrated by the above data analysis and cost-benefit analysis. The efficiency of the project is illustrated by the cost effectiveness which shows a low cost per WUA strengthened and a low rehabilitation cost per hectare. 3.5 Overarching Themes, Other Outcomes and Impacts a) Poverty Impacts, Gender Aspects, and Social Development An improved performance of the agricultural sector, being the main source of income in the rural areas, will have an impact on poverty alleviation. On the social side, through the establishment of WUAs the occurrence of conflicts related to the use and distribution of water was reduced leading to more favorable social conditions. As a result of the nature of the project, almost all land owners within the command area of the established WUAs benefited from the better water management and delivery regardless of their ethnic origin and socio-economic conditions. Further conclusions regarding poverty, gender and social development are difficult to draw since during the project lifespan there was no assessment of the socio-economic conditions of water users within the WUAs, and how different socio-economic groups benefitted from the project s institution strengthening and rehabilitation activities. During OIP-2, it would be useful to carry out farm-level surveys in order to better monitor and understand possible changes and impact of migration trends on farming systems and irrigation. Such surveys should include the changes in local governance of water resulting from the mechanisms set up by the project to ensure that water users have a major role in decision-making in the O&M of the on-farm system. (b) Institutional Change/Strengthening An institutional framework for management of the on-farm systems has been established, including mechanisms to ensure that water users are involved in decision-making. This comprises clearly defined procedures, election of WUA Councils, and promotes the adequate representation of water users through assemblies. The creation of Zonal Representatives allows an adequate and operational representation of the water users. Moreover, the usefulness and efficiency of WUAs for management, operation and maintenance of on-farm systems is recognized by authorities at different levels. This is indeed a tremendous change towards a system of local management of water by the users. However, it is important that WUA representatives continue to represent all water users in the future. It will be important to keep water users informed about WUA activities and decisions made by the WUA Council. The project is leaving behind a much strengthened DWR with improved capacity in assisting the development of WUAs thanks to a comprehensive network of SUs which are integrated in and financed by the Department. At project completion raion SUs are operational in all main irrigation raions of the country and are supported by seven oblast support units (OSU) and one CSU. At the end of the project, a total of 121 Support Unit staff was in place, six in the CSU, 21 in the OSUs, and 94 in the RSUs. More qualitatively, these WUA SU staff acquired abilities in a wide variety of fields, including general administration and financial management, water management, engineering, WUA formation and development, legal aspects of WUA activities, how to hold meetings, community development, conflict resolution, monitoring and evaluation, 12