ASIAN DEVELOPMENT BANK PPA:INO 17164

Transcription

1 ASIAN DEVELOPMENT BANK PPA:INO PROJECT PERFORMANCE AUDIT REPORT ON THE INTEGRATED IRRIGATION SECTOR PROJECT (Loans 1017-INO/1018-INO[SF]) IN INDONESIA December 2001

2 CURRENCY EQUIVALENTS Currency Unit Indonesian Rupiah (Rp) At Appraisal (August 1989) At Project Completion (April 1999) At Operations Evaluation (May 2001) Rp1.00 = $ $ $ $1.00 = Rp1,815 Rp8,250 Rp11,025 ABBREVIATIONS ADB Asian Development Bank BAPPEDA Badan Perencanaan Pembangunan Daerah (Regional Development Planning Agency) DGFCH Directorate General of Food Crops and Horticulture DGRD Directorate General of Regional Development DGRLR Directorate General of Reforestation and Land Rehabilitation DGWRD Directorate General of Water Resources Development DI Daerah Istimewa (special region) EIRR economic internal rate of return EOM efficient operation and maintenance ha hectare IOMP irrigation operation and maintenance ISF irrigation service fee M&E monitoring and evaluation O&M operation and maintenance OEM Operations Evaluation Mission PBB pajak bumi dan bangunan (land and building tax) PCR project completion report PPAR project performance audit report PRAS provincial agricultural service PRIS provincial irrigation service PTGA Indonesian On-Farm Water Management Project PWRS provincial water resources service R&U rehabilitation and upgrading Repelita Five-Year Development Plan SJFC South Java Flood Control TA technical assistance TDU tertiary development unit VO variation order WATSAL Water Sector Adjustment Loan WID women in development WOC Water Operations Center WUA water users association NOTES (i) (ii) The fiscal year (FY) of the Government ends on 31 March. In this report, $ refers to US dollars. Operations Evaluation Department, PE-586

3 CONTENTS BASIC DATA EXECUTIVE SUMMARY MAP Page ii iv vii I. BACKGROUND 1 A. Rationale 1 B. Formulation 1 C. Purpose and Outputs 1 D. Cost, Financing, and Executing Arrangements 3 E. Completion and Self-Evaluation 3 F. Operations Evaluation 4 II. PLANNING AND IMPLEMENTATION PERFORMANCE 4 A. Formulation and Design 4 B. Achievement of Outputs 5 C. Cost and Scheduling 5 D. Procurement and Construction 6 E. Organization and Management 6 III. ACHIEVEMENT OF PROJECT PURPOSE 7 A. Operational Performance 7 B. Performance of the Operating Entity 10 C. Economic Reevaluation 11 D. Sustainability 11 IV. ACHIEVEMENT OF OTHER DEVELOPMENT IMPACTS 12 A. Socioeconomic Impact 12 B. Environmental Impact 13 C. Impact on Institutions and Policy 13 V. OVERALL ASSESSMENT 14 A. Relevance 14 B. Efficacy 14 C. Efficiency 15 D. Sustainability 15 E. Institutional Development and Other Impacts 16 F. Overall Project Rating 16 G. Assessment of ADB and Borrower Performance 16 VI. ISSUES, LESSONS, AND FOLLOW-UP ACTIONS 16 A. Key Issues for the Future 16 B. Lessons Identified 18 C. Follow-Up Actions 22 APPENDIXES 23

4 PROJECT PREPARATION Technical Assistance TA 857 vii BASIC DATA Integrated Irrigation Sector Project (Loans 1017/1018-INO[SF]) Telang and Saleh Drainage Improvement Project Title Type No. of Personmonths Amount ($ million) Approval Date PPTA Feb 1987 Loans a Loan 479 Lower Citanduy Irrigation Project Nov 1980 Loan 518 Wadaslintang Multipurpose Project Jun 1981 Loan 522 Bali Irrigation Sector Sep 1983 Loan 638 Second Irrigation Sector Sep 1983 Loan 725 Technical Assistance Program Dec 1984 Loan Loans 860(SF)/861 Third Irrigation Sector Project Nov 1987 KEY PROJECT DATA ($ million) Per ADB Documents Actual Total Project Cost Foreign Exchange Cost Local Currency Cost ADB Loan Amount/Utilization ADB Loan Amount/Cancellation 5.2 KEY DATES Expected Actual Appraisal 1-18 Aug 1989 Loan Negotiations Nov 1989 Board Approval 17 Apr 1990 Loan Agreement 27 Jun 1990 Loan Effectiveness 25 Sep Aug 1990 Initial Disbursement 22 Apr 1991 Project Completion 31 Mar Mar 1997 Loan Closing 30 Sep Jan 1999 Months (effectiveness to completion) ECONOMIC AND FINANCIAL Appraisal PCR PPAR INTERNAL RATE OF RETURN (%) Economic Internal Rate of Return Financial Internal Rate of Return BORROWER Government of Indonesia EXECUTING AGENCIES Part A: Directorate General of Water Resources Development Part B: Directorate General of Public Administration and Regional Autonomy Parts C & E: Directorate General of Food Crops and Horticulture Part D: Directorate General of Reforestation and Land Rehabilitation Part F: Directorate General of Regional Development = not calculated, ADB = Asian Development Bank, PCR = project completion report, PPAR = project performance audit report, PPTA = project preparatory technical assistance. a The number of person-months spent for preparatory work for the Project under various loans cannot be determined.

5 iii MISSION DATA Type of Mission No. of Missions No. of Person-Days Appraisal b Inception 1 10 Project Administration Review Midterm Review 1 40 Project Completion 1 84 Operations Evaluation c 1 42 b c No formal loan fact-finding mission was undertaken. Preparatory work was carried out under various ADBsupported loans and TAs prior to appraisal. The Operations Evaluation Mission comprised C.B. Amerling, Senior Evaluation Specialist (Mission Leader); and C. Morris, Consultant.

6 vii EXECUTIVE SUMMARY The Project was designed to support the Government s development goals in the agriculture sector, which included consolidating rice productivity gains, broadening the agriculture base, creating rural employment opportunities, and achieving balanced regional development. The Project was specifically aimed to accelerate agricultural development in the major rice producing provinces of Central Java, Daerah Istimewa (DI) Yogyakarta, Southeast Sulawesi, South Sumatra, and West Sumatra. The Project consisted of six components: (i) irrigation development, including rehabilitation and upgrading of irrigation and drainage schemes; introduction of efficient operation and maintenance (O&M); transfer of O&M responsibilities from central to provincial agencies and water users associations (WUAs); and institutional strengthening; (ii) introduction of an irrigation service fee (ISF); (iii) agricultural development through tertiary development units (TDUs) for testing water management techniques, improvement of seed farms, land development, and strengthening of WUAs; (iv) soil and water conservation; (v) women in development; and (vi) strengthening of coordination and monitoring. Project relevance has changed considerably from the time of approval to evaluation. The objectives of increasing farm productivity, creating new employment opportunities, and improving the living standards of farmers remain highly relevant. However, the highly top-down centralized approach of project design and implementation, and lack of public consultation on policy reforms are now outdated. Major investments in isolated transmigration areas for transforming fragile swampland into irrigated rice areas have also been largely curtailed as they have generally been shown to be economically, ecologically, and technically unsound. In general, the Project achieved its physical targets and improved the welfare of poor families in its area of influence. The higher cropping intensities and rice yields generated resulted in substantial incremental output of rice, increased farmer incomes, and job creation, though at levels below the optimistic appraisal expectations. Physical completion of rehabilitation and upgrading, which dominated project expenditures, was significantly delayed, and in most cases, did not achieve full restoration of irrigation systems capacities or efficiencies. Project funding was inadequate to fully address all system needs, while funding supported works that, in some cases, may not have been technically or economically justified. The overall impact of the agricultural development component was marginal. Most TDU programs were carried out before irrigation systems were upgraded. This aggravated field operation problems and precluded most water management extension activities (often due to lack of water control structures and/or lack of water). Deterioration in most systems started soon after project completion due to less than adequate O&M funding and management. Progress on ISF has not been significant, and development of self-sustaining WUAs was largely not achieved. Sustainability of irrigation systems operations and project incremental outputs is assessed as less likely than anticipated. The unabated deterioration of project watersheds is expected to continue to accentuate flooding and drainage problems in the wet season, reduce water availability in the dry season, exacerbate the problems associated with increased sedimentation, and reduce water quality over time. Overall, the Project is rated only partly successful owing largely to (i) substantive shortfalls on O&M, WUA development, and cost recovery targets; (ii) significant shortfalls in improvements of women in development activities and executing agencies institutional capacity; (iii) accelerated deterioration of subprojects watersheds and resultant problems of sedimentation, drainage, and flooding in a number of subproject areas; (iv) insufficient attention to tertiary irrigation and drainage canal development; (v) negligible long-term impact of the TDU

7 v and seed production components; (vi) substantial noncompliance with major loan covenants for irrigation policy reforms; and (vii) questionable selection of some subprojects for investment. The evaluation has identified important issues for the future, among which are (i) sector approach to irrigation development and firm commitment to institutional development and capacity building; (ii) beneficiary and WUA participation, including delegation of responsibility for irrigation O&M to WUAs; and (iii) simplification of scope and involvement of fewer agencies. A stop-start approach to irrigation development reduces opportunities for institutional capacity building, leads to an inefficient deferred maintenance regime, and raises false hopes with water users and government agencies that development will continue in a well-planned long-term manner. A long-term commitment to irrigation sector development is required with major sectoral institutional changes addressed by using other loan modalities such as combining a sectoral investment and program modality within a long-term framework. In general, farmers, and collectively WUAs, were not involved in a participatory manner in project activities. Paternal, top-down attitudes of all executing agencies together with (i) inappropriate funding mechanisms, (ii) less than satisfactory institutional project design, and (iii) less than optimum use of consultants and nongovernment organizations all contributed to a lack of beneficiary participation. Tertiary designs and water management without WUA participation led to incomplete development of tertiary systems. The Project paid less attention to determining the readiness of WUAs to undertake O&M of tertiary systems. Difficulties in transferring O&M responsibilities were due to farmers inadequate knowledge of operational responsibilities, lack of skills for managing the schemes, insufficient involvement of WUAs, and inadequate extension services. Under the complex project design, resources of the Asian Development Bank and the Government, required to ensure efficient and effective implementation, were severely stretched. The Project, executed through six government agencies, was geographically diverse and included three unusual subprojects: (i) Batang Anai in West Sumatra with peat soils; (ii) Wawotobi in Southeast Sulawesi with high land development needs; and (iii) the core subproject Telang-Saleh in South Sumatra, a major swamp pilot project, accounting for about 25 percent of the total project area. These subprojects were not representative of the irrigation sector in Indonesia and had major technical differences from the Java irrigation subprojects. Limiting the Project to Java subprojects only, which were more homogeneous, contiguous in area, and similar in development requirements, would have significantly simplified implementation and allowed major works and activities (drains, flood prevention, tertiary development, and institutional support) to be finished. Among the lessons learned are the following: (i) (ii) Nationally managed provincial irrigation project activities need to be transferred to local provincial and district water resource services. The pressure to retain control at the center (national and provincial) rather than to delegate and decentralize continues to decrease institutional and project implementation efficiency. During project design, institutionalization of project activities needs to be given priority. The participation of beneficiaries is critical if they are to feel a sense of ownership and responsibility; this will also affect their attitudes to ISF and O&M. WUA establishment and strengthening should be accelerated with a focus on

8 vi (a) quality organizations, (b) involvement in tertiary and main system O&M, and (c) a clear linkage between irrigation service and irrigation fees. (iii) (iv) (v) (vi) (vii) (viii) Regular coordination meetings within the Regional Development Planning Agency (BAPPEDA) are important and should concentrate on the needs for interagency planning; identification of planning priorities; and assessment of proposals for scheme, subproject, and sector monitoring and evaluation, and institutional development coordination. To carry out such a program, the provincial water resources services need to strengthen their technical departments for irrigation planning and design, BAPPEDA needs to allocate regular funding (routine budgets) for funding such meetings, and beneficiaries need incentives and assistance to participate in bottom-up planning and implementation activities. Strengthening of administrative sections of provincial and district water resources services is required through the focused training of administrative staff and streamlining of procedures. The proposed introduction of efficient management information systems could also be very effective in more efficient utilization of staff time. More involvement of provincial and district authorities and beneficiaries in key policy directives is essential. Implementation of government O&M policy should be formulated, explained, and discussed jointly in a participatory manner with managers of the provincial and district water resources services, BAPPEDA officials (provincial and district levels), and other senior provincial government officials. A two-year commissioning period is generally required for adequate development programs for medium-sized schemes (1,000 5,000 hectares) with a four-year period for large and major schemes (larger than 5,000 hectares). Programs should include training, improvements in infrastructure, development of information techniques for crop planting reporting, adjustments in water allocation systems, optimization of budgets, and purchase and commissioning of equipment. A human resources development program should be the basis of the training program. Training units need to be incorporated into the structural organization. All staff holding positions should be compelled to carry out a fixed number of training days each year as instructors. A training monitoring system and information system should be established to provide feedback for future courses Community organizers are required to ensure beneficiaries participate in government-led activities. However, a sustainable mechanism should be introduced, whereby project beneficiaries are not dependent on project community organizers. Farmers need to participate and be organized into WUAs (or other suitable institutions) at the earliest possible stage of system planning. Ensuring farmer involvement in tertiary system construction and main system design, prior to agreement on the modality of subproject development and O&M, is essential. For farmers to be available for participation in planning and design activities, field motivators must be active and in place in a timely manner, coinciding with the fielding of the survey, investigation, and design consultants.

9 vii

10 I. BACKGROUND A. Rationale 1. In early 1989, the Government of Indonesia asked the Asian Development Bank (ADB) to finance part of its irrigation subsector investment program under the Rencana Pembangunan Lima Tahun (Repelita) V (1989/ /94). The loan would support the rehabilitation and upgrading (R&U) of existing irrigation and drainage schemes, and the improvement of operation and maintenance (O&M) in the key rice-growing province of Central Java, and the provinces of Daerah Istimewa (DI) Yogyakarta, South Sumatra, Southeast Sulawesi, and West Sumatra. The Project 1 was designed to support the Government s development goals in the agriculture sector at that time, which included consolidating rice productivity gains, broadening the agriculture base, creating rural employment opportunities, and achieving balanced regional development. 2 Repelita V also placed increased emphasis on promoting diversified cropping systems and improving sector efficiency. B. Formulation 2. Appraised in August 1989, the Project was prepared without a project preparatory technical assistance (TA). However, all seven subprojects, 3 including four core subprojects and three noncore subprojects, had existing preparatory studies, including feasibility studies and detailed designs, financed primarily by ADB under earlier loans and TAs, as well as by the World Bank and the governments of Italy and the Netherlands. 4 Institutional issues of the Project were also well supported by earlier ADB loans and TAs. 5 C. Purpose and Outputs 3. The Project aimed to accelerate agricultural development in the major rice-producing provinces to increase farm productivity, create employment opportunities, and improve the living standards of poor farmers. The Project was expected to have considerable impact on poverty 1 Loans 1017/1018(SF)-INO: Integrated Irrigation Sector Project, for $200 million, approved on 17 April The Project was preceded by three ADB-financed irrigation sector projects including Loan 522-INO: Bali Irrigation Sector Project, for $33.6 million, approved on 17 September 1983; Loan 638-INO: Second Irrigation Sector Project, for $85 million, approved on 22 September 1983; and Loans 860(SF)/861-INO: Third Irrigation Sector Project, for $120 million, approved on 17 November Core subprojects, evaluated and approved in the appraisal report, included South Kedu irrigation, Serayu barrage and Ijo Tipar drainage, and Lower Citanduy irrigation subprojects in Central Java; and Telang and Saleh agricultural development in South Sumatra. Noncore subprojects, subject to final approval during project implementation, included Kulon Progo (Opak-Serang) irrigation subproject in DI Yogyakarta, Batang Anai irrigation subproject in West Sumatra, and Wawotobi irrigation subproject in Southeast Sulawesi (see Map). 4 Preparatory works were financed under Loan 518-INO: Wadaslintang Multipurpose Project, for $87.7 million, approved on 23 June 1981; Loan 725-INO: Technical Assistance Program Loan, for $25.0 million, approved on 18 December 1984; TA 857-INO: Telang and Saleh Drainage Improvement Project, for $350,000 from ADB and $300,000 from the Government of the Netherlands, approved on 23 February 1987; Loan 479-INO: Lower Citanduy Irrigation Project, for $55.2 million, approved on 13 November 1980; 1980 feasibility study of Sermo Dam, 1983 hydrological study of Sermo Dam, and 1985 detailed design of Sermo Dam financed by the World Bank; 1986 Dataran Anai feasibility study for the development of water resources financed by the Government of Italy; and 1989 detailed design study of Batang Anai financed by the Government of Indonesia. 5 TA 5160-REG: Study of Food Demand and Supply and Related Strategies for Developing Member Countries (Phase II), for $500,000, approved on 23 October 1984; TA 673-INO: Study of Irrigation Management, for $350,000, approved on 27 March 1985; TA 937-INO: Efficient Irrigation Management and System Transfer, for $600,000, cofinanced by the Ford Foundation for $300,000, approved on 17 December 1987; Loan 860(SF)/861- INO: Third Irrigation Sector Project, for $60 million, approved on 17 November 1987; and Loans 1014/1015- INO(SF): Food Crop Sector Program, for $250 million, approved on 13 March 1990.

11 2 reduction in its area of influence as it would directly benefit about 260,000 poor and near-poor farm families. Average per capita income was anticipated to increase, at full development, from $55 to $125 per annum, which would then be above the projected poverty level. Increased farm income would be generated by expanding the irrigated rice area, and increasing cropping intensities and yields per hectare (ha). Construction activity was expected to generate additional employment opportunities: 6,500 person-years for skilled labor and 39,000 person-years for unskilled labor. The Project was also designed to provide institution building to strengthen the technical, management, and coordination capabilities of government agencies for planning and coordination, and for irrigation and agricultural development, as well as of irrigation committees and water users associations (WUAs). The project components included the following. 1. Part A: Irrigation Development 4. R&U of irrigation and drainage facilities. This included tertiary systems and other related infrastructure covering an aggregate area of 110,000 ha; and surveys and mapping, planning and detailed designs, and civil works. 5. Introduction of efficient O&M (EOM) 6 in irrigation and drainage schemes. Covering an aggregate area of about 132,000 ha, this component included (i) review of existing staffing levels and O&M systems, procedures, and practices; (ii) determination of O&M requirements and preparation of manuals and training materials; (iii) training of O&M staff and WUAs; (iv) provision of equipment; and (v) introduction of EOM. The transfer of O&M responsibilities from the central Government to provincial irrigation services (PRIS) required project financing of incremental O&M costs, equipment, buildings, and training of irrigation service personnel; and the transfer of irrigation schemes with areas of less than 500 ha to WUAs. 6. Institutional strengthening. Focusing on provincial, district, and subdistrict irrigation services, district irrigation committees, and WUAs, the scope included establishing a project monitoring and evaluation unit in each PRIS, redeploying staff; and providing training (including specialized training overseas), equipment, and facilities. 2. Part B: Irrigation Service Fee 7. Introduction of an irrigation service fee (ISF). The Project provided consulting services, equipment, and facilities for introducing ISF for about 129,000 ha of the project area with the aim of achieving full recovery of O&M costs by fiscal year 1999/ Part C: Agricultural Development 8. Strengthening of existing agricultural services. The scope included (i) establishing and operating tertiary demonstration units (TDUs); (ii) improving seed production; (iii) developing land, including clearing and leveling; and (iv) providing institutional strengthening. The activities included establishing about 22 TDUs, with each intended to (i) demonstrate proper on-farm water management techniques and related organizational arrangements at the tertiary level; (ii) serve as a field training ground for extension staff, local officials, and farmers; (iii) improve three existing seed farms and other facilities; and (iv) develop 4,000 ha into rice fields. 6 Defined as (i) the level of operation required to ensure the delivery of the right amount of irrigation water to the crop or the removal of excess water at the right time so as to make possible optimum crop production in an irrigation scheme; and (ii) the level of regular and periodic maintenance required to keep the system facilities and equipment in such condition that efficient operation can be achieved.

12 3 4. Part D: Soil and Water Conservation 9. Establishment of soil erosion control measures. To safeguard the existing Wadaslintang reservoir in South Kedu, Central Java, project-specific interventions included (i) implementation of erosion control measures, such as terraces and waterways on 4,500 ha of critical land, and the construction of check dams; (ii) establishment of soil conservation demonstration farms; and (iii) training of farmers in erosion control techniques. 5. Part E: Women in Development 10. Support of the contribution of women farmers. Focusing on contributions to agricultural development and family welfare, a pilot subproject in Telang and Saleh was designed to encompass (i) inclusion of women in workshops, demonstration, training, and extension related to the O&M of improved drainage systems; (ii) strengthening of agricultural extension services to women farmers; (iii) establishment and strengthening of women s farmer groups; (iv) provision of assistance to setting up home industries; and (v) training of women village leaders and key women farmers. 6. Part F: Strengthening of Coordination and Monitoring 11. Strengthening of coordination and monitoring capability of development agencies. Focusing on regional development planning agencies (BAPPEDAs) at the provincial and district levels, the Project was to provide office buildings, equipment, other facilities, training, and consulting services. D. Cost, Financing, and Executing Arrangements 12. The total estimated project cost at appraisal was $264 million, including a $103 million foreign exchange component (Appendix 1). ADB financing of $200 million equivalent was provided through loan 1017 for $170 million from ADB s ordinary capital resources, and loan 1018 for $30 million equivalent from ADB s Special Funds resources. The ADB loans represented about 76 percent of total project costs and were to provide financing for 97 percent ($100 million) of the estimated foreign exchange costs, and 62 percent of the local currency costs ($100 million). The Government of the Netherlands was expected to make available about $8.0 million in cofinancing (half grant and half loan) for the Telang and Saleh core subproject. The Government of Indonesia was to finance the remaining local currency costs of $56 million. The loans were approved by ADB on 17 April 1990 and became effective on 10 August The Directorate General of Water Resources Development (DGWRD) was the principal executing agency. E. Completion and Self-Evaluation 13. The Project was completed in March 1997, about two years later than envisaged at appraisal. The project completion report (PCR), 7 circulated to the Board in October 1999, rated the Project partly successful. It concluded that the physical targets of rehabilitating and upgrading irrigation and drainage schemes were essentially met. However, problems relating to inadequate O&M were experienced. The Project paid less attention to determining the readiness of WUAs to undertake O&M of tertiary systems. Difficulties in transferring O&M responsibilities related to 7 ADB Indonesia. Integrated Irrigation Sector Project, Project Completion Report. Manila.

13 4 inadequate farmers knowledge of operational responsibilities, lack of skills for managing the schemes, insufficient involvement of the WUAs, and inadequate extension services. Capacity building to institutionalize ISFs and thus reach full cost recovery for O&M was not successful. The ISF program experienced operational and administrative difficulties. Procedures for ISF collection were complicated and time-consuming for the Government and WUAs involved. Despite the involvement of BAPPEDAS, effective monitoring and coordination of project implementation was difficult. Coordination was complex and multisectored across three levels of government. The economic benefits of the Project would be sustained only when EOM mechanisms were in place, supported by well-functioning WUAs and attainment of full cost recovery for O&M. While the estimated economic internal rates of return (EIRRs) of the subprojects and the Project were acceptable, continuing problems related to O&M, if not rectified, would likely curtail or substantially reduce future project benefits. F. Operations Evaluation 14. The project performance audit report (PPAR) focuses on pertinent aspects of the Project and presents the findings of the Operations Evaluation Mission (OEM), which visited Jakarta and the project areas in May The PPAR includes an assessment of the effectiveness of the Project in achieving its objectives, generating and maintaining benefits, and sustaining the operations of subproject irrigation systems and support services. It also deals with other important issues related to the results of the Project. The PPAR is based on (i) a review of project files and related documents; (ii) discussions with the staff of executing and implementing agencies at the central, provincial, and district levels including staff from the project monitoring and evaluation unit established under each PRIS; (iii) interviews with farmer stakeholders, spouses, WUA members, landless field workers, and key informants; and (iv) physical inspection of all subproject sites and major civil works. The PPAR also took into consideration the data and analysis presented in the PCR. Copies of the draft PPAR were submitted for review to the executing and implementing agencies, as well as to ADB staff concerned. Comments received were taken into consideration in finalizing the PPAR. II. PLANNING AND IMPLEMENTATION PERFORMANCE A. Formulation and Design 15. The Project s goals and purpose were consistent with the Government s development needs and ADB s operational strategy for Indonesia at the time of approval and remained relevant at the time of evaluation. The Project followed the sector loan approach, which was considered the most appropriate and cost-effective method of ADB assistance in view of the number of subprojects. In addition, Repelita V was generally considered to be well conceived and appropriately formulated to meet the development needs of the irrigation subsector. DGWRD had been the executing agency for 28 loans and TAs from ADB, three under the sector lending modality, and as such was judged to have the necessary experience and capability to implement a sector loan. However, a detailed institutional assessment of DGWRD was not undertaken. 16. Project design strengths included the involvement of BAPPEDAs and the introduction of O&M as a project-supported activity. A project weakness was the inappropriate use of ADB s sector loan modality, due to the limited number (seven) of very large, geographically dispersed subprojects already identified and appraised (including feasibility studies and detailed designs) before project implementation. The methodology and selection criteria established for approval of noncore subprojects were sound and comprehensive. However, approval of the Batang Anai,

14 5 and Telang and Saleh subprojects should have been withheld based on technical and economic considerations. Less investment in large civil works, and more emphasis on investments in institutional aspects of O&M, would have likely resulted in better and more sustainable gains in O&M efficiencies and crop yields. A design limited to the three large contiguous subprojects in Central Java province alone, which accounted for about 75 percent of the project area, would have been simpler and more effective. Pressures to add seemingly small subcomponents (i.e., additional projects, pilot women in development [WID] activities, paddy seed production farms, and pilot watershed protection) disproportionately diverted limited resources from primary project activities. B. Achievement of Outputs 17. The physical targets were essentially met for R&U, EOM, TDU development, and watershed improvement works; and exceeded appraisal targets for the technical transfer of O&M for small-scale irrigation systems to WUAs. A list of physical achievements is detailed in Appendix 2. In general, tertiary infrastructure (farmer responsibility) and drainage and flood protection works were not fully completed. Land development fell significantly short of targets, primarily due to shortfalls in the Batang Anai subproject as a result of large areas of deep peat soils unsuitable for rice production. Training programs were partially successful, although more practical courses with field-level, on-the-job follow-up would have been more effective. The minor project components were too small to have a measurable impact and did not yield replicable models for the future. Details of project outputs are included in paras C. Cost and Scheduling 18. Actual project cost was about $250 million, 5.3 percent below the appraisal estimate of $264 million. Differences were primarily due to (i) cancellation of the cofinancing arrangement of $8 million with the Government of Netherlands for the Telang and Saleh subproject, 8 (ii) cancellation of various other project infrastructure works (particularly drainage), and (iii) depreciation of the rupiah from Rp1,815 to the dollar at appraisal to Rp2,342 in In addition, most civil works contracts contained no price escalation clauses. ADB financed all foreign exchange costs of $110.6 million (appraisal estimate $103 million), including $23.8 million in interest during construction and related charges, and about 63 percent ($87.3 million equivalent) of total local costs of $139.5 million (appraisal estimate $161 million). Government funding amounted to an estimated $52.1 million equivalent, or about 21 percent of total project costs. However, neither the PCR nor OEM could confirm the $16.3 million in Government project expenditures for administration and land acquisition because accounts were consolidated with other projects. 19. Compared with the original five-year project implementation schedule, actual physical completion was delayed by about two years, necessitating two loan extensions. Loan accounts were kept open for delayed payments until March 1998, and accounts were not officially closed until January 1999 (a total delay of 40 months). Delays in recruiting consultants, adverse weather (particularly major floods in 1992), and slow action by ADB (based on agency staff views) in issuing no-objection letters on award of many civil works contracts all contributed to lengthening implementation. Serious difficulties were experienced in sequencing and coordinating interrelated activities. In general, agricultural development activities preceded R&U works and adversely affected this project subcomponent. In the Serayu core subproject, canal 8 The Netherlands development assistance program for Indonesia was temporarily suspended shortly after the Project was approved by the ADB Board.

15 6 R&U works were completed well before the commissioning of the new Serayu barrage. Upon commissioning, new canal works suffered significant damage, as conveyance tests at full capacity were not completed. Additionally, following the cancellation of funding from the Netherlands, the recruitment of consultants and, therefore, the civil works for the Telang and Saleh subproject, were delayed for two years. D. Procurement and Construction 20. Procurement of consultants and contractors was overly complex because of the large number of packages used by the executing agencies. Bidding implementation was generally satisfactory. A number of executing agencies raised concerns regarding the length of time required to obtain ADB s no-objection letters for several civil works contracts. However, many of these delays resulted from (i) the executing agencies failure to adequately comply with ADB guidelines; (ii) inadequately prepared summary subproject reports 9 in some cases; and (iii) legitimate technical engineering issues (i.e., design aspects of Sermo Dam in the Yogyakarta subproject). 21. Delays in the completion of subprojects and several cases of low standards of construction were generally attributable to weak supervision and poor organization of work by contractors. High construction contract cost overruns were not uncommon, and while they did not necessarily mean poor value for money, they could be indicative of (i) inadequate or inaccurate initial design surveys; (ii) poor design based on inadequate or inaccurate surveys; (iii) inaccuracies in taking-off of quantities, bills of quantities, and contract preparation; (iv) poor unit rate engineering estimates; (v) no mutual check zero 10 carried out after the start of construction; (vi) inadequate site survey capabilities and use of day works measurement and routine certification procedures by site staff; (vii) variation orders (VOs) due to poor design/survey, contractor suggestions, local farmer requests, site staff proposals, and changes in specifications; (viii) excessive cost estimates in VOs or lack of preparation of VOs (inadequate or no design section in VO preparation); (ix) lack of willingness or technical ability of site supervisors to make obvious cost saving adjustments on-site; (x) poor management and site staff control of volume of work in the contract, in particular contractor manipulation of earthwork import, disposal, and transport items; (xi) the ability of experienced, well-managed contractors to dominate site construction and administrative procedures; and (xii) unrealistic bid prices leading to contractor claw back during construction. 22. Cost overruns on individual civil works contracts could have been significantly reduced within the existing procedural framework if staff resources and implementation techniques employed had strictly followed contractual procedures and routine information system maintenance. E. Organization and Management 23. Project components required implementation through six central Government agencies, five provincial offices, and dozens of district and subdistrict offices of each agency (Appendix 3). 9 Feasibility studies produced by the executing agency with assistance from project consultants. Government and ADB approval of the report was required for noncore subprojects to be formally included for financing under the Project. 10 A joint review by the contractor and the owner of quantities of work required at the beginning of a contract.

16 7 This, combined with a large number of contracts 11 and a wide geographic spread (see Map), effectively made the Project unmanageable within Indonesia s centralized framework. ADB fielded 16 review missions over the seven years of project implementation. Continuity from ADB was maintained because an experienced project engineer administered the Project for most of that period. However, while reviews were substantive and intensive, they could not adequately cover all components of all subproject areas. 24. Although the extent of possible corrupt practices cannot be known with any degree of certainty, the Project operated under conditions that would create an environment conducive for corruption. ADB review missions could not possibly oversee the award or construction supervision of the hundreds of large and small contracts dispersed over such a large geographic area. Implementation consultants had no role in overseeing contract awards and also lacked direct construction supervision control. Lack of accountability, poor record keeping, and highly underpaid agency staff with little linkage of performance to promotion persist in the Project s implementing agencies. Circumstantial evidence, based on a review of EOM contracts awarded under local competitive bidding procedures, indicates that collusion in bidding may have occurred, such that competing firms each received a fair share of the tendered civil works. Conducting the mutual check zero to determine agreed quantities of work on individual civil works contracts after the contractor had already started work was reported to be a common practice. Similarly, major contract variations occurred frequently Overall compliance by the Government with loan covenants was generally unsatisfactory (Appendix 4). Of the 30 major loan covenants, 14 were fully complied with. Partial or noncompliance was noted for major covenants including (i) submission of audited accounts and progress reports, (ii) recruitment of additional staffing, (iii) strengthening of the South Kedu Water Operations Center, (iv) issuance of land titles and transfer of ownership to farmers, (vi) O&M funding, (vii) ISFs, and (viii) establishment of viable WUAs. III. ACHIEVEMENT OF PROJECT PURPOSE A. Operational Performance 1. Part A: Irrigation Development 26. Physical targets of the R&U of irrigation and drainage schemes were generally met and exceeded appraisal estimates for the transfer of O&M responsibility to WUAs on schemes of less than 500 ha. Construction quality varied among subprojects with good results noticeable on Sermo dam, Serayu barrage, and some Wawatobi schemes. The irrigation development component suffered from implementation shortcomings due to inadequate project planning and design. These included (i) critical water shortages experienced in a number of subprojects; (ii) inadequate drainage and flood prevention facilities; and (iii) reduction in scope of some R&U works due to expensive design options, insufficient funds, or lack of time. 27. Inadequate procedures for system commissioning and handover to operational staff resulted in some systems operating below the design level. While EOM programs were 11 The PCR (footnote 7) Appendix 1, Table A1.6 lists a total of 893 contracts, including 303 civil works, 100 O&M, 221 equipment/vehicles, 63 consulting services, and 27 overseas and 179 local training contracts. Contract files, reports, and other loan documentation fill 52 boxes. 12 Although occurring after the fact, three civil works contractors who participated under the Project have been subsequently sanctioned by ADB s Office of the General Auditor as a result of corrupt or fraudulent acts on a succeeding project also located in one of the subproject areas.

17 8 introduced under the Project, most of the subprojects did not have O&M budget allocations. This led to infrastructure deterioration such as rusting gates, broken parts, siltation, and heavy weed growths on main canals. The impact of the institutional strengthening component and performance of the WUAs varied among the subprojects. Field-level institutions were established in most Central Java and West Sumatra subproject areas, and most of the training programs provided field staff with technical knowledge on infrastructure and O&M. Appendix 5 presents a detailed discussion of the achievements and performance of the seven subprojects. 2. Part B: Irrigation Service Fees 28. This component comprised introducing ISF for about 129,000 ha of the project area with the aim of full cost recovery of O&M by fiscal year 1999/2000. ISF for main system O&M was introduced under the Project in Central Java and Southeast Sulawesi. 13 However, collection rates were not set to recover full O&M costs and typically Rp15,000 Rp20,000/ha, less than 10 percent of O&M needs, was collected. ISF funds during the Project were collected by local government and were not linked to the provision of O&M services in the area where the funds were collected. WUAs had no control over the use of ISF collected from members. ISF was generally perceived as an additional government tax. 29. The OEM noted that in areas where WUAs were active, kilograms of paddy per crop (in addition to the above ISF) were contributed by WUA members to WUA funds for internal operations and tertiary O&M. 3. Part C: Agricultural Development 30. This component comprised (i) establishing and operating TDUs, (ii) improving seed production, (iii) developing 4,000 ha of rice fields, and (iv) providing institutional strengthening. A total of 22 TDUs and 14 satellite TDUs were established, meeting appraisal targets. Seed farms were rehabilitated in Tegalgondo and Serayu in Central Java, Telang and Saleh in South Sumatra, and Wawotobi in Southeast Sulawesi. At the tertiary level, the main agencies involved in irrigation development have key functions, working closely with the WUAs. TDUs generally were able to demonstrate significantly higher yields that were mainly attributed to improved agricultural inputs and practices rather than to improved water management techniques. Constraints to TDU activities included (i) timing of TDU implementation vis-a-vis main system activities, (ii) farmer resistance to construct new tertiary infrastructure, (iii) lack of quality topographic and soil data, and (iv) lack of trained water management extension workers. 31. Only 1,638 ha were developed compared with the appraisal target of 4,000 ha: 1,400 ha in four Wawotobi subprojects and 238 ha in Serayu. Development of a further 1,929 ha postproject in Batang Anai was stated to be complete, but this could not be demonstrated to the OEM. All project land development areas visited by the OEM appeared appropriate and utilized. However, some areas in the downstream reaches of Wawotobi were suffering from water shortages, double cropping was not practiced, and available labor was sometimes constrained for these subprojects. In addition, major secondary swamp forest was visible in areas reportedly cleared under the Project on Batang Anai. 13 Central Java has one of the most successful collection areas in the country. Recent regulations in Central Java reversed a 1998 governor s instruction to stop ISF collection; in many locations ISF collection reportedly continues but under WUA management.

18 9 32. The overall impact of the agricultural development component was marginal. Most TDU programs were carried out before irrigation systems were upgraded. This aggravated field operation problems and precluded most water management extension activities (often due to lack of water control structures and/or lack of water). TDU operations that began after R&U works generally lasted only 1 2 years, and few TDUs continued after project funding stopped. Although data are not available on farmer outreach, in terms of adoption rates of improved practices, the OEM estimates that the TDU program did not result in a significant increase in yields in the seven subproject areas. Rehabilitation significantly increased the capacity of the seed farms to produce higher yielding certified seed. However, widespread increased use of certified seed by project farmers is not evident due to (i) uncertainty about the quality and benefits of certified seed, (ii) higher prices than for seed purchased from other farmers, and (iii) timely availability of seed. 4. Part D: Soil and Water Conservation 33. This component comprised establishing soil erosion control measures to safeguard the existing Wadaslintang Reservoir in South Kedu, Central Java. The risk to Wadaslintang Reservoir was documented by ADB in the PPAR for the Wadaslintang Multipurpose Project (footnote 4). The older Sempor Reservoir is designed to augment supplies to the South Kedu irrigation subprojects but suffers from water shortages because of reduced reservoir storage. Small-scale physical interventions were included under the Project for 4,500 ha of critical land in the watershed. Although adequately constructed, the works are considered inadequate in nature and scope to have any significant positive impact on the overall watershed condition. 34. Sediment monitoring is not carried out by Wadaslintang Reservoir operators, despite the long-standing concerns about reservoir storage capacity and the availability of echo-sounding equipment on the nearby Sermo Reservoir where monitoring surveys are undertaken biannually. The ongoing South Java Flood Control Sector Project 14 (SJFC) includes catchment protection activities in the Wadaslintang Reservoir catchment area. However, consultant reports indicate that ongoing activities are unlikely to seriously address the sedimentation problems emanating from the watershed. 5. Part E: Women in Development 35. WID activities commenced in late 1993 on a pilot basis in Telang and Saleh. The South Sumatra Provincial Agricultural Service implemented the component with assistance from the consultants, who designed and implemented training activities for 1,300 participants in 10,930 training days. In general, technology transfer and institutional development programs were directed at the head of the household, although only a small percentage of them were women. The most successful intervention was the project-supplied drinking-water storage tanks that reportedly had a significant impact on reducing waterborne diseases. WID activities in the project area ceased after the Project. 6. Part F: Strengthening Coordination and Monitoring 36. Under ADB TA 1363-INO, 15 four monitoring and evaluation (M&E) models were developed and tested in , using the Project as a pilot area. The models all included methodology for measuring (i) financial progress, (ii) physical progress, (iii) water management, 14 Loan 1479-INO: South Java Flood Control Sector Project, for $103 million, approved on 7 November TA 1363-INO: Strengthening Project Benefit Monitoring and Evaluation Activities of the Directorate General for Water Resources Development, for $458,000, approved on 22 August 1990.

19 10 (iv) project benefits, and (v) environmental impact. The TA provided a robust, simple, and flexible M&E system that minimized new field data collection and therefore allowed high geographic coverage at low incremental cost. During the TA, ADB consultants recommended that the M&E system be introduced into project provinces. Instead, the M&E activities under part C of the Project designed a detailed database based on rapid rural surveys and devised an M&E technique based on logical framework techniques. The M&E system was to monitor and evaluate the impact of budgetary support interventions in any sector. The method, which was adopted nationwide, used, in sequential order, a set of development parameters to track the expenditures. This system was not designed specifically for the irrigation sector and the M&E proposal designed under TA 1363 was not directly or indirectly used. The two systems were conceived to perform different tasks: (i) the TA used time-based systems for M&E, and (ii) the Project used a logical framework general procedure to monitor the impact of individual budget expenditures. 37. Under part C, the project M&E units of the provincial water resources service (PWRS) were not strengthened in monitoring irrigation system performance. They were directed away from the models prepared under TA 1363 toward collecting primary data through surveys, focusing on detailed information on sample schemes rather than global coverage, annual performance monitoring against predetermined targets (measured as a ratio rather than absolute parameters), and discontinuing the simple time series presentation of M&E reporting. As a result of the change in focus of the project M&E away from the irrigation sector, the advances made under the World Bank Irrigation Sector Support Project and TA 1363 were severely curtailed. The OEM found no continuation of routine M&E of the irrigation sector in any of the project provinces except Southeast Sulawesi. The Central Java M&E Unit is considered to be active but is preparing detailed expensive reports focusing on measuring performance against short-term budget expenditure rather than irrigation sector M&E. 38. Coordination activities of BAPPEDA under part F were institutionally difficult (perhaps inappropriate) as project funds were channeled through national-level funding mechanisms in all sectoral agencies. BAPPEDA and provincial and district irrigation and agriculture organizations had, during project implementation, no direct management role or responsibilities in implementing project activities. Terminology in the appraisal report does not differentiate between nationally funded, provincially based irrigation projects from provincially based, provincially funded (Dinas) organizations. B. Performance of the Operating Entity 39. DGWRD and its provincial, district, and subdistrict irrigation agency counterparts from the subproject areas have successfully executed and implemented many funding agencyassisted projects. Together, they have thousands of engineers who have been the recipients of numerous government and funding agency-sponsored formal training programs, as well as technical knowledge transfer from specialized consultants over the past 20 years or more. However, the Project still relied heavily on the use of domestic and international consultants in its implementation, with the executing agencies focusing on their administrative role and construction supervision. For survey and design of irrigation and drainage works, the shortcomings included (i) low status of the provincial hydrology units and the failure to process incoming hydrology data at the provincial level; (ii) lack of staff time to monitor developments by other agencies, such that programs for paddy development were not well coordinated with overall special planning activities; (iii) relative inexperience of technical staff who checked designs by domestic consultants; (iv) lengthy administrative procedures and poor information systems; (v) insufficient time in the field to ensure that main system designs were adequate;