Indonesia: Integrated Pest Management for Smallholder Estate Crops Project

Transcription

1 Validation Report Reference Number: PCV: INO Project Number: Loan Number: 1469 December 2009 Indonesia: Integrated Pest Management for Smallholder Estate Crops Project Independent Evaluation Department

2 ABBREVIATIONS AARD Agency for Agricultural Research and Development ADB CAQ Asían Development Bank Center for Agricultural Quarantine DGE Directorate General of Estate Crops EA executing agency FFS farmer field school IA IAARD IED IPM M&E implementing agency Indonesian Agency for Agricultural Research and Development Independent Evaluation Department integrated pest management monitoring and evaluation MTR mid-term review PBME project benefit monitoring and evaluation PCR PPTA RRP project completion report project preparatory technical assistance report and recommendation of the President NOTE In this report, $ refers to US dollars. Key Words adb, asian development bank, farmer field schools, ied, independent evaluation department, integrated pest management, project completion report, project benefit monitoring and evaluation, research Director Team leader Team member R. B. Adhikari, Independent Evaluation Division 1, Independent Evaluation Department (IED) C. Infantado, Portfolio Evaluation Officer, Independent Evaluation Division 1, IED V. Melo, Operations Evaluation Assistant, Independent Evaluation Division 1, IED In preparing any evaluation report, or by making any designation of or reference to a particular territory or geographic area in this document, the Independent Evaluation Department does not intend to make any judgments as to the legal or other status of any territory or area.

3 PROJECT COMPLETION REPORT VALIDATION A. Basic Project Data PCR Validation Date: December 2009 Project and Loan/ 27357, 1469-INO Grant Number: Approved Actual Project Name: Integrated Pest Total Project Costs ($M): Management for Smallholder Estate Crops Project Country: Indonesia Loan/Grant ($M): Sector: Primary - Environment Total Cofinancing ($M): ADB Financing ($M): ADF: 0.0 Borrower ($M): OCR: 44.0 Beneficiaries ($M): Cofinanciers: Others ($M): Approval Date: 26 Sep 1996 Effectiveness Date: 7 Jan Apr 1997 Signing Date: 6 Nov 1996 Closing Date: 30 Sep Apr 2006 Project Officers: Name: A.T. Perez A. Rijk W. Menninger M. Jayawant M. E. Khan M. N. Islam Location (HQ or RM): HQ HQ HQ HQ HQ HQ From Sep 1996 Dec 2000 Jul Jun 2002 Mar 2003 Sep 2006 To Nov 2000 Jun 2001 Dec 2001 Feb 2003 Aug 2006 Dec 2006 Validator: C. Dingcong Consultant Director: R. B. Adhikari, IED1 Quality Control Reviewer/Peer C. Infantado Portfolio Evaluation Reviewer: Officer, IED1 ADB = Asian Development Bank, ADF = Asian Development Fund, HQ = headquarters, IED = Independent Evaluation Department, IED1 = Independent Evaluation Division 1, INO = Indonesia, M = million, OCR = ordinary capital resources, PCR = project completion report, RM = resident mission. B. Project Description (summarized from the report and recommendation of the President [RRP]) (i) Rationale. The Project 1 targeted smallholder farmers whose incomes were constrained by considerable yield losses. These yield losses were largely caused by pests, diseases, and poor farming practices, mainly because of farmers lack of knowledge of improved crop and pest management practices. Since conventional strategies to enhance smallholder estate crop yields had been traditionally pesticide-driven, farmers needed an alternative environmentally-sound option that would enable them to intensify their production. The Project envisioned achieving this through the adoption of a cost-effective, environmentally-sound integrated pest management (IPM) approach. This approach was based on the following considerations: (a) smallholder production of estate crops could be intensified through an integrated farm management approach, instead of depending purely on pesticides; (b) institutional constraints in the sector were a major impediment to the Government s plan to promote IPM among smallholder estate crop farmers nationwide and needed to be addressed; (c) the training capacity of relevant government agencies and farmer groups needed to be upgraded, while the responsibilities of the crop protection institutions linked to extension services needed to be streamlined; (d) institutional linkages between crop protection services, agricultural research, and plant quarantine needed to be improved; (e) the use of existing human resources and the physical infrastructure put in place under the first ADB-assisted National Estate Crop Protection 1 ADB Report and Recommendation of the President to the Board of Directors: Proposed Loan to Indonesia for the Integrated Pest Management for Smallholder Estate Crops Project. Manila.

4 2 Project 2 could be maximized to support government policy to expand IPM in the estate crops sector; and (f) the experiences and lessons learned from the successful Indonesian IPM rice program based on farmer field schools (FFSs) and phased implementation could be taken into consideration in designing the Project. (ii) Impact. The envisioned primary impact of the Project was the widespread adoption of IPM practices by trained smallholder estate crop farmers as a result of the increased institutional training capability of government agencies and farmer groups. This would contribute significantly to the Project s goals: (a) to enhance and protect the environment; and (b) to improve product quality, and increase productivity and the incomes of smallholder farmers. At project completion, about 106,000 farmers were expected to benefit from the IPM training. Of these, 10 percent (10,600) were expected to be women. (iii) Objectives or Expected Outcomes. The expected outcome of the Project was the adoption of costeffective environmentally-sound IPM practices by smallholder estate crop farmers, by strengthening selected government institutions and farmer groups. The Project targeted existing smallholder farms devoted to estate crops (coffee, cacao, tea, pepper, cotton) in the provinces of East Java, Central Java, West Java, Bali, North Sumatra, South Sumatra, East Sumatra, East Kalimantan, West Kalimantan, Southeast Sulawesi, South Sulawesi, Lampung, Bangka-Belitung and West Nusa Tenggara. (iv) Components and/or Outputs. The Project consisted of three parts: Part A: Capacity Building for IPM would provide specialized training to farmers, extension workers, and plant quarantine staff to meet farmers needs (including farming systems diversification) and to upgrade Indonesia s national plant quarantine standards. The subcomponents of Part A were (a) human resource development, (b) farming systems diversification, (c) upgrading of national IPM facilities, (d) upgrading of provincial IPM facilities, and (e) strengthening of IPM support services. Part B. The IPM Research Program would provide IPM solutions, and strengthen feedback linkages among farmers, researchers, and crop protection services. This component focused mainly on developing and testing IPM technologies suited to smallholder farming systems by supporting: (a) the national long-term IPM research program, (b) the provincial adaptive IPM research program, and (c) farmer-driven IPM research. Part C. Project Management Support would provide effective, decentralized, and responsive management of the project investments and services. The subcomponents of Part C were: (a) coordination between the Directorate General of Estates (DGE), Agency for Agricultural Research and Development (AARD), and Center for Agricultural Quarantine (CAQ); (b) monitoring and evaluation (M&E) units in DGE, AARD, and CAQ; (c) support to DGE for policy formulation; and (d) special studies to examine policy and technical issues related to the use of pesticide packages. C. Evaluation of Design and Implementation (PCR assessment and validation) (i) Relevance of design and formulation. The Project was consistent with the Ministerial Decree No. 390 (September 1994) that established the modalities for implementation of the national IPM program, and defined IPM as the management of the population or level of infestation of organisms harmful to plant growth, through the management techniques applied in an integrated manner, in order to prevent economic loss and damage to environment. The promotion of IPM has been a declared policy of the Government, resulting from the positive past experience in solving the severe problems with pests in rice. ADB s operational strategy for Indonesia supported the objectives of the 2 ADB Report and Recommendation of the President to Board of Directors: Proposed Loan to Indonesia for the National Estate Crop Protection Project. Manila (Loan 0682, for $63.0 million).

5 3 Government s Five-Year Development Plan of efficient, equitable, and environmentally sustainable growth. In this regard, the Project was also consistent with ADB s strategy for Indonesia by focusing on environmental enhancement and institutional strengthening. There were four major changes in the project design. First, cotton, as one of the featured crops, was discontinued in the Training Expansion Phase because (a) there were only limited cotton areas in either West Nusa Tenggara or Central Java, where it was planned to be undertaken; (b) cotton was primarily grown under a contract system, and farmers do not have much choice on how the commodity would be grown; and (c) cotton did not have a comparative advantage in production. Coffee was therefore selected for the FFS-IPM training in West Nusa Tenggara and Central Java. Second, the farm diversification component was removed and replaced with crop rehabilitation because of the midterm review (MTR) finding and assertion of the executing agency (EA) that farmers, in reality, were highly diversified. The project completion report (PCR) mentioned that the inclusion of cotton as a featured crop and the subcomponent on farm diversification, were recommended in the PPTA but these turned out to be infeasible in actual project implementation. Third, the responsibility for the research program component was transferred from the Indonesian Agency for Agricultural Research and Development (IAARD) to the DGE because the IAARD s outputs were insufficient and not fully aligned with the farmers needs under this Project. Fourth, changes in the scope of the Project required an extension of 15 months in the loan closing date. (ii) Project Outputs. The Project s key output was farmer training through the FFSs. The PCR discussion on this is complete and fair. Specific details, comparing targets with achievements, were shown in Appendix 1 of the PCR. In summary, 1,016 field leaders were equipped and trained. The total number of farmers trained in FFSs was 122,610, which exceeded the appraisal target of 106,000. The total number of women participants in FFSs was 20,235, compared to the appraisal target of 10,600. Training was provided to 332 DGE staff, another 427 staff received hands-on training in biological control laboratories, and a further 262 staff received training in plant quarantine. As regards the upgrading of IPM facilities, the PCR reported that all civil works related to IPM facilities were completed satisfactorily. This is supported by the specific details on the achievements presented in Appendix 1. The EA's PCR also provided specific details to support the satisfactory completion of all civil works. In connection with the subcomponent on the strengthening of IPM support services, the PCR provided an adequate report on the outputs. In the research subcomponent, shortfalls in the research program targets were sufficiently pointed out. Except for the farmer-driven research, research programs at the national and provincial levels did not meet the targets set at appraisal. The conduct of studies relevant to IPM and farmer needs improved only during the latter part of project implementation (2003) when the research subcomponent was transferred from the IAARD to DGE. For most of the Project, little of the output from the national long-term research program and the provincial adaptive research program addressed the problems of small estate crop farmers. 3 However, the subcomponent on farmer-driven research became more focused on IPM and relevant to farmers needs during the latter part of the project, eventually meeting the Project s targets. In the project management support subcomponent, the PCR appropriately pointed out the weakness in the M&E system, and benefit estimation. This observation was also made during the Project's MTR. (iii) Project Cost, Disbursements, Borrower Contribution, and Conformance to Schedule. Tables and discussions on project costs (Basic Data), disbursements (Basic Data and Appendix 5), and borrower contribution (Basic Data and Appendix 4) contain adequate data. The PCR reported four partial loan cancellations (para. 19), but the underlying reasons for the cancellations were inadequately explained. Nonetheless, the Government attributed these cancellations to the Asian financial crisis that led to substantial reductions in counterpart funds (Government PCR). The Project required two loan extensions owing to the Asian financial crisis and the resulting devaluation of the Indonesian Rupiah 3 The Government PCR reported that a few topics germane to IPM began to be produced only in 2003, including two ecosystem studies for coffee and cacao.

6 4 (Rp). The PCR adequately explained the reasons for these extensions. While the Project implementation schedule, which involved two phases, was presented in the PCR, the initial delays in implementation, resulting from the 1-year delay in fielding consultants and the limited Government counterpart funds, were not discussed in that section. Furthermore, the 15-month extension of the loan closing date, because of revisions in the Project scope, was not adequately discussed. (iv) Implementation Arrangements, Conditions and Covenants, related Technical Assistance, and Procurement and Consultant Performance. The PCR provided a very fair assessment of the implementation arrangements. There was less effective coordination among agencies because information was not readily shared, and there was a different understanding of project goals and activities. The disconnection between the research activities and the objectives of the Project in the early stages of implementation were highlighted in the MTR. Except for the transfer of the supervision of the research component from IAARD to DGE, the implementation arrangements were in line with those set out in the RRP. However, implementation was affected by the frequent changes in DGE project directors, managers, and staff; irregular coordination meetings; infrequent coordination between implementing agencies (IAs); and promulgation of the 1999 decentralization laws. The combination of these factors resulted in less effective levels of project coordination and M&E. The PCR rated compliance with loan covenants as generally satisfactory. Para. 27 provided adequate discussion on the delayed compliance with some covenants. Appendix 8 clearly showed the status of compliance with the loan covenants. However, the PCR failed to report in the discussion that some of the loan conditions are still being complied with, since they are not due until 3 years after project completion. The PCR mentioned that a project preparatory technical assistance (PPTA) 4 was carried out, but this was not evaluated in the report. PAI 6.07A stipulates that a PPTA resulting in a loan should be evaluated in the PCR for the loan project. There was no section in the PCR that discussed technical assistance related to the Project. Recruitment of consultants took longer than expected, resulting in the delay of activities by 1 year. Government regulations and procedures did not allow for timely recruitment of consultants. Apparently, this was not anticipated in the preparatory stages of project implementation. The PCR provided information on the changes from the appraisal in the actual person-month inputs of international and local consultants. It also provided an overall assessment of the consultants performance in relation to the work plan targets, which was rated partly satisfactory. Except for the international M&E specialist, there was no discussion on how the other consultants affected the quality and timeliness of project outputs. It would have been helpful if these issues had been incorporated into the discussion and if the PCR had provided information on how the consultants performance was evaluated, and how they were made accountable for their assigned tasks and targets. Further, it was reported in the MTR that the consortium identified to provide consultancy inputs consisted of four international and four domestic consulting firms. This number of firms and people involved in a single contract made it unwieldy and difficult to manage, backstop, and supervise. The MTR recommended that the consultancy inputs be reviewed and re-examined based on priorities. As a result, the services of short-term consultants were deemed no longer necessary, since their tasks were already covered by the long-term consultants. Also, the total of 242 person-months used by international consultants was lower than the 267 person-months proposed in the appraisal. 4 ADB Technical Assistance to Indonesia for the Second National Estate Crop Protection Project. Manila. The Food and Agriculture Organization was contracted by ADB to undertake the TA. The rationale for the Project, the general terms of reference, and the budget were laid out in this paper.

7 5 (v) Performance of the Borrower and Executing Agency. IED agrees with the satisfactory rating of the Government (the Borrower) in view of its compliance with ADB s reporting requirements and establishment of the necessary imprest accounts. The PCR also rates the performance of the EA (DGE) as satisfactory. While there were, indeed, many challenges faced by the EA, which included the broad scope of project activities that needed to be coordinated and developments beyond their control, such as the new decentralization law and the Asian financial crisis, further justification is needed for the satisfactory rating. There were concerns regarding project implementation that relates directly to the performance of the EA. First, the research activities and studies lagged behind throughout project implementation, except for the farmer-driven research. Second, frequent changes in project directors, managers, and staff affected timely implementation of project activities. Third, weak coordination with and among IAs was repeatedly raised in mission reports and in the MTR. Fourth, an effective project benefit, monitoring and evaluation (PBME) system was lacking for most of the project implementation, becoming operational only towards the end of the Project. Despite these shortcomings in implementation, the Project was still able to achieve its overriding objective of adoption by small estate farmers of environmentally-sound IPM practices. The project effectively achieved most of its intended outcomes. In this regard, the performance of the Borrower and EA is rated by IED as satisfactory. (vi) Performance of the Asian Development Bank. The PCR described the missions undertaken and raised the issue of changes in project officer. Since the project officers had different approaches to implementation, this may have contributed to the confusion in project direction. The PCR also noted that the mission teams often lacked multi-disciplinary specialists. Nevertheless, ADB was able to (a) respond promptly to the effects of the 1997 Asian financial crisis on the Project by re-evaluating project costs and reducing the size of the ADB loan; (b) make the necessary adjustments in the scope of the Project to ensure attainment of the intended outcomes; and (iii) respond to the reprogramming of a portion of the loan to undertake rehabilitation and reconstruction work in tsunami- and earthquakeaffected areas in Nanggroe Aceh Darussalam province. The PCR reported the performance of ADB as satisfactory. IED concurs with this rating. D. Evaluation of Performance (PCR assessment and validation) (i) Relevance. The PCR rated the Project as highly relevant, in view of its relevance to the policies of the Government and ADB, and its contribution to the Government s and ADB s objectives of promoting economic growth, raising farm incomes, and reducing the incidence of poverty in rural areas. The PCR also pointed out that the Project helped streamline crop protection institutions and improved the institutional linkages between crop protection services, agricultural research, and plant quarantine. Agricultural research capacity was strengthened, and support for farmer-driven research was institutionalized. Further, the use of FFS enabled the farmers to achieve greater empowerment and control of their own learning processes. All these positive outcomes were mentioned in rating the Project as highly relevant. IED agrees with all these positive outcomes and concurs that the Project was relevant at the time of approval. However, there were issues that arose during project implementation that affected the sustained relevance of the Project. First, the original design of the Project included a farm diversification subcomponent, which was later replaced by crop rehabilitation. The project design assumed an unbalanced, mono-cropping smallholder enterprise, but this was not the reality of estate crop smallholder farming systems in Indonesia. In reality, smallholder farming systems were highly diversified. This was not foreseen in the project design and formulation stage. Second, the research component of the Project continued to lag behind despite the establishment of the independent ad hoc committee following the MTR. Hence, research outputs were not generated on time for use in IPM solutions or for strengthening feedback linkages between farmers, researchers, and crop protection services. The Government's PCR even reported that over the life of the Project, the long-term research program had achieved little that would address the problems of small estate crop farmers. 5 Third, the absence of an effective and operational PBME for the most part of the implementation limited the 5 Para. 39 of the Government PCR, April 2006.

8 6 Project's ability to monitor project progress, outcomes, issues, and problems. Hence, the Project was constrained in responding to operational issues and problems in a timely and effective manner. Considering all these and the positive outcomes realized by the Project, IED rates the Project as relevant. (ii) Effectiveness in Achieving Outcome. The PCR rated the Project as effective in achieving its outcome. While the PCR stated that most of the intended outcomes, in accordance with the quantitative objectives were achieved, there was no further discussion on these outcomes. Presumably, these were the outcomes already presented in para. 11 and Appendix 1. It would have been more helpful if the reader was referred to a related paragraph or appendix for the outcomes of key quantitative objectives. Also, discussions on the qualitative outcomes were lacking. The PCR stated that though the research component initially lagged behind, it progressed satisfactorily after the MTR with the establishment of the independent ad hoc committee (para. 42).With better coordination, the committee was able to significantly change the direction of the AARD research program (para. 42). This appears in conflict with the PCR of the Government which stated that: The research component of the Project continued to lag behind and remained weak despite the establishment of the independent ad hoc committee after the MTR. The committee was not able to significantly change the direction of AARD research program especially those that had budget appropriations from the Annual Budget (Daftar Isian Proyek). Over the life of the Project, the approved annual national long-term research program had little to address the problems of small estate crop farmers. This could be attributed to the poor coordination among the IAs. (para. 39, PCR of EA). With this conflicting view, the PCR needs to justify further its assessment of the research component. Again, it needs to be noted that the delayed implementation of an operational PBME deprived the Project from making adjustments during the implementation based on feedback from farmers and field personnel. Nonetheless, despite the shortfalls in the research component and shortcomings in the PBME, mission reports and the impact assessment study indicated that majority of the farmers trained, through the Project, adopted IPM practices. There were also spillover effects wherein neighboring farmers had, to some extent, adopted at least some IPM practices. Targets for training of farmers were achieved as well as for the capacity building training programs of selected Government institutions. In essence, the Project was able to achieve its overriding development objective of adoption by small estate farmers of environmentally-sound IPM practices, despite the shortfalls and shortcoming earlier mentioned in some components. In view of this, IED concurs with the effective rating of the PCR. (iii) Efficiency in Achieving Outcome and Outputs. IED concurs that the routine change of DGE officials and project managers affected the smooth implementation of the Project. It is important to mention also that the Asian financial crisis in 1997 reduced the Government fund allocations for the Project, resulting in delays each year of counterpart funds. This adversely affected the timely implementation of Project activities. IED concurs with the higher economic internal rate of return estimate at Project completion of 25.3%, compared to 12.7% at appraisal. This estimate is plausible in view of the higher crop yields at Project completion, and higher commodity prices compared to projections at appraisal. IED agrees with the efficient rating of the PCR in achieving the Projects outcome and outputs. (iv) Preliminary Assessment of Sustainability. IED rates the sustainability of the Project as less likely. The PCR Review Mission observed that the number of FFS alumni groups have not been active after project closure, and less than 20% of the farmer groups are engaged in any post FFS activities. 6 It would require sustained effort on the part of DGE to motivate these groups to continue with their IPM activities. It appears that there s a demand for estate crops covered by the Project, especially with the clamor for organically grown or pesticide free products in the world market. However, greater efforts are needed in processing, marketing, and business enterprise development in order to take advantage of opportunities in the market. 6 BTOR of Project Completion Review Mission, 22 April2 May 2008.

9 7 The support of local governments is an important factor in sustaining and promoting IPM practices and activities. However, the level of support varies widely among districts and provinces. The small budget available to them limits their ability to provide adequate funding for sustained IPM activities. Also, the decentralization law has led to different priorities between the central and local governments. In view of all these, IED agrees with the rating of the PCR that the sustainability of the Project is less likely. (v) Impact (both intended and unintended). The IPM approach has helped enhance and protect the environment. The PCR clearly specified the benefits for the environment (para. 53). The Project envisioned the widespread adoption of IPM practices by estate crop farmers. In the impact assessment study 7 conducted for the Project, the average score on the knowledge test of FFS graduates was about 74%, which was higher than the average score of less than 55% of the exposed and control groups. The study reports that FFS graduates were able to identify at least three pests and five natural enemies. These graduates also used more biological than chemical based methods to control pest infestations. Reportedly, they were more likely to practice good crop culture techniques and farm sanitation than their exposed and control group counterparts. All these are indications that farmers participating in the FFS were adopting IPM practices in estate crops. This impact on farmers was highlighted by the PCR (para. 55). The Project, to some extent, was able to increase farm productivity and income (Appendix 11). Graduates of the FFS were able to raise their crop yield by 20% to 50%. In terms of income, the weighted average farm income increased by 42%. However, despite these improvements, the income of farmers remained low because of the small size of the crop area. The small crop area of less than 1 ha farmed by most farmer beneficiaries had limited impact on farm income. Despite the increases in productivity, yields remained way below potential. The PCR outlined the factors causing this kind of performance in yields under smallholder conditions (Appendix 1, para. 11). Improving product quality in estate crops among smallholder farmers was one of the goals of the Project. The PCR s assessment of this was somewhat sketchy and scattered. This was not adequately discussed in the section on impact. The PCR did not rate the impact of the Project. IED assesses the impact to be significant. E. Overall Assessment, Lessons, and Recommendations (Validation of PCR assessment) (i) Overall Assessment. The PCR rated the Project as highly relevant, effective, efficient, and less likely to be sustainable. Overall, it rated the Project as successful. IED concurs with the overall successful rating but differs in its rating of the relevance of the Project. (ii) Lessons. The Project provided valuable lessons on targeting and setting the selection criteria for participation. The Project showed that those who directly manage estate crop production, and who derive 50% or more of their household income from it, tend to practice IPM more seriously than those whose income is derived from other activities. Specifically, targeting this group would have greater development impact and would likely contribute to an even more widespread adoption of IPM. The lack of a component on processing, marketing, and enterprise development, constrained farmers from taking advantage of market opportunities. In view of the growing demand for pesticide-free and organic farm produce, this would have benefited the farmers economically. The PBME system should have been established at the initial phase of implementation so that the progress and status of the Project could have been easily tracked, and necessary actions taken on time. Typically, the PBME system should include project performance indicators and a baseline 7 Sombilla, M. A Impact Assessment Study of the Integrated Pest Management for Smallholder Estate Crops Project. Final Report.

10 8 socioeconomic survey that will be used for assessing socioeconomic impact. IED finds the lessons presented in the PCR helpful and valuable. ADB needs to consider these in future similar projects. (iii) Recommendations. IED concurs with project-related and general recommendations set forth by the PCR. Of particular importance is the continued monitoring of IPM technology dissemination and adoption. A more simplified, relevant, and practical PBME system that can be implemented at the regional and field level is needed. IED agrees with the PCR that further training in conducting surveys and data collection is needed to improve the quality of information gathered. While it is recommended that greater support is needed from the local government to continue FFS activities, the decentralization law implies that districts and provinces will determine their own budgets and priorities, which may or may not be aligned with further promotion of IPM technology. In this regard, greater advocacy directed towards the local government is needed on the environmental and economic benefits of IPM adoption. F. M&E Design, Implementation, and Utilization (PCR assessment and validation) The M&E subcomponent has been a weak point of the Project. For most of the Project s implementation, no M&E system was in place to monitor either the socioeconomic and environmental impact or progress in implementation. Even though it was established towards the end of the Project s implementation, it still needs to be strengthened. Its quality, timeliness, and relevance are yet to be clearly established. The recommendation to integrate the Project s M&E with that of the Ministry of Agriculture, as with other ADB projects, was not pursued. G. Other (e.g., Safeguards, including governance and anticorruption; Fiduciary aspects; Government assessment of the Project, as applicable) (PCR assessment and validation) The Government submitted its PCR to ADB, in compliance with the loan covenant. No issues were raised by the PCR on safeguards, fiduciary aspects, and other concerns. Hence, IED could not comment on these other aspects. H. Ratings Relevance: Effectiveness in Achieving Outcome: Project Completion Report Highly Relevant Effective Independent Evaluation Department Review Relevant Effective Reason for Disagreement/Comments There were issues during implementation that affected sustained relevance of the Project. First, the project design and formulation did not foresee the reality of diversified smallholder systems. Hence, the farm diversification subcomponent was replaced by crop rehabilitation. Second, the research outputs were not generated on time for use in IPM solutions or for strengthening feedback linkages between farmers, researchers, and crop protection services. Third, the absence of an effective and operational PBME system for most of the Project s implementation constrained the EA from providing timely and effective responses to operational and performance issues.

11 9 Efficiency in Achieving Efficient Efficient Outcome and Outputs: Preliminary Less Likely Less Likely Assessment of Sustainability: Borrower and Satisfactory Satisfactory Executing Agency: Performance of Asian Satisfactory Satisfactory Development Bank: Impact: Not rated Significant The Project helped enhance and protect the environment. It has resulted in the adoption of IPM practices by estate crop farmers. Farmers that graduated from the FFS were able to increase their crop yield and average farm income. Overall Assessment: Successful Successful Quality of Project Completion Report: Satisfactory I. Comments on PCR Quality The quality of the PCR is satisfactory. It is concise and generally follows PAI However, it did not include a review of the PPTA that preceded the loan. The revisions in the scope of the Project that led to the 15-month extension of the loan closing date were not sufficiently discussed. Assessment of the improvement in product quality, which was one of the Project goals, was somewhat sketchy. Economic analyses (economic and financial internal rates of return) at project completion were based on plausible and robust assumptions. On the whole, there is adequate data and information to support reported outcomes. Lessons and recommendations are sound and are based on the findings of the report. J. Recommendation for IED follow up No follow-up action is required. K. Data Sources for Validation Back-to-office mission reports, the MTR report, PCRs (ADB and Government), RRP, Impact Assessment Study of the Project, TA 2154 Inception Report, minutes of the management review meeting on the Project, minutes of the Board meeting on the proposed loan. 8 ADB Project Completion Report. Project Administration Instructions. PAI No. 6.07A. Manila.

12 REGIONAL DEPARTMENT S RESPONSE TO THE PROJECT COMPLETION REPORT VALIDATION REPORT On 17 August 2009, the Independent Evaluation Department 1 (IED) circulated a draft project completion report validation report for interdepartmental comments. On 31 August 2009, IED received comments from the Agriculture, Environment, and Natural Resources Division (SEAE), Southeast Asia Department. IED provided clarifications for the validation's ratings. Based on these clarifications, SEAE accepted the validation's relevant rating as opposed to the PCR's highly relevant rating. 1 The Independent Evaluation Department was named the Operations Evaluation Department, or OED, until December 2008.