TERMS OF REFERENCE. Integration of Climate Change Risks and Resilience into Forestry Management in Samoa (ICCRIFS) Project Mid Term Review

Transcription

1 TERMS OF REFERENCE Integration of Climate Change Risks and Resilience into Forestry Management in Samoa (ICCRIFS) Project Mid Term Review UNDP Samoa Multi-country Office (MCO) seeks to hire a consultant to conduct a Mid-Term Evaluation of the UNDP/GEF Integration of Climate Change Risks and Resilience into Forestry Management in Samoa (ICCRIFS) Project. Curriculum vitae (CVs) and related experiences of the consultant must form part of the proposal. Post title: ICCRIFS project Midterm Evaluation Consultant Agency/Project: UNDP/GEF ICCRIFS Project Countries : Samoa Duration of contract: 20 days of work Expected starting date 21 November Mode of application: All candidates must apply through the UNDP Multi-country office in Samoa. Applications must be sent via to registry.ws@undp.org. All applications should include updated curriculum vitae of the consultant, as well as a technical and financial bid. 1. INTRODUCTION: The Monitoring and Evaluation (M&E) policy of the UNDP/GEF at the project level, has four objectives including to: i) monitor and evaluate results and impacts; ii) provide a basis for decision making on necessary amendments and improvements; iii) promote accountability for resource use; and iv) document, provide feedback on, and disseminate lessons learned. A mix of tools is used to ensure effective Project Monitoring and Evaluation (M&E). These might be applied continuously throughout the lifetime of the project e.g. periodic monitoring of indicators through the annual Project Implementation Reports (PIR), Project Steering Committee meetings or as specific and timebound exercises such as Mid-Term Reviews (MTR), Audit Reports and Final 1

2 Evaluations (FE). In accordance with UNDP/GEF policies and procedures, all projects are with exception of the preparatory grants mandated to conduct midterm and final evaluations. The evaluation is responsive to GEF Council decisions on transparency and better access of information during the implementation. Mid-term evaluations are intended to identify project design problems and to recommend corrective measures. They are to be conducted by an independent evaluator not associated with the implementation of the project at any stage. 2. Project background Samoa is experiencing increasing climate change-induced damage to human and economic development in key sectors, with adverse effects already experienced by its village communities. There is a high risk that further climatechange related damage will strongly impact on the country s economy, social infrastructure and natural environments, and significantly affect the livelihoods of rural communities. The combined effects of sea level rise, increased frequency and intensity of tropical cyclones, coastal erosion, disease impacts on crops, drought, spread of invasive species and reductions of fresh water supply seriously undermine Samoa s forestry and agro-forestry resource base and the related livelihood opportunities of its communities. Climate change exacerbates current environmental pressures on forestry resources, due to clearing and encroachment in native upland forestry areas, unsustainable landuse practices in lowland agro-forestry areas, couples with lack of understating and knowledge of government institutions and rural communities on current and potential impacts of climate change on forestry resources and effective adaptation options. Through this project, the Government of Samoa (GoS) will strengthen institutional capacities to systematically identify and address the climate changedriven risks for the management of native forests and agroforestry areas, in order to increase the resilience of rural communities and protect their livelihoods from dynamic climate-related damage, pursuant to the attainment of Samoa s Millennium Development Goals (MDGs). Project components: The ICCRIFS project consists of three major components: 2

3 1. Climate change risks and resilience are integrated into forestry policy frameworks Revised national forestry policy and plan through enhanced institutional coordination Development of forestry-tailored climate early warning and information systems Development of a Forest Fires Prevention Strategy and Manual Training of government officers on climate risk analysis, adaptive policies and planning techniques 2. Climate resilient agro-forestry and forestry techniques are demonstrated in lowland agroforestry and upland native forest areas Detailed vulnerability assessments in native upland forest ecosystems and lowland agroforestry areas and preparation of site-specific adaptation plans using climate early warning and information systems Rehabilitation of degraded native forestry areas using climate-resilient native species, introduction of climate resilient crops and cultivation techniques, climate-sensitive grazing land management techniques and training of farmers Promote linkages with sustainable livelihood support mechanisms, value added produce and products Capacity and coordination strengthening of national and community stakeholders 3. Project knowledge and lessons learned are captured, analysed and disseminated Lessons learned and best practices are generated and shared between local communities, and national and regional stakeholders through appropriate mechanisms Feeding of on-the-ground experience to inform policy mainstreaming processes Development of a project communication and awareness raising strategy involving range of media Project experience in forestry adaptation is transmitted to education institutions to incorporate knowledge generated in training materials, curricula and school programs 3

4 These components will ensure to enhance overall institutional capacity of the Forestry Division and institutional coordination with related Ministries and Divisions, through revising existing forestry policy frameworks, developing and building capacity on planning and decision making tools integrating climate risks. These measures are aiming at creating an enabling environment to support the implementation of the on-the-ground adaptation interventions under Outcome 2, develop decision making tools covering the country s entire forest areas, policy frameworks and related institutional capacity that will facilitate the upscaling and replication of climate-resilient forestry practices nationwide. 3. GENERAL OBJECTIVES OF THE EVALUTION: The mid-term evaluation of the UNDP/GEF ICCRIFS is initiated by the UNDP Samoa Office and it is being undertaken in accordance with the UNDP/GEF Project Monitoring and Evaluation Policy - edures.html. The principal purpose of the mid-term evaluation is to assess the project s implementation results and impacts as required by the UNDP/GEF Monitoring and Evaluation Policy. It is also mandatory to evaluate and review any UNDP project of the magnitude of USD 1 million or more, at mid-term and when the assistance is about to phase out called final evaluation. 4. PROJECT PERFORMANCE 4.1 OBJECTIVES OF THE MID-TERM EVALUATION: The mid-term evaluation is intended to provide a comprehensive overall assessment at mid-term of the project and provides an opportunity to critically assess administrative and technical strategic issues and constraints. The evaluation should provide recommendations for strategies, approaches and/or activities to improve the potential of the project to achieve expected outcomes and meet objectives within the project timeframe. Findings of this evaluation will be incorporated as recommendations for enhanced implementation of the current project phase as well as for the design of the subsequent project phases. The purpose of the Mid-Term Evaluation is: a. To assess overall performance against the project objective and outcomes as set out in Project Document and other related documents (this includes assess whether the project design is clear, logical and commensurate with project time-line and resources available); 4

5 b. To assess Project outputs to date and review planned strategies and plans for achieving the overall outcomes of the Project within the timeframe c. To assess the substantive effectiveness and efficiency of the Project d. To assess Project relevance to national priorities including the extent to which the project is a meaningful response to Climate Change adaptation needs in Samoa with the potential for significant opportunity to reduce vulnerabilities and enhance resilience of communities in the country; To critically analyze the implementation and management arrangements of the Project To provide guidance for the future Project activities and, if necessary, for the implementation and management arrangements. To list and document initial lessons concerning Project design, implementation and management In particular, this evaluation will assess progress in establishing the baseline, achievement of the outcomes and outputs, and identifying any difficulties in project implementation and their causes, and recommend corrective courses of action. Project performance will be measured based on the quantitative and qualitative indicators defined in the Logical Framework and the Results Framework of the Project Document. The Report of the Mid-Term Evaluation will be stand-alone document that substantiates its recommendations and conclusions. The evaluation will in particular assess: (1) Project Design review the original project intervention strategy including objectives, outcomes, and outputs and assess quality of the design and delivery of planned outcomes. The review should also assess the conceptualization, design, effectiveness, relevance and implementation feasibility of the project. The review should also include the updated logical framework. (2) Project Progress and Impact assess the achievements of the ICCRIFS Project to date against the original objectives, outcomes, outputs and activities using the indicators as defined in the logical framework contained in the project document as well as any valid amendments made thereafter. Achievements should be measured against the indicators as described in the log frame. 5

6 (3) Project Implementation assess: a) Project management arrangements, i.e., effectiveness of the UNDP (HQ, Regional Centre and Country Office), the Executing Agency for the project (MNRE) and the Project Management Unit (ICCRIFS PMU), and Implementing Partner arrangements at the national level; b) Quality and timeliness of delivering outputs and activities; c) Financial situation (i.e., budget and expenditure status). Financial audits were done and consultants have access to all audit reports; d) Cooperation among partners including but not limited to: GEF-supported projects, UNDP, Government counterparts; as well as those listed in the project document in the stakeholder participation plan as project cofinanciers; e) Responsiveness of project management to adapt and implement changes in project execution, based on partner and stakeholder feedback; Based on the above points, the evaluation should provide a document of approximately 50 pages indicating what project activities, outputs, outcomes and impacts have been achieved to date, and specifically: (1) Assess the extent of the progress which the ICCRIFS Project has made to achieve its objectives and where gaps are evident; (2) Draw lessons from the experiences of the ICCRIFS Project, in particular those elements that have worked well and those that have not, requiring adjustments and; (3) Provide recommendations to strengthen the effectiveness, efficiency, impact, implementation, execution and sustainability of the ICCRIFS Project. 4.2 SCOPE OF THE EVALUATION: While the specific issues of concern are listed in the following paragraphs, a reference to the UNDP programming manual and UNDP/GEF guidelines to conduct mid-term evaluations should be made for addressing the issues not covered below. The evaluation will include ratings on the following two aspects: (1) Sustainability and (2) Outcome/Achievement of objectives (the extent to which the projects immediate and development objectives were achieved). The consultant should 6

7 provide ratings for three of the criteria included in the Mid-Term Evaluations: (1) Implementation Approach; (2) Stakeholder Participation/Public Involvement; and (3) Monitoring and Evaluation. The ratings will be: Highly Satisfactory, Satisfactory, Marginally Satisfactory, Unsatisfactory, and N/A. 4.2a) Project Conceptualization/Design: 1. whether the problem the project is addressing is clearly identified and the approach soundly conceived. 2. whether the target beneficiaries and end-users of the results of the project are clearly identified. 3. whether the outcomes and outputs of the project were stated explicitly and precisely in verifiable terms with observable success indicators. 4. whether the relationship between outcomes, outputs, activities and inputs of the project are logically articulated. 5. whether the project started with a well-prepared work-plan and reasons, if any, for deviations. 4.2b) Project Relevance: 1. whether the project is relevant to the development priorities of Samoa and to increase the adaptation capacity and resiliency to climate change 2. given the outcomes of the project whether appropriate institutions have been involved. 4.2c) Project Implementation: The consultant will examine the quality and timeliness in regard to: 1. the delivery of inputs specified in the project document, including selection of sub-projects, institutional arrangements, interest of beneficiaries, the scheduling and actual implementation. 2. the fulfillment of the success criteria as outlined in the project document. 3. the responsiveness of the project management to significant changes in the environment in which the project functions (both facilitating or impeding project implementation). 4. lessons from other relevant projects if incorporated in the project implementation. 5. the monitoring and backstopping of the project as expected by UNDP. 6. the delivery of Government counterpart inputs in terms of personnel, premises and indigenous equipment. 7

8 4.2d) Project Performance: 1. whether the management arrangements of the project were appropriate. The evaluation will assess the effectiveness of project management and administration functions at the national and regional levels, involving National Coordinators and the Project Management Unit, the Execution Agency functions and oversight functions and quality assurance role of UNDP (at the country office, regional level and HQ). In the context of the above, this MTE should assess the following: o National Project Management Unit - Nature and quality of support provided by National Coordinators and team in the 3 areas of activity for the achievement of outputs - The effectiveness of the monitoring mechanisms currently employed by the project manager in monitoring on a day to day basis the progress in achieving project outputs; - Evaluation of the extent to which the PMU has strategically managed the programme of including sequencing of project activities, and quality and timeliness of programme actions on the ground o UNDP - Quality assurance function of UNDP (at the country, regional and HQ level) In addition, the review will consider; - Administrative, operational and/or technical problems and constraints that influenced the effective implementation of the project and present recommendations for any necessary operational changes; and - Financial management of the project, including the balance between expenditures on administrative and overhead charges in relation to those on the achievement of substantive outputs. - The role of the Project Steering Committee, as well as technical support mechanisms and partnerships established. 2. whether the project resources (financial, physical and manpower) were adequate in terms of both quantity and quality. 8

9 3. whether the project resources are used effectively to produce planned results. 4. whether the project is cost-effective compared to similar interventions. 5. whether the technologies selected (any innovations adopted, if any) were suitable 4.2e) Results/Success of the project applied to each Specific Outcomes and Outputs: The overall outputs and their meaning are as defined in the project document that should form the main basis for this evaluation. In addition to the mid-term targets in the logical framework, the details of the specific project impact to be provided are: 1. what are the major achievements of the project vis-à-vis its objectives, outcomes and outputs. 2. what are the potential areas for project success? Please explain in detail in terms of impact, sustainability of results and contribution to capacity development. 3. what major issues and problems affected the implementation of the project, and what factors could have resolved them. 4. given an opportunity, what actions the evaluation team members would recommend to ensure that this potential for success translates into actual success. 5. level of institutional networking achieved and capacity development of key partners, if being done in a structured manner at different stages from inception to implementation. 6. environmental impacts (positive and negative) and remedial actions taken, if relevant. 7. social impacts, including impact on the lives of women at each demonstration site. 8. any underlying factors, beyond control, that are influencing the outcome of the project. Project implementation a. An evaluation of the project by reviewing major components undertaken to date and a determination of progress towards achievement of its overall objectives; 9

10 b. An evaluation of project performance in relation to the indicators, assumptions and risks specified in the logical framework matrix and the project document. c. An assessment of the scope, quality and significance of the project outputs produced to date, in relation to expected results and impacts on the ground ie a safer and more resilient community and adaptation mainstreamed into development planning processes. d. An evaluation of the extent to which project implementation has benefitted from cutting edge knowledge on adaptation, science and technology and best practices from around the world. e. An evaluation of the benefits for communities derived from the implementation of the project. An analysis of the perception of communities will additionally provide qualitative information about benefits to communities 4.3 METHODOLOGY/EVALUATION APPROACH: With the aim of having an objective and independent evaluation, the successful Contractor is expected to conduct the project evaluation according to international criteria and professional norms and standards as adopted by the UN Evaluation Group (UNEG). The Standards for Evaluation in the UN System is available here: aluation_annex_iii.pdf/uneg%2bstandards%2bfor%2bevaluation_annex%2bii I.pdf. UNDP/GEF Project Monitoring and Evaluation Policy must be strictly adhered to. The Mid-Term Review will be conducted in a participatory manner working on the basis that its essential objective is to assess the performance of the project and to document both successes and failures and to provide a basis for improvement during the next half of the project s lifespan. Methods will involve a wide ranging desk review of documentation, stakeholder consultations through missions, interviews, phone and exchanges and field visits to project sites: a. Desk review will include but shall not be limited to: 10

11 Project document, outputs, monitoring reports such as the Project Document, Project Inception Report, Work Plans, Quarterly Progress Reports, documents and proceedings of Project Steering Committee meetings, PIRs, financial reports, Reports from technical training and operational support workshops, meetings and missions Datasets, management and action plans, publications, audiovisual and knowledge materials, other reports; b. Consultations with key project stakeholders, including: Interviews conducted during missions with UNDP staff involved (CO, RTA, HQ), ICCRIFS PMU (including Project Coordinator and Sub-coordinators), Project Steering Committee members and partners Consultations and/or interviews with relevant stakeholders involved, including governments representatives, local communities, NGO s, private sector, donors, other UN agencies and organizations. c. Field visits to project sites to observe activity results and consult with local stakeholders. 6. CONSULTATIONS: The consultant is open to consult all reports, files, manuals, guidelines and resource people they feel essential, to make the most effective findings, conclusions and recommendations. The mission will maintain close liaison with the UNDP MCO in Samoa, UNDP/GEF Regional Technical Advisor and the ICCRIFS PMU. 7. REPORTING: The consultant will report directly to UNDP MCO Samoa, UNDP/GEF RCU (Bangkok). The consultant shall work in close collaboration with the ICCRIFS PMU. The consultant will prepare and submit the draft report of the evaluation to UNDP and MNRE. A presentation and debriefing of the report to UNDP, the project beneficiaries (country representatives), and implementing MNRE agencies). The reporting schedule will be finalized during the inception meeting between the evaluation team and key stakeholders. 11

12 8. Expected Deliverables of the Assignment The Contractor is expected to produce the following main outputs: a) debriefing to UNDP/MNRE at the end of the consultation period in Samoa; b) submission of the draft final report c) final report (which has benefitted from views of UNDP/PMU etc) Draft Mid-term Evaluation Report a) In order to highlight the major findings of the evaluation, the Contractor shall prepare debriefing notes, based on preliminary findings, conclusions and recommendations from the consultations, phone and interviews and consultations b) Discuss preliminary draft debriefing notes with appropriate personnel from UNDP Samoa MCO, GoS (MNRE & MoF) and UNDP/GEF. Prepare minutes of the meetings and submit for comment and approval of the participating parties. Mid-term Evaluation Report Prepare a detailed evaluation report covering Clause 5 above - Scope of the Midterm review with detailed attention to lessons learned and recommendations according to the suggested outline as below. Recommended structure of the Mid-Term Evaluation Report The main final output of the evaluation will be an independent and comprehensive MTE report with annexes as needed. The minimum requirements for the content of the final report are: 1. Executive Summary a. Brief description of project b. Context and purpose of the evaluation c. Main conclusions, recommendations and lessons learned 2. Introduction a. Purpose of evaluation b. Key issues addressed c. Methodology of the evaluation 12

13 d. Structure of the evaluation 3. The project and its development context a. Problems that the project seek to address b. Immediate and development objectives of the project c. Planned outputs and sub-outputs d. Results expected 4. Findings and Conclusions a. Implementation Financial planning Monitoring and evaluation Execution, implementation modalities (including suitability of original project design and adjustments made during the project) Project management. Specifically analyzing the performance and value added of PMU and UNDP and the MCO Coordination and operational issues b. Results Attainment of planned objectives & outcomes Financial delivery Sustainability of impacts (including policy impact and evidence of mainstreaming disaster risk management and Climate Change adaptation into sustainable development strategies and programmes) Quantitative and qualitative results in reducing vulnerabilities and enhancing resilience of communities Contribution to national capacity development c. Lessons learned 5. Recommendations a. Corrective actions for the design, implementation, monitoring and evaluation of the project b. Actions to follow up or reinforce initial benefits from the project c. Proposals for future directions underlining main objectives 13

14 6. Annexes, including i) TOR; ii) mission itinerary; iii) list of persons interviewed; iv) list of documents reviewed; v) questionnaire other instruments used and summary of results; vi) Co-financing and Leveraged Resources and vi) comments by stakeholders (only in case of discrepancies with evaluation findings and conclusions. If there are discrepancies between the impressions and findings of the Successful Contractor and the key project partners these must be explained in an annex attached to the final report). 9. Qualifications of review expert A consultant (drawn from International/regional and national pools) with the following qualifications shall be engaged to undertake the evaluation working in accordance with the planned schedule. The consultant has the overall responsibility for collecting of documents, completing the desk review prior to the country mission and for submitting the interim and final documents and reports. The consultant will sign an agreement with UNDP and will be bound by its terms and conditions set in the agreement. Qualifications: 1. International/regional consultants with academic and professional competencies in fields related to Climate Change Adaptation/Disaster Management and Forestry. The consultant should have a minimum of 7 years of relevant experience; 2. Substantive experience in socio-economic analysis and evaluation of community involvement 3. Substantive experience in reviewing and evaluating similar projects, preferably those involving UNDP/GEF or other United Nations development agencies or major donors; 4. Excellent English writing and communication and analytical skills. The team must be equipped with own computing equipment; 5. Demonstrate ability to assess complex situations, succinctly distilling critical issues, and drawing forward-looking conclusions and recommendations; 6. Highly knowledgeable of participatory monitoring and evaluation processes, and experience in evaluation of technical assistance projects with major donor agencies; 7. Ability and experience to lead multi disciplinary and national teams, and deliver quality reports within a given timeframe; 14

15 8. Familiarity with the challenges developing countries face in adapting to climate change and approaches they are taking; 9. Familiarity/experience with the Pacific countries 10. Excellent in human relations, coordination, planning and team work and 11. Willingness to travel to and work in Samoa if residing overseas. 10. Proposed schedule 20 days of work. This will include in-country mission to Samoa, telephone interviews as well as a desk review (prior to the country mission) and drafting and finalization of the report (following the country mission). 11. Estimated costs and payment schedule Along with their technical bids, applicants for the consultancy shall prepare financial bids which will be presented in a separate sealed envelope. The most financially competitive bid from applicants adjudged to possess the technical capacity to undertake the evaluations shall be awarded the contract. Payment schedule 30 % upon de-briefing note following country missions and consultations 35% upon submission of draft MTR 35% upon approval of final report 12. The consultant will be supported with the following inputs UNDP Samoa MCO a) Organize the consultancy and serve as the UNDP/GEF contracting entity b) Liaise with the successful Contractor to set up stakeholder interviews; c) Assist with logistics concerning missions including meetings; d) Ensure the timely provision of payments as per contract with the successful Contractor e) Provide relevant background information and documents to the successful Contractor f) Provide comments on all draft deliverables; and g) Participate in meetings as appropriate UNDP/GEF 15

16 a) Provide guidance on relevant UNDP/GEF procedures, policies and practices b) Provide relevant background information including copies of relevant documentary sources; c) Provide input on draft documents d) Participate in meetings as appropriate ICCRIFS PMU a) Provide relevant background information and documents to the successful Contractor b) Assist with logistics concerning mission including meetings c) Provide feedback on draft deliverables d) Participate in meetings as appropriate 13. Submission of proposals The proposed overall technical approach, including specific mix of methodological tools to be applied as part of the evaluation must be included in: a. A technical bid outlining how the evaluator proposes to conduct the assessment b. A financial bid that outlines the evaluator s proposed cost for conducting the evaluation. The successful Contractor will be expected to submit a note which provides further details on the methodology that will be applied during the evaluation. Subsequently these will be discussed and agreed upon between the successful Contractor and UNDP evaluation. The applicants will present: a) CV of the proposed consultant and letter of justification of his/her capacity to develop the tasks b) Financial and technical proposal 14. Application Process 16

17 Qualified candidates are requested to apply by the 4th of November 2013 by sending their application packages to with the subject line Integration of Climate Change Risks and Resilience into Forestry Management in Samoa. The application should contain: Letter of interest and availability using the standard template. plate_confirmation_interest_submission_financial_proposal.pdf Cover letter explaining why you are the most suitable candidate for the advertised position and a brief methodology on how you will approach and conduct the work (if applicable). Filled P11 form including past experience in similar projects and contact details of referees, please upload the P11 instead of your CV. (a template can be downloaded from Financial Proposal* - specifying a total lump sum amount for the tasks specified in this announcement. The financial proposal shall include a breakdown of this lump sum amount (number of anticipated working days in home office and on mission, travel international and local, per diems and any other possible costs). DISCLOSURE Although the consultant is free to discuss with the authorities on anything relevant to the assignment, under the terms of reference, the consultant is not authorized to make any commitments on behalf of UNDP, the Governments of the countries participating in the UNDP/GEF project). 17

18 Annex 2: CONTACT POINTS UNDP Samoa Country Office Ms Marta Moneo UNDP Asia-Pacific Regional Center Mr. Gabor Vereczi ICCRIFS PMU Moafanua Tolusina Pouli UNDP in Samoa Apia Samoa Tel: /1/2 Fax: