ACTIVITY MONITORING AND EVALUATION PLAN (AMEP)

Transcription

1 ACTIVITY MONITORING AND EVALUATION PLAN (AMEP) SECOND EDITION, NOVEMBER 30, 2017 This publication was produced for review by the United States Agency for International Development. It was prepared by Tetra Tech ARD.

2 This publication was prepared for review by the United States Agency for International Development under Contract # AID-497-TO It was produced by Tetra Tech ARD. The period of this contract is from July 2015 to July Implemented by: Tetra Tech ARD P.O. Box 1397 Burlington, VT 05402

3 ACTIVITY MONITORING AND EVALUATION PLAN (AMEP) SECOND EDITION, NOVEMBER 30, 2017 DISCLAIMER This publication is made possible by the support of the American People through the United States Agency for International Development (USAID). The contents of this publication are the sole responsibility of Tetra Tech ARD and do not necessarily reflect the views of USAID or the United States Government. USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 1

4 TABLE OF CONTENTS ACRONYMS Introduction Project Background AMEP Development LESTARI M&E Frameworks Situation Model LESTARI Results Framework and Theories of Change Logical Framework and Performance Indicator Tracking Table (PITT) Plan for Activity Monitoring and Evaluation LESTARI Learning Loop: Data Collection, Storage, Quality Assurance, Analysis, and Utilization Internal Evaluations Plan Schedule for Performance Monitoring Tasks Monitoring and evaluation structures, functions, and capabilities Annex I: Performance Indicator Reference Sheets (PIRS) Annex II: Results Chains Annex III: Data Quality Assessment Checklists Annex IV: LANDSCAPE SITUATION MODELS USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 2

5 ACRONYMS ADS AMEP APBD APL BAPPEDA BAPPENAS BAU BIG BMPs C4J CAs CCAs CIFOR CMMPs CMP CO 2 CO 2e COP CSOs DAK DCOP DNPI DQ Automated Directives System Activity Monitoring and Evaluation Plan Anggaran Pendapatan dan Belanja Daerah (Local Budget Revenues and Expenditures) Area Penggunaan Lain (Other Uses Zones) Badan Perencanaan Pembangunan Daerah (Regional Development Planning Agency) Badan Perencanaan Pembangunan Nasional (National Development Planning Agency) Business as Usual Badan Informasi Geospasial (Geospatial Information Agency) Best Management Practices Changes for Justice Conservation Areas Community Conservation Agreements Center for International Forestry Research Conservation Management and Monitoring Plans Conservation Measures Partnership Carbon Dioxide CO 2 equivalent Chief of Party Civil Society Organizations Dana Alokasi Khusus (Special Allocation Fund) Deputy Chief of Party Dewan Nasional Perubahan Iklim (National Council on Climate Change) Data Quality USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 3

6 DQAs FACTS FMU FREL FY GAP GHG GIS GOI GPS Ha HCS HCV HO HPH ICCTF ICRAF IFACS IPCC IR IUCN KPH-L KPH-P KPA KR KRP Data Quality Assessments Foreign Assistance Coordination and Tracking Systems Forest Management Unit (or KPH) Forest Reference Emission Level Fiscal Year Good Agricultural Practice Greenhouse Gas Geographic Information System Government of Indonesia Global Positioning System Hectare High Carbon Stock High Conservation Value Home Office Hak Pengusahaan Hutan (Forest Concession) Indonesia Climate Change Trust Fund International Center for Research in Agroforestry/World Agroforestry Center Indonesian Forest and Climate Support Intergovernmental Panel on Climate Change Intermediate Result International Union for Conservation Nature Kesatuan Pengelolaan Hutan Lindung (Protected Forest Management Unit) Kesatuan Pengelolaan Hutan Produksi (Production Forest Management Unit) Kawasan Pelestarian Alam (Nature Conservation Area) Key Result Kebijakan Rencana Program (Policy Planning Program) USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 4

7 KSA LBA LCPs LEDS LOP LSM LULUCF M&E ME & L METT MIS MOEF MOU MRV MSF MSI NGO NRM PA PEA PES PHPL PIRS PITT PPPs QCA Kawasan Suaka Alam (Nature Reserve Area) Landscape Baseline Analysis Landscape Conservation Plans Low Emissions Development Strategy Life of Project Landscape Situation Model Land Use, Land Use Change, and Forestry Monitoring & Evaluation Monitoring Evaluation & Learning Management Effectiveness Tracking Tool Management Information System Ministry of Environment and Forestry Memorandum of Understanding Measuring Reporting Verification Multi Stakeholder Forum Multi Stakeholder Initiative Non-Governmental Organization Natural Resource Management Protected Areas Political Economy Analysis Payment for Environmental Services Pengelolaan Hutan Produksi Lestari (Sustainable Production Forest Management) Performance Indicator Reference Sheets Performance Indicator Tracking Table Public-Private Partnerships Qualitative Comparative Analysis USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 5

8 RAN GRK REDD+ RF REL RIL-C RKPD RPJM RPHJP RPJPd RTRWK RTRWN RTRWP SEAs SDI SMART SOP SOW STI TFA ToC UN-REDD USAID USG WCPA Rencana Aksi Nasional Gas Rumah Kaca (National Action Plan for Greenhouse Gas Emission) Reducing Emissions from Deforestation and Forest Degradation (including conservation, sustainable management of forests, and enhancement of forest carbon stocks) Results Framework Reference Emission Level Reduced Impact Logging-Carbon Rencana Kerja Program Daerah (Annual Local Government Development Plan) Rencana Pembangunan Jangka Menengah (Mid-Term Development Plan) Long Term Term Forest Management Planning Annual Forest Management Planning Rencana Tata Ruang Wilayah Kabupaten (District Spatial Plan) Rencana Tata Ruang Wilayah Nasional (National Spatial Plan) Rencana Tata Ruang Wilayah Provinsi (Provincial Spatial Plan) Strategic Environmental Assessments Spatial Data Infrastructure Spatial Monitoring and Reporting Tool Standard Operating Procedure Scope of Work Sustainable Travel International Tropical Forest Alliance Theory of Change United Nations-Reducing Emissions from Deforestation and Forest Degradation United States Agency for International Development United State Government World Commission of Protected Area USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 6

9 WCS WWF Wildlife Conservation Society World Wildlife Fund USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 7

10 1. INTRODUCTION 1.1 Project Background USAID LESTARI supports the Government of Indonesia to reduce greenhouse gas (GHG) emissions and conserve biodiversity in carbon rich and biologically significant forest and mangrove ecosystems. Built on the strong foundation of USAID s IFACS project, LESTARI applies a landscape approach to reduce GHG emissions, integrating forest and peatland conservation with low emissions development (LEDS) on other, already degraded land. This is achieved through improved land use governance, enhanced protected areas management and protection of key species, sustainable private sector and industry practices, and expanded constituencies for conservation among various stakeholders. LESTARI is implemented under the leadership of Tetra Tech ARD and a consortium of partners including WWF-Indonesia, Winrock International, Wildlife Conservation Society (WCS), Blue Forests, Yayasan Sahabat Cipta, PT Hydro South Pole Carbon, Sustainable Travel International (STI), Michigan State University, and the FIELD Foundation. LESTARI runs from July 2015 through July LESTARI activities are targeted in six strategic landscapes on three of Indonesia s largest islands, where primary forest cover remains most intact and carbon stocks are greatest. In northern Sumatra, the Leuser Landscape comprises significant portions of Aceh Selatan, Gayo Lues, Aceh Tenggara, and Aceh Barat Daya districts, and includes the Aceh portion of Leuser National Park and Singkil Wildlife Sanctuary. In Central Kalimantan, LESTARI works in the Katingan-Kahayan Landscape, comprising Pulang Pisau, Katingan, and Gunung Mas districts; Palangkaraya municipality; and Sebangau and Bukit Baka Bukit Raya National Parks. LESTARI also works in four landscapes in Papua. Sarmi and Cyclops Landscapes are located along the northern coast and comprise Sarmi district as well as Jayapura district and municipality. The Lorentz Lowlands Landscape, comprising Mimika and Asmat districts plus a large portion of Lorentz National Park, and the Mappi-Bouven Digoel Landscape are located along Papua s southern coast. LESTARI is managed from its headquarters in Jakarta, with offices in each landscape as well as the provincial capitals of Aceh, Central Kalimantan, and Papua. Overall Results of LESTARI are: Goals: Key Result 1: At least 41% of total CO2-equivalent emissions reduced from land use, land use change and deforestation averaged across all landscapes within the project scope; Key Result 2: At least 8.42 Million hectares of primary or secondary forest (and peatland), including orangutan habitat, under improved management; USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 8

11 IR 1: Improved Forest Management: Key Result 3: Management of at least six Conservation Areas (CAs) improved, resulting in the conservation of valuable orangutan and other key species habitat, and the reduction in poaching of threatened and endemic species; Key Result 4: At least ten Public-Private Partnerships (PPPs) promoting low-emissions conservation oriented development established; Key Result 5: Funding leveraged from public and private sources, representing coinvestment in project outcomes; IR 2: Improved Land Use Governance: Key Result 6: Increased commitment of key private sector, government, and community stakeholders regarding the positive benefits of conservation and sustainable use of forests and the species they encompass; Key Result 7: Policies, laws, regulations, and procedures in support of low emission development and forest conservation and management increased, promulgated, and enforced at all levels; and Key Result 8: Model(s) for successful integration of district, provincial, and national low emissions development and forest conservation strategies developed and shared at all levels of government and with other key stakeholders. To achieve these results, LESTARI will implement 8 Strategic Approaches in three complementary technical thematic areas: 1. Forest & Land Use Governance & Advocacy; 2. Conservation Co-Management; and 3. Private Sector Engagement Following a Landscape Approach 1 to implementation, each strategic approach will be nuanced to address the biophysical, social, political, and cultural context of each landscape. LESTARI will be implemented primarily with congressionally mandated Biodiversity Conservation funds. All Biodiversity Earmark-funded activities must align with U.S. Agency for International Development s (USAID) Biodiversity Policy and Biodiversity Code. The Code requires that funded activities must a) have an explicit biodiversity objective; b) be identified based on an analysis of drivers and threats to biodiversity and a corresponding theory of change; c) must have the intent to positively impact biodiversity in biologically significant areas; and d) must monitor indicators associated with each stated theory of change for biodiversity conservation results. To support learning, adaptive management, and to comply with the Biodiversity Code and ADS 203, Tetra Tech ARD has developed the LESTARI Activity Monitoring and Evaluation Plan (AMEP) to provide standards and guidelines for monitoring, evaluating, and communicating our progress toward achievement of the biodiversity and emissions targets, project results, and higher level USAID and GOI goals. 1 Further information about Landscape Approach, please refer to LESTARI Landscape Baseline Analysis (LBA) document. USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 9

12 1.2. AMEP Development The LESTARI AMEP is a living document, designed to ensure accountability and learning which will be used to facilitate and promote evidence-based decision making. The document outlines Monitoring & Evaluation (M&E) systems, protocols, and data collection methodologies. Throughout implementation, this framework will ensure continuous learning while improving coordination and collaboration to track, evaluate, and articulate results and impacts. The LESTARI M&E approach also ensures the adaptive management of project activities through the use of sound, reliable, and timely data. Furthermore, the approach to performance management integrates the five Open Standards for the Practice of Conservation steps (hereafter, Open Standards). This enables the LESTARI AMEP to be used as not only a monitoring tool but also a management tool that will be referenced by staff, and used for training and capacity building to ensure comprehension and compliance with sound performance management. The design of LESTARI AMEP is further founded upon lessons learned from the IFACS M&E system, and key recommendations from the IFACS Final Assessment. A key recommendation was the need to incorporate a Theory of Change (ToC) approach in M&E design. The assessment found that key critical assumptions about the causal linkages between activities and higher level results had not been considered due to limited use of program theory. Following this recommendation, the LESTARI AMEP incorporates a theory of change approach to define all building blocks required to bring about positive change whereby multistakeholders interests and actions are better aligned with sustainable outcomes. LESTARI s theories of change not only illustrate the causal linkages between project interventions, but also how these interventions address drivers to deforestation to reduce high level threats to ecosystems, species, and human well-being targets. In addition, performance indicators and key impact evaluation questions have been selected to monitor progress toward achievement of outcomes and impacts along each Results Chain. Theories of Change will also serve as a project management tool to review critical assumptions driving LESTARI strategic approaches, facilitate early identification of technical challenges, manage stakeholder expectations, and serve as a framework for gathering evidence. This document outlines the detailed blueprint designed to monitor and evaluate the project s performance. First, the LESTARI Situation Model, Results Framework, Strategic Approaches, and Theories of Change are described. This is followed by outlines project indicators and targets which further detailed in Performance Indicator Reference Sheets (PIRS) formatted to USAID standards. PIRS will provide identification of staff responsible for different tasks, as well as timelines. The document then describes the LESTARI Learning Loop which explains how the LESTARI M&E system be integrated into project management and how project impact will be routinely monitored and assessed. Finally, the operationalization of M&E system through the establishment of M&E Team and the development of Management Information System (MIS). The structure and content of this AMEP is congruent with ADS 203 and includes both USAID Foreign Assistance Coordination Tracking Systems (FACTS) and custom indicators for monitor performance and evaluating the theories of change. USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 10

13 This second edition of the LESTARI AMEP provides adjustments to indicators and targets in a manner that better guides the LESTARI team and partners toward contract deliverables and outcomes. The need for these changes first emerged at the end of Year 1, during the annual reporting cycle. The LESTARI team focused on understanding and adjusting these targets and indicators during the internal mid-term assessment conducted in February and March 2017, and provided this revised AMEP to USAID/Indonesia on September 30, Significant changes to the AMEP include the addition of landscape situation models that more effectively reflect integrated landscape activities implementation, revised technical theme strategic approaches, and revised wording, milestones and targets for indicators 5, 6, 7, 8, 9, 11 and 15. These changes more accurately reflect LESTARI s progress towards contract deliverables and outcomes, and in no way reduce any of these contractual commitments. Finally, Targets are reported as Actuals through Year 2, and then as Targets for Years 3, 4 and 5. USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 11

14 2. LESTARI M&E FRAMEWORKS 2.1 Situation Model The LESTARI AMEP was developed following the Conservation Measures Partnership (CMP) Open Standards for Adaptive Management. Design began with identification of the project scope (geographic focus), conservation targets (key species and ecosystems, as well as human well-being targets linked to these species and ecosystems), direct and indirect threats, and drivers to be included in the LESTARI Situation Model. Upon completion of the Situation Model, theories of change were developed by analyzing how LESTARI interventions will address factors within each results chain to reduce key threats and lead to the conservation of priority targets and emissions reductions. In September 2015, a participatory M&E Workshop was held with LESTARI Staff, USAID representatives, and representatives from USAID s Measuring Impact project to review and further nuance the Theories of Change for inclusion in the AMEP. This process resulted in the identification of eight Strategic Approaches which will be implemented within three Technical Themes: Forest & Land Use Governance & Advocacy, Conservation Co-Management, and Private Sector Engagement. Throughout the course of the project, LESTARI will lead participatory reviews of theories of change with partners, staff across all landscapes, and USAID to assess the validity of initial assumptions and casual linkages, to refine project planning as necessary, to address gaps and maximize results, and to update the AMEP to support ongoing learning and adaptive management. Reviews will address key evaluation questions, including: To what extent have LESTARI interventions been effective in reducing biodiversity threats and carbon emissions in targeted areas? To what extent have LESTARI intervention been effective in improved land-use governance? To what extent has the project addressed barriers of political will and buy-in at the national and regional levels? To what extent has improved collaboration between government, private sector and community contributed to improved forest management? During the second year of the project (February-March 2017), those evaluation questions had been learned and tested through internal mid-term assessment and political economy analysis (PEA) process. The most important finding was in order to reduce threats, different interventions might be applied to different landscapes. Hence, the need to understand context and political economic situation in each LESTARI landscape became crucial. Draw on those assessments, LESTARI developed more focused landscape-level situation models as guidance for staff to select the most strategic approach to reduce threats applicable to their landscape. These are presented in Annex IV of the AMEP. USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 12

15 Figure 1. LESTARI Situation Model USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 13

16 Table 1 LESTARI Situation Analysis Problem Statement Threats, Drivers, and Opportunities As one of the richest biological countries in the world, Indonesia is considered mega diverse country, contributing, along with six other Asian countries, up to 70 percent of the globe s biological diversity. Indonesia has held Asia s largest and continuous track of tropical rain forests and 10% of the world s forests. Important terrestrial ecosystems include mangroves, lowland forests, montane forests, and peat/swamp forests, the protection of which is considered critical for conserving biodiversity and preventing or reducing land based carbon emissions. Healthy intact forest ecosystems are also an important safety net for local communities living nearby, presenting economic opportunities through forest dependent livelihoods that prevent their slide into extreme poverty and improving their resilience to climate and other external shocks. While at least 30 million people depend directly on Indonesia s forests and on the ecosystem services they provide, these ecosystems have been and continue to be under extreme threats. Indonesia s deforestation rate is well known to be among the highest in the world. It was reported million Ha of forest cover loss for period (Margono, 2014). Deforestation is especially critical in carbon rich peat and swamp forests of Sumatra, Kalimantan, and Papua. These forests are also the primary habitat for key species of global importance, particularly those listed on CITES Appendix I (especially rhinos, elephants, tigers, and orangutans) and endemics found in Papua. These key species are threatened by large-scale destruction of tropical forests, habitat loss, unsustainable harvesting, and trafficking. Deforestation, peatland degradation, and forest fire have put Indonesia among the top three largest emitters of greenhouse gases in the world. The emissions resulting from forest fires and degradation are five times greater than those ensuing from non-forestry emissions. A 2010 report suggests that 85% of Indonesia s greenhouse gas (GHG) emissions stem from land use activities with 37% due to deforestation and 27% due to peat fires (National Council on Climate Change, 2010). The most recent research on forest fire and emission it has resulted demonstrates that since the beginning of September 2015, Indonesia s fires have been emitting carbon at a rate of million tons per day, or more than the 14 million tons emitted on a daily basis by the whole American economy. Forest fires also destroy large areas that serve as habitat for biodiversity. Plants and animals are eliminated directly by the fires and those survive will diminish gradually. For example, the fires in caused the population of orangutans to decline by 33 percent on the island of Borneo (Rijksen et al.,2006). Over the last 20 years deforestation has been driven predominantly by agricultural expansion, especially of oil palm plantation. A history of weak governance combined with opportunistic private sector actors has created unsustainable economics based on natural resource exploitation. Land clearing for oil palm plantations is one of the key drivers of deforestation. Palm oil is also an important element of Indonesia s national development strategy, as it is the world s largest producer and exporter of crude palm oil. Oil palm plantation area has doubled in Indondesia in the past decade, and pressures for expansion continues as GOI has instituted targets for doubling CPO production to 40 million tons by Other commodity-driven pressures on land include unsustainable industrial scale logging and conversion for pulp, rubber, and coffee/cacao plantations. An inequitable system of land use decision-making is threatening forests, as smallholders that cannot compete for access with large private sector actors are forced to encroach onto forested areas. USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 14

17 In regards with key species, expansion of oil palm and pulpwood plantations in Sumatra was responsible for nearly two-thirds of the destruction of tiger habitat from 2009 to When rainforests are cleared, it becomes increasingly easier for poachers to track and kill tigers, which are considered prized trophies for any hunters. Used in traditional Chinese medicine and as ornaments, each Sumatran tiger can sell for up to $5,000, offering poachers a greater income than they would earn working in paddy fields or on rubber tree plantations. The World Wildlife Fund (WWF) estimates that orangutan populations have significantly declined from 230,000 nearly a century ago to 7,500 in Sumatra and 55,000 in Borneo. A significant factor in the decline of orangutan numbers is the fragmentation of their habitat, which makes them vulnerable to wildlife traffickers. 2 The LESTARI Situation Model and Analysis illustrate how deforestation and forest degradation in Indonesia is a result of a complex dynamic of political, economic and institutional drivers. Decision-making on land use involves a large array of laws and actors and the process is rarely transparent or consistent. Land use decisions are driven by economic incentives that prioritize extractive industries and agriculture over low emissions and conservation-oriented business models. However, enabling conditions for a shift to more equitable and sustainable forest and land use management are emerging after years of contradictory talk about forest clearing moratoriums while oil palm and industrial forest plantations rapidly expanded. The new President administration is taking clear, firm, and ambitious steps to improve forest and land use governance in a sustainable and equitable manner. Three new ministries have been formed to elevate and reinforce the goals of better governance of natural resources and the rural poor who depend upon them for most livelihoods: Environment and Forestry; Agrarian and Spatial Planning; and Villages, Development of Backward Regions, and Transmigration. The private sector is also demonstrating substantial leadership toward reducing large-scale deforestation. The US Ambassador recently facilitated the Palm Oil Pledge, signed by CEOs of Asian Agri, Cargill, Golden Agri Resources, Wilmar, and Indonesian Chamber of Commerce (KADIN). Reaching all the way to their third-party suppliers, the pledge aims to make palm oil supply chains more sustainable. Further, Sinar Mas Asia Pulp & Paper has committed to halt deforestation in their concessions and to conserve 1,000,000 hectares of High Conservation Value (HCV) and High Carbon Stock (HCS) landscapes in and around their operations across Sumatra and Kalimantan; it has undergone an independent audit of these areas by Rainforest Alliance. Increasingly, companies are demonstrating that good business integrates sustainable landscape and conservation management into their operations. Civil society, particularly forest-dependent communities, are increasingly advocating for more responsible land and forest governance. This is demonstrated in Central Kalimantan, where communities have become disenchanted with the empty promises of economic benefits from oil palm expansion while suffering from the omnipresent haze of land clearing and peatland fires. Forest-dependent communities across Indonesia are initiating actions to counter this situation mapping customary (adat) forests, integrating these maps into the OneMap initiative, and negotiating boundaries with concession holders and spatial planners. LESTARI aims to catalyze such positive change, and is strategically designed to do so. LESTARI targets a 41% GHG emissions reduction in its landscapes 2 USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 15

18 by The Government of Indonesia has also committed to reducing GHG emissions by 29% below the business as usual condition (baseline) by 2030 (previously 26% by 2020) unilaterally, and up to 41% below the baseline with bi-lateral or international assistance. LESTARI will work within the same time frame as the GoI commitments and has adopted the same targets of working towards 41% reduction of Baseline emissions by As such the project can effectively harmonize efforts with the GoI within the LESTARI landscapes. A thorough understanding of where LESTARI will contribute to the GoI targets is presented in the Landscape Baseline Analysis Plan (LBA). Historical deforestation and degradation in protected areas, production forest and development (non-forest estate zones), as well as on peat and non-peat areas was analysed to indicate where forest loss and degradation is likely to occur in the future, thus targeting LESTARI s efforts. LESTARI targets of GHG emission reductions are based on the LBA. LESTARI Landscape Approach Indonesia is also a Party to the Convention on Biological Diversity and as such, has made commitments to achieve the Aichi Biodiversity Targets by Indonesia has set specific targets for protection of key endangered species such as tigers, rhinos, and orangutan. This clear alignment in goals and interests is expected to give LESTARI strong buy-in and support from all stakeholders and contribute to the project s successes in protecting Indonesia s forests. LESTARI applies a landscape approach an integrated land use management framework that seeks to integrate policies and actions across sectors in order to harmonize development and conservation objectives. In other words, the landscape approach aims to ensure that all the uses of land and all the users of that land are being addressed in an integrated way. LESTARI activities are targeted in six strategic landscapes on three of Indonesia s largest islands, where primary forest cover remains most intact and carbon stocks are greatest. The LESTARI landscapes are largely defined by national parks and forests supporting mega diversity and key species. To achieve both sustainable landscapes and conservation objectives, LESTARI has targeted area with HCV and HCS. To address key threats, key activities to be implemented under LESTARI will focus on improving spatial planning for better land management, improved collaborative management, alternative funding for forest conservation or payment for environmental services, preventing encroachment, reduce poaching, community conservation agreements, forest education initiatives as well as strengthening local partners landscape vision and commitment for low emissions development LESTARI Results Framework and Theories of Change An overarching Results Framework (RF) has been developed (Figure 2a and 2b) to explain how LESTARI Strategic Approaches will address the drivers of deforestation and forest degradation, and reduce emissions and threats to biodiversity presented in the Situation Model. The RF demonstrates causality and the logical linkages between shorter-term, intermediate, and longer-term results. The Results Framework is founded upon the overarching theory that if there is increased awareness and advocacy of local stakeholders (government officials, communities, CSOs and private sector) for conservation and natural resource management in ways which contribute to their long term interests; if Conservation Co-Management is actually enhanced to the benefit of key stakeholders, and if the private sector is engaged, there will be improved governance and natural resource management in biologically significant USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 16

19 and critical areas 3, leading to a 41% reduction in GHG emissions by Our Results Framework is then translated into a matrix showing how LESTARI plans to integrate LESTARI s Technical Components into 8 strategic approaches that can be nuanced according to the context of each targeted landscape. This matrix also demonstrates our theory of change for each Strategic Approach, in accordance with the Results Chains provided in 3 Critical areas: areas identified for conservation because of their high conservation value or their high carbon value USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 17

20 Annex II: Results Chains. These Results Chains were updated at the end of Year 1, during the Annual Report and Year 2 Work Plan preparation process, and are included in this second edition of AMEP. USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 18

21 Figure 2a. LESTARI Results Framework with Key Results USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 19

22 Figure 2b. LESTARI diagrammatic Results Framework USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 20

23 Table 2 Theory of Change Matrix Strategic Approaches Technical Components and Tasks: (SOW Technical Components: 1. Land Use Governance; 2. CAs & Key Species; 3. Private Sector; 4. Constituencies) Theory of Change Technical Theme I: Forest and Land Use Governance and Advocacy Awareness and Advocacy Operationalize SEAs & LCPs to Improve Land Use Governance C1.1. Increasing government level awareness, appreciation, and advocacy to ensure that government decision-makers embrace a conservation and sustainable use vision for land use planning. C4.1. Improve local level awareness, appreciation, and advocacy. Improve key stakeholders awareness, appreciation and advocacy for healthy ecosystems, leveraging increased availability of mobile and internet based technology. C4.2. Empower media coverage of environmental issues. Strengthen the ability of media and citizen journalists to monitor, document, and publicize the positive and negative consequences of land use decisions, including those made by private sector actors in the landscapes. C1.2. Operationalizing Strategic Environmental Assessments (SEAs) and Landscape Conservation Plans (LCPs) developed by USAID IFACS within land use plans Improved local level awareness, appreciation, and advocacy to promote citizen action is one of enabling conditions for other LESTARI strategic approaches to run effectively. Through the implementation of this strategy, we expect that local stakeholders will adopt and continue desired behavior thus results in the formation of constituent for conservation. We assume that if the citizens demand for healthy ecosystems and have the capacities for advocacy and lobbying, the government will respond by embracing a conservation and sustainable development vision to better address demand from its citizens. As outlined in the Result Chains for this strategy (Annex IV), it is critical for LESTARI to first identify target audience, message, desired behavior, and appropriate media to deliver the message. LESTARI needs to ensure that target audiences receive the messages through a series of awareness and advocacy activities, retain the desire knowledge which will lead to improvement on attitudes and values. Draw on IFACS experience, it is critical to confirm that knowledge and attitude be translated into effective desired behavior. Therefore, during the implementation LESTARI will assess factors that impede change to occur and address them immediately. Improved land use governance is a cornerstone of achieving improved forest management and biodiversity conservation. Good governance is characterized by the respect for legal certainty, transparency and free flow of information, significant public participation (expression of voice and choice), equality, high accountability, effective and coordinated management of public resources. LESTARI assumes that SEAs and LCPs initiated under IFACS are effective entry points to improve land use governance and forest USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 21

24 Strategic Approaches Technical Components and Tasks: (SOW Technical Components: 1. Land Use Governance; 2. CAs & Key Species; 3. Private Sector; 4. Constituencies) Theory of Change management, including support for FMU development. The incorporation of SEA/LCP recommendations into government planning (notably spatial planning, RTRW, RPJMD/P, Forestry Renstra and other KRP) will involve significant public participation. As such, a comprehensive, up to date and accurate SEA is crucial to developing a spatial plan that incorporates LEDS. In conjunction, LCPs is a tool to give local communities quality information about critical conservation values of their landscape and ensure that government plans consider local conservation needs. Such data can also serve as valuable evidence for communities in their advocacy and lobbying work, which may be an important element in holding decision makers accountable. LESTARI will operationalize both SEA-LEDS and LCPs recommendations for improved forest and land use management through the following: Work through local government, multi-stakeholder initiatives to integrate SEA-LEDS recommendations into local government policy, planning and program (KRP) especially those concerning for improving forest management, biodiversity conservation and other land conservation management Work with Sub-national governments to facilitate incorporation of SEA/LCP recommendations in to KRP and sectoral budgets to safeguard forest areas from infrastructure development and other sectoral threats Improve the understanding and awareness of local stakeholders about SEA, priority environmental issues and sustainable landscape governance opportunities. Incorporate SEA recommendations in to advocacy initiatives to strengethen impact USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 22

25 Strategic Approaches Sustainable Landscape Governance Technical Components and Tasks: (SOW Technical Components: 1. Land Use Governance; 2. CAs & Key Species; 3. Private Sector; 4. Constituencies) C1.3. Improving licensing and permitting processes so they are public and accessible, subject to public oversight, compliant with laws and regulations, and violations are enforced C1.4. Monitoring and enforcing land use, including increasing district and/or provincial government prosecution and adjudication capacity, building on USAID C4J activities, and capacity to use remote sensing and other geospatial tools, building on the spatial data infrastructure units established previously by USAID IFACS C4.3. Strengthen citizen based mechanisms for public input on land use. Support citizen-based mechanisms for public input and monitoring of land use decisions by key groups, especially local communities, indigenous groups, and women. This task is closely aligned with the Component 1 task to encourage sub-national government to solicit and adopt input from citizen based mechanisms C1.6. Increasing district and/or provincial governments willingness to adopt input from citizen-based mechanisms. This is connected to the work with multistakeholder forums under Component 4. Theory of Change Land use planning and licensing of natural resource use is the single largest contributor to GHG emissions in the LESTARI landscapes. Sustainable landscape governance is LESTARI Strategic Approach that contributes to improved planning, permitting, and enforcement of natural resource use within landscapes. It is an interlinked approach that combines introducing citizen-based mechanisms for input on land use, as well as increasing the sub-national government willingness to adopt such input, with improved mechanisms for licensing, monitoring, and enforcing the use of natural resources. Multi Stakeholder Initiatives (MSI) will serve as a bridge to facilitate communication between government and citizen. LESTARI will also facilitate other multistakeholder initiatives or civil society advocacy initiatives to ensure participatory and transparent land use decision making. The strategic approach also links to operationalizing SEA and LCP recommendations. Numerous governance challenges exist including the limited availability of data which several stakeholder groups deem credible represents one of the most serious. Robust and accurate data is needed to inform wellmeasured action plans, especially in terms of reforming legal and policy frameworks and improving the capacities of government, civil society, indigenous communities and private sector stakeholders. Therefore, LESTARI will continue to build upon IFACS progress on developing Spatial Data Infrastructure (SDI) units, such as SIMTARU linked to OneMap, to improve the accuracy, consistency, and availability of land use information especially public transparency and accessibility of natural resource use processes and licenses through relevant subnational institutions, e.g., One Stop Shops for Licensing. Technical Theme II: Conservation Co-Management USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 23

26 Strategic Approaches Improve Forest Management Through Co- Management and FMU Strengthening Technical Components and Tasks: (SOW Technical Components: 1. Land Use Governance; 2. CAs & Key Species; 3. Private Sector; 4. Constituencies) C2.3 Improve co-management adjacent to Conservation Areas by building staff knowledge and implementation of improved collaborative management strategies with adjacent districts and local communities. This task is closely linked to Component 4 s collaborative management task. C3.2. Pilot innovative financing for critical areas. Mobilize financing opportunities, including payments for ecosystem services, to incentivize sustainable enterprises that meet required criteria for environment and social soundness C4.4. Improve co-management adjacent to critical areas. Provide technical assistance to establish co-management arrangements between communities living in and near critical areas and the appropriate government entities. Includes defining acquiring clear tenure. Closely linked to Strategic Approach 2. SEAs. See IUCN definition of "co-management". Theory of Change Key to project success is building capacity and commitment for comanagement among relevant actors in areas adjacent to PAs and critical areas. This is essential for pressure reduction on the PA through more transparent, negotiated roles and responsibilities. Relevant actors will include Ministry of Environment and Forestry Park staff, provincial and district governments, Forest Management Unit (FMU), relevant CSOs, and villages living adjacent to and dependent on PA and critical areas resources and services. LESTARI will facilitate dialogue between key stakeholders and communities, develop the capacity of villages to be able to negotiate with district, provincial and national agreements, comply with, and benefit from co-management agreements to protect their local natural resources. The strategic approach supported by awareness and advocacy so as to increase the willingness of key stakeholders to develop comanagement agreements with communities. It also links to strategic approach green enterprise and innovative financing mechanism whereas the co-management agreements provide incentives for communities to participate in forest protection. LESTARI will work with leaders that represent both men and women and other disadvantaged groups to ensure differing gender considerations and viewpoints are addressed through co-management. In addition, LESTARI also supports the development and operation of FMU as the spearhead of forest management at the grassroot level. With stakeholder involvement in each stage of FMU development and communication at various levels, LESTARI wants to ensure that the FMUs in adjacent landscapes have implement a multi-benefit and multistakeholder forest management concept. USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 24

27 Strategic Approaches Protected Area Management Technical Components and Tasks: (SOW Technical Components: 1. Land Use Governance; 2. CAs & Key Species; 3. Private Sector; 4. Constituencies) C2.1. Improve Conservation Area management by determining priority needs and actions for addressing threats specific to each CAs and the species they harbor in consultation with key stakeholders. Once these have been identified, support the implementation of priority actions to address threats and strengthen the protection and/or management of CA and the species they harbor. These may include management planning; improving data, monitoring and information systems; increasing the use of technology to monitor and enforce wildlife encroachment and poaching; enhancing patrols; improving public relations with sub-national government and communities inside or adjacent to CAs; and other capacity building C2.2. Pilot innovative financing for Conservation Area by identifying and trailing CA sustainable financing and revenuegenerating strategies C1.7. Enhance national level policy coordination Component 2 Theory of Change Key to success in improving conservation areas management capacity is building political leadership; implementing improved management approaches including protection of core areas to combat poaching and deforestation; and adopting M&E systems to assess the effectiveness of management interventions and adapt approaches as needed. LESTARI will focus on the socialization and application of the METT tool to assess management effectiveness in protected areas embedded in its landscapes. Baseline METT will be conducted in a participatory way so as to enable CA managers taking ownership of METT score and committed to improving their management weaknesses. Awareness and advocacy will be emphasized to increase public pressures on CA authority to improve their management weaknesses. A training program with a focus on technical skills and modern tools that managers and PA staff require for effective day-to-day duties will be delivered based on the outcome of the METT assessments. LESTARI will also equip CAs with the capacity to undertake SMART (Spatial Monitoring and Reporting Tool) as a tool to monitor and enforce wildlife encroachment and poaching. This strategic approach links to improve co-management adjacent to CAs, especially because threats are coming from area adjacent to CAs. Further assumption is CAs could improve and implement key aspects of management plan if they are adequate financing. Therefore, LESTARI will identify and develop effective innovative financing plans for CAs. Furthermore, the success of LESTARI in implementing a landscape approach requires effective policy engagement at multiple levels across multiple stakeholders. LESTARI will coordinate with the national level government and provincial level to ensure local impact sustainability, particularly since Law 23 of 2014 recentralizes a significant amount of natural resource decision-making to the province. USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 25

28 Strategic Approaches Technical Components and Tasks: (SOW Technical Components: 1. Land Use Governance; 2. CAs & Key Species; 3. Private Sector; 4. Constituencies) Theory of Change Technical Theme III. Private Sector Engagement Green Enterprises C3.1. Develop public private partnerships for green enterprises. This includes reviewing existing or conducting new market and feasibility studies to explore the potential for ecotourism and other environment-friendly sustainable enterprises that will directly benefit conservation and local communities living within and adjacent to high conservation area forests. Equality important is cultivating relationships with key private sector actors in the project s landscapes to identify sources of co-investment in the above conservation efforts. Finally, the Contractor should develop and pilot sustainable business models that can have broad impacts, including outside the specified landscapes, and disseminate the results of pilots C4.5. Improved sustainable livelihoods and alternative livelihoods for communities adjacent to critical areas. Provide technical assistance to communities living in or near critical areas to improve sustainable natural resources management practices, including those that might lead to eventual third party certification, that enhance productivity and income or adopt locally appropriate alternative livelihoods or green enterprises in order to reduce encroachment. Communities receiving livelihood support must acknowledge and agree to reduce encroachment through formal agreements such as the Community Conservation Agreements (CCAs) developed under the USAID IFACS project As with all other Technical Themes, Private Sector Engagement takes an integrated approach to the development of livelihoods, responsibilities of environmental governance, and shared roles in comanagement of natural resources by reducing spatial uncertainty especially for local communities. The capacity of indigenous communities is naturally impacted by the actions of the business sector: when a business company follows good governance practices, this will have a positive impact on indigenous communities, and vice versa. Therefore, LESTARI will facilitate both parties to seek opportunities for cooperation and mutual reinforcement. Through establishing publicprivate partnerships for green enterprises, and coupled with commitments to safeguards for forest resources (such as through comanagement) and improve environmental governance, LESTARI will support alternative livelihoods while reducing pressure on forest resources. USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 26

29 Strategic Approaches Private Sector BMP Innovative Financing for Sustainable Land and Forest Management Technical Components and Tasks: (SOW Technical Components: 1. Land Use Governance; 2. CAs & Key Species; 3. Private Sector; 4. Constituencies) C3.3. Industry certification, best management practices (BMPs) and Conservation Management and Mitigation Plans (CMMPs). Support private sector-led efforts to make explicit commitments towards zero deforestation and conservation of HCV and other critical areas including support for third party certification, implementation of BMPs and CMMPs C1.5.Enhance district and/or provincial readiness to access financing (DAK, REDD+, etc) by building necessary capacity, supporting MRV systems, and informing policies, rules, and systems for the equitable distribution of funds generated from payments for carbon or other ecosystem services Theory of Change The private sector provides significant opportunities for conservation of biodiversity and reduction of deforestation and degradation within LESTARI landscapes. Subsequently, LESTARI will expand on its work under IFACS to promote BMPs (that include Reduced Impact Logging that reduces GHG emissions - RIL-C) in natural forest timber concessions and the promotion of HCV forest set-asides in fiber and oil palm plantations, both important aspect in third-party certification schemes. Through co-management agreements, LESTARI will facilitate the establishment of LEDS-based community partnerships with villages adjacent to concessions area aimed at improving value chains critical to improved livelihoods. Our assumption is that if companies apply BMPs, companies enhance co-management with villages, and both parties gain benefits from BMPs and co-management implementation, then unsustainable use of natural resource will be reduced and discontinued thus emissions will be reduced and biodiversity conserved. Public discourse will be encouraged towards the removal of policy constraints to increasing the value of natural resources to the private sector in ways that are both economically and environmentally sustainable. With the creation of REDD+ funding mechanisms such as the Indonesia Climate Change Trust Fund (ICCTF), Global Environment Fund, and the Green Climate Fund, there is potentially increased opportunity for districts and/or province to access funds based on performance of reduced emissions from deforestation and degradation.lestari will provide technical assistance to local government to prepare project design documents and safeguards to access REDD+ funding. However, an important and more omnipresent source of funding is the government budget. LESTARI will work with national and subnational government agencies and advocate for LEDS, forest conservation, and sustainable landscapes programs to be funded in the annual budget to USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 27

30 Strategic Approaches Technical Components and Tasks: (SOW Technical Components: 1. Land Use Governance; 2. CAs & Key Species; 3. Private Sector; 4. Constituencies) Theory of Change meet reduced emission commitments, such as through the Payment for Environmental Services (PES) and societal valuing of such services in general. LESTARI will also identify and engage related ministries to gain more access and support additional financing sources and to explore potential collaboration with similar programs from the government in optimizing the benefits to the communities and environment. If funding can be secured, then impact on emissions reduction could be magnified. USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 28

31 2.3 Logical Framework and Performance Indicator Tracking Table (PITT) Our Logical Framework presents an overview of LESTARI Goals, Strategic Objectives, indicators, means of verification, critical assumptions, and annual and life of project (LOP) targets for monitoring progress toward achievement of results and evaluating the validity of LESTARI s Theories of Change. Output, outcome, and impact level indicators that provide a sound benchmark for performance management and reporting on project results have been selected, including required standard indicators for biodiversity and sustainable landscapes. Detailed information for each indicator is provided in Annex I: Performance Indicator Reference Sheets (PIRS). USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 29

32 Table 3 Logical Framework and Targets Note: Please refer to Performance Indicator Reference Sheets (PIRS) in Annex I for indicator disaggregation STRATEGIC OBJECTIVES INDICATOR MEANS OF VERIFICATION ASSUMPTION TARGETS B Y1 Y2 Y3 Y4 Y5 LOP LESTARI Goals Reduced GHG emissions from land-use sector 1. Percentage reduction in GHG emissions as a result of USG assistance measured using actual emissions compared to REL (OUTCOME/IMPACT) Note: for the purpose of FACTS reporting, quantity of GHG emissions will be reported in metric tons of CO2equivalent Comparison of actual emissions to REL following GoI methodology for Land-Based GHG Emission and Sequestration Baseline Calculations. The GOI FREL is used to calculate land cover changes and related emissions. Reductions in deforestation and degradation will lead to reductions in GHG emissions. 0% reduc tion Not measure Not measure Not measu re Not measu re 30% reducti on 45 M tco2- eq 40% reducti on 60 M tco2- eq 41% reducti on 75 M tco2- eq 41% reducti on 75 M tco2- eq Key species protected See #3 See #3 See #3 See #3 See #3 See #3 See #3 See #3 See #3 See #3 USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 30

33 STRATEGIC OBJECTIVES INDICATOR MEANS OF VERIFICATION ASSUMPTION TARGETS B Y1 Y2 Y3 Y4 Y5 LOP Carbon rich forest, peatland, mangrove ecosystems and the habitat of key species conserved 2. Number of hectares of biological significance and/or natural resources under improved natural resource management as a result of USG assistance (FACTS ) (OUTCOME/IMPACT) Sub-national: Comanagement plan in place and under implementation as evidenced by incorporation into government program and budget. Improved management will lead to the conservation of key species habitats as well as contribute to reduction in GHG emissions M 1.7 M 3.3 M M FMUs assisted by LESTARI operational. Certain area setaside for conservation as result of SEA-LEDS and/or LCP recommendations into local development plans. CA: Increased METT score Concessions: CMMP, RIL-C or other conservation plan in place and under implementation as evidenced by incorporation into SOP and company budget. USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 31

34 STRATEGIC OBJECTIVES INDICATOR MEANS OF VERIFICATION ASSUMPTION TARGETS B Y1 Y2 Y3 Y4 Y5 LOP Reduced rate of deforestation and degradation See #1 See #1 See #1 See #1 See #1 See #1 See #1 See #1 See #1 See #1 Reduced threat on key species 3. Percentage reduction in poaching in focus area (OUTCOME) Analysis of Spatial Monitoring and Reporting Tool (SMART) data. Comparison of baseline and endline. LESTARI will undertake baseline in Q2 Year 1 to determine number of current poaching incident in focus area. Reduction in poaching will lead to conservation of key species in focus area TBD Not measure 10% reducti on 20% reducti on 30% reducti on 40% reducti on 40% reducti on Improved Land Use Governance Forest & Land Use Governance & Advocacy 4. Number of public policies introduced, adopted, repealed, changed or implemented consistent with citizen input (FACTS /OUTCOME) Review the process of policies development, minutes of public consultation meetings, comparison of policies introduced, adopted, repealed, changed or implemented with citizen input. When stakeholder s voices continue to be heard by their government, it will ensure sustainability of MSF and lead to improved land use regulatory framework as prerequisite of improved land use governance USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 32

35 STRATEGIC OBJECTIVES INDICATOR MEANS OF VERIFICATION ASSUMPTION TARGETS B Y1 Y2 Y3 Y4 Y5 LOP 5. Number of sub-national government with improved licensing and permitting mechanism (OUTCOME) The term improved is taken to subsume the five core principles of good governance (transparency, participation, accountability, responsiveness, and timeliness). Standard Operational Procedures (SOP) will be developed to ensure that all licenses and permits follow the government regulation and procedures. Verification will be made through reviewing the implementation of SOP, licensing process, interview licensing officers, business enterprises and community to ensure good governance principles are adhered. When licensing and permitting mechanism adopt five core principles of good governance, the sustainable use of natural resource can be secured USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 33

36 STRATEGIC OBJECTIVES INDICATOR MEANS OF VERIFICATION ASSUMPTION TARGETS B Y1 Y2 Y3 Y4 Y5 LOP 6. Number of SEAs/LCPs recommendations related with land use, spatial plan, forest management, and biodiversity conservation incorporated into subnational government policy, planning and program (OUTCOME) Comparison of SEA-LEDS & LCPs recommendation to government policy, planning and program. Explicit reference to adopting SEA/LEDS recommendation of improved-case scenario (not BAU) in policy, planning and program. Incorporation of SEA- LEDS & LCPs recommendations into government policy, planning and program will ensure budget and implementation and lead to improved land use governance, specifically on forest management and biodiversity conservation Number of Multi Stakeholder Initiatives (MSIs) or civil society advocacy initiatives which formulate and deliver policy-relevant inputs on sustainable forest and land management to decision makers (OUTCOME) Review of multi stakeholder initiatives report and produced policy paper or any form of policy input, and review policy development processes. The success of multi stakeholder initiatives as mechanism to bridge communication between local government and wider public in land use will lead to more transparent, participatory and accountable land use decisions. Those principles are prerequisite of good governance USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 34

37 STRATEGIC OBJECTIVES INDICATOR MEANS OF VERIFICATION ASSUMPTION TARGETS B Y1 Y2 Y3 Y4 Y5 LOP 8. Number of champions engaged in advocacy interventions (OUTPUT) Review advocacy plans or strategies, recording of press conference, copy of testimony or press release, advocacy campaign materials, record of public consultations. Advocacy should be understood as a means for individuals and constituencies to shape public agendas, change public policies, and influence other processes that impact their lives. Advocacy interventions are essential aspects of democratic policy making, citizen participation, and oversight of government USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 35

38 9. Number of people reached by LESTARI communication programs to improve awareness and understanding of LEDS and biodiversity conservation (OUTPUT) Sign-in sheet from communication trainings and meetings. Review communication products and outreach. Monitoring social media. Qualitative assessments (e.g., case study) of increase awareness and understanding. Increase awareness and understanding, supported by continuous advocacy will lead to increased commitment of key stakeholders regarding the positive benefits of conservation and sustainable use of forests and the species they encompass, which in turn lead to improved land use decision making, thus contributes to improved land use governance , ,374 55,500 57,000 47, ,000 Improved Forest Management USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 36

39 STRATEGIC OBJECTIVES INDICATOR MEANS OF VERIFICATION ASSUMPTION TARGETS B Y1 Y2 Y3 Y4 Y5 LOP Conservation Co- Management 10. Number of Conservation Areas (CAs) with at least 70 point in METT scores across LESTARI landscapes as result of USG assistance (OUTCOME) METT baseline and endline assessment; enriched by qualitative analysis To maximize the potential of protected areas, it is important to understand the strengths and weaknesses of their management and the threats that they face. Addressing the weaknesses and reducing threats will lead to increase effectiveness of CAs management. See PIRS Number of Forest Management Unit (FMU) strengthened as a result of USG assistance Review the process of FMU development, minutes of meetings, comparison of FMU operationalization standards improved with baseline condition. When FMU operationalization standard are met and strengthen, it will ensure the optimum operationalization of FMU and lead to improved the multi-use of forest management See PIRS Number of people receiving USG supported training in natural resources management and/or biodiversity conservation (FACTS /OUTPUT) Sign-in sheets, training reports. Assessments of the effectiveness of trainings and the application of knowledge and skills gained. Increased capacity to manage natural resources and/or biodiversity conservation will lead to improved land use governance and forest management ,946 3,663 1, ,000 USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 37

40 STRATEGIC OBJECTIVES INDICATOR MEANS OF VERIFICATION ASSUMPTION TARGETS B Y1 Y2 Y3 Y4 Y5 LOP Private Sector Engagement 13. Amount of investment mobilized (in USD) for climate change and biodiversity conservation as supported by USG assistance (OUTPUT) Finance documents (e.g., APBD, finance reports, etc). Interviews the financers. Assess the use of funds and the issues being addressed with the funds. LESTARI programs should attract additional funds that are necessary to increase capacities for addressing land use and forest conservation. Such funds represent knowledge and commitment of local stakeholders to, and institutionalizati on of, the issues being addressed with the funds. 0 3 M 1 M 6 M 6 M 4 M 20 M 14. Number of people receiving livelihood co-benefits (monetary or non monetary) (OUTCOME) Random survey and on-site observations Economic and noneconomic benefits derived from sustainable use of forest resource will provide incentives for people to be committed to LEDS and forest conservation ,721 9,500 8,200 30,000 USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 38

41 STRATEGIC OBJECTIVES INDICATOR MEANS OF VERIFICATION ASSUMPTION TARGETS B Y1 Y2 Y3 Y4 Y5 LOP 15. Number of private sector firms that have improved management practices as a result of USG assistance (FACTS /OUTCOME) Review the RIL-C and/or CMMP/conservatio n plan document, incorporation into SOP and budget line item allocated by concessionaires to implement the plan. Spot-check to observe field implementation Implementation of RIL- C and/or CMMPs/conservation plan will ensure that HCV areas in concessions are managed in sustainable manner Number of new USGsupported public-private partnerships (PPPs) formed (FACTS PPP 5) (OUTPUT) PPP document signed by both public and private entities PPP will provide incentives both for governments, private sectors and communities to embrace LEDS and conservation oriented practices USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 39

42 Table 4 LESTARI Targets per Landscape Note: This is illustrative table. The target per landscape will be updated in LESTARI Annual Work Plans based on annual achievement. INDICATOR TARGET OVER THE LIFE OF PROJECT LANDSCAPE ANNUAL TARGETS B Y1 Y2 Y3 Y4 Y5 TOTAL Leuser 0% Not measure Not measure Not measure Not measure 30% 40% 41% 41% 4,525,619 2,709,302 3,616,449 4,525,619 M tco2-eq 1. Percentage reduction in GHG emissions as a result of USG assistance measured using actual emissions compared to REL (OUTCOME/IMPACT) Note: for the purpose of FACTS reporting, quantity of GHG emissions will be reported in metric tons of CO2equivalent 41% reduction Katingan- Kahayan Lorentz Lowlands 0% 0% Not measure Not measure Not measure Not measure Not measure Not measure Not measure Not measure 30% 40% 41% 41% 23,535, ,390, ,249,820 39,249,820 M tco2-eq 30% 40% 41% 41% 7,063,326 9,476,706 11,919,556 11,919,556 M tco2-eq Mappi-Bouven Digoel 0% Not measure Not measure Not measure Not measure 30% 40% 41% 41% 10,153, ,563, ,985,845 16,985,845 M tco2-eq Sarmi 0% Not measure Not measure 30% 40% 41% 41% USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 40

43 INDICATOR TARGET OVER THE LIFE OF PROJECT LANDSCAPE ANNUAL TARGETS B Y1 Y2 Y3 Y4 Y5 TOTAL Not measure Not measure 1,526,926 2,040,547 2,556,490 2,556,490 M tco2-eq Cyclops 0% Not measure Not measure Not measure Not measure 30% 40% 41% 41% 50,155 30,093 40,124 50,155 M tco2-eq Leuser , , , ,275, Number of hectares of biological significance and/or natural resources under improved natural resource management as a result of USG assistance (FACTS ) (OUTCOME/IMPACT) 8.7 Million Katingan- Kahayan Lorentz Lowlands Mappi-Bouven Digoel , , , ,000 2,088, ,019, ,805 1,156, ,000 3,041, , , ,000 1,749,904 Sarmi , , ,595 Cyclops , , ,980 Leuser TBD Not measure 10% 20% 30% 40% 40% 3. Percentage reduction in poaching in focus area (OUTCOME) 40% reduction Katingan- Kahayan TBD Not measure 10% 20% 30% 40% 40% Lorentz Lowlands TBD Not measure 10% 20% 30% 40% 40% USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 41

44 INDICATOR TARGET OVER THE LIFE OF PROJECT LANDSCAPE ANNUAL TARGETS B Y1 Y2 Y3 Y4 Y5 TOTAL Mappi-Bouven Digoel n/a n/a n/a n/a n/a n/a n/a Sarmi n/a n/a n/a n/a n/a n/a n/a Cyclops TBD Not measure 10% 20% 30% 40% 40% Leuser Number of public policies introduced, adopted, repealed, changed or implemented consistent with citizen input (FACTS /OUTCOME) 28 policies Katingan- Kahayan Lorentz Lowlands Mappi-Bouven Digoel Sarmi Cyclops Leuser Katingan- Kahayan Number of sub-national government with improved licensing and permitting mechanism (OUTCOME) 8 sub-national governments Lorentz Lowlands Mappi-Bouven Digoel Sarmi USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 42

45 INDICATOR TARGET OVER THE LIFE OF PROJECT LANDSCAPE ANNUAL TARGETS B Y1 Y2 Y3 Y4 Y5 TOTAL Cyclops Leuser Number of SEAs/LCPs recommendations related with land use, spatial plan, forest management, and biodiversity conservation incorporated into subnational government policy, planning and program (OUTCOME) 6 sub-national governments Katingan- Kahayan Lorentz Lowlands Mappi-Bouven Digoel Sarmi Cyclops Leuser Number of Multi Stakeholder Initiatives (MSIs) or civil society advocacy initiatives which formulate and deliver policy-relevant inputs on sustainable forest and land management to decision makers (OUTCOME) 20 MSIs Katingan- Kahayan Lorentz Lowlands Mappi-Bouven Digoel Sarmi Cyclops Leuser USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 43

46 INDICATOR TARGET OVER THE LIFE OF PROJECT LANDSCAPE ANNUAL TARGETS B Y1 Y2 Y3 Y4 Y5 TOTAL Katingan- Kahayan Number of champions engaged in advocacy interventions (OUTPUT) 500 landscape champions Lorentz Lowlands Mappi-Bouven Digoel Sarmi Cyclops Leuser 0 2,562 7,477 15,000 15,000 15,000 55, Number of people reached by LESTARI communication programs to improve awareness and understanding of LEDS and biodiversity conservation (OUTPUT) 500,000 people Katingan- Kahayan Lorentz Lowlands Mappi-Bouven Digoel 0 7,143 10,254 20,000 15,000 15,000 67, ,222 2,460 3,000 8,500 5, , ,276 2,000 8,500 5,075 30,577 Sarmi Cyclops ,907 15,500 10,000 7, , Number of Conservation Areas (CAs) with at least 70 point in METT scores across LESTARI landscapes as result of USG assistance (OUTCOME) 6 CAs Leuser Katingan- Kahayan See PIRS See PIRS Not measure Not measure USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 44

47 INDICATOR TARGET OVER THE LIFE OF PROJECT LANDSCAPE ANNUAL TARGETS B Y1 Y2 Y3 Y4 Y5 TOTAL Lorentz Lowlands See PIRS Not measure Mappi-Bouven Digoel n/a n/a n/a n/a n/a n/a n/a Sarmi n/a n/a n/a n/a n/a n/a n/a Cyclops See PIRS Not measure Leuser Katingan- Kahayan Number of Forest Management Unit (FMU) strengthened as a result of USG assistance 7 FMUs Lorentz Lowlands Mappi-Bouven Digoel Sarmi Cyclops Number of people receiving USG supported training in natural resources management and/or biodiversity conservation (FACTS /OUTPUT) 10,000 people Leuser ,086 1, ,800 Katingan- Kahayan Lorentz Lowlands , , USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 45

48 INDICATOR TARGET OVER THE LIFE OF PROJECT LANDSCAPE ANNUAL TARGETS B Y1 Y2 Y3 Y4 Y5 TOTAL Mappi-Bouven Digoel Sarmi Cyclops Leuser 0 18,664 29,912 4,151,424 2,500,000 1,250,000 7,950, Amount of investment mobilized (in USD) for climate change and biodiversity conservation as supported by USG assistance (OUTPUT) USD 20 Million Katingan- Kahayan Lorentz Lowlands Mappi-Bouven Digoel 0 3,382, , ,922 2,000,000 1,250,000 7,050, , ,671 1,318,575 1,200,000 1,000,000 3,700, , , , , ,000 Sarmi Cyclops , ,000 50, ,000 Leuser ,349 4,200 3,500 12,150 Katingan- Kahayan ,672 4,100 3,500 13, Number of people receiving livelihood co-benefits (monetary or non monetary) (OUTCOME) 30,000 people Lorentz Lowlands , ,900 Mappi-Bouven Digoel Sarmi USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 46

49 INDICATOR TARGET OVER THE LIFE OF PROJECT LANDSCAPE ANNUAL TARGETS B Y1 Y2 Y3 Y4 Y5 TOTAL Cyclops Leuser Katingan- Kahayan Number of private sector firms that have improved management practices as a result of USG assistance (FACTS /OUTCOME) 10 private sector firms Lorentz Lowlands Mappi-Bouven Digoel Sarmi Cyclops n/a n/a n/a n/a n/a n/a n/a Leuser Katingan- Kahayan Number of new USG-supported public-private partnerships (PPPs) formed (FACTS PPP 5) (OUTPUT) 20 PPPs Lorentz Lowlands Mappi-Bouven Digoel Sarmi Cyclops USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 47

50 3. PLAN FOR ACTIVITY MONITORING AND EVALUATION 3.1 LESTARI Learning Loop: Data Collection, Storage, Quality Assurance, Analysis, and Utilization Monitoring and evaluation (M&E) and reporting are essential parts of the LESTARI project. The ability to rapidly collect, review, analyze, and communicate monitoring and evaluation information is essential for maximizing LESTARI results. To facilitate this process, LESTARI will develop a tailored Management Information System (MIS). The MIS will serve as an accessible and transparent reporting tool to guide future activities and learning. It will also connect the development of activity scopes of work to the LESTARI approval process and link the outcomes of these activities to results as outlined in this AMEP as well as annual work plans. To increase the efficiency, the MIS will also include features that allow coordination of technical activities with necessary administrative and operations support. Variable levels of MIS access will be provided to USAID and LESTARI partners for reporting, operations support, and invoicing purposes. Progress captured in the MIS will be used to facilitate LESTARI Adaptive Management approach. DESIGN: The first step of the project s Learning Loop is design, which includes building consensus amongst project staff, USAID, and partners on the project s theory of change, expected results, and indicators. As described in Section 3.1, this AMEP is founded upon the results of a Measuring Impact Workshop, where LESTARI staff and USAID collaborated in the development of theories of change and indicators for each LESTARI Strategic Approach. The outcomes of this workshop were presented to regional staff and partners in breakout sessions during the first Annual Work Planning Meeting, so that theories of change could be further refined and used to develop Landscape Level Work Plans. Figure 3 LESTARI Adaptive Management Data Cycle USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 48

51 Step 1: DESIGN USAID/Indonesia Measuring Impact LESTARI Team Home Office M&E Specialist Local Partners Construct Situation Model to understand project context Develop Theories of Change for each Strategic Approach Develop a Results Framework to show linkages between Strategic Approaches and higher-level results Identify performance indicators and targets. Identify key evaluation questions. COLLECT : Most M&E data will be collected using electronic data collection form. During project startup, LESTARI Monitoring, Evaluation & Learning (ME&L) Coordinator, with support from Tetra Tech ARD s Home Office M&E Specialist, will use mobile data collection tools to create customized electronic data collection forms (based on the Year 1 Work Plan and AMEP) whereas data collected will be put into LESTARI MIS. These electronic forms will include required fields, skips, and ranges to improve standardization and data quality, as well as open-ended fields to capture qualitative data. It is important to note that changes in the methodology for collecting data may compromise LESTARI s ability to compare results over time. However, as the project continues to learn how to improve data quality, changes to data collection tools or methods may become necessary. In these cases, the ME&L Coordinator will be able to use the web-based interface to update data collection forms. However, the ME&L Coordinator must document these changes over time in revisions to the AMEP. Additional details on our specific data collections methods for LESTARI indicators are outlined in Annex I: Performance Indicator Reference Sheets (PIRS). Step 2: COLLECT LESTARI Staff (include Landscape M&E Specialists) Collects data and data documentation for all project activities using approved data collection forms and following the requirements outlined in the PIRS Submits data to ME&L Coordinator as data available ME&L Coordinator Develops data collection forms Collects additional qualitative data as necessary Trains Technical Staff on data collection methods, standards, and requirements Trains Technical Staff on how to collect data using data collection forms USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 49

52 Step 2: COLLECT Documents changes to data collection tools/methods resulting from ongoing learning COP HO M&E Specialist Provides support to the ME&L Coordinator in ensuring that data are collected and submitted in a timely manner. Provides initial support and training in data collection form and development of data collection tools. REVIEW: Mobile data collection tool will improve data quality through facilitating real-time reviews by the ME&L Coordinator. The ME&L Coordinator will receive a notification each time new data are added to MIS and will then review the quality of data against ADS 203 data quality criteria (validity, reliability, timeliness, precision and integrity), and reassign the form to the field-based data collector if corrections are required. If ongoing data quality issues persist, the ME&L Coordinator will work with the CoP, Tetra Tech ARD s home office M&E Specialist as well as USAID/Indonesia to develop new data collection strategies to strengthen data quality. In addition to this continuous review of data quality, at least two internal data quality assessments (DQAs) will be conducted by the Home Office M&E Specialist during the project following the format outlined in Annex III: Data Quality Assessment Checklists. The internal DQA described here is an internal quality control mechanism and does not substitute for any formalized, USAID-initiated DQA of project data. Step 3: REVIEW Technical Staff & Landscape M&E Specialists ME&L Coordinator COP TT ARD Home Office M&E Specialist Makes necessary changes to incorrect/incomplete data reviewed by ME&L Coordinator. Reviews all data against ADS 203 data quality standards Ensures that at least two (2) internal Data Quality Assessments (DQAs) are conducted during the project. Conducts two (2) internal DQAs during the project. USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 50

53 MANAGE: Following the quality review, the Jakarta M&E Specialist will assist ME&L Coordinator to enter disaggregated data into the LESTARI Management Information System (MIS). For each data point entered, supporting documentation (e.g., sign in sheets, photographs, meeting agendas, etc.) must be provided. Step 4: MANAGE Jakarta M&E Specialist Inputs data into MIS Ensures each data point is supported by documentation Files hard and electronic data documentation Maintain MIS in day to day basis ANALYZE: On a quarterly basis, LESTARI s ME&L Coordinator will review disaggregated data to assess progress toward targets and identify implementation challenges. When analyzing LESTARI data, the ME&L Coordinator will consider both INTERNAL and EXTERNAL factors that may be influencing results. Internal factors include challenges or opportunities related directly to project implementation (for example, staff turnover) while external factors refer to those outside of the control of LESTARI staff and USAID (for example, negative factors might include political instability or natural disasters; positive factors might include new opportunities for collaboration). An important part of the analysis step is not only providing data to explain the what, but also to provide context to explain the why. This context is extremely important in informing how LESTARI may need to adjust activities. It will also be beneficial to future USAID projects, as they determine what worked in the past, and why. Step 5: ANALYZE Technical Staff & Landscape M&E Specialists ME&L Coordinator Work with the ME&L Coordinator to contextualize the results: What other INTERNAL and EXTERNAL factors (risks and challenges, or enabling events or opportunities) may be influencing our results? Assesses progress toward targets Analyzes quantitative and qualitative data to tell the story of project successes, challenges, and lessons learned. USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 51

54 REPORT AND COMMUNICATE RESULTS: Once data have been analyzed, they will be ready for reporting to USAID/Indonesia. The M&E Sections of reports will include: Any M&E-related activities (e.g., trainings, data collection activities, internal data quality assessments and findings) conducted that quarter; An indicator table(s) showing quantitative results achieved that quarter (targets vs. actuals); A narrative describing overall analysis of project achievements, lessons learned, and implications for implementation; Project Success Stories, including photographs and quotes from beneficiaries (starting in year two); Discussion of any data limitations, reporting challenges, and proposed changes to the AMEP. The findings of any special studies/analyses performed to inform project learning. While the minimum requirement is an indicator table showing disaggregated actuals against targets for each indicator, it is sometimes difficult for a reader to easily assess what is/isn t working in this format. The goal of the report is to provide a clear picture of what the project has achieved, how much the project has achieved, and why the project has or has not met its goals. Whenever possible, the project will use visual representations of data (such as pie charts, bar graphs, or line graphs) to report project results. In addition to reporting results to USAID, the ME&L Coordinator will support the development and dissemination of Success Stories and Lessons Learned with LESTARI partners and stakeholders at the national and regional levels during annual meetings. Step 6: REPORT Technical Staff & Landscape M&E Specialists ME&L Coordinator Provides any additional clarification/information requested by ME&L Coordinator for inclusion in M&E Sections of Quarterly/Annual Reports Drafts M&E sections of Quarterly/Annual reports Assists with development of Success Stories (beginning in Year 2) COP Reviews M&E Sections of Quarterly/Annual reports for quality prior to reporting to USAID. USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 52

55 LEARN: A critical step in the adaptive management process is learning. While the process described above will promote learning throughout all phases of the project, the ME&L Coordinator will facilitate a formal learning process at Annual Strategic Reviews. Following annual internal evaluations conducted by the ME&L Coordinator, Strategic Reviews will seek the participation of project staff, partners, and USAID to review evaluation data and findings to address the following questions: Which activities were successful, and why? Which activities fell short of their anticipated results? Why? How could any activity be redesigned to increase efficiencies, effectiveness, scope, and satisfaction? How have activities had a positive effect on women and other vulnerable/disadvantaged groups? Team members will also review key aspects of the AMEP (theories of change, indicators and targets) and consider whether these remain aligned with project activities. If the process reveals misalignment between activities, indicators and expected results; changes in initial assumptions (or causal logic) supporting the theory of change, unanticipated factors influencing achievement of results, or changes in scope, elements of the AMEP may need to be refined or redesigned. Table 5 Annual Strategic Reviews Question Response Decisions Made/ Actions Required Items Not Resolved from previous period- why? Which activities during previous quarter were successful, and why? What activities were planned but didn t occur? If they did not occur, why? For activities planned but did not occur, when will they be re-programmed? Are we achieving our performance indicators targets for key performance indicators? If we are not, why? What activities can be planned for the following year to meet our performance indicator targets? Are our target assumptions still valid? If not, why? USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 53

56 Question Response Decisions Made/ Actions Required Items Not Resolved from previous period- why? Which activities are falling short of their anticipated results and why? Is the causal logic of the theories of change still valid? If not, why? Are our assumptions related to our ability to implement Strategic Approaches and meet our performance indicators targets still valid? If not, what has changed? What effects do these changed assumption(s) have on the project? What is our strategy to address any changed assumption(s)? Are there any red-flags if so; are there any corrective actions that need to be taken? Step 7: LEARN Technical Staff & Landscape M&E Specialists ME&L Coordinator Participate in Annual Strategic Reviews Contribute useful contextual information that contributes to the learning process Facilitate learning sessions at Annual Strategic Review Document and disseminate the findings to project staff Include the findings in the next Quarterly/Annual Report COP Ensure that findings from learning forums are included in annual work plans and are considered in the development of activities. USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 54

57 REFINE: If the learning process results in recommended changes to the AMEP, the ME&L Coordinator (under the guidance of the COP), will document any major changes in the Executive Summary of the revised AMEP along with the rationale for these adjustments. Major revisions could include changes to frameworks, indicators, or the PIRS. If minor AMEP elements change, the AMEP will be updated to reflect these changes. While required changes to the AMEP may become apparent at any point in the project cycle, the project must continue to follow the standards and methods outlined in the current AMEP until the revised document is officially approved by USAID. Once approved, the project may then follow the new AMEP standards and protocols throughout the adaptive management process. Step 8: REFINE Technical Staff & Landscape M&E Specialists ME&L Coordinator Review AMEP elements Recommend changes to AMEP based on project learning Recommend changes to Work Plan and/or activity implementation based on project learning Review AMEP elements Recommend changes to AMEP based on project learning Recommend changes to Work Plan and/or activity implementation based on project learning Make necessary changes to the AMEP COP Review and approve recommended changes to AMEP prior to submission to USAID Submit revised AMEP/Work Plan to USAID. 3.2 Internal Evaluations Plan Evaluation is an applied inquiry process for collecting and synthesizing evidence that culminates in conclusions about the state of affairs, value, merit, worth, significance, or quality of a program, product, person, policy, proposal, or plan. Conclusions made in evaluations encompass both an empirical aspect (that something is the case) and a normative aspect (judgement about the value of something). It is the value feature that distinguishes evaluation from other types of inquiry, such as basic science research, clinical epidemiology, investigative journalism, or public polling (Fournier, 2005, p.140) In accordance with the 2011 USAID Evaluation Policy, LESTARI will use evaluation methodologies with the dual purposes of accountability to USAID, government partners and other stakeholders; and learning about key results to improve effectiveness. In the simplest terms, evaluations planned for LESTARI will answer three questions: What? So what? Now what? The answers of these questions will not only aim to understand project effectiveness but also to generate new knowledge for the wider biodiversity and sustainable landscapes USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 55

58 community. It is worth noting that evaluation for LESTARI is not an end in itself. The purpose is to inform thought and action. Moving from what, to so what, to now what means moving from data to interpretation to action. Regular monitoring of LESTARI performance indicators will provide the project with the answers of what questions as well as demonstrate accountability. However, several key impact questions will need to be assessed to answer the so what question and track changes over time so as to LESTARI can determine its effectiveness. An example for this case could be: a national park has increased its METT score to 70 point (what), but does this score reflect on improved biodiversity condition? (so what). If yes, what can we learn to replicate the success (now what). If not, how can we modify our approach? (now what). Internal evaluations for LESTARI will be designed in a way that meet the principles of USAID evaluation policy (i.e., unbiased, relevant, based on the best methods, transparent, high utility) and address the most important questions about project impact, how LESTARI is impacting the direct threats and the key species of interest. Tools, methods and design for evaluating LESTARI performance indicators will be developed and agreed upon in a consultation with LESTARI COR on a timeframe that allows for LESTARI interventions to demonstrate impact. These evaluations will be performed during mid and end of project life, and timed in order to ensure that they are completed before and thus can contribute to external mid-term and final evaluations. The SOW for the mid-term internal evaluation will be developed with the USAID COR by the end of the second quarter of Year 2, and the evaluation will be completed and written-up by the end of Year 2. The SOW for the final internal evaluation will be developed with the USAID COR by the end of the first quarter of Year 5, and the evaluation will be completed and written-up by the end of the third quarter of Year 5. In both cases, adjustments could be made to ensure these internal evaluation documents are available in time for external evaluations. Table 6 presents the illustrative evaluation tools, suggested methods and designs relate to the evaluation questions posed. The proposed key evaluation questions (KEQs) are selected to assess the validity of assumptions illustrated in LESTARI Results Chains (Annex IV). USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 56

59 Table 6 Illustrative summary of tools, methods and design implications for proposed evaluation questions Key Evaluation Question Suggested tools, methods, and designs Timing of Data Collection To what extent have LESTARI interventions been effective in reducing biodiversity threats (e.g., poaching, fragmentation of key habitat) and positively impacted populations of key identified species? Collecting data from population survey done by WCS, WWF, or other organization (e.g. BOSF). When data on population is not available, LESTARI may want to consider funding population surveys for targeted key and indicator species (e.g., tiger and orangutan). Impact evaluation detailed land use and land use change (LULUC) assessments to determine the impact of LESTARI interventions on key habitat. Mid-term (2017) and End of Project (2020) Note: Population survey is available every three years depend on the key species assessed (e.g. a WCS population survey for tiger will be conducted in 2016). To what extent has improved collaboration between government, private sector and community contributed to improved forest management? Are economic benefits sustained by LESTARI beneficiaries government, private sector and community? Are economic benefits sufficiently incentivizing those beneficiaries to minimize their encroachment of forests? To what extent have LESTARI intervention been effective in improved land-use governance? To what extent has the project addressed barriers of political will and buy-in at the national and regional levels? Cost-benefit analysis Mid-term (2017) and End of Project (2020) A combination of tools such as: Assessing land cover and policy changes Stories of change a case study method to determine pathways of success Bellwether method an interview method to determine an issue s position on the policy agenda Stakeholder analysis method to determine which stakeholders are invested in the intervention Stories of change, bellwether method, and stakeholder analysis will be used in regular basis (rolling as data available) to monitor the process of policy influence. The data collected from the tools will feed into QCA and Contribution Analysis which will be used in mid-term and end of project evaluation. USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 57

60 Key Evaluation Question Suggested tools, methods, and designs Timing of Data Collection What changes have been made to land use policy as result of advocacy interventions and strengthening citizen based mechanism input? Have beliefs, opinions, and behaviours of stakeholders changed as result of advocacy and campaign activities? Have improved land use governance impacted on reduction in large scale and small scale deforestation and degradation? Knowledge generation: enhance general understandings and identify generic principles about effectiveness Qualitative Comparative Analysis (QCA) 4 and other theory-based evaluation design, e.g. contribution analysis 5 will be used to gain a better understanding of project attribution. QCA is a middle way that combines certain features of the qualitative approach (case orientation, interest in complexity) with those of quantitative research (interest in generalization). An example of QCA utilization can be seen further in CIFOR research that aims to compare national REDD+ policy processes in 12 countries ( Meta-analyses Lessons learned Case studies Mid-term and end of project year 4 QCA is an alternative approach to examining contributory causes that uses a comparative framework perspective. QCA provides a credible causal claim that the intervention is a contributory cause. QCA can answer the question Did it make a difference?, but not the question Why has the result occured? 5 Contribution analysis is one of theory-based approaches to evaluation. Contribution analysis confirms: a) that the expected result occured; b) that the causal package is sufficient; c) that the intervention is a necessary part of the causal package. USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 58

61 3.3 Schedule for Performance Monitoring Tasks Table 7 Key Performance Monitoring and Evaluation Tasks and Timeline Tasks FY 2015 FY 2016 FY 2017 FY 2018 FY 2019 FY 2020 Quarter: Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Responsible Submit Draft/Final AMEP Develop MIS and train staff Draft M&E Section of Quarterly Report Support development of Success Stories Draft M&E Section of Annual Report Conduct Internal Evaluation Facilitate M&E Session of Strategic Annual Review Revise AMEP Draft M&E Session of Final Project Report Assess Data Quality Conduct internal DQA Review Internal Performance Hold Adaptive Management Meeting LESTARI Collects Data M&E M&E/HO M&E M&E M&E M&E M&E M&E M&E M&E M&E M&E/HO M&E M&E M&E ALL USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 59

62 3.4. Monitoring and evaluation structures, functions, and capabilities Monitoring, evaluation, and learning are core priorities across LESTARI s technical and administrative positions. While the ME&L Coordinator (under the direction of the COP) will hold primary responsibility for developing and implementing the M&E system, all LESTARI staff must participate in monitoring and evaluation in order for the system to be successful. This section provides a general description of the M&E roles and responsibilities for LESTARI staff. Specific M&E roles and responsibilities for each indicator can be found in the PIRS in Annex I. Table 8 M&E Roles and Responsibilities M&E Contributor ME&L Coordinator Roles and Responsibilities Provide professional guidance and hands-on assistance to all LESTARI staff on the LESTARI AMEP; MIS; USAID ADS 203 guidelines; and data collection techniques, standards, and best practices. Provide training and technical assistance to local partners in performance monitoring and information sharing. With assistance from the HO M&E Specialist, develop and maintain the LESTARI MIS, including standardized data collection forms, cloudbased M&E database, and a Performance Monitoring Dashboard. Assess the quality of collected data and provide feedback to parties responsible for data collection. Draft and submit M&E sections of quarterly and final reports to the COP for submission to USAID. Conduct special studies and analyses as necessary to inform project learning. Lead Quarterly Results Reviews. Support development of Success Stories and other communications and outreach materials. Work with the DCOP and regional and technical staff to ensure that M&E data are informing LESTARI management and decision making. Conduct regular data verification activities. Conduct annual internal evaluations. Share internal evaluation findings and facilitate review and feedback sessions with stakeholders at Annual Strategic Reviews. Facilitate internal impact evaluations, and dissemination of findings. Jakarta M&E Specialist Provide inputs to ME&L Coordinator in the development of LESTARI AMEP; standardized data collection forms; and MIS. Maintain MIS operation in daily basis and train staff on its use. USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 60

63 M&E Contributor Roles and Responsibilities Supports Jakarta technical staff to perform their monitoring tasks and responsibilities. Maintain up-to-date hard and electronic files of all data and documentation. Work with Landscape M&E Specialists to ensure timely reporting on LESTARI activities. Landscape M&E Specialists In the activity design level, provide hands-on assistance to regional team to develop activity logic align with project level ToC. Train LESTARI regional team, grantees, and sub-contractors to accurately and reliability collect data, ensure that data collection forms and templates are used correctly. Verify and validate all data including reports from grantee and subcontractors for submission to ME&L Coordinator. Provide inputs to ME&L Coordinator for quarterly and final reports. Under guidance of ME&L Coordinator, conduct special studies and analyses as necessary to inform project learning. Provide inputs to ME&L Coordinator for the development of Success Stories and other communications and outreach materials. Work with the regional team to ensure timely reporting on LESTARI activities. Under guidance of ME&L Coordinator, conduct regular data verification activities. Facilitate review and feedback sessions with regional team at quarterly strategic reviews. Regional Coordinators/ Technical Staff Work with the M&E team to gain consensus on how LESTARI activities will contribute to indicators and objectives. Collect data and data documentation (photos, GPS coordinates, agendas, sign in sheets, etc.) using data collection form. Provide recommendations to the M&E Team on any needed changes to the AMEP or M&E process, such as revisions to data collection forms, methods, indicators, or targets. Participate in forums (Quarterly Results Reviews; Annual Strategic Reviews) intended to facilitate learning from project data. Incorporate findings from learning forums into future activities and work plans. DCOP Tetra Tech Home Office M&E The DCOP will be responsible for ensuring that project activities contribute to indicator targets and expected results, are gender-responsive, and that M&E data and findings are used to inform planning and decision making. In Year 1, Tetra Tech ARD s home office M&E Specialist, Ms. Robynne Locke, will be responsible for assisting with development of the AMEP and USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 61

64 M&E Contributor Specialist (Robynne Locke) Roles and Responsibilities providing training to the M&E team on data collection form development and AMEP operationalization. Thereafter, she will provide training and technical assistance to the M&E team as needed. In addition, she will be responsible for conducting at least one internal DQA within the first two years of the project. USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 62

65 ANNEX I: PERFORMANCE INDICATOR REFERENCE SHEETS (PIRS) INDICATOR 1 GOAL RESULT INDICATOR TITLE Reduced GHG emissions from land-use sector KR 1. At least 41% of total CO2-equivalent emissions reduced from land use, land use change and deforestation averaged across all landscapes within the project scope Percentage reduction in GHG emissions as a result of USG assistance measured using actual emissions compared to REL (For the purpose of FACTS reporting, quantitiy of GHG emissions will be reported in metric tons of CO 2equivalent) PERFORMANCE INDICATOR DESCRIPTION INDICATOR LEVEL Outcome INDICATOR TYPE Custom Indicator contributes to FACTS PRECISE DEFINITION UNIT OF MEASURE This indicator reports the percentage of GHG emissions reduced, sequestered, and/or avoided, as a result of USG activities, as compared to a baseline level of GHG emissions. The baseline is the business-as-usual reference for GHG emissions that would have occured during the reporting period if there had been no USG intervention. At least 41% of total CO2-equivalent emissions will be reduced from land use, land use change and deforestation averaged across all landscapes within the project scope, based on the use of the IPCC Good Practice Guidance for Land Use, Land-Use Change and Forestry, and taking into consideration appropriate relevant national or subnational methods. Reduced emission will be achieved through LESTARI direct assistance, originating from reducing forest encroachment and improving peat land management with communities and protected area authorities, developing effective communities conservation agreements and better agricultural practices, engaging the private sector to implement best management practices in their concessions, and governance advocacy in changing licensing and embracing sustainable development visions that will stop deforestation. It is noted that for the purpose of FACTS reporting, quantity of GHG emissions will be reported in metric tons of CO2equivalent. Percentage (and metric tons of CO2equivalent) DISAGGREGATIONS Landscape RATIONALE OR JUSTIFICATION FOR INDICATOR Indicator used to track progress on reducing emissions from land use sector. The indicator is in line with GoI target to reduce emissions nationally as outlined in National Action Plan for Greenhouse Gas Emissions (RAN-GRK). USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 63

66 INDICATOR 1 DESIRED DIRECTION LINKAGE TO LONG TERM OUTCOME OR IMPACT Decreasing Reductions in deforestation and degradation will lead to reduction in GHG emissions. Reducing, sequestering, or avoiding GHG emissions will slow the rate of climate change and reduce the impacts. DATA COLLECTION APPROACH DATA COLLECTION METHOD DATA SOURCE FREQUENCY AND TIMING OF DATA ACQUISITION ESTIMATED COST OF DATA ACQUISITION INDIVIDUAL RESPONSIBLE DATA STORAGE LOCATION The data collection methodology for calculating reduced emissions is described in the Landscape Baseline Analysis (LBA) report update version (2017). In this revised LBA, the LESTARI project has chosen to use a different methodology based on the National Forest Reference Emissions Level (FREL) based on an evaluation of current methodologies and approaches used in Indonesia. The FREL methodology was chosen as it is the most acknowledged and recognized by international, national and local institutions in the forestry sector; and because FREL has been showing steady progress on proportioning targets to the provincial level. Additionally, the methodology used in FREL is consistent with its associated development of Measurement, Reporting and Verification (MRV) System developed by the Ministry of Environment and Forestry (MoEF). The FREL was not published in 2015 when the initial LBA was developed. The methodology applied to LESTARI that works in the forestry and land-use sector is based on: 1. Living biomass (above ground) GHG emissions from gross deforestation, and 2. Below ground GHG emissions from the decomposition of organic soils (peat) For the calculation of GHG emissions from Above Ground Biomass (AGB) carbon pools the National Forest Inventory Database developed by IPSDH is used. To calculate GHG emissions from peat decomposition similar LULUC maps from IPSDH and National Peat Land Map from BBSDLP. Peat decomposition is counted from all land cover types from the peat land area that was forested in Emission from fires have not been calculated in the FREL. The current methodology uses land cover data from Ministry of Environment and Forestry from the years 1990 through to Measurement of reduced emissions will use the same sourced data. Methodology applied for measuring the forest degradation will be referring to the LBA. Land Cover Data from the Ministry of Environment and Forestry Annual Publicly available data from the Ministry of Environment and Forestry Spatial Planning and GIS Specialist; Deputy Chief of Party LESTARI MIS; with supporting files kept at the LESTARI Office in Jakarta DATA QUALITY USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 64

67 INDICATOR 1 INITIAL DQA DATE OF FUTURE DQA PROCESS FOR FUTURE DQA KNOWN DATA LIMITATIONS ACTION TAKEN TO ADDRESS LIMITATIONS TBD Internally by the Tetra Tech ARD M&E team in October 2018 The project will employ a system of continuous adaptive management and therefore will audit data on a systematic basis before input into the project s MIS. For each data point reported to USAID, supporting documentation will be identified and reviewed using the process outlined in USAID s DQA Template. USAID LESTARI s ME&L Coordinator will ensure that each data point is supported with documentation and that data are assessed against data integrity standards as outlined in ADS The baseline data used in the analysis of deforestation and degradation and associated GHG emissions uses Ministry of Environment and Forestry (MOEF) wallto-wall land cover (penutupan lahan) data that has a reasonable accuracy in some parts of LESTARI landscapes especially Papua. GIS operators have frequently misclassified land use types and this is the most problematic but since this data will be used on a landscape scale such inaccuracies should not alter the overall accuracy of the emission reduction calculations. A potential source of inaccuracy is in peatland data (sourced from Soil Research Centre, Ministry of Agriculture) and land cover (MOEF). For instance, the incidence of dry land forest types occurring on peat in questionable. It is hard to state which data set is more accurate. At the scale of LESTARI landscape, this is not a major issue for the project, but administrations wishing to report on their site level emissions reductions should seek accurate data. Where unlikely combinations of land cover types and soils do not align, estimation of emissions should be treated with caution. To calculate emissions from fire, compromises must be made and average emissions from areas using available monitoring data have been applied to develop the LESTARI baseline. This is in line with provincial government approaches. Forest and land fire is a significant issue in Central Kalimantan and LESTARI has applied methodology adopted by the Central Kalimantan Province. Ground check and verification with stakeholders (especially established Spatial Data Infrastructure groups and networks that have been established through USAID IFACS) will be undertaken to ensure accuracy of land cover data. LESTARI will use percentage of reduction as project target, but will provide information in metric tons of CO2-equivalent to serve USAID global reporting purpose The time frame may not be much improved over the current 5-6 month data publishing time frame. DATA ANALYSIS REPORTING AND REVIEW DATA ANALYSIS Data analysis will be done by Spatial Planning and GIS Specialist Tabular by landscape PRESENTATION OF DATA Landscape Target Achievement % Completion xx % xx % USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 65

68 INDICATOR 1 xx tco2-eq xx tco2-eq REVIEW OF DATA REPORTING OF DATA Annually by ME&L Coordinator Annual Report BASELINE AND TARGETS BASELINE DATA SOURCE BASELINE NOTES LESTARI Landscape Baseline Analysis report. Baseline calculated following GoI methodology for Land-Based GHG Emission and Sequestration Baseline Calculations. The methodology used to calculate the baseline for LESTARI is based on the national approach. It is expected that this national level methodology will be improved in the future. For the LESTARI project, methodology used must evolve for the simple reason of delivering achievement at the end of the project. The data used for this methodology (land cover based on Landsat Imagery and developed by the MoEF) is normally available to the public at least 5-6 month after it was captured. LESTARI applied the base years of The table below presents a summary of emissions targets from deforestation, degradation and peat decomposition. No Landscape Reduced Deforestation AGB Emissions Target (-41%) (CO 2e) Total reduced Emission Target over life of Project Reduced Degradation AGB Emissions Target (- 41%) (CO 2e) Total reduced Emission Target over life of Project Range in Reduced Peat Decomp Emissions Target (- 41%) (CO 2e) Total reduced Emission Target over life of Project BASELINE VALUE 1 Leuser 604,775 3,023,875 57, ,773 2 Katingan Kahayan 4,889,915 24,449, , , , ,240 2,827,946-2,846,474 1,215,968 14,186,050 3 Lorentz Lowlands 589,937 2,949, ,938 3,594,689 1,016,098-1,133,975 5,375,181 4 Mappi Bouven Digoel 2,168,701 10,843, ,251 3,251, , ,250 2,891,085 5 Sarmi 87, , ,829 1,844,144 50,166-59, ,054 6 Cyclops 9,243 46, , BASELINE YEAR TARGETS NOTES The target set for LESTARI averaged across the landscapes is a 41% reduction from the baseline emissions by LESTARI proposes a progressive approach increasing the reductions of to reach this target by Annual emissions will be reduced progressively by 10%, 20%, 30%, 40% and 41% each year of the project. But emissions reductions will only start to be reported in the third year of the project. USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 66

69 INDICATOR 1 YEAR TARGETS ACTUAL OBSERVATIONS FY 2016 Not measured Not measured FY 2017 Not measured Not measured FY % reduction 45 M tco2-eq % reduction M tco2-eq FY % reduction 60 M tco2-eq % reduction M tco2-eq FY % reduction 75 M tco2-eq % reduction M tco2-eq Targets LOP 41% reduction 75 M tco2-eq % reduction M tco2-eq NOTES ABOUT CHANGES TO THE INDICATOR IMPORTANT: New LBA has been updated with new methodology and baseline year. Please refer to the document for detailed methodology and target calculations based on the update methodology. LAST SHEET UPDATED BY: Rachman Pasha ME&L Coordinator DATE: November 10, 2017 USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 67

70 INDICATOR 2 GOAL RESULT INDICATOR TITLE Carbon rich forest, peatland, mangrove ecosystems and the habitat of key species maintained KR 2. At least 8.42 million hectares of primary or secondary forest, including orangutan habitat, under improved management Number of hectares of biological significance and/or natural resources under improved natural resource management as a result of USG assistance PERFORMANCE INDICATOR DESCRIPTION INDICATOR LEVEL Outcome INDICATOR TYPE FACTS PRECISE DEFINITION Improved natural resource management includes activities that promote enhanced management of natural resources for one or more objectives, such as conserving biodiversity, sustaining soil or water resources, mitigating climate change, and/or promoting sustainable agriculture. Management should be guided by a stakeholder-endorsed process following principles of sustainable NRM and conservation, improved human and institutional capacity for sustainable NRM and conservation, access to better information for decision-making, and/or adoption of sustainable NRM and conservation practices. For LESTARI, an area is considered under "improved management when any one of the following occurs: 1. Forest Management Units (KPH-L and KPH-P) areas that include HPH, social forestry, and the other partnership formats; FMUs strengthened defined by RPHJP, RPHJPd, institutional capacity improved, co-management agreements implemented and effective protected forest management. Conservation Management and Monitoring Plan (CMMP), RIL-C or other conservation plan within private sector natural resources concessions (HPH) are in place and under implementation as evidenced by incorporation in business SOP, company budget and monitoring reports (or through sustained certification audits implemented by a third party) Co-management agreements and management arrangements among key stakeholders (area manager, community and/or provincial/district government other than FMUs) are in place and under implementation. 2. Conservation areas such as KSA/KPA areas, national park, TAHURA, hunting game reserve, nature preserve, wildlife sanctuary; An increased in the Management Effectiveness and Tracking Tool (METT) score for Protected Areas (PAs) 3. All other areas outside state forest including essential ecosystem in APL, buffer zone area (adjacent) Adaptive management is demonstrated; or on-the-ground management impacts are demonstrated (e.g., conservation and cultural set-asides created in Oil Palm, illegal roads closed, snares removed, no-fishing zones demarcated, etc) USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 68

71 INDICATOR 2 Forestry sector development is prioritized in local development plans as result of SEA-LEDS and/or LCP recommendations, which affect program, budget and/or improved zonation (e.g., certain area setaside for conservation, etc) especially in forested areas that are not yet covered by FMUs in Papua Reported as total number of hectares improved during the fiscal year in question, which can include maintained improvement in previously reported hectares and/or new, additional hectares. Improved management should be reported for activities where the LESTARI supported program was plausibly linked to the improvements observed. UNIT OF MEASURE DISAGGREGATIONS RATIONALE OR JUSTIFICATION FOR INDICATOR DESIRED DIRECTION LINKAGE TO LONG TERM OUTCOME OR IMPACT Hectares Areas under improved management will be disaggregated by landscape following further disaggregation of management category and landscape values (primary and secondary forest types, and peatland) as define in the LBA. Areas of Biologically significant areas (areas identified as important for biodiversity through national, regional, or global priority-setting processes such as Orangutan habitats, HCV in concession) will be noted for reporting against the biodiversity earmark. These areas will be a sub-set of the above areas under improved management and will not be counted in addition to them to avoid double counting. Measures of this indicator demonstrate progress towards sustainable natural resources governance and institutions, and can inform adaptive management of programs. This indicator is a reliable annual measure that demonstrates the magnitude of USG investments in biodiversity conservation and other natural resource sectors. Increasing A spatial indicator is an appropriate measure of the scale of impact of biodiversity conservation and/or NRM interventions. Good management of natural resources is a prerequisite for achieving improved biophysical condition of natural resources and contributes to reduction in deforestation and degradation. DATA COLLECTION APPROACH DATA COLLECTION METHOD DATA SOURCE FREQUENCY AND TIMING OF DATA ACQUISITION Field staff will compile for review implementation of co-management agreement, review sub-national government spatial plans, development plans and land use policy process (to identify changes in legal status), review implementation of CMMPs, review implementation of FMUs management plan, review of METT report and site-observation to check demonstrated adaptive management (for example as results of SMART Patrol in CAs). The GIS team will conduct geospatial analysis of remotely sensed imagery and GIS layers. GIS, co-management agreement, social forestry management plans, METT report, CMMP document, PHPL report, FMUs management plan, conservation management plans, managements plans, SMART report, site-visit reports, land cover maps from Ministry of Environment and Forestry (MoEF), spatial plans, procurement of remotely sensed imagery (when feasible). Annual USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 69

72 INDICATOR 2 ESTIMATED COST OF DATA ACQUISITION INDIVIDUAL RESPONSIBLE DATA STORAGE LOCATION Partial cost of remotely sensed imagery allocated to this activity, ground check, staff time for data analysis and interpretation, and cost of photocopies for conservation management plans and spatial plans. Spatial Planning and GIS Specialist; Component Team Leaders LESTARI MIS; with supporting files kept at the LESTARI Office in Jakarta DATA QUALITY INITIAL DQA TBD DATE OF FUTURE DQA Internally by the Tetra Tech ARD M&E team in October 2018 PROCESS FOR FUTURE DQA KNOWN DATA LIMITATIONS ACTION TAKEN TO ADDRESS LIMITATIONS The project will employ a system of continuous adaptive management and therefore will audit data on a systematic basis before input into the project s MIS. For each data point reported to USAID, supporting documentation will be identified and reviewed using the process outlined in USAID s DQA Template. USAID LESTARI s ME&L Coordinator will ensure that each data point is supported with documentation and that data are assessed against data integrity standards as outlined in ADS Precision is low: improved management is a relative term, and narrative is required to explain the quality of this management improved. Equal weight is given to unequal improvements along a continuum: eg. creating, adopting and implementing management plans may each be an improvement over a baseline. Likewise, a small management improvement across a large area may be as important as a large improvement across a small area. Conservation management plans may not be well described spatially. Conservation management plans will be interpreted by trained LESTARI staff. Locations of all activity areas carried out by partners will be located by LESTARI field staff using GPS and recorded into electronic data form. Benchmark that are being used within the project to gauge success will be articulated clearly and a short narrative to describe the benchmarks that have been reached in the past year will be provided. DATA ANALYSIS REPORTING AND REVIEW DATA ANALYSIS Field staff will compile for review conservation management plans, maps and subnational government spatial plans and other relevant documents mentioned in Data Source. The GIS team will conduct geospatial analysis of remotely sensed imagery and GIS layers. Landscape hectare totals will be aggregated into a tabular format, with accompanying narrative. The link to USG assistance will be established through periodic assessments conducted by LESTARI staff. For those areas already improved under IFACS, LESTARI will report these areas as project achievement should there is new level of improvement made under LESTARI. Narratives will be provided to describe new level of improvement. Data presentation will be disaggregated between improved areas under IFACS (e.g., concession area, area under Community Conservation and Livelihood Agreements, etc) and new areas under LESTARI. Analysis will also be made to present data on orangutan habitats. Orangutan habitats will be reported as a sub-set of the areas as they may occur inside USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 70

73 INDICATOR 2 conservation areas or FMU areas etc. these areas will not be accrued to the areas already counted under improved management so avoiding double counting. Tabular PRESENTATION OF DATA LANDSCAPE Primary Forest Secondary Forest Primary Swamp Forest Secondary Swamp Forest Secondary Mangrove Forest Non-forested Peat Land Non-Forested / Non- Peat # Hectares of Orang Utan Habitat # Hectare of HCV Conservati on Area (ha) FMU (KPHL / KPHP) (ha) Other Areas (APL) % completion REVIEW OF DATA Annually by ME&L Coordinator REPORTING OF DATA Annual Report BASELINE AND TARGETS BASELINE DATA SOURCE BASELINE NOTES LESTARI Landscape Baseline Analysis report, METT baseline report, SMART baseline report, existing spatial planning and project activity progress report to determine the level of current management. Project activities to improve current management will be started in FY 1, therefore baseline for this indicator is assumed as zero because no improved management yet observed. Analysis of existing condition will include business-as-usual scenario in order to be able to attribute to USG assistance. BASELINE VALUE 0 BASELINE YEAR 2015 TARGETS NOTES Hectares under improved management will only start to be reported in the second year of the project with the assumption that sufficient time is allowed for impact to occur. YEAR TARGETS ACTUAL OBSERVATIONS FY 2016 FY 2017 FY 2018 FY 2019 FY 2020 Not measure 2.4 M 1.7 M 3.3 M 1.3 M USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 71

74 INDICATOR 2 Targets LOP 8.7 M NOTES ABOUT CHANGES TO THE INDICATOR Learning through 2 years project implementation, LESTARI has developed Roadmap to Hectares that described the realistic target that can be achieved on annual basis. The target area of under improved management were consolidated into from six into three categories; The disaggregation of hacters reported is now based on the values within landscapes as defined by the revised Landscape Baseline Analisis (LBA). LAST SHEET UPDATED BY: Rachman Pasha ME&L Coordinator DATE: November 10, 2017 USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 72

75 INDICATOR 3 GOAL RESULT INDICATOR TITLE Reduced threats on key species IR 1. Improved forest management Percentage reduction in poaching in focus area PERFORMANCE INDICATOR DESCRIPTION INDICATOR LEVEL INDICATOR TYPE PRECISE DEFINITION Outcome Custom Indicator Poaching is the illegal hunting, killing or capturing of animals, a practice that occurs in a variety of ways. Poaching can refer to the failure to comply with regulations for legal harvest, resulting in the illegal taking of wildlife that would otherwise be allowable. Examples include: hunting, killing or collecting wildlife that is listed as endangered by IUCN and protected by law such as CITES, taking without a license or permit, use of a prohibited weapon or trap, taking outside of the designated time of day or year, and taking of a prohibited sex or life stage. Poaching can also refer to the taking of animals from a gazzetted wildlife sanctuary, such as a national park, game reserve, or zoo. Wildlife biologists and conservationists consider poaching to have a detrimental effect on biodiversity both within and outside protected areas as wildlife populations decline, species are depleted locally, and the functionality of ecosystem is disturbed. Focus area are defined as the area within protected areas targeted for regular patrols due to the high level of threats (e.g., accessibility). Patrols are targeted and not all areas need to be patrolled. To monitor a reduction in poaching, LESTARI will use SMART (Spatial Monitoring and Reporting Tool). SMART is designed to improve anti-poaching efforts and overall law enforcement effectiveness in established conservation area management zones. SMART enables the collection, storage, communication, and evaluation of data on: patrol effort (e.g. the time spent on patrols, effective area patrol and the distance of patroling), patrol results (e.g. number of snares removed, arrests made), and threat levels (e.g. the number of paths into the protected areas, areas deforested or number of tress felled illegally). A keystone species are defined a species that has a disproportionately large effect on its environment relative to its abundance or thread by large scale poaching for business reason. Such species are described as playing a critical role in maintaining the structure of an ecological community, affecting many other organisms in an ecosystem and helping to determine the types and numbers of various other species in the community. List of keystone animal species in LESTARI Landscape as follows: 1. Leuser National Park: Sumatran Tiger, Sumatran Elephant, Sumatran Rhino, Sumatran Orang Utan 2. Rawa Singkil Wildlife Santuary: Sumatran Orang Utan 3. Sebangau National Park: Borneo Orang Utan 4. Bukit Baka Bukit Raya National Park: Borneo Orang Utan 5. Cyclop Natural Reserve: Cendrawasih, Dingiso Tree Kangoro 6. Lorentz National Park: Pig-nosed Turtle USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 73

76 INDICATOR 3 UNIT OF MEASURE Number of Poaching Incidents per Km DISAGGREGATIONS Conservation Area RATIONALE OR JUSTIFICATION FOR INDICATOR DESIRED DIRECTION LINKAGE TO LONG TERM OUTCOME OR IMPACT Indicator used to track increased commitment of key stakeholders regarding the protection of key species and conservation of their habitat. Decreasing When effectively monitored and enforced, reduce in poaching can help to substantially improve protection of wildlife and their habitats. DATA COLLECTION APPROACH DATA COLLECTION METHOD DATA SOURCE FREQUENCY AND TIMING OF DATA ACQUISITION ESTIMATED COST OF DATA ACQUISITION INDIVIDUAL RESPONSIBLE DATA STORAGE LOCATION Review regular patrol reports and patrol maps (monthly or quarterly); undertake in-depth data analysis with an evaluation of various trends in patrol perfromance and threat-levels, as well as an evaluation of the entire patrol management system (annually). Patrol reports; direct observation; annual evaluation Rolling as data available Staff labor costs; monitoring and evaluation cost Biodiversity Conservation Advisor LESTARI MIS; with supporting files kept at the LESTARI Office in Jakarta DATA QUALITY INITIAL DQA DATE OF FUTURE DQA PROCESS FOR FUTURE DQA TBD Internally by the Tetra Tech ARD M&E team in October 2018 The project will employ a system of continuous adaptive management and therefore will audit data on a systematic basis before input into the project s MIS. For each data point reported to USAID, supporting documentation will be identified and reviewed using the process outlined in USAID s DQA Template. USAID LESTARI s ME&L Coordinator will ensure that each data point is supported with documentation and that data are assessed against data integrity standards as outlined in ADS USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 74

77 INDICATOR 3 KNOWN DATA LIMITATIONS Using the SMART software and establishing a patrol database will not, on its own, improved protection of a conservation area. In addition to the SMART software and database, basic enforcement capacity and infrastructure must be in place. ACTION TAKEN TO ADDRESS LIMITATIONS LESTARI will combine the utilization of SMART with introduction of adaptive patrol management practices through promotion of feedback mechanisms between managers and rangers to discuss patrol efforts and results and set new patrol targets. This intervention is closely linked with METT intervention as to support overall improvement of conservation area management capacity. DATA ANALYSIS REPORTING AND REVIEW SMART allows the calculation of poaching incidence, per unit of effort such as the number of snares removed per kilometer patrolled in high risk areas, and therefore can be used to show a change in paoching incidence even if the number of patrols varies over time. During the first 6 months, SMART patrols and observation in the field will identify areas of high risk, and develop a baseline of the average number of incidents per kilometer traveled on foot patrols. The base unit will be no. incidents / km measured semi-annually. DATA ANALYSIS Application of effort-related base units is due to the fact that the number of poaching incidents is correlated with patrol effort. The ore one patrols, the more incidents are found. With sustained patrol, however, the assumption is that the incidents will start to decrease as the law is enforced. A semi-annual evaluation to determine trends in poaching levels will be made, as well as semi-annual evaluations of the threat levels within conservation areas to maintain targeting of patrol effort in high risk areas. The reduction in poaching will only be reported for areas where patroling is programed and implemented on a regular basis. The individual Poaching reduction will be reported individually for each conservation areas and for the project as a whole. If different protected areas are including in the anti-poaching effort facilitated by the project, overall effort in patroling, a weighted average will be applied to report the LESTARI project-wide poaching redcution effort. Tabular with supporting narrative, as appropriate PRESENTATION OF DATA Landscape Conservation Area Target Achievement % of Completion REVIEW OF DATA REPORTING OF DATA Semi-annually by ME&L Coordinator Annual Report BASELINE AND TARGETS BASELINE DATA SOURCE SMART patrol result in Year 1 USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 75

78 INDICATOR 3 BASELINE NOTES BASELINE VALUE LESTARI will undertake baseline in Year 1 to determine number of current poaching incident in targeted conservation areas. TBD BASELINE YEAR 2015 TARGETS NOTES Numbers of poaching will be reduced progressively by 10%, 20%, 30%, and 40% each year of the project compared to baseline. But achievement will only start to be reported in the second year of the project. YEAR TARGETS ACTUAL OBSERVATIONS FY 2016 Not measure FY % FY % FY % FY % Targets LOP 40% NOTES ABOUT CHANGES TO THE INDICATOR LAST SHEET UPDATED BY: Rachman Pasha ME&L Coordinator DATE: September 25, 2017 USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 76

79 INDICATOR 4 GOAL RESULT INDICATOR TITLE Reduced rate of deforestation and degradation IR 2. Improved land use governance KR 6. Increased commitment of key private sector, government, and community stakeholders regarding the positive benefits of conservation and sustainable use of forests and the species they encompass KR 7. Policies, laws, regulations, and procedures in support of low emission development and forest conservation and management increased, promulgated, and enforced at all levels Number of public policies introduced,adopted, repealed, changed or implemented consistent with citizen input PERFORMANCE INDICATOR DESCRIPTION INDICATOR LEVEL Outcome INDICATOR TYPE FACTS PRECISE DEFINITION UNIT OF MEASURE Public policies include any law, regulation, policy or similar directive that is formally adopted by either the legislative branch or a unit of the executive branch at any level. Citizen inputs means that the public, citizens and/or civil society organizations have proposed language used in, provided comments incorporated into, or monitored the implementation of the policy. Introduced refers to draft legislation formally being presented and accepted for consideration by a legislative or executive institution. Adopted refers to the process by which a public policy, whether a completely new policy or one that has been changed, is formally adopted through decree, passage of law or other formal process. Repealed refers to existing or draft policies that are removed or prevented from establishment. Changed refers to an existing policy that has been substantively changed. Implemented means that the policy has been operationalized. Policies may cover LESTARI landscapes, e.g., acceleration of community forestry licensing in support of the national goal of 12.7 million hectares. Policies may explicitly target LESTARI landscapes, e.g., collaborative management of Cyclops. For interpretation of this indicator, a qualitative description will be provided to explain what the number represents, particularly: What is the title of the measure? At what stage is it? (e.g., officially introduced, adopted, repealed, changed, or implemented?) How does the measure contribute to sustainable landscapes or biodiversity conservation? What is/are the institution(s) that will be implementing and/or enforcing the measure and at what scale (e.g., national, provincial, district) Number of public policies USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 77

80 INDICATOR 4 DISAGGREGATIONS RATIONALE OR JUSTIFICATION FOR INDICATOR DESIRED DIRECTION LINKAGE TO LONG TERM OUTCOME OR IMPACT District; Provincial; and National Sustainable landscapes policy and biodiversity policy The indicator is used by LESTARI to ensure that there is willingness from local government to adopt input from citizen-based mechanisms, or in other word, to ensure that development policies are responsive to the public. Increasing Democratic governance concerns are integral to improving policy reform and implementation across a wide range of development sectors. More effective policy change across development sectors requires government policy makers (in both the executive and legislative branches) to consider and incorporate citizen input into the policy development process. Also, an improved enabling environment through legal and policy reform is essential for ensuring that efforts and investments in sustainable landscapes and/or biodiversity conservation have legal and strategic backing and institutional ownership. DATA COLLECTION APPROACH DATA COLLECTION METHOD DATA SOURCE FREQUENCY AND TIMING OF DATA ACQUISITION ESTIMATED COST OF DATA ACQUISITION INDIVIDUAL RESPONSIBLE DATA STORAGE LOCATION Review project reports, policy documents, tracking policy development process, direct observation, analysis of secondary data and third party observation (i.e. monitoring of public policy by local NGO). Activity reports, policy documentation, assessment reports, policy monitoring reports Rolling, as data are available Staff labor costs; policy monitoring cost; photocopies of documents Decentralized Governance Specialist; Stakeholder Engagement Advisor LESTARI MIS; with supporting files kept at the LESTARI Office in Jakarta DATA QUALITY INITIAL DQA DATE OF FUTURE DQA PROCESS FOR FUTURE DQA TBD Internally by the Tetra Tech ARD M&E team in October 2018 The project will employ a system of continuous adaptive management and therefore will audit data on a systematic basis before input into the project s MIS. For each data point reported to USAID, supporting documentation will be identified and reviewed using the process outlined in USAID s DQA Template. USAID LESTARI s ME&L Coordinator will ensure that each data point is USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 78

81 INDICATOR 4 supported with documentation and that data are assessed against data integrity standards as outlined in ADS KNOWN DATA LIMITATIONS ACTION TAKEN TO ADDRESS LIMITATIONS There are some limitations to validity since the quality of citizen input is very subjective. Reliability is also of concern since policy reform is not always a linear process. Because the intended result is an improved enabling environment, the number of public policies provides only a partial measure of success, given that effective implementation and enforcement are also critical. Public policies might also not be well-designed or effective. Narrative is critical for interpreting this indicator. Also if the policies may affect climate or biodiversity indirectly, LESTARI will provide narrative to explain the connection. DATA ANALYSIS REPORTING AND REVIEW DATA ANALYSIS A citation analysis method will be used to evaluate uptake and use. This method involves looking at the extent to which citizen recommendations are visibly picked up and used by local governments. Analysis will include triangulation of secondary data and third party observation (when available). Case study will be deployed to assess the progress made along the way in terms of convening stakeholders, drafting, approving, and implementing/enforcing policies. It should be noted that the policy introduced, changed or adopted may take concern provincial and national regulations. In this case LESTARI will still count it as a single policy. Data analysis for this indicator will involve assessing a series of milestones. We will count this indicator achieved once the policy is either introduced, adopted, repealed, changed or However, the project will still monitor and report the number of policies that are implemented or enforced. Key analytical questions include: how many (percentage) of policies get adopted? Of those that get adopted, what percentage are implemented? How many of the policies that were adopted and implemented had substantial participatory events as part of the development process? Type of Policy Introduced Adopted Repealed Changed Implemented Tabular with supporting narrative, as appropriate PRESENTATION OF DATA Landscape District, Provincial, National Target Achievement % Completion REVIEW OF DATA REPORTING OF DATA Quarterly by ME&L Coordinator Quarterly and Annual Report BASELINE AND TARGETS USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 79

82 INDICATOR 4 BASELINE DATA SOURCE BASELINE NOTES N/A For this indicator, the baseline is zero BASELINE VALUE 0 BASELINE YEAR TARGETS NOTES N/A It is assumed that in each of the 6 landscapes and national level government at least four policies (concerning sustainable landscapes and biodiversity) will be influenced as a result of advocacy activities. However, it should be noted that the intended result is an improved enabling environment. Therefore the numbers provide only a partial measure of success. Effective implementation and enforcement are much more critical which in turn could be reflected through achievement of LESTARI key goals: reducing emissions and conserving biodiversity. YEAR TARGETS ACTUAL OBSERVATIONS FY FY FY FY FY Targets LOP 28 NOTES ABOUT CHANGES TO THE INDICATOR LAST SHEET UPDATED BY: Rachman Pasha ME&L Coordinator DATE: November 10, 2017 USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 80

83 INDICATOR 5 GOAL RESULT INDICATOR TITLE Reduced rate of deforestation and degradation IR 2. Improved land use governance KR 7. Policies, laws, regulations, and procedures in support of low emission development and forest conservation and management increased, promulgated, and enforced at all levels Number of sub-national government with improved licensing and permitting mechanism PERFORMANCE INDICATOR DESCRIPTION INDICATOR LEVEL INDICATOR TYPE PRECISE DEFINITION UNIT OF MEASURE Outcome Custom Indicator Sub-national government includes province and district/municipality. The term improved is taken to subsume the five core principles of good governance. First and foremost transparency, inclusivity/participation (e.g., village voice in license approval), accountability (clear focal point of responsibility for legally-approved licenses and their consequences), responsiveness (responding to public or sectoral complaints) and timeliness. For new licenses, improved also means that all new requests for licenses and permits for a new concession area for natural resources extraction activities (i.e., mining, oil palm plantation and logging/industrial timber plantation) follow Standard Operational Procedures (SOP) that developed or updated based on the government regulation and procedures for granting licenses and permits. These also referring that new licenses should be more transparent and accessible by public for monitoring. Improved licensing and permitting mechanism are targeted (but not limited) to Izin Prinsip, Izin Lokasi, AMDAL, and others according to the type of the private sectors that are operating within LESTARI landscapes. Number of sub-national government units DISAGGREGATIONS Province, District RATIONALE OR JUSTIFICATION FOR INDICATOR DESIRED DIRECTION LINKAGE TO LONG TERM OUTCOME OR IMPACT The indicator is used by LESTARI to track the progress of strengthening governance process in the respective sub-national government regarding the issuance of forest licenses and permits. The indicator also illustrates the extent to which spatial plans policy enforced. Increasing Improved licensing and permitting process will ensure the certainty of land use and transparency, reduce potential conflicts between private sectors and communities, and address driver of deforestation and degradation. DATA COLLECTION APPROACH DATA COLLECTION METHOD Review the maps indicates existing licenses, case study to assess adoption of good governance principles into licensing and permitting mechanism, review the licensing process and SOP, interview government licensing officers and business USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 81

84 enterprises, analysis of secondary data and third party observation (i.e. monitoring of forest licensing by local NGO). DATA SOURCE FREQUENCY AND TIMING OF DATA ACQUISITION ESTIMATED COST OF DATA ACQUISITION INDIVIDUAL RESPONSIBLE DATA STORAGE LOCATION Maps, SOP of licensing and permitting process, activity reports, monitoring/ assessment reports Rolling, as data are available Staff labor costs; case study cost; photocopies of documents Sustainability Screening Specialist; Private Sector Engagement Coordinator, Stakeholder Engagement Advisor LESTARI MIS; with supporting files kept at the LESTARI Office in Jakarta DATA QUALITY INITIAL DQA DATE OF FUTURE DQA PROCESS FOR FUTURE DQA KNOWN DATA LIMITATIONS ACTION TAKEN TO ADDRESS LIMITATIONS TBD Internally by the Tetra Tech ARD M&E team in October 2018 The project will employ a system of continuous adaptive management and therefore will audit data on a systematic basis before input into the project s MIS. For each data point reported to USAID, supporting documentation will be identified and reviewed using the process outlined in USAID s DQA Template. USAID LESTARI s ME&L Coordinator will ensure that each data point is supported with documentation and that data are assessed against data integrity standards as outlined in ADS Incomplete regulatory rules and the lack of synergy between different plans (the National Spatial Land use Plan (RTRWN), the Provincial Spatial Land use Plans (RTRWP) and the District Spatial Land use Plans (RTRWK), opens the door to weak implementation of the regulations and corruption. As a result land may be inappropriately zoned or zones may be changed in order to allow logging or other extraction. Without adequate plans or consultation with local communities, the current zoning of land does not provide a basis on which to make licensing decisions, and therefore such decision-making processes are open to manipulation and undue influence by loggers or extractors. Furthermore, there are no official institutions tasked with monitoring the zoning of land and in the absence of accurate data and maps on logging/extraction zones independent monitoring is difficult. LESTARI will develop capacity of local CSOs to monitor the forestry license activities through research and analysis supported by advocacy and Geographic information System (GIS) analysis. Spatial Data Infrastructure (SDI) initiated under IFACS will be strengthened to improve the accuracy, consistency, and availaibility of land use information for public use. DATA ANALYSIS REPORTING AND REVIEW DATA ANALYSIS Technical review will be carried out to determine transparency and accessibility of licensing and permitting information. Also to review if new licenses have followed USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 82

85 the SOP/screening process. The analysis will be enriched by qualitative assessment through interview processes toward government officers and business enterprises. LESTARI will track intermediate milestones when counting progress toward meeting these targets. Milestones Subnational level Review and align licenses and permits with spatial plan Screening tool developed - Protocol and procedures for public monitoring developed Communication and consultation with local government and approval received for screening tool and monitoring protocol Official public socialization & training for the screening tool The SST incorporated in SOP for licensing and permitting Tabular with supporting narrative, as appropriate PRESENTATION OF DATA Landscape Sub-national level Target Achievement % Completion REVIEW OF DATA REPORTING OF DATA Quarterly by ME&L Coordinator Quarterly and Annual Report BASELINE AND TARGETS BASELINE DATA SOURCE BASELINE NOTES N/A For this indicator, the baseline is zero BASELINE VALUE 0 BASELINE YEAR TARGETS NOTES N/A Achievement of target will only start to be reported in the second year of the project because a considerable time is needed to develop SOP and enhance local officers capacity to implement SOP. YEAR TARGETS ACTUAL OBSERVATIONS FY 2016 Not measure FY FY FY USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 83

86 FY Targets LOP 8 NOTES ABOUT CHANGES TO THE INDICATOR The fifth milestone is revised based on the realistic condition on the SST development. The adoption of SST into SOP of issuance permit and license by local government is considered realistic within project period. The target also reduced from 14 districts into 8 districts. LAST SHEET UPDATED BY: Rachman Pasha ME&L Coordinator DATE: September 25, 2017 USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 84

87 INDICATOR 6 GOAL RESULT INDICATOR TITLE Reduced rate of deforestation and degradation IR 2. Improved land use governance Number of SEAs/LCPs recommendations related with land use, spatial plan, forest management, and biodiversity conservation incorporated into subnational government policy, planning and program PERFORMANCE INDICATOR DESCRIPTION INDICATOR LEVEL INDICATOR TYPE PRECISE DEFINITION UNIT OF MEASURE Outcome Custom Indicator High quality SEA-LEDS and/or LCP recommendations that are relevant to LESTARI are defined as: The SEA-LEDS document produced or updated follow the government standard (e.g., Permendagri 67/2012, PP 46/2016); Relevant recommendations incorporated into Policy, Planning, and Program (Kebijakan, Rencana, Program or KRP) especially those that are programmed and budgeted by sub-national government which explicitly referred in the RPJMD or Renstra (Dinas/SKPD), RTRW, executive decrees which directly improve forest management and reduce external threats SEA recommendations include but not be limited to: - Aceh: RPJMD and Renstra SKPD focus on KPH forest program and budgeting, co-management and livelihood, wildlife conservation, water resources, and improved licensing of natural resources management; - Central Kalimantan: RPJMD and Renstra SKPD focus on KPH forest program and budgeting, co-management and livelihood, peatland hydrology and improved licensing of natural resources management - Papua: RTRW focus on mangrove conservation, co-management and livelihood, wildlife conservation, water resources, and improved licensing of natural resources management, land use rationalization, incorporation of LCP in SEA Number of SEA/LCP recommendations. DISAGGREGATIONS Sub-national governments RATIONALE OR JUSTIFICATION FOR INDICATOR DESIRED DIRECTION LINKAGE TO LONG TERM OUTCOME OR IMPACT The indicator is used to measure progress of KRP for improved land use governance. Due to long periods required for KRP implementation across the various levels of government, it is crucial to keep monitoring the process and find other potential entry points to maximize impact through incorporating SEA-LEDS & LCPs into KRP which will secure financing for implementation. Increase adoption of SEA recommendations improves forest management and safeguards forest areas from external threats. Increasing Integrating environmental consideration into strategic decision-making can improve the quality of local planning, ensure sustainability, and change the land-use practices thus contributing to emission reduction and biodiversity conservation. USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 85

88 INDICATOR 6 DATA COLLECTION APPROACH DATA COLLECTION METHOD DATA SOURCE FREQUENCY AND TIMING OF DATA ACQUISITION ESTIMATED COST OF DATA ACQUISITION INDIVIDUAL RESPONSIBLE DATA STORAGE LOCATION Technical review of SEA-LEDS and/or LCPs recommendations in KRP (RPJMD, Renstra, RTRW, Executive decrees). Interviews will be conducted with relevant government officers to evaluate recommendations for uptake and use in KRP and obtaining sufficient budget allocation. SEA document, LCP document, Spatial Plan document, Mid-term development Plan document, assessment report of SEA development/updates, Strategic plan (Renstra), Executive decrees, annual local government plan (RKPD) and other relevant policy document. Rolling, as data are available Staff labor costs; assessment cost; photocopies of documents Forest Governance Advisor LESTARI MIS; with supporting files kept at the LESTARI Office in Jakarta DATA QUALITY INITIAL DQA DATE OF FUTURE DQA PROCESS FOR FUTURE DQA KNOWN DATA LIMITATIONS ACTION TAKEN TO ADDRESS LIMITATIONS TBD Internally by the Tetra Tech ARD M&E team in October 2018 The project will employ a system of continuous adaptive management and therefore will audit data on a systematic basis before input into the project s MIS. For each data point reported to USAID, supporting documentation will be identified and reviewed using the process outlined in USAID s DQA Template. USAID LESTARI s ME&L Coordinator will ensure that each data point is supported with documentation and that data are assessed against data integrity standards as outlined in ADS This indicator measures SEA-LEDS & LCP recommendations incorporated into government policy, planning and program (KRP) for forest and land use management,especially FMU development and conservation management, as well as KRP from other sectors that potentially threaten forest areas. Build sustainability within the process to ensure that the recommendations are accepted and then implemented. Building sustainability is increased as a result of specific and targeted public consultation by LESTARI to bring the community and private sector and local government together to arrive at consensus and commitment to implement the plan. DATA ANALYSIS REPORTING AND REVIEW DATA ANALYSIS Citation analysis method will be used to evaluate uptake and use. This method involves looking at explicit reference to adopting SEA recommendation of USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 86

89 INDICATOR 6 improved-case scenario in government policy, planning and program (KRP) document. Because policy improvement involves a multi-step process, LESTARI will track intermediate milestones when counting progress toward meeting these targets. Milestones Subnational level Recommendation of high quality SEAs utilized for improve forest and land use management SEA recommendations actively conveyed to sub-national decision makers SEAs operationalized by being incorporated into KRP impacting improved forest and land use management (directly and indirectly through safeguarding from external threats) including but not limited to spatial plan revision, RPJMD, Renstra SKPD and other agency planning documents. Highest priority given to incorporating SEA recommendation into programing and budgeting for forest and land use management,especially FMU development and conservation management Tabular with supporting narrative, as appropriate PRESENTATION OF DATA Landscape Sub-national Target Achievement % Completion REVIEW OF DATA REPORTING OF DATA Quarterly by ME&L Coordinator Quarterly and Annual Report BASELINE AND TARGETS BASELINE DATA SOURCE BASELINE NOTES SEA/LCP recommendations which initiated since IFACS and implemented during LESTARI and/or new recommendation that being incorporate into KRP and being implemented during LESTARI For this indicator, the baseline is zero BASELINE VALUE 0 BASELINE YEAR TARGETS NOTES N/A N/A USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 87

90 INDICATOR 6 YEAR TARGETS ACTUAL OBSERVATIONS FY FY FY FY FY Targets LOP 6 NOTES ABOUT CHANGES TO THE INDICATOR This indicator is revised to better demonstrate operationalization of SEA/LEDS and/or LCP recommendations. Operationalization will occur when one or more recommendations are incorporated into government policy, planning and program (KRP) relevant to forest management and safeguarding from external threats to the forest. Incorporation into KRP will secure budget for implementation, thus assuring operationalization. The target also reduced from 14 districts into 6 recommendations within LESTARI landscapes LAST SHEET UPDATED BY: Rachman Pasha ME&L Coordinator DATE: November 10, 2017 USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 88

91 INDICATOR 7 GOAL RESULT INDICATOR TITLE Reduced rate of deforestation and degradation IR 2. Improved land use governance Number of Multi Stakeholder Initiatives (MSIs) or civil society advocacy initiatives which formulate and deliver policy-relevant inputs on sustainable forest and land management to decision makers PERFORMANCE INDICATOR DESCRIPTION INDICATOR LEVEL INDICATOR TYPE PRECISE DEFINITION Outcome Custom Indicator Civil society advocacy initiatives are events or processes through which more than one civil society organisation (CSOs) or citizen group formulate policyrelevant recommendations (based on studies, investigation, or their own needs), and deliver them to the government, or publicize them through local media (print/radio/etc.). Recommendations must be relevant to policies on land and forest management. An initiative must be more than a single discussion or public statement; for instance, a series of at least two discussions with several stakeholders (who can be from the same type of stakeholder such as CSOs). A multi-stakeholder initiative (MSI) is a series of interactions (discussions, etc.) through which a set of relevant stakeholders (for example including local government officials, CSOs, citizens, private sector representatives) debate/discuss/analyze an issue of land and forest management or biodiversity conservation and then formulate recommendations relevant to policy design, revision, implementation or enforcement. Recommendations must be delivered to relevant decision making policy makers (if not already participating in the MSI). MSIs may occur in the form of Multi-Stakeholder Fora, with a stable institutional structure and regular discussion, or they may be more temporary processes created around a particular issue and relevant set of stakeholders. The key is that the MSI leads to substantive interaction between stakeholders, clear recommendations which can be implemented, and the delivery of these recommendations to the relevant policy makers. In some contexts, it may not be feasible to create a full MSI, and thus civil society advocacy (as above) may be the appropriate route for amplifying citizen voice on this issue. Where there is strong government engagement on the issue, an MSI should be established to make further progress towards solutions which consider the needs and interests of multiple stakeholder groups. MSIs thus serve as a bridge between citizens and local government and foster trust-based relationships built upon mutual respect. They could be focused on monitoring policy implementation, implementing policy in effective ways, or devising/revising suitable policies. The MSIs and civil society initiatives will focus on particular topics of relevance to LESTARI objectives, but also be aligned with the interests of local stakeholders and the political context of the province or district. An indicative list of such topics would include but not be limited to: - Aceh: Forest program and budgeting, co-management and livelihood, wildlife conservation, water resources, and SST; USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 89

92 INDICATOR 7 - Central Kalimantan: Co-management and livelihood, wildlife conservation, peat restoration, forest concessions; - Papua: SST, FMU and mangrove management, land rationalization in Boven Digoel UNIT OF MEASURE Number initiatives DISAGGREGATIONS District or province RATIONALE OR JUSTIFICATION FOR INDICATOR DESIRED DIRECTION LINKAGE TO LONG TERM OUTCOME OR IMPACT This indicator is used to measure the extent to which citizen-based mechanisms for public input are successfully strengthened as a result of USG assistance. Increasing The success of civil society advocacy and MSIs to bridge communication between local government and wider public in land use will lead to more transparent, participatory and accountable land use decisions and policies, which weigh the needs of various stakeholders. Those principles are prerequisite of good governance. DATA COLLECTION APPROACH DATA COLLECTION METHOD DATA SOURCE FREQUENCY AND TIMING OF DATA ACQUISITION ESTIMATED COST OF DATA ACQUISITION INDIVIDUAL RESPONSIBLE DATA STORAGE LOCATION Review reports from regional technical team on MSIs and civil society advocacy; review the process and products of MSF activities, stakeholders involved, recommendations, delivery of recommendations, and reception by decision makers; qualitative assessment to determine if the definition above is satisfied. Activity reports, SEA/LCPs process, assessment reports Rolling, as data are available Staff labor costs; assessment cost; photocopies of documents Stakeholder Engagement Advisor LESTARI MIS; with supporting files kept at the LESTARI Office in Jakarta DATA QUALITY INITIAL DQA DATE OF FUTURE DQA PROCESS FOR FUTURE DQA TBD Internally by the Tetra Tech ARD M&E team in October 2018 The project will employ a system of continuous adaptive management and therefore will audit data on a systematic basis before input into the project s MIS. For each data point reported to USAID, supporting documentation will be USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 90

93 INDICATOR 7 identified and reviewed using the process outlined in USAID s DQA Template. USAID LESTARI s ME&L Coordinator will ensure that each data point is supported with documentation and that data are assessed against data integrity standards as outlined in ADS KNOWN DATA LIMITATIONS ACTION TAKEN TO ADDRESS LIMITATIONS Activity reports may not always be complete Will require supporting documentation and observation to verify results DATA ANALYSIS REPORTING AND REVIEW The advocacy process can be delivered through two ways; Civil society advocacy initiatives or A multi-stakeholder initiatives (MSI). These process involves multiple steps as follows: DATA ANALYSIS 1. Involving at least two institutions/organisation or citizen group which can be from the same or different type of stakeholder. 2. Discuss the different topics or issues through debate/discuss/analyze an issue that relate to land and forest management. 3. The process involved more than a single discussion or public statement. 4. Formulate policy-relevant recommendations (design, revision, implementation or enforcement) which must be relevant to land and forest management. 5. Recommendations must be delivered to relevant decision making policy makers (if not already participating in the MSI). We will count this indicator achieved once the recommendation is delivered to the government; however, the project will still monitor and report the implementation of MSI. Tabular with supporting narrative, as appropriate PRESENTATION OF DATA Landscape MSI Target Achievement % Completion REVIEW OF DATA REPORTING OF DATA Quarterly by ME&L Coordinator Quarterly and Annual Report BASELINE AND TARGETS BASELINE DATA SOURCE BASELINE NOTES N/A While IFACS established some MSFs but many then became dormant.these are not included as a baseline; instead achievements under LESTARI will be calculated from a baseline of zero. BASELINE VALUE 0 BASELINE YEAR N/A USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 91

94 INDICATOR 7 TARGETS NOTES Because the process and content of the initiative is the focus rather than the institutionalization of a forum, the target was set assuming that there may be more than one initiative in a district or province. Thus the target has been raised higher than it was under the old indicator (which focused on fora). YEAR TARGETS ACTUAL OBSERVATIONS FY FY FY FY FY Targets LOP 20 NOTES ABOUT CHANGES TO THE INDICATOR Building on project learning so far, the indicator was changed from multi-stakeholder fora to multistakeholder initiatives, in order to ensure that the indicator captures project efforts to facilitate citizenbased input into policy processes. Fora are one such multi-stakeholder initiatives, but an indicator aimed at fora only risks unnecessarily restricting and formalizing the format of these initiatives, which may hold back achievements. There is precedent for this kind of indicator in USAID Democracy and Governance guidance, even though there will be some subjectivity in applying the definition of initiatives ; see The target number is increased from 14 MSF into 20 MSI LAST SHEET UPDATED BY: Rachman Pasha ME&L Coordinator DATE: November 10, 2017 USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 92

95 INDICATOR 8 GOAL RESULT INDICATOR TITLE Reduced rate of deforestation and degradation IR 2. Improved land use governance Number of champions engaged in advocacy interventions PERFORMANCE INDICATOR DESCRIPTION INDICATOR LEVEL INDICATOR TYPE PRECISE DEFINITION UNIT OF MEASURE Output Custom Indicator Champions include individuals who influence others (by persuasion or by example) to adopt more sustainable behaviours or policies in the landscape (i.e. behaviours or policies in line with LESTARI objectives) or individual who committed to forest conservation as demonstrated through significant actions. They can be from communities, civil society, government or private sector. They can be existing champions whose impact is strengthened by LESTARI, or new champions who act because of LESTARI intervention. In order to count a champion under this indicator, LESTARI must have strengthened their impact on others behaviour or on forest and land use policy advocacy. Given that successful advocacy efforts result in change, this indicator is highly connected with Indicator #4 (public policies) and Indicator #5 (more transparent and accountable licensing and permitting). Number of individuals DISAGGREGATIONS Sex (female/male); Province; District; Landscape RATIONALE OR JUSTIFICATION FOR INDICATOR DESIRED DIRECTION LINKAGE TO LONG TERM OUTCOME OR IMPACT The indicator is used to track the progress of strengthening citizen-based mechanisms for public input on land use. It is also used for measuring the extent to which the project is able to build constituencies for conservation. Increasing Advocacy interventions are essential aspects of democratic policy making, citizen participation, and oversight of all branches of government. These interventions play an important role in giving voice to citizens and historically marginalized groups. Successful advocacy will improve land use decisions. DATA COLLECTION APPROACH DATA COLLECTION METHOD DATA SOURCE FREQUENCY AND TIMING OF DATA ACQUISITION Review advocacy plans or strategies, recording of press conference, copy of testimony or press release, advocacy campaign materials, record of public consultation. Case study will be conducted to track the process of advocacy and policy change. Advocacy plans or strategies, press conference, press release, advocacy campaign materials, minutes of public consultation, assessment reports. Rolling, as data are available USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 93

96 INDICATOR 8 ESTIMATED COST OF DATA ACQUISITION INDIVIDUAL RESPONSIBLE DATA STORAGE LOCATION Staff labor costs; assessment cost; photocopies of documents Communication and Advocacy Specialist, Stakeholder Enggagement Advisor LESTARI MIS; with supporting files kept at the LESTARI Office in Jakarta DATA QUALITY INITIAL DQA DATE OF FUTURE DQA PROCESS FOR FUTURE DQA KNOWN DATA LIMITATIONS ACTION TAKEN TO ADDRESS LIMITATIONS TBD Internally by the Tetra Tech ARD M&E team in October 2018 The project will employ a system of continuous adaptive management and therefore will audit data on a systematic basis before input into the project s MIS. For each data point reported to USAID, supporting documentation will be identified and reviewed using the process outlined in USAID s DQA Template. USAID LESTARI s ME&L Coordinator will ensure that each data point is supported with documentation and that data are assessed against data integrity standards as outlined in ADS Activity reports may not always be complete Will require supporting documentation and observation to verify results DATA ANALYSIS REPORTING AND REVIEW DATA ANALYSIS Counting number of individuals supported to have strengthened impact on others behaviour or on forest and land use policy advocacy, supported by documentation. Tabular with supporting narrative, as appropriate PRESENTATION OF DATA Landscape Province District Target Achievement (disaggregated by Female/Male) % Completion REVIEW OF DATA Quarterly by ME&L Coordinator REPORTING OF DATA Quarterly and Annual Report BASELINE AND TARGETS BASELINE DATA SOURCE N/A USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 94

97 INDICATOR 8 BASELINE NOTES For this indicator, the baseline is zero BASELINE VALUE 0 BASELINE YEAR TARGETS NOTES N/A Target is determined based on potential CSOs in the landscape, population, membership of MSF during IFACS project. YEAR TARGETS ACTUAL OBSERVATIONS FY FY FY FY FY Targets LOP 500 NOTES ABOUT CHANGES TO THE INDICATOR Criteria was revised to recognize that champions can be from all stakeholder groups (not just communities) and to allow for strengthening of impact from existing champions. Previous definition also had superfluous information on advocacy. LAST SHEET UPDATED BY: Rachman Pasha ME&L Coordinator DATE: September 25, 2017 USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 95

98 INDICATOR 9 GOAL RESULT INDICATOR TITLE Reduced rate of deforestation and degradation IR 2. Improved land use governance Number of people reached by LESTARI communication programs to improve awareness and understanding of LEDS and biodiversity conservation PERFORMANCE INDICATOR DESCRIPTION INDICATOR LEVEL INDICATOR TYPE PRECISE DEFINITION UNIT OF MEASURE Output Custom Indicator Reached by LESTARI communication programs is defined as individuals who have been reached via LESTARI communication and outreach campaigns, including printed sustainable landscapes and forest conservation materials, radio campaigns, influential religious leader raising sustainable landscapes and forest conservation issues to their congregations, social media, video, communication trainings, and exhibitions. People could be reached through passive participation (receive or consume information) or empowerment, where primary stakeholders are capable and willing to become involved in the process and take part in advocacy and decision-making. This indicator is an early step to measure the scale and reach of communication activities more focused on increased awareness. Increased awareness be assessed when those people are measured as champions as indicated by Indicator #8. Number of individuals DISAGGREGATIONS Sex (female/male); Province; District; Landscape RATIONALE OR JUSTIFICATION FOR INDICATOR DESIRED DIRECTION LINKAGE TO LONG TERM OUTCOME OR IMPACT These results will help to estimate the coverage of LESTARI communications activities. Increasing Increase awareness and understanding of LEDS and biodiversity, supported by continous advocacy will lead to increase commitment of key stakeholders regarding the positive benefits of conservation and sustainable use of forests and the species they encompass. This in turn leads to improved land use decision making contributes to improved land use governance. DATA COLLECTION APPROACH DATA COLLECTION METHOD Data collection will vary widely based on the particular activity that reaches people with LESTARI messages/information on sustainable landscapes and biodiversity conservation issues. For communication training, sign-in sheets will validate the result. For radio campaign, LESTARI will assess the number of people that could potentially be reached to this message through population and geographical data. Using the data, the project will estimate the % of people (disaggregated by sex) within that geographical area that likely heard that message. A similar methodology will be used for mass media such as information USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 96

99 INDICATOR 9 in newspapers and other forms of print media. The project will also review communication products developed by stakeholders and deliver case study to assess increase awareness and changes in behaviours. For social media, the process of gathering data, as well as what data we gather, differs for each channel. In Facebook, LESTARI focus on three main areas: likes, comments and links on posts, and shares. If someone likes LESTARI page, we will know that they re listening to us. When followers add comments to our posts, we ve made the conversation two-way by engaging them. And when they share our posts with their Facebook friends, we ve expanded our audience. Gathering this information will help us understand what people are doing on Facebook, and how to tap into it. Twitter is similar engagement is a more accurate measure of our influence than follower count. Do followers retweet our posts, or share information of relevance and interest to our organization and constituents? Measure engagement against the time we spend on the site to get a sense of the channel s value. For LESTARI website, we will use Google Analytics to monitor traffic and referral data. DATA SOURCE FREQUENCY AND TIMING OF DATA ACQUISITION ESTIMATED COST OF DATA ACQUISITION INDIVIDUAL RESPONSIBLE DATA STORAGE LOCATION Sign-in sheet from trainings, secondary data (population and geographical), communication products (e.g., articles, shermon sheets, etc) developed by stakeholders, activity reports, assessment report, social media monitoring report. Rolling as data available Staff and labor cost, assessment cost, photocopies of activities documentation. Communication & Advocacy Specialist LESTARI MIS; with supporting files kept at the LESTARI Office in Jakarta DATA QUALITY INITIAL DQA DATE OF FUTURE DQA PROCESS FOR FUTURE DQA KNOWN DATA LIMITATIONS TBD Internally by the Tetra Tech ARD M&E team in October 2018 The project will employ a system of continuous adaptive management and therefore will audit data on a systematic basis before input into the project s MIS. For each data point reported to USAID, supporting documentation will be identified and reviewed using the process outlined in USAID s DQA Template. USAID LESTARI s ME&L Coordinator will ensure that each data point is supported with documentation and that data are assessed against data integrity standards as outlined in ADS Calculating direct beneficiaries as a result of communication training is rather easy. However, when LESTARI issues a radio spot or puts information in a local newspaper, measuring the precise number is extremely difficult and imprecise. USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 97

100 INDICATOR 9 ACTION TAKEN TO ADDRESS LIMITATIONS LESTARI will provide a rational justification for data counted against this indicator that obtained through mass media such as radio and newspaper information dissemination. LESTARI will access the reach of the message (e.g., the newspaper s geographic circulation or how far a radio signal can carry) and determine the population within that area. LESTARI will then apply a percentage of people that are likely reached to that message, using a rather conservative percentage (e.g., 30% of total population within that range). DATA ANALYSIS REPORTING AND REVIEW DATA ANALYSIS Counting number of individuals reached in LESTARI communication programs. LESTARI will assess the efficacy of outreach and other communication activities using the case study as well as assess the achievement of other related indicators that indicate changes in behaviors. Tabular with supporting narrative, as appropriate PRESENTATION OF DATA Landscape Province District Target Achievement (disaggregated by Female/Male) REVIEW OF DATA REPORTING OF DATA Quarterly by ME&L Coordinator Quarterly and Annual Report BASELINE AND TARGETS BASELINE DATA SOURCE BASELINE NOTES N/A For this indicator, the baseline is zero BASELINE VALUE 0 BASELINE YEAR TARGETS NOTES N/A Target is determined based on potential population reached per landscape and IFACS lesson learned. LOP target is 500,000. Target for the fourth and fifth year of the project decreased following the activity declines. YEAR TARGETS ACTUAL OBSERVATIONS FY ,551 FY ,374 FY ,500 FY ,000 USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 98

101 FY ,575 Targets LOP 500,000 INDICATOR 9 NOTES ABOUT CHANGES TO THE INDICATOR Total target of this indicator is raised from to people. The revision was made considering that the current number are too low while LESTARI work in the 17 sub-national government and province with dense population as potential target. LAST SHEET UPDATED BY: Rachman Pasha ME&L Coordinator DATE: September 25, 2017 USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 99

102 INDICATOR 10 GOAL RESULT INDICATOR TITLE Reduced rate of deforestation and degradation IR 1. Improved forest management KR 3. Management of at least six conservation areas improved, resulting in the conservation of valuable orangutan and other key species habitat, and the reduction in poaching of threatened and endemic species Number of Conservation Areas (CAs) with at least 70 point in METT scores across LESTARI landscapes as result of USG assistance PERFORMANCE INDICATOR DESCRIPTION INDICATOR LEVEL INDICATOR TYPE PRECISE DEFINITION UNIT OF MEASURE Outcome Custom Indicator Conservation Areas include National Parks, Wildlife sanctuaries, natural reserves, all areas of high biodiversity under the jurisdiction of the Ministry of Forestry. The Management Effectiveness Tracking Tool (METT) is a tool developed by the World Wildlife Foundation and The World Bank to track and monitor the effectiveness of protected area management. It provides a rapid and replicable assessment designed to reveal trends, standardize assessment and reporting, and aid in adaptive management. METT consists of two primary sections datasheets and an assessment form. The datasheets contain contextual information such as size and location, local designation, and budget, as well as a ranking of threats. The assessment form consists of a 30-question scorecard that quantifies performance based on a 4-point scale (0-3). Each question also requires an explanation for qualitative judgments, such as detailing the level of staff knowledge or results from external studies. Overall METT encompasses 6 key elements (context, planning, inputs, process, outputs, and outcomes) crucial to effective protected area management. METT allows park managers to identify needs, constraints, and priority actions to improve protected area management. Number of CAs DISAGGREGATIONS CAs, Landscape RATIONALE OR JUSTIFICATION FOR INDICATOR DESIRED DIRECTION METT was selected, not only because it allows the LESTARI team to monitor a portfolio of sites with a single, cost-effective tool, but also because it has been adopted by GoI as a tool to assess the management effectiveness of protected areas across Indonesia. The ability to manage protected areas effectively relies on a combination of good governance, sufficient capacity, well-trained staff and enough money to pay for essential management activities and equipment. Therefore, METT will provide a quick and simple picture of effectiveness in individual protected areas, through repeated application in sites over time. Increasing USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 100

103 INDICATOR 10 LINKAGE TO LONG TERM OUTCOME OR IMPACT Success in maintaining biodiversity is linked to a well-regulated and managed protected area, where staff are assessing progress and making changes as necessary. The strongest association is with law enforcement, control of access, resource management, monitoring and evaluation, maintenance of equipment, budget management and existence of annual work plans. Increase effectiveness of CAs management will contribute to conservation goal and reduce GHG emissions. DATA COLLECTION APPROACH DATA COLLECTION METHOD DATA SOURCE FREQUENCY AND TIMING OF DATA ACQUISITION ESTIMATED COST OF DATA ACQUISITION INDIVIDUAL RESPONSIBLE DATA STORAGE LOCATION METT baseline and endline survey; monitoring the implementation of CAs management plan Survey report; CAs management plan; direct observation Annual Staff labor costs; METT survey cost; photocopies of necessary documents Biodiversity Conservation Advisor LESTARI MIS; with supporting files kept at the LESTARI Office in Jakarta DATA QUALITY INITIAL DQA DATE OF FUTURE DQA PROCESS FOR FUTURE DQA KNOWN DATA LIMITATIONS TBD Internally by the Tetra Tech ARD M&E team in October 2018 The project will employ a system of continuous adaptive management and therefore will audit data on a systematic basis before input into the project s MIS. For each data point reported to USAID, supporting documentation will be identified and reviewed using the process outlined in USAID s DQA Template. USAID LESTARI s ME&L Coordinator will ensure that each data point is supported with documentation and that data are assessed against data integrity standards as outlined in ADS METT has been developed to provide a quick overview of progress in improving the effectiveness of management in individual protected areas, to be filled in by the protected area manager or other relevant site staff. As such it is clear that there are limitations on what it can achieve: it should not for example be regarded as an independent assessment, or as the sole basis for adaptive management. The Tracking Tool is also too limited to allow a detailed evaluation of outcomes and is really aimed at providing a quick overview of the management steps identified in the WCPA (World Commission of Protected Area) Framework up to and including outputs. Clearly, however good management is, if biodiversity USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 101

104 INDICATOR 10 continues to decline, the protected area objectives are not being met. Therefore the questions on condition assessment have disproportionate importance in the overall Tracking Tool. ACTION TAKEN TO ADDRESS LIMITATIONS Additional questions may need to be added to suit local circumstances rather than modify the Tracking Tool. The Tracking Tool contains a set of questions that have been designed to be easily answered by those managing the protected area without any additional research. However, it is useful to review the results of existing monitoring and to spend sufficient time discussing each aspect of management being assessed to arrive at a considered judgement. A group of protected area staff from the reserve, project staff or other agency staff should be involved in answering the questions in the Tracking Tool; where possible additional external experts, local community leaders or others with knowledge and interest in the area and its management should also be involved. DATA ANALYSIS REPORTING AND REVIEW DATA ANALYSIS Analysis will be done through quantitative (scoring) and qualitative methods. METT assessment is made by assigning a simple score ranging between 0 (poor) to 3 (excellent). A series of four alternative answers are provided against each question to help assessors to make judgements as to the level of score given. The maximum score of the 30 questions and supplementary questions is 99. A final total of the score from completing the assessment form will be calculated as a percentage of 99 or of the total score from those questions that were relevant to a particular protected area. Thus if a protected area scores 65 out of a maximum score of 87 the percentage can be calculated by dividing 65 by 87 and multiplying by 100 (i.e x 100 = 75%). Qualitative judgements will be applied whereas the information come from local staff knowledge (in many cases, staff knowledge will be the most informed and reliable source of knowledge), a reference document, monitoring results or external studies and assessments. Tabular with supporting narrative, as appropriate PRESENTATION OF DATA Landscape Conservation Area Target Achievement % Completion REVIEW OF DATA REPORTING OF DATA Annually by ME&L Coordinator Annual Report BASELINE AND TARGETS BASELINE DATA SOURCE BASELINE NOTES BASELINE VALUE METT baseline survey undertaken by Ministry of Environment and Forestry (MOEF) for six conservation areas None Conservation Areas METT score Gunung Leuser National Park 67 % Sebangau National Park 62 % Lorentz National Park 52 % Cycloops Nature Reserve 43 % USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 102

105 INDICATOR 10 Rawa Singkil 55 % Bukit Baka Bukit Raya National Park 64 % BASELINE YEAR 2015 TARGETS NOTES LESTARI will assist six conservation areas to improve their METT scores. YEAR TARGETS ACTUAL OBSERVATIONS FY 2016 Not measure FY FY FY FY Targets LOP 6 NOTES ABOUT CHANGES TO THE INDICATOR LAST SHEET UPDATED BY: Rachman Pasha ME&L Coordinator DATE: November 10, 2017 USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 103

106 INDICATOR 11 GOAL RESULT INDICATOR TITLE Reduced rate of deforestation and degradation IR 1. Improved forest management Number of Forest Management Unit (FMU) Strengthened as a result of USG assistance PERFORMANCE INDICATOR DESCRIPTION INDICATOR LEVEL INDICATOR TYPE PRECISE DEFINITION Output Custom Indicator This indicator measures the number of Forest Management Unit (FMU) receiving USG assistance that improved their management practices. FMU defined as forest management areas according to its function principal (protection or production) and designation that can be managed on a regular basis in efficient and sustainable. There are various level of FMU capacity within LESTARI landscapes. Some FMUs are recently established, many do not have management plan, running with limited budget and thus have limited impact on sustainable forest management practices on the ground. LESTARI will support strengthening of FMU in two aspects: enhance FMU institutional capacity and advocate for multi-use and multi-stakeholder forest management principles to be adopted by FMU. Government regulations and other references (e.g., Forest Management Toolkit) will be used as guidance. Improved FMU management practices encompasses three (3) key elements: Absolute requirements, Basic requirements and Supporting conditions. Each key element consist of several aspects that should be met by the FMU as the basis of fulfilling operationalization criteria. Under this indicator, improved management is defined when a FMU within LESTARI landscape at least have three of five of the following components that are supported through LESTARI support with participatory process. 1. Long-term management plan (RPHJP) including: Clear mission and purpose that described desired forest condition including landscape, vegetation, wildlife, biophysical elements at multiple scales Sustainability strategy based on management of forest resources A conservation strategy that will protect biodiversity, environmental services and social and cultural values. Sustainable economic strategy that aims to create job and reduce poverty. A clear strategy that recognizes and supports the rights (Comanagement) and interest forest-dependent communities A workforce management strategy that includes clear organizational structure, required expertise, roles and responsibilities 2. Annual short term management plan (RPHJPd) that operationalizes the above strategies mentioned in point (1). 3. Improved institutional capacity measured through adequate human resource expertise and financial resources 4. Co-management of forest implemented with community and relevant stakeholders. Co-management schemes define sharing of roles & responsibilities and benefits as well as monitoring. 5. Effective protected forest management (hutan lindung) that include collaborative forest patrol, inventory and data management USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 104

107 UNIT OF MEASURE Number of FMU DISAGGREGATIONS Landscape RATIONALE OR JUSTIFICATION FOR INDICATOR DESIRED DIRECTION LINKAGE TO LONG TERM OUTCOME OR IMPACT The indicator is used to measure progress of FMU strengthening in LESTARI landscape. Due to long periods required for FMU operationalization across the various levels of requirements, it is crucial to keep monitoring the process and find the way to filling the gap to optimizing the operationalization of the FMU institution as the forest manager rather than forest administrator. Enhance quality of FMU standards/toolkits will improve forest management in the area. Increasing Improved FMU management practices will benefit community adjacent to FMU area (through the adoption of multi-use and multi-stakeholder principles) and contribute to reduction on deforestation and degradation. DATA COLLECTION APPROACH DATA COLLECTION METHOD DATA SOURCE FREQUENCY AND TIMING OF DATA ACQUISITION ESTIMATED COST OF DATA ACQUISITION INDIVIDUAL RESPONSIBLE DATA STORAGE LOCATION Technical review of Forest Management Unit (FMU) plans and reports concerning RPHJP, RPJPd, Institutional Capacity, Co-management and Protected Forest Management. Interviews will be conducted with relevant FMU staff and stakeholders to evaluate the quality of these standards and the application on the ground RPHJP, RPJPd, Monitoring reports, co-management scheme. To be compiled quarterly and annually Staff labor costs including field staff and transportation to FMU sites to observe and verify the improved capacity and management practices. Assigned field staff will report to Landscape M&E Specialist with cc: ME&L Coordinator LESTARI MIS; with supporting files kept at the LESTARI Office in Jakarta DATA QUALITY INITIAL DQA DATE OF FUTURE DQA PROCESS FOR FUTURE DQA TBD Internally by the Tetra Tech ARD M&E team in October 2018 The project will employ a system of continuous adaptive management and therefore will audit data on a systematic basis before input into the project s MIS. For each data point reported to USAID, supporting documentation will be identified and reviewed using the process outlined in USAID s DQA Template. USAID LESTARI s ME&L Coordinator will ensure that each data point is supported with documentation and that data are assessed against data integrity standards as outlined in ADS USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 105

108 KNOWN DATA LIMITATIONS ACTION TAKEN TO ADDRESS LIMITATIONS Government regulation on FMU operationalization standard has not been in place in order to evaluate FMU readiness across Indonesia. USAID Bijak and USFS are still working to develop this standard at the national level. LESTARI used Perdirjen P.7/2016 to set the standard combined with Forest Management Toolkits from USFS Manuals. DATA ANALYSIS REPORTING AND REVIEW DATA ANALYSIS Data analysis for this indicator will involve assessing a series of milestones. Due to various FMU capacity level, this indicator will be counted once FMU has achieved four (4) criteria set up above. Key analytical questions will refer to the four criteria set up in the precise definition what has changed within FMU compared to baseline condition as a result of USG assistance? Do they have RPHJP? How is the quality? Is it implemented? Do they adopt workforce management? How is the adoption in practice? Do they have comanagement with community and relevant stakeholders? How is the level of implementation and monitoring? FMU RPHJP RPHJPd Institutional capacity Co- Management Management of Protected Area Tabular with supporting narrative, as appropriate PRESENTATION OF DATA Landscape FMU Target Achievement % Completion REVIEW OF DATA REPORTING OF DATA Quarterly by ME&L Coordinator Quarterly and Annual Report BASELINE AND TARGETS BASELINE DATA SOURCE N/A USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 106

109 BASELINE NOTES Baseline for each landscape is different, considering the level of readiness for each FMU as follow: Landscape FMU RPHJP RPHJPd Aceh Institutional Capacity Co- Management Management of Protected Area FMU III FMU V FMU VI FMU XVI Central FMU XVII Kalimantan FMU XXXI Papua FMU VI BASELINE VALUE 0 BASELINE YEAR N/A TARGETS NOTES Target is determined based on the level of FMU readiness in each landscape using the assessments result which previously conducted Landscape FMU RPHJP RPHJPd Aceh Institutional Capacity Co- Management Management of Protected Area FMU III - FMU V FMU VI FMU XVI - Central FMU XVII Kalimantan FMU XXXI Papua FMU VI - - YEAR TARGETS ACTUAL OBSERVATIONS FY FY FY FY FY Targets LOP 7 NOTES ABOUT CHANGES TO THE INDICATOR USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 107

110 This indicator was changed from previously titled Number of Co-Management Agreements Signed that Secure Community Rights and Benefit into the new title Number of Forest Management Unit (FMU) that have Improved Management Practices as a Result of USG Assistance. The changes was made to avoid the possibilities of double count on the area or co-management with hectares under improves management (Indicator#2) as well as to ensure the co-management agreement are well implemented after signed. The works on FMU strengthening will contribute with a significant number or area under improves management. However, due to long periods required for FMU operationalization across the various levels of requirements, then it is needed to provide the intermediate indicator to show the progress of the FMU strengthening process along the way. LAST SHEET UPDATED BY: Rachman Pasha ME&L Coordinator DATE: November 30, 2017 USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 108

111 INDICATOR 12 GOAL RESULT INDICATOR TITLE Reduced rate of deforestation and degradation IR 1. Improved forest management Number of people receiving USG supported training in natural resources management and/or biodiversity conservation PERFORMANCE INDICATOR DESCRIPTION INDICATOR LEVEL Output INDICATOR TYPE FACTS PRECISE DEFINITION This indicator will measure the number of individuals trained through learning activities in natural resources management (NRM) and/or biodiversity conservation. Training will primarily be traditional classroom training and workshops (led by designated instructors or lead persons, with a learning objective and defined curricula). Training topics will include but are not limited to: Best Management Practices, forest fire management, Good Agricultural Practice (GAP), etc. This indicator reports against the Biodiversity Fund. UNIT OF MEASURE DISAGGREGATIONS RATIONALE OR JUSTIFICATION FOR INDICATOR DESIRED DIRECTION LINKAGE TO LONG TERM OUTCOME OR IMPACT Number of people Sex (male/female); District; Province; Landscape; and Affiliation (government, non-governmental organizations, private sector, and community, as appropriate). Tracking training in NRM and biodiversity conservation provides information about the reach and scale of training and capacity building efforts. Increasing Increased capacity to manage forest resources and adapt to climate change will lead to improved land use management and strengthened mitigation and adaptation strategies to overall ecosystem management. DATA COLLECTION APPROACH DATA COLLECTION METHOD DATA SOURCE FREQUENCY AND TIMING OF DATA ACQUISITION Review of training attendance records, with a proportion of the trainings attended by LESTARI staff. Pre and post survey to seek information about changes in knowledge resulting from the training. Monitor other indicators to justify changes in application of knowledge and skill. Training records/reports, training grantee/subcontractor reports, monitoring reports. At the end of each training, to be compiled quarterly and annually USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 109

112 INDICATOR 12 ESTIMATED COST OF DATA ACQUISITION INDIVIDUAL RESPONSIBLE DATA STORAGE LOCATION Staff labor costs including field staff and transportation to training sites to observe and verify the training. Assigned field staff will report to Landscape M&E Specialist with cc: ME&L Coordinator LESTARI MIS; with supporting files kept at the LESTARI Office in Jakarta DATA QUALITY INITIAL DQA DATE OF FUTURE DQA PROCESS FOR FUTURE DQA KNOWN DATA LIMITATIONS ACTION TAKEN TO ADDRESS LIMITATIONS TBD Internally by the Tetra Tech ARD M&E team in October 2018 The project will employ a system of continuous adaptive management and therefore will audit data on a systematic basis before input into the project s MIS. For each data point reported to USAID, supporting documentation will be identified and reviewed using the process outlined in USAID s DQA Template. USAID LESTARI s ME&L Coordinator will ensure that each data point is supported with documentation and that data are assessed against data integrity standards as outlined in ADS Training record/report may not always be accurate. Independent verification through attendance by LESTARI staff. DATA ANALYSIS REPORTING AND REVIEW DATA ANALYSIS Counting number of people participating in LESTARI NRM and/or biodiversity trainings Tabular with supporting narrative, as appropriate PRESENTATION OF DATA Landscape District and/or Province Target Achievement (disaggregated by sex and affiliation) % Completion REVIEW OF DATA REPORTING OF DATA Quarterly by ME&L Coordinator Quarterly and Annual Report BASELINE AND TARGETS BASELINE DATA SOURCE N/A USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 110

113 INDICATOR 12 BASELINE NOTES For training, the baseline is zero BASELINE VALUE 0 BASELINE YEAR TARGETS NOTES N/A Target is determined based on potential population and IFACS lesson learned YEAR TARGETS ACTUAL OBSERVATIONS FY FY ,946 FY ,663 FY ,910 FY Targets LOP 10,000 NOTES ABOUT CHANGES TO THE INDICATOR LAST SHEET UPDATED BY: Rachman Pasha ME&L Coordinator DATE: September 25, 2017 USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 111

114 INDICATOR 13 GOAL RESULT INDICATOR TITLE Reduced rate of deforestation and degradation IR 1. Improved forest management KR 5. Funding leveraged from public and private sources, representing coinvestment in project outcomes Amount of investment mobilized (in USD) for climate change and biodiversity conservation as supported by USG assistance PERFORMANCE INDICATOR DESCRIPTION INDICATOR LEVEL Output INDICATOR TYPE Custom Indicator - contributes to FACTS PRECISE DEFINITION This indicator includes finance mobilized (or leveraged), enabled by USG assistance, for actions, activities, projects or programs that avoid, reduce, or sequester GHGs and conserve biodiversity. Finance may be mobilized from the public sector (e.g., local government), communities (e.g., village fund) or private sector (e.g., corporate investment) and should help to advance the objectives established by LESTARI program. Finance mobilized under this indicator also include those allocated for sustainable forest management, spatial planning, and conservation activities in targeted landscapes. Finance can be mobilized through a variety of instruments and vehicles, including common funding instruments, parallel investments, or in-kind support as a result of USG support. Examples of the types of U.S. assistance (under LESTARI) that could mobilize finance include: Finance interventions, such as: Grants (or in-kind support) for technical assistance Equity or investment shares through PPP scheme Investments made possible by policy interventions and technical assistance interventions, such as: Facilitate organizations/local governments to access financing from ICCTF (Indonesia Climate Change Trust Fund) Market value chains for cocoa that link farmers to wider market Regulatory policy support for transparent and fair permitting and approvals Regulatory policy support for Payment for Environmental Services (PES) Co-management interventions Allocation of village funds for forest and environment sector Examples of what mobilized funds may support include improving the enabling environment for mitigation actions, funding the costs of climate change and conservation activities advanced by the program, monitoring climate change progress or outcomes, or sensitizing stakeholders to climate risks and land use issues and opportunities addressed through the program. USD amount will be determined on a quarterly basis for all Indonesian rupiah values within that quarter using a currency converter USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 112

115 INDICATOR 13 ( This will be printed and filed with the indicator documentation. UNIT OF MEASURE DISAGGREGATIONS RATIONALE OR JUSTIFICATION FOR INDICATOR DESIRED DIRECTION LINKAGE TO LONG TERM OUTCOME OR IMPACT US Dollars Domain of interest: Sustainable Landscapes and Biodiversity Source of funds (public, private) dissagregated further into domestic and international sources of investment Provincie; District; Landscape Indicator used to track amount of investment leveraged the project could reach under initiatives fitting together public and private entities Increasing LESTARI programs are intended to be catalytic and to have sustainable benefits. The mobilization of additional financial resources can be an important indicator for assessing the success of LESTARI in catalyzing resources needed for transformational change. This indicator can also help to provide a baseline of data needed to test hypotheses as to the most effective strategies, techniques, or necessary capacities for mobilizing the funds required to address climate change and biodiversity conservation, leading to lessons learning over time. DATA COLLECTION APPROACH DATA COLLECTION METHOD DATA SOURCE FREQUENCY AND TIMING OF DATA ACQUISITION ESTIMATED COST OF DATA ACQUISITION INDIVIDUAL RESPONSIBLE DATA STORAGE LOCATION LESTARI will gather data about the amount of finance mobilized from all types of potential activities intervention (please see the examples under precise definition section). Documentation should include a rationale for how LESTARI support has facilitated the mobilization of additional resources and include information such as source of funds and use of funds. Financial records/statements verify amounts leveraged and documentation that prices out labor and material per period of time that was mobilized to enhance LESTARI objectives in climate change and biodiversity conservation. Rolling as data available Staff labor costs; photocopies of documentation Landscape M&E Specialists LESTARI MIS; with supporting files kept at the LESTARI Office in Jakarta DATA QUALITY USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 113

116 INDICATOR 13 INITIAL DQA DATE OF FUTURE DQA PROCESS FOR FUTURE DQA KNOWN DATA LIMITATIONS ACTION TAKEN TO ADDRESS LIMITATIONS TBD Internally by the Tetra Tech ARD M&E team in October 2018 The project will employ a system of continuous adaptive management and therefore will audit data on a systematic basis before input into the project s MIS. For each data point reported to USAID, supporting documentation will be identified and reviewed using the process outlined in USAID s DQA Template. USAID LESTARI s ME&L Coordinator will ensure that each data point is supported with documentation and that data are assessed against data integrity standards as outlined in ADS Proprietary information some organizations providing funding may consider some information on their funding support proprietary. Precision measuring funding leveraged does not necessarily indicate the magnitude of impact or results achieved. Accept self-reporting (with narrative caveats) and validate with estimates of public or private sector investments using documented similar investments for which data are known. DATA ANALYSIS REPORTING AND REVIEW DATA ANALYSIS Use standard audit methods on available data, calculate descriptive statistics - Disaggregation by domain of interest - Comparison with targets. When LESTARI intervention resulting on government allocating budget for certain program (e.g., fire prevention) in APBD, we will make a comparison between current APBD and previous APBD to allow justification of achievement. PRESENTATION OF DATA Tabular with supporting narrative, as appropriate Landscape Province District Target Achievement (disaggregated by domain of interest and source of funds public, private domestic vs international) % Completion REVIEW OF DATA REPORTING OF DATA BASELINE DATA SOURCE Quarterly by ME&L Coordinator Quarterly and Annual Report BASELINE AND TARGETS N/A BASELINE NOTES The baseline is zero. The project is at a starting point and no activity has been developed by this time. USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 114

117 INDICATOR 13 BASELINE VALUE 0 BASELINE YEAR TARGETS NOTES N/A Target is determined based on potential investment that can be leveraged in each landscape. YEAR TARGETS ACTUAL OBSERVATIONS FY ,477,921 FY ,320 FY ,282,759 FY ,250,000 FY ,750,000 Targets LOP 20,000,000 NOTES ABOUT CHANGES TO THE INDICATOR LAST SHEET UPDATED BY: Rachman Pasha ME&L Coordinator DATE: September 25, 2017 USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 115

118 INDICATOR 14 GOAL RESULT INDICATOR TITLE Reduced rate of deforestation and degradation IR 1. Improved forest management Number of people receiving livelihood co-benefits (monetary or non monetary) PERFORMANCE INDICATOR DESCRIPTION INDICATOR LEVEL Outcome INDICATOR TYPE Custom Indicator contributes to FACTS PRECISE DEFINITION UNIT OF MEASURE DISAGGREGATIONS RATIONALE OR JUSTIFICATION FOR INDICATOR DESIRED DIRECTION LINKAGE TO LONG TERM OUTCOME OR IMPACT This indicator identifies the number of people in LESTARI landscapes where sustainable landscapes activities as well as conservation and sustainable use of forest resources activities are implemented, who have received livelihood cobenefits associated with these activities. People included in the metric should be part of populations or households identified by a project with a documented relationship to the project. Beneficiaries should be reasonably assumed to have received a documented benefit or service enabled by USG assistance. Beneficiaries may include, but are not limited to: members of a household with an increased income or a newly secured land title, members of a cooperative who have increased sales due to increased market access, or even children attending a school renovated with payments for ecosystem services. Examples of monetary benefits may include, but are not limited to: increased income due to government policies related to climate change mitigation, payments for avoided emissions or carbon sequestration, monetary benefits from ecosystem services, public-private partnership, etc, facilitated by LESTARI. Examples of non-monetary benefits may include, but are not limited to: access to services, access to markets, land titling or registration, increased access to environmental services, protection of traditional livelihoods and customary rights, or increased productivity from climate-smart agricultural practices, etc, facilitated by LESTARI. Number is specific to each year, not cumulative Number of people Sex (male/female); Landscape; District Type of activity: Sustainable landscape; Biodiversity Reducing emission from deforestation and degradation can stimulate a win-win situation, by delivering carbon sinks and at the same time guaranteeing the conservation of biodiversity and livelihoods for forest-dependent communities. It is important to not only monitor emission reduced or avoided, but also to keep track of biodiversity and the livelihoods of local people. This measure demonstrates project reach. Increasing This indicator links sustainable natural resources management to economic growth objective. When people receive tangible monetary or non-monetary benefits from natural resource management or conservation, they are more likely USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 116

119 INDICATOR 14 to value and support these activities into the future, well after the project ends, creating a sustainable impact. DATA COLLECTION APPROACH DATA COLLECTION METHOD DATA SOURCE FREQUENCY AND TIMING OF DATA ACQUISITION ESTIMATED COST OF DATA ACQUISITION INDIVIDUAL RESPONSIBLE DATA STORAGE LOCATION Beneficiary interview, random survey and on-sight observations to approximate impact across households or population Project and subcontractor/grantee records; survey reports Annually Staff labor costs; survey costs Landscape M&E Specialist LESTARI MIS; with supporting files kept at the LESTARI Office in Jakarta DATA QUALITY INITIAL DQA DATE OF FUTURE DQA PROCESS FOR FUTURE DQA KNOWN DATA LIMITATIONS ACTION TAKEN TO ADDRESS LIMITATIONS TBD Internally by the Tetra Tech ARD M&E team in October 2018 The project will employ a system of continuous adaptive management and therefore will audit data on a systematic basis before input into the project s MIS. For each data point reported to USAID, supporting documentation will be identified and reviewed using the process outlined in USAID s DQA Template. USAID LESTARI s ME&L Coordinator will ensure that each data point is supported with documentation and that data are assessed against data integrity standards as outlined in ADS This indicator is not intended to capture the quality of co-benefits received or the degree to which livelihoods are improved. During the quantitative survey, the survey team will also solicit qualitative assessment to capture performance and other relevant assumptions into the calculations. DATA ANALYSIS REPORTING AND REVIEW DATA ANALYSIS The analysis will involve quantitative (statistical) such as random survey and qualitative analysis through beneficiaries interview, on-sight observation, etc. Qualitative will enrich explanation of project impact. Analysis includes gender and location of those receiving benefits, which will more closely assess are those benefits equitably distributed between men and women. USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 117

120 INDICATOR 14 Individuals receiving benefits from more than one sustainable landscapes activity, or receiving multiple benefits from a single activity, should be counted once per fiscal year. Tabular with supporting narrative, as appropriate PRESENTATION OF DATA Landscape District Type of Activity (Sustainable Landscape; Biodiversity) Target Achievement (disaggregated by sex) % Completion REVIEW OF DATA Annually by ME&L Coordinator REPORTING OF DATA Annual Report BASELINE AND TARGETS BASELINE DATA SOURCE BASELINE NOTES N/A N/A BASELINE VALUE 0 BASELINE YEAR TARGETS NOTES N/A Target achievement for this indicator will only start to be reported in the second year of the project to allow impact to realize. YEAR TARGETS ACTUAL OBSERVATIONS FY FY FY ,721 FY ,500 FY ,200 Targets LOP 30,000 NOTES ABOUT CHANGES TO THE INDICATOR LAST SHEET UPDATED BY: Rachman Pasha ME&L Coordinator DATE: September 25, 2017 USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 118

121 INDICATOR 15 GOAL RESULT INDICATOR TITLE Reduced rate of deforestation and degradation IR 1. Improved forest management Number of private sector firms that have improved management practices as a result of USG assistance PERFORMANCE INDICATOR DESCRIPTION INDICATOR LEVEL Outcome INDICATOR TYPE FACTS PRECISE DEFINITION This indicator measures the number of firms receiving USG assistance that improved their management practices. There are numerous private sector companies in LESTARI landscapes, ranging from large, multi-national, publicly traded Tropical Forest Alliance (TFA) members to smaller Indonesian privately held firms. Firms include forestry concenssions, oil palm and fiber plantations, mining and agroforestry/commodity producers, and their buyers. Companies, especially those involved in international trade, are under growing pressure to establish and meet sustainability and reduced emissions standards. Most companies lack the tools to assess potential GHG impacts, costs, and efficacy of these emerging goals, nor do they necessarily incorporate best management practices (BMPs) to meet these goals. The project will facilitate development (or update) of Conservation Management and Monitoring Plans (CMMPs) and support their implementation. These plans are used as a means to guide the management and monitoring of HCV and HCS areas in concessions. In the natural forest industry timber private sector, the project will assist companies to institutionalize Best Management Practices (BMPs) such as Reduce Impact Logging that measurably reduces GHG emission compared to conventional/bussines as usual practices. Improved management practices are achieved when companies incorporate Standard Operating Procedures (SOP) for BMP/RIL-C, allocate sufficient budgets for implementation, and implement some or all of the action plans. UNIT OF MEASURE Number of firms DISAGGREGATIONS Landscape RATIONALE OR JUSTIFICATION FOR INDICATOR DESIRED DIRECTION LINKAGE TO LONG TERM OUTCOME OR IMPACT Indicator used to track private sectors commitment regarding the positive benefits of conservation and sustainable use of forests. Increasing Adoption and implementation of BMPs will ensure that HCV and HCS areas in concessions are managed in perpetuity thus result on reduction in GHG emissions. USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 119

122 INDICATOR 15 DATA COLLECTION APPROACH DATA COLLECTION METHOD DATA SOURCE FREQUENCY AND TIMING OF DATA ACQUISITION ESTIMATED COST OF DATA ACQUISITION INDIVIDUAL RESPONSIBLE DATA STORAGE LOCATION Review the CMMP document, RIL-C document, SOP, and budget line item allocated by concessionaires to implement the plan. Spot-check to observe field implementation. RIL-C document, CMMP document, SOP, direct observation Rolling as data available Staff labor costs; photocopies of documentation; field visit cost Private Sector Engagement Coordinator LESTARI MIS; with supporting files kept at the LESTARI Office in Jakarta DATA QUALITY INITIAL DQA DATE OF FUTURE DQA PROCESS FOR FUTURE DQA KNOWN DATA LIMITATIONS ACTION TAKEN TO ADDRESS LIMITATIONS TBD Internally by the Tetra Tech ARD M&E team in October 2018 The project will employ a system of continuous adaptive management and therefore will audit data on a systematic basis before input into the project s MIS. For each data point reported to USAID, supporting documentation will be identified and reviewed using the process outlined in USAID s DQA Template. USAID LESTARI s ME&L Coordinator will ensure that each data point is supported with documentation and that data are assessed against data integrity standards as outlined in ADS Concessionaires may refuse to release their financial reports LESTARI field staff to spot-check concessionaires claims of implementation through field visits and site observations DATA ANALYSIS REPORTING AND REVIEW DATA ANALYSIS Triangulation of findings will be made through compilation of CMMP, SOP, budget documents review, direct observation, and interview concessionaires managers and staff. When counting progress toward meeting this target, LESTARI will track the intermediate milestones. LESTARI will also report the number of hectares under improved management practices. Achievement of this indicator will contribute to Indicator #2. USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 120

123 INDICATOR 15 Milestones MoU with private sector BMP package agreed upon Training for BMP provided to company staff BMP SOP finalized Budget for BMP activities incorporated into SOP BMP activities adopted and implemented BMP monitoring and evaluation conducted Tabular with supporting narrative, as appropriate PRESENTATION OF DATA Landscape Target Achievement (number of private firms and hectare under improved management practices) % Completion REVIEW OF DATA REPORTING OF DATA Quarterly by ME&L Coordinator Quarterly and Annual Report BASELINE AND TARGETS BASELINE DATA SOURCE BASELINE NOTES N/A The baseline is zero. The Project is at a starting point and no activity has been developed by this time. BASELINE VALUE 0 BASELINE YEAR TARGETS NOTES N/A Target is determined based on potential private firms to collaborate in each landscape. YEAR TARGETS ACTUAL OBSERVATIONS FY FY FY FY FY Targets LOP 10 USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 121

124 INDICATOR 15 NOTES ABOUT CHANGES TO THE INDICATOR Milestones are slightly revised into more logical order according to actual processes happening on the ground. LAST SHEET UPDATED BY: Rachman Pasha ME&L Coordinator DATE: November 30, 2017 USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 122

125 INDICATOR 16 GOAL RESULT INDICATOR TITLE Reduced rate of deforestation and degradation IR 1. Improved forest management KR 4. At least ten public-private partnerships (PPPs) promoting low-emissions conservation oriented development established Number of new USG-supported public-private partnerships (PPPs) formed PERFORMANCE INDICATOR DESCRIPTION INDICATOR LEVEL Outcome INDICATOR TYPE FACTS PPP 5 PRECISE DEFINITION UNIT OF MEASURE Number of public-private partnerships (PPPs) formed during the reporting year. Partnerships can be long or short in duration (length is not a criteria for measurement). A partnership with multiple partners should only be counted as a single partnership. However, an institution may form more than one partnership with the same entity and each partnership should be counted separately. A partnership is considered formed when there is a clear agreement, written and signed, to work together to achieve a common objective. Public-private partnership will be understood as a formal agreement between public and private entities with the purpose to provide and /or fulfill a public service in a schema defined to allow to the private enterprise to reach return on investment. Private partners could be for-profit enterprises, NGOs, private company, a community group, or a state-owned enterprise which seeks to make a profit (even if unsuccessfully). Public-private partnership also include partnerships between LESTARI (on behalf of USAID) and the private sector entities in its landscape. LESTARI will work directly with businesses and business associations from the onset in the creation of the PPPs to foster their support and to broaden impact to increase economic benefits while reducing GHG emissions and deforestation. PPP scheme will require some amount of leveraged funding from the private sector that contributes to low emissions conservation oriented activities. The ratio of funding between two parties will be determined during the process of developing agreement. Though it is expected that there is one to one match of funding. Investment mobilized through PPP scheme will contribute to Indicator #13 (investment mobilized). Number of new partnerships. The indicator should only measure the new partnerships in the reporting year. DISAGGREGATIONS Landscape; Type of PPP RATIONALE OR JUSTIFICATION FOR INDICATOR DESIRED DIRECTION Indicator used to track capacity of public and private entities to reinforce the legal frameworks for facilitating PPP, establish partnerships and support implementation of these kind of partnerships. The indicator is also used to measure sustainability of efforts and reinforce initiative for resilience and local benefits. Increasing USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 123

126 INDICATOR 16 LINKAGE TO LONG TERM OUTCOME OR IMPACT PPP will provide incentives for governments, private sectors and communities to embrace LEDS and conservation oriented practices. DATA COLLECTION APPROACH DATA COLLECTION METHOD DATA SOURCE FREQUENCY AND TIMING OF DATA ACQUISITION ESTIMATED COST OF DATA ACQUISITION INDIVIDUAL RESPONSIBLE DATA STORAGE LOCATION Review of partnership signed, implementation documentation, activities reports and proofs of disbursements PPP agreement, activities reports and proofs of disbursements Annually Staff labor costs; photocopies of documentation Private Sector Engagement Coordinator LESTARI MIS; with supporting files kept at the LESTARI Office in Jakarta DATA QUALITY INITIAL DQA DATE OF FUTURE DQA PROCESS FOR FUTURE DQA KNOWN DATA LIMITATIONS ACTION TAKEN TO ADDRESS LIMITATIONS TBD Internally by the Tetra Tech ARD M&E team in October 2018 The project will employ a system of continuous adaptive management and therefore will audit data on a systematic basis before input into the project s MIS. For each data point reported to USAID, supporting documentation will be identified and reviewed using the process outlined in USAID s DQA Template. USAID LESTARI s ME&L Coordinator will ensure that each data point is supported with documentation and that data are assessed against data integrity standards as outlined in ADS None None DATA ANALYSIS REPORTING AND REVIEW DATA ANALYSIS Compilation of data - Disaggregation by type of PPP - Comparison with targets - Analyze on improvements and impacts on climate change and biodiversity conservation aspects. USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 124

127 PRESENTATION OF DATA INDICATOR 16 Tabular with supporting narrative, as appropriate Landscape Target Achievement (disaggregated type of PPP) % Completion REVIEW OF DATA REPORTING OF DATA BASELINE DATA SOURCE BASELINE NOTES Annually by ME&L Coordinator Annual Report BASELINE AND TARGETS N/A N/A BASELINE VALUE 0 BASELINE YEAR TARGETS NOTES N/A Target is determined based on potential PPP in each landscape YEAR TARGETS ACTUAL OBSERVATIONS FY FY FY FY FY Targets LOP 20 NOTES ABOUT CHANGES TO THE INDICATOR LAST SHEET UPDATED BY: Rachman Pasha ME&L Coordinator DATE: September 25, 2017 USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 125

128 ANNEX II: RESULTS CHAINS USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 126

129 USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 127

130 USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 128

131 USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 129

132 USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 130

133 USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 131

134 USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 132

135 USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 133

136 ANNEX III: DATA QUALITY ASSESSMENT CHECKLISTS This Data Quality Assessment (DQA) Checklist is provided as a recommended tool that an operating unit (OU) may use to complete its DQAs. If the OU prefers or has successfully used a different tool for conducting and documenting its DQAs in the past, they are free to continue the use of that tool instead. The checklist below is intended to assist in assessing each of the five aspects of data quality and provide a convenient manner in which to document the OU s DQA findings. USAID Mission or Operating Unit Name: Title of Performance Indicator: [Indicator should be copied directly from the Performance Indicator Reference Sheet] Linkage to Foreign Assistance Standardized Project Structure, if applicable (i.e. Project Area, Element, etc.): Result This Indicator Measures [For USAID only] (i.e., Specify the Development Objective, Intermediate Result, or Project Purpose, etc.): Data Source(s): [Information can be copied directly from the Performance Indicator Reference Sheet] Partner or Contractor Who Provided the Data: [It is recommended that this checklist is completed for each partner that contributes data to an indicator it should state in the contract or grant that it is the prime s responsibility to ensure the data quality of sub-contractors or sub grantees.] USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 134

137 Period for Which the Data Are Being Reported: Is This Indicator a Standard or Custom Indicator? Standard Foreign Assistance Indicator Custom (created by the OU; not standard) Data Quality Assessment methodology: [Describe here or attach to this checklist the methods and procedures for assessing the quality of the indicator data. E.g. Reviewing data collection procedures and documentation, interviewing those responsible for data analysis, checking a sample of the data for errors, etc.] Date(s) of Assessment: Assessment Team Members: USAID Mission/OU Verification of DQA Team Leader Officer approval X VALIDITY Data should clearly and adequately represent the intended result. 1 Does the information collected measure what it is supposed to measure? (E.g. A valid measure of overall nutrition is healthy variation in diet; Age is not a valid measure of overall health.) 2 Do results collected fall within a plausible range? YES NO COMMENTS USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 135

138 3 Is there reasonable assurance that the data collection methods being used do not produce systematically biased data (e.g. consistently over- or under-counting)? 4 Are sound research methods being used to collect the data? RELIABILITY Data should reflect stable and consistent data collection processes and analysis methods over time. 1 When the same data collection method is used to measure/observe the same thing multiple times, is the same result produced each time? (E.g. A ruler used over and over always indicates the same length for an inch.) 2 Are data collection and analysis methods documented in writing and being used to ensure the same procedures are followed each time? TIMELINESS Data should be available at a useful frequency, should be current, and should be timely enough to influence management decision making. 1 Are data available frequently enough to inform project management decisions? 2 Are the data reported the most current practically available? 3 Are the data reported as soon as possible after collection? PRECISION Data have a sufficient level of detail to permit management decision making; e.g. the margin of error is less than the anticipated change. 1 Is the margin of error less than the expected change being measured? (E.g. If a change of only 2% is expected and the margin of error in a survey used to collect the data is +/- 5%, then the tool is not precise enough to detect the change.) 2 Has the margin of error been reported along with the data? (Only applicable to results obtained through statistical samples.) 3 Is the data collection method/tool being used to collect the data fine-tuned or exact enough to register the expected change? (E.g. A yardstick may not be a precise enough tool to measure a change of a few millimeters.) INTEGRITY Data collected should have safeguards to minimize the risk of transcription error or data manipulation. 1 Are procedures or safeguards in place to minimize data transcription errors? USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 136

139 3 Is there independence in key data collection, management, and assessment procedures? 3 Are mechanisms in place to prevent unauthorized changes to the data? SUMMARY Based on the assessment relative to the five standards, what is the overall conclusion regarding the quality of the data? Significance of limitations (if any): Actions needed to address limitations prior to the next DQA (given level of USG control over data): IF NO DATA ARE AVAILABLE FOR THE INDICATOR COMMENTS If no recent relevant data are available for this indicator, why not? What concrete actions are now being taken to collect and report these data as soon as possible? When will data be reported? RECOMMENDATIONS FOR CONDUCTING DATA QUALITY ASSESSMENTS 1. Data Quality (DQ) assessor should make sure that they understand the precise definition of the indicator by checking the Performance Indicator Reference Sheet. Please address any issues of ambiguity before the DQA is conducted. 2. DQ assessor should have a copy of the methodology for data collection in hand before assessing the indicator. For USAID Missions, this information should be in the AMEP s Performance Indicator Reference Sheets for each indicator. Each indicator should have a written description of how the data being assessed are supposed to be collected. USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 137

140 3. Each implementing partner should have a copy of the method of data collection in their files and documented evidence that they are collecting the data according to the methodology. 4. DQ assessor should record the names and titles of all individuals involved in the assessment. 5. Does the implementing partner have documented evidence that they have verified the data that has been reported? Partners should be able to provided USAID with documents (process/person conducting the verification/field visit dates/persons met/activities visited, etc.) which demonstrates that they have verified the data that was reported. Note: Verification by the partners should be an ongoing process. 6. The DQ assessor should be able to review the implementing partner files/records against the methodology for data collection laid out in the AMEP (for USAID Missions only). Any data quality concerns should be documented. 7. The DQ should include a summary of significant limitations found. A plan of action, including timelines and responsibilities, for addressing the limitations should be made. USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 138

141 ANNEX IV: LANDSCAPE SITUATION MODELS USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 139

142 USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 140

143 USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 141

144 USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 142

145 USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 143

146 USAID LESTARI Activity Monitoring and Evaluation Plan P a g e 144