IMPROVING WATER QUALITY: A REVIEW OF THE MISSISSIPPI RIVER BASIN HEALTHY WATERSHEDS INITIATIVE (MRBI) TO TARGET U.S. FARM CONSERVATION FUNDS

Transcription

1 Working Paper IMPROVING WATER QUALITY: A REVIEW OF THE MISSISSIPPI RIVER BASIN HEALTHY WATERSHEDS INITIATIVE (MRBI) TO TARGET U.S. FARM CONSERVATION FUNDS MICHELLE PEREZ AND SARA WALKER SUMMARY Historically, federal conservation programs have focused on solving environmental and natural resource problems on individual farms. While improvements have been made in water quality and wildlife habitat at the farm scale, landscape-scale environmental benefits in streams, lakes, and bays, for example, are less commonly documented. Excess nutrients (nitrogen, N, and phosphorus, P) continue to impair thousands of waterways, and eutrophication leads to hypoxia (low oxygen levels that harm aquatic life) or dead zones in water bodies around the country. Currently, approximately 1 10 percent of the U.S. Department of Agriculture s (USDA) Natural Resource Conservation Service s (NRCS) conservation budget is spent on targeting conservation efforts in high priority areas to achieve environmental outcomes at the landscape scale (i.e., across a geographic region facing similar water quality issues such as a watershed). However, focusing more conservation efforts in this manner, as opposed to the predominant approach, which disperses rather than concentrates funds across farms in each state, has the potential to achieve greater environmental improvements per dollar spent. In 2009, NRCS launched the Landscape Conservation Initiatives to more effectively address priority environmental and natural resource concerns by focusing on the most important geographic areas. These initiatives hold great promise for cost-effectively achieving significant outcomes at the landscape scale. CONTENTS Summary...1 Introduction...3 Background...5 Methods...6 Findings and Discussion...8 Conclusion and Recommendations...24 Endnotes...30 References...32 Appendix: Assessment Factors, Criteria, and Ratings...34 Disclaimer: Working Papers contain preliminary research, analysis, findings, and recommendations. They are circulated to stimulate timely discussion and critical feedback and to influence ongoing debate on emerging issues. Most working papers are eventually published in another form and their content may be revised. Suggested Citation: Perez, Michelle, and Sara Walker Improving Water Quality: A Review of the Mississippi River Basin Healthy Watersheds Initiative (MRBI) To Target U.S. Farm Conservation Funds. Working Paper. Washington, DC: World Resources Institute. Available online at: wri.org/ publication/mrbi The World Resources Institute (WRI) reviewed the Mississippi River Basin Healthy Watersheds Initiative (MRBI), one of NRCS s largest water quality-focused Landscape WORKING PAPER January

2 Conservation Initiatives to determine how well it was designed to achieve measurable improvements in water quality. WRI selected MRBI because of its focus on local water bodies in the Mississippi River Basin that drain into the Gulf of Mexico, the country s largest water body suffering from hypoxia. MRBI is a Cooperative Conservation Partnerships Initiative (CCPI) 2 that strives to accelerate adoption of conservation practices to achieve landscape-scale outcomes in high priority locations by leveraging financial and technical resources from project partners. Such partners include state and local agricultural and environmental agencies, universities, soil and water conservation districts (SWCDs), and nongovernmental organizations (NGOs). Given that it is too soon to evaluate the achievement of measurable improvements in water quality, WRI focused its analysis on the design of MRBI and the awarded proposals from 2010 and WRI reviewed the literature and interviewed experts 3 to produce six factors deemed to be indicators of effective targeting: stakeholder and producer buy-in; presence of specific, measurable, achievable, results-oriented, timebound, and quantitative (SMART-Q) goals; geographic targeting; monitoring and evaluation; cost effectiveness; and adaptive management. WRI then developed between one and four criteria for each of the six targeting factors to assess the degree of fulfillment of each factor. Most, but not all, of the targeting factors and their criteria reflect what NRCS requires or encourages in its request for proposals (RFPs) for MRBI projects. Cost effectiveness was not reflected in the RFPs, and adaptive management was mentioned indirectly. WRI reviewed MRBI at the programmatic level and project level for the presence of these targeting factors and their associated criteria. WRI assigned the following ratings based on the degree to which MRBI, as a whole, addressed the factor s criteria, where 5 = Exemplary. All criteria for the factor are met. 4 = Good. Most criteria for the factor are met. 3 = Fair. Some criteria for the factor are met. 2 = Poor. Few criteria for the factor are met. 1 = Very Poor. No criteria for the factor are met. To assign the ratings, WRI reviewed MRBI s available literature (Web sites, press releases, annual reports, and RFPs), evaluated 60 percent of the 2010 and 2011 project proposals, and interviewed NRCS staff associated with the initiative. To assess how effectively MRBI is designed to target available funds to achieve landscape-scale water quality outcomes, WRI defined effective targeting. WRI concluded that effective targeting, in general, and also specifically by MRBI, would result in achievement of improved landscape-scale water quality outcomes associated with reduced nutrient and sediment pollution, such as reductions in in-stream N, P, and sediment concentrations. Given that press releases and RFPs for MRBI state that the initiative was developed to address local and Gulf of Mexico water quality problems and that measurable conservation results would be evaluated on a watershed basis, WRI believes this vision of effective targeting is also shared by NRCS. 4 WRI commends NRCS s efforts to begin focusing conservation funds to maximize water quality outcomes through this partnership-based, targeted watershed project approach. Overall, MRBI received an average rating of fair across the six factors we used to rate how likely the program s design is to achieve improved landscape-scale water quality outcomes for nutrient and sediment pollution. MRBI excelled at geographically targeting conservation activities, receiving a good rating. It received fair ratings for including stakeholders and producers in MRBI, for setting SMART-Q goals, for measuring and evaluating progress, and for reflecting principles of adaptive management. It received a poor rating for cost effectiveness. These ratings are described in more detail throughout this report. Based on these findings, WRI has identified specific recommendations for MRBI that may assist NRCS and its project partners in achieving measurable improvements in landscape-scale water quality outcomes. Recommendations Stakeholder and Producer Buy-in 1. Clarify which stakeholders are involved in what aspects of MRBI. 2. Enable agricultural producers and rural landowners to participate in the development of MRBI and the projects. 3. Prioritize awards to future projects that leverage and formalize significant resources from non-usda sources. 2

3 A Review of the Mississippi River Basin Healthy Watersheds Initiative Figure 1 Before and After Conservation Practice Implementation BEFORE AFTER The business-as-usual approach to federal conservation programs works by solving individual water quality problems on individual farms. On a field in Stark County, Ohio, the before photo indicates that the farm s dairy cows were walking outside of the access road, tearing up the field, causing gully erosion and surface water runoff. In the after photo, several conservation practices were installed to solve the livestock, erosion, and runoff problems, including a dairy cow access road, an electric fence, and a grassed waterway. Source and photo credit: NRCS SMART-Q Goals 4. Lead by example and write a clear and SMART-Q goal statement for MRBI that aims to achieve landscapescale outcomes and require projects to do the same. 5. Prioritize future MRBI funds to those projects that aim to achieve already existing landscape-scale policy objectives. Geographic Targeting 6. Provide targeting narratives for the MRBI-designated focus areas and the MRBI project watersheds. Monitoring and Evaluation 7. Improve leadership and accountability for landscapescale outcomes. 8. Establish advisory teams for water quality monitoring, metrics, and modeling. 9. Prioritize projects with already existing baseline monitoring data or that propose to use a paired watershed approach. 10. Consider requiring watershed-based planning to help ensure attainment of improved landscape-scale water quality outcomes. Cost Effectiveness 11. Require projects to provide at least a narrative discussion about the cost effectiveness of their targeted watershed projects, and accelerate improvement of methods to quantitatively estimate cost effectiveness of practices and projects. Adaptive Management 12. Develop a formal framework on adaptive management to more effectively implement MRBI, and require projects to include plans for adaptive management in their proposals. INTRODUCTION For more than three decades, 5 federal conservation programs have helped provide solutions to environmental and natural resource problems associated with individual fields and farms. Farmers have adopted scores of conservation practices with financial and technical assistance from federal, state, and local conservation programs, university extension programs, and nongovernmental organizations (NGOs). These practices have addressed soil erosion, manure management, nutrient management, pesticide management, wildlife habitat, and other environmental and natural resource concerns. On individual farms, there is evidence that on-site environmental problems are being solved, and cleaner water and more wildlife habitat are being provided (see Figure 1). 6 WORKING PAPER January

4 However, water pollution, particularly eutrophication, the condition wherein excess nitrogen (N) and phosphorus (P) result in algae blooms that create hypoxic areas or dead zones with insufficient oxygen to support aquatic life, remains one of the most widespread environmental problems in the country. More than 15,000 water bodies remain impaired for nutrients, 7 and for the majority of these, agriculture is listed as the first or one of the top sources of these impairments. 8 Historically, U.S. Department of Agriculture (USDA) has worked primarily on solving water quality problems within individual farm boundaries, and therefore the focus has not been on generating or assessing landscape-scale environmental outcomes (e.g., delisting an impaired water body). Working at the landscape scale to improve water quality involves prioritizing watersheds and specifically, subsections of watersheds from which the majority of pollutants are delivered and are thus in need of the greatest conservation. By targeting funds to these areas, the collective efforts of individual farm conservation efforts are more likely to lead to in-stream water quality improvements. 9 USDA s primary approach to aid individual farms often results in conservation practices being implemented on farms that are dispersed across the landscape, an approach that helps address individual resource concerns but does not ensure that those individual efforts will collectively amount to solving priority water quality problems. By concentrating more resources in high priority sections of a watershed, better environmental outcomes can occur for the same or lower costs (see Figure 2). The business-as-usual approach to conservation funding depicted in the left panel in Figure 2 disperses conservation practices to a few fields located in various places on the landscape, both inside and outside of a watershed that is known to have an impaired river segment (depicted by the fish symbol and red stream coloring). Thus, the business-as-usual approach is solving environmental problems on individual farms and fields, but it is not directly addressing the impaired river segment. In contrast, a targeted approach to conservation funds, as depicted in the right panel in Figure 2, indicates that a decision has been made to concentrate conservation funds Figure 2 Business as Usual vs. Targeting Conservation BUSINESS AS USUAL Solves individual farm water quality problems TARGETING CRITICAL SUB-AREAS Solves landscape-scale water quality problems The business-as-usual approach (left panel) disperses conservation projects across the landscape, while a targeting approach (right panel) concentrates conservation projects upstream from the impaired water body to improve water quality. Source: WRI 4

5 A Review of the Mississippi River Basin Healthy Watersheds Initiative directly above the impaired stream segment. By implementing the right practices, in the right places, at the right scale, and at the right time, 10 the impaired stream segment can become cleaner over time and able to host many more fish and other aquatic organisms. After 30 years, only a handful of measurable landscapescale environmental outcomes have been documented in the United States, largely due to a lack of large-scale agricultural conservation targeting efforts. For example, the U.S. Environmental Protection Agency (EPA) Nonpoint Source Control Program that leads targeted watershed projects to clean up impaired water bodies has credited USDA financial and technical assistance to farmers as key factors in about one-third of the EPA s 443 success stories of partially or fully restored nutrient-impaired water bodies. 11 In addition, several university studies have documented increased populations of several endangered or threatened bird species due to concentrating funds from federal conservation programs in priority wildlife areas. 12 In 2009, USDA s Natural Resources Conservation Service (NRCS) launched a series of partnership-based, landscape-level targeting initiatives to address water, wildlife, and ecosystem resource concerns. 13 This effort to better target conservation funds to achieve environmental outcomes could improve the effectiveness of federal conservation programs that seek to concentrate funds in particular watersheds or other geographically defined areas to achieve environmental outcomes at the landscape scale. The purpose of this working paper is to assess how well the Mississippi River Basin Healthy Watersheds Initiative (MRBI), one of more than a dozen NRCS Landscape Conservation Initiatives, 14 is designed to achieve landscape-scale water quality outcomes. The World Resources Institute (WRI) focused on MRBI for two reasons: First, it is a large, landscape-scale conservation effort focused on improving water quality that is impaired due largely to nutrient and sediment pollution from agricultural sources. Second, MRBI is associated with the Gulf of Mexico, the largest water body in the United States that is suffering from hypoxia. Decades of data and modeling analysis attribute 70 to 80 percent of the nitrogen and phosphorus in the Gulf to agricultural activities in the Mississippi River Basin. 15 Thus, WRI s analysis of MRBI is restricted to the effect that the initiative can have on water quality (rather than also looking at its impact on wetlands or wildlife habitat, the other two initiative concerns). WRI assessed MRBI against six indicators that were identified as being critical factors to achieving environmental improvements: stakeholder buy-in, goal statements, geographic targeting efforts, monitoring and evaluation efforts, demonstration of cost effectiveness, and adaptive management principles. This review provides a snapshot of how well-designed the initiative and its projects are, as it is too early to evaluate environmental outcomes. Nevertheless, the analyses will be useful to NRCS s initiative leaders, the targeted watershed project leaders, and policymakers who want federal conservation programs to help achieve landscape-scale water quality improvements. We hope such efforts will help demonstrate that scarce taxpayer funds are being spent in the most cost-effective manner. BACKGROUND NRCS launched MRBI in 2009 under the Cooperative Conservation Partnerships Initiative (CCPI), which was authorized by the 2008 Farm Bill. 16 WRI was particularly interested in the CCPI projects due to their primary focus on water quality rather than wetland protection and wildlife habitat. CCPI is a 2008 initiative that is designed specifically to foster coordination with other partners, and to encourage additional non-federal investment in natural resource conservation through the use of in-kind services or matching funds in a geographic area, such as a watershed. In so doing, CCPI projects aim to accelerate conservation assistance to improve water quality, protect, restore, and enhance wetlands, enhance wildlife habitat;. 17 By statute, CCPI is allocated about 6 percent of NRCS s funds for some of its working lands conservation programs 18 the Environmental Quality Incentives Program (EQIP), Wildlife Habitat Improvement Program (WHIP), and Conservation Stewardship Program (CSP) 19 which has been estimated to translate roughly to more than US$100 million per year. 20 CCPI provides this money to farmers and landowners in the form of financial assistance to implement and maintain conservation practices. CCPI does not provide any technical assistance funding to the project partners (e.g., soil and water conservation districts, SWCDs, and nonprofit conservation groups) to implement the project or to measure outcomes. Instead, CCPI relies on project partners to cover their own staff time by leveraging existing or new funding from other sources like state conservation and environmental programs, charitable foundations, or other sources of funding from farm and conservation NGOs. WORKING PAPER January

6 MRBI has committed about $80 million per year over four years 21 for voluntary CCPI, Wetlands Reserve Enhancement Program (WREP), and Conservation Innovation Grant (CIG) projects in priority watersheds located in 13 key states: Arkansas, Illinois, Indiana, Iowa, Kentucky, Louisiana, Minnesota, Mississippi, Missouri, Ohio, Tennessee, South Dakota (beginning in 2011), and Wisconsin. 22 In 2009, NRCS state conservationists, state technical committees, 23 and state water quality agencies selected 43 designated focus areas (8-digit HUC watersheds) 24 within the participating states, and partners submitted targeted watershed project proposals for sub-watersheds (12-digit HUC) 25 within the designated focus areas. Fifty-eight MRBI-CCPI projects were funded in 2010 and 17 in Additional projects were awarded in 2012 but were not reviewed by WRI. MRBI is exceptional in many regards but particularly in its focus on environmental outcomes. NRCS boldly stated in its 2010 request for proposals (RFP) that the Mississippi River carries an average of 436,000 tons of sediment each day. Nutrients, like nitrogen and phosphorus, contribute to both local water quality problems and the hypoxic zone in the Gulf of Mexico. To help solve these water quality problems, NRCS developed MRBI. 26 The press release in 2009 emphasized that The initiative is performance oriented, which means that measurable conservation results are required in order to participate. In addition, Assessment of progress in implementing MRBI will be critical, as will evaluation of outcomes at the field scale/edge of field and on the watershed basis. 27 METHODS To assess how effectively MRBI is designed to target available funds to achieve landscape-scale water quality outcomes, WRI defined effective targeting. Specifically, for MRBI, effective targeting would result in achievement of improved landscape-scale water quality outcomes associated with reduced nutrient and sediment pollution, such as reductions in in-stream N, P, and sediment concentrations. Given that MRBI literature states that the initiative was developed to address local and Gulf of Mexico water quality problems and that measurable conservation results would be evaluated on a watershed basis, WRI believes this vision of effective targeting for achieving landscape-scale water quality outcomes is also shared by NRCS for MRBI. Hence, the success of MRBI in achieving these landscapescale water quality outcomes depends largely on the success of its targeted watershed projects. But for the projects to succeed and for their collective success to be evaluated and communicated to the public, there must be effective leadership at what WRI will refer to as the program level. The program level of MRBI refers to the NRCS staff that designed and continues to oversee MRBI. Among other things, the NRCS staff provides guidance to the targeted watershed projects through the RFP process. Conversely, efforts by the individual MRBI project leaders and partners of the 45 reviewed targeted watersheds will be referred to as being at the project level. WRI s assessment reflects MRBI in its entirety, from the program level down to the project level. Evaluation Factors Because at the time of this assessment, MRBI had only been under way for two years, it is too soon to assess outcomes, such as improvements in water quality, achieved at the landscape scale. 28 Due to natural lag times in surface and subsurface flow from fields to streams to watershed outlets, it could take years to observe water quality improvements at a watershed scale. 29 Therefore, WRI identified factors related to the initiative s design and the awarded project proposals, as a proxy for how likely they are to result in such targeting success in the future. These factors were culled from interviews with agricultural and water quality professionals 30 as well as literature on topics relevant to targeting and water quality outcomes, including scientific, economic, and policy journal articles; governmental and nongovernmental reports; and news articles. From this information, WRI identified six factors important to success in achieving landscape-scale water quality outcomes: 1. involving stakeholders and agricultural producers and achieving their buy-in; 2. setting policy-oriented goals that are SMART-Q (specific, measurable, achievable, results-oriented, time-bound, and quantitative); 3. employing geographic targeting of conservation efforts; 4. establishing monitoring and evaluation systems for assessing progress toward meeting goals; 5. being cost effective; 6. managing adaptively. 6

7 A Review of the Mississippi River Basin Healthy Watersheds Initiative To assess MRBI s programmatic efforts against the six factors, WRI reviewed MRBI Web sites and the available annual program performance reports. WRI also interviewed NRCS staff representing MRBI, the Landscape Conservation Initiatives, and EQIP, which is one of the largest program sources of funding for MRBI. In addition, WRI reviewed the 2010 and 2011 RFPs to determine what guidance and requirements the agency specified for its partners who implement the projects. Rating System WRI then developed a review and rating system for the MRBI program and its projects, based on the six factors. For each factor, WRI identified important rating criteria that should be present in the program and/or project proposals. Checking for these criteria in initiative materials and project proposals ensured that MRBI would be rated as objectively as possible. This assessment uses five-circle symbols to illustrate the degree to which individual criteria were fulfilled by the initiative, where = Exemplary. Criteria met. = Good. Criteria mostly met. = Fair. Criteria met to some degree. = Poor. Criteria largely not met. = Very Poor. Criteria not met. Then, these criteria ratings were averaged to assign an overall rating per factor. The ratings for each factor were then again averaged to rate MRBI as a whole across all six factors, where 5 = Exemplary. All criteria for the factor are met. 4 = Good. Most criteria for the factor are met. 3 = Fair. Some criteria for the factor are met. 2 = Poor. Few criteria for the factor are met. 1 = Very Poor. No criteria for the factor are met. To calculate the overall rating for each of the six targeting factors, WRI took the mathematical average of each factor s criterion scores. WRI used standard rounding rules to round to the nearest whole number. Even though this paper uses the six factors to evaluate the success of MRBI, it should be noted that NRCS has not, thus far, clearly indicated that it shares the same vision of success for MRBI as defined by our six factors. Although the factors largely represent the MRBI RFP requirements, WRI included additional elements that we believe are important for targeting success. Most notably, the RFPs did not require any discussion or analysis of cost effectiveness in the project proposals, nor did they require an adaptive management plan. As a result, our assessment holds the MRBI program and projects to higher standards than those currently set by NRCS. Assessment Methods To assess the individual projects under MRBI, WRI randomly selected 60 percent of the awarded MRBI-CCPI project proposals from 2010 and 2011 (see Table 1 and Figure 3). WRI also reviewed the project performance reports that NRCS had available at the time of this study. Because project effectiveness could only be assessed based on how the proposals were written, WRI interviewed 13 leaders from five projects about implementation in the first year or two of the projects to gain some understanding of how implementation matched or diverged from what was intended and stated in the proposal. These interviews also shed light on obstacles faced by the projects. These findings are discussed in detail in a follow-up WRI paper on the barriers to targeting and how to overcome them. Table 1 Projects Reviewed for this Assessment NUMBER OF AWARDED PROJECTS WRI S REVIEW TOTAL # REVIEWED % REVIEWED MRBI Projects % Because MRBI is still a young program, the assessment is largely based on NRCS documentation. In some instances, it was unclear whether there were flaws in the design of the program or simply a lack of program and project documentation. While we recognize that a lack of transparency does not equate to a lack of action, our assessment of MRBI is based on the readily available information. We hope that the results of this research and associated recommendations may help improve how the MRBI program and project leaders design, implement, and evaluate the initiative and the projects. WORKING PAPER January

8 Figure 3 Number of MRBI Proposals Reviewed by WRI Out of the Number of Proposals Awarded Per State and Their Geographic Distribution 31 Source: WRI MN 5/7 FINDINGS AND DISCUSSION WI IA IL IN MO 6/11 10/18 AR 6/10 LA 2/3 1/2 3/3 In this section, findings for the MRBI program and its projects for each of the six targeting factors are presented and discussed. 1. Stakeholder and Producer Buy-in MS 3/4 4/7 TN 2/3 2/3 OH KY 1/2 Portions of MRBI states within the Mississippi River Basin Portions of MRBI states outside the Mississippi River Basin Addressing large, landscape-scale environmental problems, such as water bodies impaired for drinking, swimming, or fishing, calls for flexible and transparent decision making by a diversity of stakeholders. By engaging stakeholders broadly, solutions to these problems are likely to be more sound and durable. 32 Targeting works best when it is locally driven by landowners and other stakeholders who work together toward a common outcome and believe that they have ownership over the effort. 33 Moreover, when farm producers are involved in identifying the environmental problem and planning the project to address it, the project is likely to see greater farmer participation. 34 Finally, partnership approaches to conservation that leverage nonfederal funds are increasingly being recognized as important, due to their ability to empower local partners with decision-making abilities, accountability, and tools. 35 By committing their own funds, project partners signal their support for the targeted watershed project solution to water quality problems. Based on this understanding, the MRBI program and projects were evaluated using the list of criteria in Box 1 for how well they attempt to engage stakeholders and producers. Box 1 Criteria for Assessing MRBI s Stakeholder and Producer Buy-in Efforts Findings 1a. Relevant stakeholders were identified, analyzed, and represented systematically for all stages of the project and empowered to participate in decision making. 1b. Producers were involved in identifying the environmental problem and planning the project. 1c. Funding was leveraged from nonfederal partners. 36 WRI assigned a rating of fair for stakeholder and producer involvement. NRCS and its project partners have made commendable efforts to involve a diversity of stakeholders and leverage additional funding. MRBI could be improved by clarifying the affiliation and function of stakeholders in the development of the MRBI program and individual projects. MRBI could also make a greater effort to involve producers in the project planning process to gain their buy-in and ownership of the project and pursue formal relationships with entities able to provide Table 2 Stakeholder and Producer Buy-in Rating DEGREE TO WHICH EACH CRITERION WAS MET: 1a. The initiative does a good job involving relevant stakeholders. 1b. Producers could be more involved in MRBI efforts. 1c. MRBI makes some effort to leverage nonfederal funds. Overall rating: Fair 8

9 A Review of the Mississippi River Basin Healthy Watersheds Initiative nonfederal funds to help ensure the success of the targeted watershed projects. Table 2 displays the degree to which the various criteria were fulfilled that led to the overall rating on this factor. Each criterion is discussed in detail in Table 2. 1a. Relevant stakeholders were identified, analyzed, and represented systematically for all stages of the project and empowered to participate in decision making The available information provided about the program indicates that several key stakeholders were involved in designing MRBI and in selecting the 43 priority focus areas, including staff from NRCS headquarters and state conservationists with input from the state technical committees and state water quality agencies. 37 Note that each state has varied membership on its state technical committee, but generally the committee can include representatives from federal, state, and local agricultural and water quality agencies; farm groups; environmental or conservation groups; and university extension staff. If these individuals are also members of each of the 13 state technical committees who participated in this process, then it is likely that many relevant stakeholders were involved in development of MRBI program. However, the program has not provided a description of the stakeholders involved in each of the 13 states and the extent to which they were involved in the design of MRBI. Doing so would likely enable analysis of whether the right stakeholders were or are involved in the program and might encourage invitation of additional stakeholders that should be involved if obvious omissions are observed. Again, this might be occurring, but there is a lack of documentation; or it may not be occurring. This ambiguity about stakeholder participation and defined roles in designing MRBI leads to questions about which individuals are serving which functions for the initiative and within each state (i.e., providing leadership, oversight and accountability, measurement, evaluation, reporting, and coordination). During the research and writing of this paper, NRCS hired a permanent MRBI coordinator, which seemed to put an end to the rotation of interim coordinators during the initiative s first two years. However, NRCS did not announce which of the aforementioned functions this permanent coordinator was empowered to provide. As a result, WRI is uncertain how much oversight and guidance the coordinator provides. At the project level, both the 2010 and 2011 RFPs required project applicants to describe who their project partners are and what roles, responsibilities, and capabilities they are providing. WRI s review of 45 project proposals suggests that nearly all projects listed a large and impressive array of stakeholders partnering to implement the watershed project. The median number of partners per project was 9.5 with a range from 2 to 30. Partners included federal, state, and local government institutions in both the agricultural and environmental sectors; universities; and farm or environmental/conservation NGOs. This diversity of stakeholders is key to ensuring that MRBI projects are designed and carried out by people with the skills and knowledge necessary to be successful. For example, 98 percent of proposals had partners with connections to producers, 99 percent had partners with water quality monitoring skills, and all proposals had partners with conservation technical assistance skills. Figure 4 Distribution of MRBI Project Leaders Given the focus on improving water quality, it is commendable that each of the 13 state conservationists appear to have sought input from their state s water quality agencies to select the designated focus areas for MRBI. However, it is unclear if, during initial design of MRBI, NRCS staff reached out to counterparts at EPA or the U.S. Geological Survey (USGS) agencies specializing in similar efforts of watershed protection, restoration, and monitoring efforts for feedback, guidance, or requests for assistance. As will be described in later sections, NRCS staff reviewed programmatic data from the EPA 319 nonpoint source program and scientific data from USGS regarding the Gulf of Mexico as inputs into the selection of the 43 focus areas, but additional interagency collaboration has not been described at the program level. Source: WRI 20% 10% 2% 4% (N = 45 PROJECT PROPOSALS) 58% SWCD NGO State Agency University Other WORKING PAPER January

10 Regarding project leadership, just over half of the MRBI projects were led by local SWCDs (see Figure 4); then by farm or environmental NGOs; state agricultural, conservation, or water quality agencies; or universities. 38 SWCDs are important leaders of conservation efforts, as they have a long history of working one-on-one with farmers. Despite projects having a plethora and diversity of stakeholders on board, less than half of the proposals specified that their project would involve stakeholders in decision making at the outset, through implementation of the project, and during the evaluation stage. This lack of specificity about stakeholder engagement is likely due to the fact that the RFPs did not specifically call for proposals to explain how stakeholders would be involved in decision making throughout the project life spans. This expectation could be included in future RFPs, and NRCS could inform current MRBI project leaders that they expect this issue to be discussed in the annual reports. 1b. Producers were involved in identifying the environmental problem and planning the project There is no evidence that NRCS engaged producers and landowners in the development or implementation of MRBI at the programmatic level, except to the extent that members of the state technical committees include producers and landowners. NRCS did require the projects in both RFPs to describe how project partners will encourage producer participation to increase the likelihood of project success. But neither RFP called for producers to be involved in the project design process. An impressive majority (87 percent) of reviewed projects did describe an outreach and marketing strategy to gain producer and landowner participation in the project. Many projects mentioned using radio and newspaper announcements, direct mailings, meetings, workshops, farm field days, and engagement through relationships with certified crop advisers, agricultural cooperatives, etc., to reach out to producers. However, less than half (44 percent) of the reviewed projects called for producer involvement in the design of the project and problem identification. Involving producers directly or indirectly through their farm trade associations in problem definition, project planning, and decision making often leads to greater buy-in and participation from the key constituents in the targeted watershed project. Ensuring that the producers have seats at the decision-making table would help MRBI to more effectively identify (a) where the resource concerns are, (b) what the likely causes and possible solutions are, (c) where the project efforts should be targeted, and (d) how the outcomes of the project should be monitored and measured all key elements of effective conservation targeting. 39 1c. Funding was leveraged from nonfederal partners Both 2010 and 2011 RFPs announced that MRBI would prioritize proposals that significantly leverage nonfederal financial and technical resources. The 2010 RFP stated that it would prioritize projects in the selection process that provided additional funding for conservation practice payments (in addition to the federal funds provided by EQIP, CSP, or WHIP). The 2011 RFP went further to specify that such nonfederal resources could also provide for water quality monitoring and evaluation of conservation practices and for coordination with other local, state, or federal efforts. 40 Despite this encouragement, less than half of the reviewed projects had statements indicating that they would leverage nonfederal conservation funds from a variety of sources. Of the projects that did leverage additional funds, the median amount of matching funds represented 57 percent of the projects total budgets. This number is surprisingly low and indicates underreporting. Given that MRBI funding goes to producers (i.e., not to the project partners), projects with nonfederal partners have to leverage nonfederal funds at least in the way of in-kind staff time. In reality, it is possible that this percentage is higher, but project leaders did not consistently describe or itemize these funds in their proposals. Considering that federal conservation programs are not flush with funds, leveraging nonfederal funds is not only necessary to ensure that projects have adequate funding but is also necessary to gain the buy-in needed by the many stakeholders critical to a targeted watershed project s success. Some projects included plans to use extra funds to pay for additional in-stream water quality monitoring stations or supplement conservation payments to install edge-of-field monitoring stations. Because NRCS MRBI funding does not provide funds to project partners for these tasks, other projects included statements that some leveraged funds would cover the staff time for stakeholders to implement the project by conducting outreach, providing technical assistance to producers, and providing monitoring or modeling services to measure and report on the outcomes of the project. 10

11 A Review of the Mississippi River Basin Healthy Watersheds Initiative 2. Policy-Oriented Smart-Q Goals WRI used SMART-Q criteria to assess the quality of the program and project goals. Guidelines for setting specific, measurable, achievable, results-oriented, 41 and time-bound (SMART) goals were introduced over two decades ago. 42 WRI believes these indicators for successful objectives 43 are appropriate for evaluating the goals set by MRBI. SMART goals help individuals, programs, and projects establish a clear direction, identify results, and perform at a higher level than would otherwise be achieved. 44 WRI added a sixth letter, Q for quantitative, to the SMART mnemonic, given that modern water quality policy goals are increasingly quantitative (e.g., total maximum daily load (TMDL) allocations, numeric nutrient criteria, thresholds to remove water bodies from impaired waters lists). A quantitative goal statement places higher standards on the measurable component of SMART goals. Although a goal to improve water quality is measurable, it could technically be met with only the slightest of water quality improvements. recommendation of a 45 percent nutrient reduction goal for the Gulf of Mexico). 45 WRI noted if the program or the projects mentioned being designed to achieve some or all of the specific nutrient or sediment load reductions. The MRBI program and projects were evaluated using the list of criteria in Box 2 to assess how policy-oriented and SMART-Q MRBI s goals are. Findings MRBI received a fair rating for having policy-oriented SMART-Q goals (see Table 3). MRBI lacked a clear and thorough goal statement and had different requirements in the two RFPs for SMART-Q goals. However, two-thirds of the reviewed projects wrote ambitious, outcome-oriented goals that contained all or most SMART-Q components (see Table 3). Table 3 Policy-Oriented Smart-Q Goal Rating DEGREE TO WHICH EACH CRITERION WAS MET: WRI also assessed whether the MRBI program and projects mentioned the presence of any water quality policy drivers, either in the form of water quality objectives (e.g., TMDLs, impaired water bodies lists, and the scientific Box 2 Criteria for Assessing MRBI s Policy-Oriented SMART-Q Goals 2a. Both the program s and the projects stated goals are somewhat SMART-Q. 2b. MRBI could more systematically link its efforts to overarching water quality policies and recommendations. Overall rating: Fair 2a. Goals are SMART-Q. 46 Specific: Goals state who is involved, what will be accomplished, where, and by when. Measurable: Goals provide criteria to assess progress toward specified goal(s). Achievable: The necessary skills, finances, and abilities are in place, and the project has a clear plan of action. Results-oriented: Goals focus on the desired end result(s) of the activities (i.e., the outcomes), not on the activities that will be implemented. Time-bound: Goals include a time frame for completion. Quantitative: Goals have a numeric target that the project is aiming to achieve. 2b. Goals aim to address already existing water quality policies or recommendations. It is important for NRCS to set an overarching goal for MRBI under which partners can design projects that work to address the initiative s overall mission. More transparent goals that foster better management and enable evaluation will include most or all of the SMART-Q elements. And ideally, if water quality policies or recommendations already exist in the targeted watersheds, projects should set goals that complement the existing water quality improvement efforts. 2a. Goals are SMART-Q For the MRBI program, although WRI discovered various objectives stated in a variety of press releases, Web sites, and other program literature, no single clear goal statement emerged that encapsulated the purpose of MRBI or the overarching goal under which the projects should be WORKING PAPER January

12 designed. The following statement from the 2010 MRBI RFP was selected to serve as the initiative s goal statement because WRI believed it best represented MRBI: To improve the health of the watersheds within the Mississippi River Basin, NRCS and its partners will help producers to voluntarily implement conservation practices that avoid, control, and trap nutrient runoff; improve wildlife habitat; restore wetlands; and maintain agricultural productivity. 47 MRBI s goal statement had half of the desired SMART-Q characteristics specific because it focuses on a subset of water quality concerns: nutrient pollution; measurable because it is arguably possible to measure reductions in nutrient runoff; and results-oriented because it seeks to improve the health of the watersheds through attainment of interim and environmental outcomes (reduce nutrient runoff, improve wildlife habitat, restore wetlands). Given MRBI s progressive design to achieve outcomes, rather than simply outputs (e.g., acres of conservation practices, contracts signed), WRI further analyzed the degree to which the goal statement was results-oriented, or outcome-based. In our assessment, we considered four types of goals, which are summarized below and explained in more detail in Box 3. Output goals aim to implement conservation practices Interim outcome goals aim to achieve in-field or edgeof-field environmental improvements Environmental outcome goals aim to achieve in-stream water quality improvements Ecological balance outcome goals aim to achieve instream water quality improvements beyond a specified threshold needed for achieving ecological restoration Because of MRBI s stated intention to improve the health of watersheds, WRI categorized this goal as being an environmental outcome goal. The remainder of the statement, however, focuses more on the necessary outputs (e.g., implemented practices) and interim outcomes (e.g., avoidance, control, and trapping of nutrient runoff) than a desired environmental outcome at the landscape scale. In WRI s view, the initiative should enumerate a specific, Box 3 Goals to Drive Water Quality Outcomes The current business-as-usual measures of success associated with NRCS conservation programs (e.g., EQIP, CSP, WHIP) focus on administrative metrics such as the acres of installed practices or the number of signed contracts with producers. While these administrative metrics are important stepping-stone outputs, they are not the same as actual environmental and natural resource outcomes. Because a key purpose of MRBI is to achieve water quality outcomes at the landscape scale, the goals and corresponding metrics should reflect that ambition at both the programmatic level and the project level. WRI built on definitions found in the scientific literature and in NRCS s Landscape Conservation Initiatives literature to define four levels of goals based on the degree to which they focus on achieving long-term, water quality outcomes. 1. Output goals aim to achieve implementation of in-field and edge-of-field conservation activities (e.g., livestock exclusion, planting of cover crops) as measured by administrative metrics, such as numbers of acres receiving conservation treatment, or dollars spent. 2. Interim outcome goals aim to achieve in-field and edge-of-field improvements (e.g., reduction in manure application rates, reduction in sediment runoff from edge of field) as measured by farm-site agricultural practice performance metrics. Interim outcome goals are short-term goals and can be tracked via modeling or monitoring. 3. Environmental outcome goals aim to achieve in-stream improvements (e.g., reductions in nutrient concentrations in streams, improved water quality) as measured by off-farm-site natural resource or environmental performance metrics. Environmental outcome goals are intermediate-term goals and can be tracked via modeling or monitoring. At this level, the focus of conservation effort is on changing behavior and outcomes on many agricultural fields with sufficient density and intensity of effort to result in improved water quality indicators in area streams. 4. Ecological balance outcome goals aim to achieve major thresholds of ecosystem health in water bodies (e.g., restoration of native species to stream, removal of water body from impaired waters list) as measured by off-farm-site natural resource or environmental performance metrics. Ecological balance goals are long-term outcome goals and can be tracked via modeling or monitoring. To achieve ecological balance goals, conservation activities not only need to be dense and intense, they also need to be tied to specific thresholds for quantitative improvements in specific water quality indicators that must be met for the water body to be restored to ecological health. 12

13 A Review of the Mississippi River Basin Healthy Watersheds Initiative quantitative, water quality-related, ecological balance goal for the impaired local streams that feed into the Mississippi River Basin and Gulf of Mexico. However, the initiative s current goal statement lacks achievable, time-bound, and quantitative elements. Given the ambiguity about what it means to improve the health of the watersheds, it is unclear if the goal is achievable. There is no mention of a time frame for accomplishing the goal, and there is no quantitative target associated with this goal that enables all observers to recognize when the goal has been reached. In the 2010 RFP, NRCS required projects to describe SMART objectives while the 2011 RFP only required SMAR objectives (dropping the time for completion criterion, although, later in the RFP, requiring projects to state an implementation schedule when different objectives and conservation practices will be completed). In both RFPs, NRCS required that project duration could not exceed five (2010 RFP) or four (2011 RFP) years in length. However, neither RFP discussed the need to maintain the installed conservation practices and to monitor water quality for the desired improvements beyond the four to five years of project implementation. While the project implementation was time-bound, the goals were not. And project life spans are likely not long enough to realize outcome goals. Future RFPs could clarify this timing distinction between changes in land management and water quality responses. More discussion on this topic is provided in the Monitoring and Evaluation section under criteria 4.c. Though neither RFP required quantitative goals, the 2010 RFP gave greater priority to proposals that Provide evidence of a watershed planning process which identifies quantifiable project goals for field-scale nutrient management and watershed-scale nutrient load reduction. 48 However, this element was missing from the 2011 RFP. At the project level, about half of the project plans specified quantitative interim outcomes or environmental outcomes (see Box 3 for definitions). Overall, WRI found that a third of projects met all of the SMART-Q characteristics, a third met most of the SMART-Q characteristics, and a third had only some of the SMART-Q characteristics. WRI also analyzed the degree to which the project goals focused on outcomes as part of the results-oriented component of SMART-Q (see Box 3). Because the RFPs required descriptions of the priority practices and funding amounts employed by each project, and because applicants must implement conservation practices in order to realize environmental outcomes, all projects understandably had output-related goals. Surprisingly, most (90 percent) of the projects also had environmental or interim outcome goals. This is a significant improvement in the approach to using conservation funds by NRCS and its partners. In the future, projects should also aim to achieve ecological balance goals for individual project watersheds. The areas where projects consistently fell short, as would be expected, were the same areas where the RFP fell short in its guidance: lack of adequate timelines and quantitative targets. 2b. Goals aim to address already existing water quality policies or recommendations Although MRBI s objectives extend beyond water quality (e.g., they also include improving wildlife habitat and maintaining agricultural productivity), WRI focused primarily on water quality in this review. The initiative s goal is not currently tied to any water quality policies, such as local nutrient- or sediment-related TMDLs, nor is it associated with the Gulf of Mexico nutrient reduction goal recommended by the 2007 EPA Science Advisory Board. 49 However, WRI believes that both these local policies and regional voluntary guidance should be priority factors for consideration when selecting designated focus areas or specific 12-digit HUC watersheds for targeted watershed projects and that the projects should aim to achieve such policy goals. Given that the Mississippi River Basin drains into the Gulf of Mexico, which suffers from hypoxia, this omission of directly relating MRBI to local and/or Gulf-impaired water goals is a significant missed opportunity for prioritizing use of targeted conservation funds to directly addressing these highest priority water quality issues. Although the RFPs did not ask for discussion of these local or regional water quality policies or guidance goals, surprisingly, the majority of proposals (80 percent) did mention the presence of a policy driver in their project watersheds. However, only a few (33 percent) of the projects planned to directly address the policy goal through their conservation activities, and even fewer (16 percent) mentioned the quantitative nutrient or sediment load reductions called for in the policy driver. MRBI projects could significantly improve their use of taxpayer dollars if the projects aimed to achieve existing water quality improvement policies or regional recommendations. WORKING PAPER January