Prepared by Battelle Memorial Institute

Transcription

1 Final Independent External Peer Review Report West Sacramento Project, California, General Reevaluation Report (GRR) Flood Risk Management (FRM) Project Prepared by Battelle Memorial Institute Prepared for Department of the Army U.S. Army Corps of Engineers Flood Risk Management Planning Center of Expertise Baltimore District Contract No. W912HQ-10-D-0002 Task Order: 0072 October 6, 2014

2 This page is intentionally left blank. BATTELLE October 6, 2014

3 CONTRACT NO. W912HQ-10-D-0002 Task Order: 0072 Final Independent External Peer Review Report West Sacramento Project, California, General Reevaluation Report (GRR) Flood Risk Management (FRM) Project Prepared by Battelle 505 King Avenue Columbus, Ohio for Department of the Army U.S. Army Corps of Engineers Flood Risk Management Planning Center of Expertise Baltimore District October 6, 2014 BATTELLE October 6, 2014 i

4 This page is intentionally left blank. BATTELLE October 6, 2014 ii

5 Final Independent External Peer Review Report West Sacramento Project, California, General Reevaluation Report (GRR) Flood Risk Management (FRM) Project Executive Summary PROJECT BACKGROUND AND PURPOSE The purpose of the project is to identify flood-related issues in the West Sacramento, California, study area. The decision document will present planning, engineering, and implementation details of the recommended plan to allow final design and construction to proceed after approval of the recommended plan. The project is a General Reevaluation Report (GRR) undertaken to evaluate structural and nonstructural flood risk management (FRM) measures, including in-basin storage, re-operation of existing reservoirs, improvements to existing levees, construction of new levees, and other storage, conveyance, and non-structural options. Because of the scope of the project, an Environmental Impact Study/Environmental Impact Report (EIS/EIR) will be prepared. At direction from Headquarters, U.S. Army Corps of Engineers (USACE), the GRR is being cost shared 50 percent Federal and 50 percent non-federal with the project sponsors, the State of California Central Valley Flood Protection Board (CVFPB) and the City of West Sacramento. The Water Resources Development Act (WRDA) of 1992 and the Energy and Water Development and Appropriations Act (EWDAA) of 1999 authorized the West Sacramento Project. Unfortunately, the authorized levee improvements did not consider the underseepage deficiencies facing many of the levees that protect the City. Although the levee improvements authorized for construction were redesigned to address underseepage, the remaining levees that protect the City were not re-evaluated to determine whether they were adequate to withstand the design flood event. The project partners have requested additional investigation into the remaining flood-related issues in the study area. The study area is in eastern Yolo County in the north-central region of the Central Valley of California. The City of West Sacramento is just west of the City of Sacramento, across the Sacramento River. The Sacramento River flows north to south, from its headwaters near the California-Oregon state line, to the Sacramento-San Joaquin Delta northeast of San Francisco Bay. The study area fundamentally consists of the City of West Sacramento city limit. The city is almost completely bound by floodways and levees: the Yolo Bypass to the west, the Sacramento Bypass to the north, and the Sacramento River to the east. The city is bifurcated by the Port of Sacramento Deep Water Ship Channel and Barge Canal. The non- Federal sponsor is primarily interested in reducing flood risk to the City of West Sacramento and surrounding area. The West Sacramento Project, California GRR, FRM project has been conducted to meet the USACE modernized planning initiative (i.e., SMART [Specific, Measurable, Attainable, Risk Informed, Timely] planning), which is to complete investigations leading to a decision in less time by using a risk-informed evaluation with less detailed information. BATTELLE October 6, 2014 iii

6 This new process has not been business as usual and has required heavy involvement as well as input and decisions from the Vertical Team at multiple points throughout the study. Instead of following the traditional USACE planning milestones, the study has been divided into phases, each with key milestones and associated In-Progress Reviews (IPR). A risk register and other risk management documentation will accompany the decision document. Although one of the objectives of IEPR is to evaluate whether sufficient information was available or technical analyses were completed, the IEPR must be completed within the context of the risk-informed decision-making process. Independent External Peer Review Process Independent, objective peer review is regarded as a critical element in ensuring the reliability of scientific analysis. USACE is conducting an Independent External Peer Review (IEPR) of the West Sacramento Project, California, General Reevaluation Report (GRR) Flood Risk Management (FRM) Project (hereinafter: West Sacramento GRR IEPR). As a 501(c)(3) non-profit science and technology organization, Battelle is independent, is free from conflicts of interest (COIs), and meets the requirements for an Outside Eligible Organization (OEO) per guidance described in USACE (2012). Battelle has experience in establishing and administering peer review panels for USACE and was engaged to coordinate the IEPR of the West Sacramento GRR. The IEPR was external to the agency and conducted following USACE and Office of Management and Budget (OMB) guidance described in USACE (2012) and OMB (2004). This final report presents the Final Panel Comments of the IEPR Panel (the Panel). Details regarding the IEPR (including the process for selecting panel members, the panel members biographical information and expertise, and the charge submitted to the Panel to guide its review) are presented in appendices. Based on the technical content of the West Sacramento GRR review documents and the overall scope of the project, Battelle identified potential candidates for the Panel in the following key technical areas: geotechnical engineering, hydrologic and hydraulic (H&H) engineering, economics/civil Works planning, and biology/ecology. Battelle screened the candidates to identify those most closely meeting the selection criteria and evaluated them for COIs and availability. USACE was given the list of final candidates to confirm that they had no COIs, but Battelle made the final selection of the four-person Panel. The Panel received an electronic version of the 1,948-page West Sacramento GRR review documents, along with a charge that solicited comments on specific sections of the documents to be reviewed. USACE prepared the charge questions following guidance provided in USACE (2012) and OMB (2004), which were included in the draft and final Work Plans. The USACE Project Delivery Team briefed the Panel and Battelle during a kick-off meeting held via teleconference prior to the start of the review to provide the Panel an opportunity to ask questions of USACE and clarify uncertainties. Other than Battelle-facilitated teleconferences, there was no direct communication between the Panel and USACE during the peer review process. The Panel produced individual comments in response to the charge questions. IEPR panel members reviewed the West Sacramento GRR documents individually. The panel members then met via teleconference with Battelle to review key technical comments and reach agreement on the Final Panel Comments to be provided to USACE. Each Final Panel Comment was documented using a four-part format consisting of: (1) a comment statement; (2) the basis for the comment; (3) the significance of the comment (high, medium/high, medium, medium/low, or low); and (4) recommendations BATTELLE October 6, 2014 iv

7 on how to resolve the comment. Overall, 18 Final Panel Comments were identified and documented. Of these, one was identified as having high significance, two were identified as having medium/high significance, eight had a medium significance, four had medium/low significance, and three had low significance. Battelle received public comments from USACE on the West Sacramento GRR (approximately 11 letters and individual comments, equating to 48 total pages of comments) and provided them to the IEPR panel members. The panel members were charged with determining if any information or concerns presented in the public comments raised any additional discipline-specific technical concerns with regard to the West Sacramento GRR review documents, and if adequate stakeholder involvement had occurred to identify issues of interest and to solicit feedback from interested parties. After completing its review, the Panel confirmed that no new issues or concerns were identified other than those already covered in their Final Panel Comments. The Panel also determined that adequate stakeholder involvement had occurred. Results of the Independent External Peer Review The panel members agreed on their assessment of the adequacy and acceptability of the economic, engineering, and environmental methods, models, and analyses used (USACE, 2012; p. D-4) in the West Sacramento GRR review documents. Table ES-1 lists the Final Panel Comment statements by level of significance. The full text of the Final Panel Comments is presented in Section 4.2 of this report. The following summarizes the Panel s findings. Based on the Panel s review, the review documents are clearly written and well-organized. The West Sacramento Project, California GRR, FRM project addresses a substantial risk presented to life safety and property and offers a solid overall concept. The Panel did, however, identify elements of the project that require further analysis and evaluation and sections of the GRR and Draft EIS/EIR that should be clarified or revised. Geotechnical Engineering: Of primary concern to the Panel was that the failure probabilities described in the GRR are unreasonably high. These probabilities are then incorporated into the HEC-FDA (Hydrologic Engineering Center Flood Damage Reduction Analysis) analyses, resulting in an overestimate of project benefits. USACE can address this matter by estimating geotechnical failure probabilities using a semi-quantitative risk analysis. Revised failure probabilities should include an assessment of the uncertainty in those probabilities. The Panel was also concerned that economic residual risks associated with seismic damage are not assessed. Without an estimate of the cost of repairing cutoff wall damage in a seismic event and the cost for improving seismic resistance of the levees, the net benefit of the project may be overstated because the cost associated with the residual risk of seismic damage to cutoff walls has not been included in evaluating residual risk. USACE can address this concern by, estimating the cost of levee repairs (including damaged cutoff walls) following an earthquake, and consider developing a conceptual design and cost estimates for improvements to resist seismic damage. The Panel also noted that potential damage due to seismic events as described in the Draft EIS/EIR is sometimes contradicted by the results of analyses presented in the Geotechnical Appendix to the GRR. Clarifying the discussion of seismic hazards presented in the Draft EIS/EIR would eliminate this issue. Economics/Civil Works Planning: The West Sacramento Project adheres to sound planning principles and USACE regulations and policies. The quality and quantity of the technical analyses and data that support the economics evaluation are sufficient for the feasibility study phase; however, an important BATTELLE October 6, 2014 v

8 issue the Panel identified was that the GRR does not address potential FRM benefits the project could reasonably be expected to provide. Including the additional sources of project benefits (reductions in emergency costs and agricultural flood damages and greater reductions in flood damages resulting from future development) would provide a more accurate representation of the benefits of the project. To address this issue, USACE can (1) calculate FRM benefits that would be expected in West Sacramento due to reduced emergency costs and include them in the benefit-to-cost ratio, (2) calculate FRM benefits that would result from reduced agricultural flood damages and include them in the benefit-to-cost ratio, and (3) assess future development that is likely to occur in West Sacramento and recalculate FRM benefits based on equivalent annual damages. The Panel was also concerned that the adequacy of the internal water management system and the incremental costs and benefits of improving the system have not been evaluated. Even if the Federal levee system withstands high river and bypass flows, there could be flooding in West Sacramento if the internal water management system does not function properly during a large storm event. USACE can address this concern by evaluating the design, existing condition, and operations and maintenance practices of the West Sacramento internal water management system to verify that the system is designed appropriately and will continue to function properly in the future. USACE could also evaluate the incremental costs and benefits of improvements to the internal water management system to determine whether such improvements are justified and could increase the total net FRM benefits of the recommended plan. Hydrologic and Hydraulic (H&H) Engineering: Based on the review of the GRR and the H&H Appendices, the Panel noted the HEC-RAS (Hydrologic Engineering Center River Analysis System) and hydrologic models were applied using the best available current data. One issue the Panel noted concerned levee stability and performance, including poor soil settlement and erosion over time, presence of trees larger than 2 inches in diameter at or near the levee, and the continuous, natural activity of animal burrowing within the levee that have not been fully addressed in the GRR. USACE can address this issue by implementing an active abatement or control program to remove any animals or large trees that are located on or near the levees. Biology/Ecology: From a biological resources perspective, an appropriate range of measures are considered within the constraints of meeting the project need and objectives; however, some biological resources in the study area potentially affected by project implementation have not been presented in sufficient detail to describe the existing conditions and support the EIS/EIR analysis. The lack of clear quantitative comparisons of impacts among the alternatives limits the completeness and quality of the report, but can be addressed by adding a table that quantifies (in acres) and compares the amount of each land cover type, including waters of the U.S., assumed to be affected under each alternative. The Panel also noted that baseline conditions for invasive plants in the project area, and an effects analysis for invasive plant spread as a result of project construction, have not been presented. USACE can address this by discussing existing conditions for invasive plants/noxious weeds in the project area and considering whether mitigation to prevent invasive plant spread during construction is needed. BATTELLE October 6, 2014 vi

9 Table ES-1. Overview of 18 Final Panel Comments Identified by the West Sacramento GRR IEPR Panel No. Final Panel Comment High Significance 1 The project benefits are overestimated because the probability of geotechnical failure used in the HEC-FDA analyses is unreasonably high. Medium/High Significance 2 Potential FRM benefits have not been evaluated and project benefits are likely to be significantly greater than presented in the GRR. 3 Economic residual risks associated with seismic damage are not assessed. Medium Significance The conclusions regarding seismic hazards in relation to the California Seismic Hazards Mapping Act in the Draft EIS/EIR are contradicted by the results of analyses presented in the Geotechnical Appendix to the GRR. Decisions to upgrade the levee are sometimes based on qualitative criteria that are not clearly defined, potentially resulting in non-essential levee upgrades. The adequacy of the internal water management system and the incremental costs and benefits of improving the system have not been evaluated. The basis for the assumption that the project will receive funding for construction at a rate of $100 million per year has not been provided, and the construction period may be too short, which would result in an underestimate of the cost of interest during construction. The mitigation requirements for the alternatives and the recommended plan are not described in the GRR and it is not clear whether the cost estimates include the cost of implementing and monitoring mitigation measures. Baseline conditions for invasive plants in the project area, and an effects analysis for invasive plant spread as a result of project construction, have not been presented. Some biological resources in the study area potentially affected by project implementation have not been presented in sufficient detail to describe the existing conditions and support the EIS/EIR analysis. Issues that are important to the integrity of the levee that may affect its future performance (such as poor soil composition, presence of any large trees at or near the levee, and the likelihood of animals burrowing the soil) have not been fully addressed. BATTELLE October 6, 2014 vii

10 Table ES-1. Overview of 18 Final Panel Comments Identified by the West Sacramento GRR IEPR Panel (continued) No. Final Panel Comment Medium/Low Significance A strategy has not been presented for allocating costs and benefits for West Sacramento alternatives that might be integrated with the Locally Preferred Option being considered in the American River Common Features Project. It is not clear how evaluation metrics were used in screening preliminary alternatives or evaluating the final alternatives. It is not clear how the magnitude of impacts and level of significance were determined for effects of sedimentation and turbidity on fisheries resources. Details about dates, locations, and objectives of reconnaissance-level surveys for some biological resources are not presented. Low Significance 16 No analyses have been reported that confirm that the seepage model extent is sufficient so that boundary effects do not result in inaccurate results. 17 The use of effective peak shear strength parameters may not be appropriate for all materials. 18 The level of significance of impacts on biological resources after mitigation is not clearly presented. BATTELLE October 6, 2014 viii

11 Table of Contents Page Executive Summary... iii 1. INTRODUCTION PURPOSE OF THE IEPR METHODS FOR CONDUCTING THE IEPR RESULTS OF THE IEPR Summary of Final Panel Comments Final Panel Comments REFERENCES Appendix A. IEPR Process for the West Sacramento GRR Project Appendix B. Identification and Selection of IEPR Panel Members for the West Sacramento GRR Project Appendix C. Final Charge to the IEPR Panel as Submitted to USACE on July 31, 2014, for the West Sacramento GRR Project List of Tables Table ES-1. Overview of 18 Final Panel Comments Identified by the West Sacramento GRR IEPR Panel.... vii Table 1. Major Milestones and Deliverables of the West Sacramento GRR IEPR... 3 Page BATTELLE October 6, 2014 ix

12 LIST OF ACRONYMS ACE ADM ATR BMP COI CVFPB CWRB DrChecks EC EIS EIR ER ERDC EWDAA FRM GRR H&H HEC-FDA HEC-RAS HEP IEPR IPR NEPA OEO OMB PDT SAR SMART USACE WRDA annual chance exceedance Agency Decision Milestone Agency Technical Review best management practices Conflict of Interest Central Valley Flood Protection Board Civil Works Review Board Design Review and Checking System Engineer Circular Environmental Impact Study Environmental Impact Report Engineer Regulation Engineer Research and Development Center Energy and Water Development and Appropriations Act Flood Risk Management General Reevaluation Report hydrologic and hydraulic Hydrologic Engineering Center Flood Damage Reduction Analysis Hydrologic Engineering Center River Analysis System Habitat Evaluation Procedure Independent External Peer Review In-Progress Reviews National Environmental Policy Act Outside Eligible Organization Office of Management and Budget Project Delivery Team Safety Assurance Review Specific, Measurable, Attainable, Risk Informed, Timely United States Army Corps of Engineers Water Resources Development Act BATTELLE October 6, 2014 x

13 1. INTRODUCTION The purpose of the project is to identify flood-related issues in the West Sacramento, California, study area. The decision document will present planning, engineering, and implementation details of the recommended plan to allow final design and construction to proceed after approval of the recommended plan. The project is a General Reevaluation Report (GRR) undertaken to evaluate structural and nonstructural flood risk management (FRM) measures, including in-basin storage, re-operation of existing reservoirs, improvements to existing levees, construction of new levees, and other storage, conveyance, and non-structural options. Because of the scope of the project, an Environmental Impact Study/Environmental Impact Report (EIS/EIR) will be prepared. At direction from Headquarters, U.S. Army Corps of Engineers (USACE), the GRR is being cost shared 50 percent Federal and 50 percent non-federal with the project sponsors, the State of California Central Valley Flood Protection Board (CVFPB) and the City of West Sacramento. The Water Resources Development Act (WRDA) of 1992 and the Energy and Water Development and Appropriations Act (EWDAA) of 1999 authorized the West Sacramento Project. Unfortunately, the authorized levee improvements did not consider the underseepage deficiencies facing many of the levees that protect the City. Although the levee improvements authorized for construction were redesigned to address underseepage, the remaining levees that protect the City were not re-evaluated to determine whether they were adequate to withstand the design flood event. The project partners have requested additional investigation into the remaining flood-related issues in the study area. The study area is in eastern Yolo County in the north-central region of the Central Valley of California. The City of West Sacramento is just west of the City of Sacramento, across the Sacramento River. The Sacramento River flows north to south, from its headwaters near the California-Oregon state line, to the Sacramento-San Joaquin Delta northeast of San Francisco Bay. The study area fundamentally consists of the City of West Sacramento city limit. The city is almost completely bound by floodways and levees: the Yolo Bypass to the west, the Sacramento Bypass to the north, and the Sacramento River to the east. The city is bifurcated by the Port of Sacramento Deep Water Ship Channel and Barge Canal. The non- Federal sponsor is primarily interested in reducing flood risk to the City of West Sacramento and surrounding area. The West Sacramento Project, California GRR, FRM project has been conducted to meet the USACE modernized planning initiative (i.e., SMART [Specific, Measurable, Attainable, Risk Informed, Timely] planning), which is to complete investigations leading to a decision in less time by using a risk-informed evaluation with less detailed information. This new process has not been business as usual and has required heavy involvement as well as input and decisions from the Vertical Team at multiple points throughout the study. Instead of following the traditional USACE planning milestones, the study has been divided into phases, each with key milestones and associated In-Progress Reviews (IPR). A risk register and other risk management documentation will accompany the decision document. Although one of the objectives of IEPR is to evaluate whether sufficient information was available or technical analyses were completed, the IEPR must be completed within the context of the risk-informed decision-making process. Independent, objective peer review is regarded as a critical element in ensuring the reliability of scientific analysis. The objective of the work described here was to conduct an Independent External Peer Review (IEPR) of the West Sacramento Project, California, General Reevaluation Report (GRR), Flood Risk BATTELLE October 6,

14 Management (FRM) Project (hereinafter: West Sacramento GRR IEPR) in accordance with procedures described in the Department of the Army, USACE, Engineer Circular (EC) Civil Works Review (EC ) (USACE, 2012) and the Office of Management and Budget (OMB) Final Information Quality Bulletin for Peer Review (OMB, 2004). Supplemental guidance on evaluation for conflicts of interest (COIs) was obtained from the Policy on Committee Composition and Balance and Conflicts of Interest for Committees Used in the Development of Reports (The National Academies, 2003). This final report presents the Final Panel Comments of the IEPR Panel (the Panel) on the existing engineering, economic, and environmental and plan formulation analyses contained in the West Sacramento GRR IEPR documents (Section 4). Appendix A describes in detail how the IEPR was planned and conducted. Appendix B provides biographical information on the IEPR panel members and describes the method Battelle followed to select them. Appendix C presents the final charge to the IEPR panel members for their use during the review; the final charge was submitted to USACE on July 31, PURPOSE OF THE IEPR To ensure that USACE documents are supported by the best scientific and technical information, USACE has implemented a peer review process that uses IEPR to complement the Agency Technical Review (ATR), as described in USACE (2012). In general, the purpose of peer review is to strengthen the quality and credibility of the USACE decision documents in support of its Civil Works program. IEPR provides an independent assessment of the engineering, economic, and environmental and plan formulation analyses of the project study. In particular, the IEPR addresses the technical soundness of the project study s assumptions, methods, analyses, and calculations and identifies the need for additional data or analyses to make a good decision regarding implementation of alternatives and recommendations. In this case, the IEPR of the West Sacramento GRR was conducted and managed using contract support from Battelle, which is an Outside Eligible Organization (OEO) (as defined by EC ). Battelle, a 501(c)(3) organization under the U.S. Internal Revenue Code, has experience conducting IEPRs for USACE. 3. METHODS FOR CONDUCTING THE IEPR The methods used to conduct the IEPR are briefly described in this section; a detailed description can be found in Appendix A. Table 1 presents the major milestones and deliverables of the West Sacramento GRR IEPR. Due dates for milestones and deliverables are based on the award/effective date of July 15, Note that the work items listed under Task 6 occur after the submission of this report. Battelle anticipates submitting the pdf printout of the USACE s Design Review and Checking System (DrChecks) project file (the final deliverable) on November 19, The actual date for contract end will depend on the date that all activities for this IEPR, including Civil Works Review Board (CWRB) preparation and participation, are conducted. BATTELLE October 6,

15 Table 1. Major Milestones and Deliverables of the West Sacramento GRR IEPR Task Action Due Date Award/Effective Date 7/15/2014 Review documents available 7/22/2014 Battelle submits list of selected panel members 7/28/2014 USACE confirms the panel members have no COI 7/30/2014 Battelle convenes kick-off meeting with USACE 7/22/2014 Battelle convenes kick-off meeting with USACE and panel members 8/12/2014 Panel members complete their individual reviews 9/2/2014 USACE submits public comments to Battelle 9/9/ Battelle submits public comments to Panel 9/9/2014 Panel members provide draft Final Panel Comments to Battelle 9/16/2014 Panel members provide response to public comments 9/29/ Battelle submits Final IEPR Report to USACE 10/6/2014 Battelle convenes Comment-Response Teleconference with panel 6 a members and USACE 10/30/2014 Battelle submits pdf printout of DrChecks project file to USACE 11/19/2014 Agency Decision Milestone (ADM) Meeting b 2/2/2015 Civil Works Review Board (CWRB) Meeting b c 6/4/2015 Contract End/Delivery Date 5/31/2015 a Task 6 occurs after the submission of this report. b The ADM and CWRB meetings were listed in the Performance Work Statement under Task 3, but were relocated in this schedule to reflect the chronological order of activities. c Because the CWRB has been revised to a date that is beyond the period of performance, a time extension will be needed to accommodate CWRB preparation, participation, and project closeout activities, which includes time to close out subcontracts with panel members. Battelle identified, screened, and selected four panel members to participate in the IEPR based on their expertise in the following disciplines: geotechnical engineering, hydrologic and hydraulic engineering (H&H), economics/civil Works planning, and biology/ecology. The Panel reviewed the West Sacramento GRR document and produced 18 Final Panel Comments in response to 27 charge questions provided by USACE for the review. This charge included two questions added by Battelle that sought summary information from the IEPR Panel. Battelle instructed the Panel to develop the Final Panel Comments using a standardized four-part structure: 1. Comment Statement (succinct summary statement of concern) 2. Basis for Comment (details regarding the concern) BATTELLE October 6,

16 3. Significance (high, medium/high, medium, medium/low, or low; in accordance with specific criteria for determining level of significance) 4. Recommendation(s) for Resolution (at least one implementable action that could be taken to address the Final Panel Comment). Battelle reviewed all Final Panel Comments for accuracy, adherence to USACE guidance (EC , Appendix D), and completeness prior to determining that they were final and suitable for inclusion in the Final IEPR Report. There was no direct communication between the Panel and USACE during the preparation of the Final Panel Comments. The Panel s findings are summarized in Section 4.1; the Final Panel Comments are presented in full in Section RESULTS OF THE IEPR This section presents the results of the IEPR. A summary of the Panel s findings and the full text of the Final Panel Comments are provided. 4.1 Summary of Final Panel Comments The panel members agreed on their assessment of the adequacy and acceptability of the economic, engineering, and environmental methods, models, and analyses used (USACE, 2012; p. D-4) in the West Sacramento GRR review documents. The following summarizes the Panel s findings. Based on the Panel s review, the review documents are clearly written and well-organized. The West Sacramento Project, California GRR, FRM project addresses a substantial risk presented to life safety and property and offers a solid overall concept. The Panel did, however, identify elements of the project that require further analysis and evaluation and sections of the GRR and Draft EIS/EIR that should be clarified or revised. Geotechnical Engineering: Of primary concern to the Panel was that the failure probabilities described in the GRR are unreasonably high. These probabilities are then incorporated into the HEC-FDA (Hydrologic Engineering Center Flood Damage Reduction Analysis) analyses, resulting in an overestimate of project benefits. USACE can address this matter by estimating geotechnical failure probabilities using a semi-quantitative risk analysis. Revised failure probabilities should include an assessment of the uncertainty in those probabilities. The Panel was also concerned that economic residual risks associated with seismic damage are not assessed. Without an estimate of the cost of repairing cutoff wall damage in a seismic event and the cost for improving seismic resistance of the levees, the net benefit of the project may be overstated because the cost associated with the residual risk of seismic damage to cutoff walls has not been included in evaluating residual risk. USACE can address this concern by, estimating the cost of levee repairs (including damaged cutoff walls) following an earthquake, and consider developing a conceptual design and cost estimates for improvements to resist seismic damage. The Panel also noted that potential damage due to seismic events as described in the Draft EIS/EIR is sometimes contradicted by the results of analyses presented in the Geotechnical Appendix to the GRR. Clarifying the discussion of seismic hazards presented in the Draft EIS/EIR would eliminate this issue. Economics/Civil Works Planning: The West Sacramento Project adheres to sound planning principles and USACE regulations and policies. The quality and quantity of the technical analyses and data that support the economics evaluation are sufficient for the feasibility study phase; however, an important BATTELLE October 6,

17 issue the Panel identified was that the GRR does not address potential FRM benefits the project could reasonably be expected to provide. Including the additional sources of project benefits (reductions in emergency costs and agricultural flood damages and greater reductions in flood damages resulting from future development) would provide a more accurate representation of the benefits of the project. To address this issue, USACE can (1) calculate FRM benefits that would be expected in West Sacramento due to reduced emergency costs and include them in the benefit-to-cost ratio, (2) calculate FRM benefits that would result from reduced agricultural flood damages and include them in the benefit-to-cost ratio, and (3) assess future development that is likely to occur in West Sacramento and recalculate FRM benefits based on equivalent annual damages. The Panel was also concerned that the adequacy of the internal water management system and the incremental costs and benefits of improving the system have not been evaluated. Even if the Federal levee system withstands high river and bypass flows, there could be flooding in West Sacramento if the internal water management system does not function properly during a large storm event. USACE can address this concern by evaluating the design, existing condition, and operations and maintenance practices of the West Sacramento internal water management system to verify that the system is designed appropriately and will continue to function properly in the future. USACE could also evaluate the incremental costs and benefits of improvements to the internal water management system to determine whether such improvements are justified and could increase the total net FRM benefits of the recommended plan. Hydrologic and Hydraulic (H&H) Engineering: Based on the review of the GRR and the H&H Appendices, the Panel noted the HEC-RAS (Hydrologic Engineering Center River Analysis System) and hydrologic models were applied using the best available current data. One issue the Panel noted concerned levee stability and performance, including poor soil settlement and erosion over time, presence of trees larger than 2 inches in diameter at or near the levee, and the continuous, natural activity of animal burrowing within the levee that have not been fully addressed in the GRR. USACE can address this issue by implementing an active abatement or control program to remove any animals or large trees that are located on or near the levees. Biology/Ecology: From a biological resources perspective, an appropriate range of measures are considered within the constraints of meeting the project need and objectives; however, some biological resources in the study area potentially affected by project implementation have not been presented in sufficient detail to describe the existing conditions and support the EIS/EIR analysis. The lack of clear quantitative comparisons of impacts among the alternatives limits the completeness and quality of the report, but can be addressed by adding a table that quantifies (in acres) and compares the amount of each land cover type, including waters of the U.S., assumed to be affected under each alternative. The Panel also noted that baseline conditions for invasive plants in the project area, and an effects analysis for invasive plant spread as a result of project construction, have not been presented. USACE can address this by discussing existing conditions for invasive plants/noxious weeds in the project area and considering whether mitigation to prevent invasive plant spread during construction is needed. 4.2 Final Panel Comments This section presents the full text of the Final Panel Comments prepared by the IEPR panel members. BATTELLE October 6,

18 Final Panel Comment 1 The project benefits are overestimated because the probability of geotechnical failure used in the HEC-FDA analyses is unreasonably high. Basis for Comment The computed probabilities reported in Section 14.2 of Appendix C (Geotechnical Appendix) to the GRR, which often exceed 90%, are for poor performance of levee reaches. While the Panel agrees that the probability of poor performance in a design flood is indeed very high, this value is not the probability of failure. The GRR describes the probabilities incorrectly (p. 2-12), representing them as the probability of failure. As a result, the failure probabilities described in the GRR are unreasonably high. These probabilities are then incorporated into the HEC-FDA analyses, resulting in an overestimate of project benefits. One reason that the probability of poor performance significantly exceeds the probability of failure is that the risks associated with seepage constitute a large portion of the total risk of poor performance. As stated in Section 26 (p.26-1) of the recent joint work on Best Practices by USACE and the U.S. Bureau of Reclamation (USBR, 2012), internal erosion is a potential failure mode that cannot be completely analyzed using numerical formulae or models. Thus, although seepage gradients that exceed standard criteria are a reasonable indication of potential poor performance, they are not an accurate or reasonable measure of the probability of failure. The probability of a levee breach due to slope instability is also not the same as the probability of poor performance. Not every slope failure inevitably leads to a levee breach. Some failures are only maintenance issues; in other cases active intervention can prevent a downstream failure from developing into a levee breach. In addition to the analytical challenges of estimating failure probability, the computed probabilities reported in Appendix C (Section 14.2) do not appear to consider the potential risk reduction through intervention by active flood fighting measures. While significant risks of failure remain even with intervention, completely ignoring the benefit overstates risk. The Best Practices work (USBR, 2012) states (pp ) that the USACE approach is to evaluate and communicate the potential risk reduction that can be achieved with intervention while at the same time to not mask the seriousness of a potential dam safety issue by relying on intervention to reduce the risk. The analysis conducted for the GRR is inconsistent with this approach because it ignores intervention. The GRR also does not address the degree of uncertainty associated with estimated probabilities. Best Practices (USBR, 2012) states (p.26-1) that risk estimating procedures, although quantitative, do not provide precise or accurate numerical results. The nature of the risk evaluation should be advisory and not prescriptive. In assessing the uncertainty associated with probability estimates, consideration should be given to a general calibration provided by Christian and Baecher (2011) when they indicate that one of the 10 major questions regarding geotechnical risk and reliability is why failures are less frequent than reliability studies predict. They state that predicted failure frequencies are an order of magnitude larger than observed, and two orders of magnitude larger than the frequency of modes of failure for earth dams. An understanding of the relatively imprecise nature of probabilities estimated for geotechnical events is required so that decisions to fund projects can be made with an appropriate knowledge of the degree of reliability of the estimated benefits and costs and of the effectiveness of alternative plans, specifically BATTELLE October 6,

19 required by ER (USACE, 2000). Significance High Inaccurate geotechnical probabilities in the HEC-FDA analyses result in an overstatement of withoutproject costs that could be significant and affect the benefit-cost-ratio. Providing calculations of failure probabilities without a description of the degree of reliability of those calculations is inconsistent with policy described by ER (USACE, 2000). Recommendation for Resolution 1. Estimate geotechnical failure probabilities using a semi-quantitative risk analysis conducted in accordance with USBR (2012). It may be necessary to use expert elicitation to establish a conditional probability relationship between poor performance and levee breach. Case history data may also be informative. 2. Revised failure probabilities should include an assessment of the uncertainty in those probabilities to comply with USACE (2000), Section 10. For example, perform sensitivity studies (such as the example provided in USBR [2012], Section 12) to assist in estimating the uncertainty in calculated failure probability that results from uncertainty in input distributions. Literature Cited: USACE (2000). Planning Planning Guidance Notebook. Department of the Army, U.S. Army Corps of Engineers, Washington, D.C. Engineer Regulation (ER) April 22. Available online at USBR (2012). Best Practices in Dam and Levee Safety Risk Analysis. A Joint Publication by U.S Department of Interior, Bureau of Reclamation and U.S. Army Corps of Engineers; 3 December. Available online at: Christian, J.T. and Baecher, G.B. (2011). Unresolved problems in geotechnical risk and reliability. Geo-Risk 2011: BATTELLE October 6,

20 Final Panel Comment 2 Potential FRM benefits have not been evaluated and project benefits are likely to be significantly greater than presented in the GRR. Basis for Comment The GRR does not address potential FRM benefits the project could reasonably be expected to provide. Reductions in the following costs/damages are likely to result from the project, but are not accounted for in the economic analysis. Emergency costs Agricultural flood damages associated with crops Damages associated with future intensification of land uses in West Sacramento. Emergency costs would include Federal, state, and local government emergency measures, evacuation and subsistence costs, reoccupation costs, and commercial cleanup and restoration costs. Such costs can represent a significant portion of total damages. For example, reductions in emergency costs accounted for 10 to 15% of the total FRM benefits estimated for the Louisiana Comprehensive Master Plan for a Sustainable Coast (USACE, 2007). It is reasonable to believe that reductions in emergency costs in West Sacramento would be on a similar scale. Although less significant, another benefit category that was not addressed is agricultural crop damage. The land use map (Economics Appendix, Figure 6, p. 2-8) indicates that there is significant agriculture in West Sacramento, particularly in the South Basin. A third benefit category that is not addressed focuses on land use. The Economics Appendix states (Section 3.3.2, p. 4-3) that the study area is considered to be fully built out and, therefore, expected annual damages are equal to equivalent annual damages. However, the following factors indicate that future growth is probable: The land use map (Economics Appendix, Figure 6, p. 2-8) shows large areas of agriculture and open space that could be converted to higher intensity land uses. The GRR states that there are plans for infill development in the North Basin. The City of West Sacramento plans additional development in the South Basin. The GRR states that a 64% increase in population is projected to occur between 2007 and The EIS/EIR describes new development projects that are under way now and into the next 20 years. Based on the growth that has occurred in the last 10 years in West Sacramento, it is reasonable to believe that growth will continue into the foreseeable future. This would increase future benefits of alternative plans. Currently, the USACE budgetary guidance (USACE, 2013a) requires that a flood damage reduction project have at least a 2.5 benefit-to-cost ratio at a 7% discount rate to be included in the Administration s budget (which includes Construction General Appropriations). The benefit-to-cost ratio presented in the GRR is calculated with only a 3.5% discount rate. Therefore, based on the existing economic analysis, it is possible that even if the West Sacramento Project gets authorized, the benefit-to-cost ratio may not be adequate to qualify for Construction General Appropriations. Significance Medium/High BATTELLE October 6,

21 Including the additional sources of project benefits (reductions in emergency costs and agricultural flood damages and greater reductions in flood damages resulting from future development) would provide a more accurate representation of the benefits of the project. Recommendations for Resolution 1. Calculate FRM benefits that would be expected in West Sacramento due to reduced emergency costs and include them in the benefit-to-cost ratio. 2. Calculate FRM benefits that would result from reduced agricultural flood damages and include them in the benefit-to-cost ratio. 3. Assess future development that is likely to occur in West Sacramento and recalculate FRM benefits based on equivalent annual damages. Literature Cited: USACE (2013a). Army Programs: Corps of Engineers Civil Works Direct Program, Budget Development Guidance, Fiscal Year Engineer Circular (EC) Department of the Army, U.S. Corps of Engineers, Washington, D.C. March 31. USACE (2007). Economic Analysis and Consequences, Integrated Ecosystem Restoration and Hurricane Protection: Louisiana s Comprehensive Master Plan for a Sustainable Coast. New Orleans District, U.S. Army Corps of Engineers. April. BATTELLE October 6,

22 Final Panel Comment 3 Economic residual risks associated with seismic damage are not assessed. Basis for Comment The seismic vulnerability of levees has been assessed based on their ability to provide post-seismic flood protection, in accordance with the USACE Draft ETL , Guidelines for Seismic Evaluation of Levees (not yet published). The analyses and classification in accordance with this ETL (as summarized in the Geotechnical Appendix, p. 12-3), indicates that seismic damage to cutoff walls is possible for the Bypass Levee and very likely for the West South Levee. The Panel understands that these levees do not retain water in the non-flood season, and thus the threat of loss of life only exists when a flood occurs either simultaneously or soon after a major earthquake, a relatively improbable occurrence. However, it appears that neither potential economic benefits nor residual economic risks associated with seismic damage have been fully assessed for the project. The Geotechnical Appendix does not indicate whether the proposed project will improve seismic resistance of the levees. This would be a potential benefit to the project. It appears that costs associated with repairing seismic damage to cutoff walls have not been estimated. Thus, the residual economic risks associated with repairing seismic damage to cutoff walls have not been assessed. In addition, no consideration appears to have been given to evaluating whether it would be cost-effective to improve the seismic resistance for the Bypass Levee and the West South Levee to reduce the risk of cutoff wall damage in a seismic event. Significance Medium/High Without an estimate of the cost of repairing cutoff wall damage in a seismic event, the net benefit of the project may be overstated because the cost associated with the residual risk of seismic damage to cutoff walls has not been included in evaluating residual risk. Recommendations for Resolution 1. Estimate the probability of levee damage due to seismic shaking, and estimate the cost of subsequent repair. 2. Based on the results of the above recommendation, consider whether it would be warranted to develop a conceptual design and cost estimates for improvements to resist seismic damage. Literature Cited: USACE Draft ETL , Guidelines for Seismic Evaluation of Levees (not yet published). BATTELLE October 6,

23 Final Panel Comment 4 The conclusions regarding seismic hazards in relation to the California Seismic Hazards Mapping Act in the Draft EIS/EIR are contradicted by the results of analyses presented in the Geotechnical Appendix to the GRR. Basis for Comment The Draft EIS/EIR (p. 408) indicates that the California Seismic Hazards Mapping Act of 1990 (California Public Resources Code [PRC] Sections ) addresses seismic hazards other than surface rupture, such as liquefaction and induced landslides. The Seismic Hazards Mapping Act specifies that the lead agency for a project may withhold development permits until geologic or soils investigations are conducted for specific sites and mitigation measures are incorporated into plans to reduce hazards associated with seismicity and unstable soils. The Draft EIS/EIR then concludes that because the closest active fault is 35 miles to the northwest, there are no significant issues due to seismicity. However, the seismic assessment presented in Geotechnical Appendix (p. 12-3) indicates that some sections of the levee have medium to high vulnerability, placing the Sacramento River West South Levee in a classification associated with seismically induced flow slides. This is consistent with the Panel s belief that a distance of 35 miles from an active fault is insufficient to conclude that no significant issues exist due to seismicity. Thus, the project as currently proposed appears out of compliance with the Seismic Hazards Act because seismic hazards exist, and no mitigation measures are incorporated to reduce them. If the lead agency withholds development permits until mitigation measures are incorporated, these additional measures could incur significant additional costs, possibly reducing the net project benefit. The seismic risk is also described inconsistently elsewhere in project documents. The Draft EIS/EIR states (p. 67, second paragraph) that a 200-year seismic event could very likely compromise the levee at several locations due to lateral spreading. However, in the next paragraph, the report states that because the expected magnitude of ground shaking from large regional earthquakes is relatively low in the project area, the potential for failure or significant damage of project structures is low. The analyses in the Geotechnical Appendix indicate that the expected magnitude of ground shaking is likely to result in significant damage to some levee reaches. The statements are contradictory and the analyses described do not support the latter statement. Significance Medium The conclusions regarding seismic hazards in relation to the California Seismic Hazards Mapping Act are inaccurate. If mitigation measures were deemed necessary to obtain a development permit in accordance with the Act, the costs incurred would reduce net project benefit. Furthermore, inconsistent descriptions of the potential for cutoff wall damage due to seismic events could affect the understanding and accuracy of the project. Recommendations for Resolution 1. Clarify the discussion of seismic hazards presented in the Draft EIS/EIR (p. 67). 2. Review the conclusions related to the California Seismic Hazards Mapping Act in light of other descriptions of seismic risks (i.e., p. 67 of the EIS/EIR and the GRR, Appendix C, Section 12) and resolve any inconsistency. (The Panel does not have expertise to recommend action required for compliance with the Act.) BATTELLE October 6,