Frequently-Asked Questions (FAQs) DTSC Chemical Soil Background Study

Transcription

1 Frequently-Asked Questions (FAQs) DTSC Chemical Soil Background Study 1. What is the purpose of DTSC's Chemical Soil Background Study (Study)? The Study's purpose is to establish a regulatory agency-approved, publicly-reviewed, and technically-defensible chemical soil background dataset to establish background threshold values (BTVs). The BTVs will be used in developing the Look-Up Table values as required in the Administrative Orders on Consent (AOCs) with the United States Department of Energy (DOE) and NASA. The Look-Up Table values will become the clean-up values for DOE and NASA at the Santa Susana Field Laboratory (SSFL). Chemical background results can also be used for the Boeing portion of the site if risk-based cleanup decisions are required. The public will have the opportunity to review and comment on the Study s report prior to DTSC s final approval. 2. What process did DTSC use to select its Chemical Background Reference Areas (CBRAs) and arrive at its conclusion that the CBRAs were representative of natural background for the SSFL property and not impacted by activities at SSFL? The Study s two CBRAs are located approximately six to seven miles west of the entrance gate to the SSFL property; near Oakbrook Regional Park in Thousand Oaks, California. DTSC's siteselection process for the background Study s sampling locations followed EPA guidance (e.g., EPA 2002a). The CBRAs are representative of the physical and other natural characteristics that exist at SSFL. The CBRAs are situated on the same geologic formations (Santa Susana and Chatsworth Formations) that underlie SSFL, have soil and topographic conditions similar to those at SSFL, and similar wildfire histories. To ensure the CBRAs were representative of natural background, DTSC reviewed and evaluated available historical aerial photographs, wildfire histories, assessed available wind data from SSFL and neighboring areas, interviewed property-owner representatives to identify and evaluate areas of historical use, conducted site-reconnaissance activities, and solicited and considered input from stakeholders. Based on the information gathered, DTSC determined that the CBRAs were not impacted by operations at SSFL. DTSC formed a site conceptual model (see note below) for the two CBRAs that considers naturally-occurring processes to have affected both similarly. Those natural processes include soil formation, wildfires, air-fall deposition, that either caused or account for the presence of the Study's inorganic (e.g., constituents that do not contain carbon- like metals) and organic (e.g., constituents that do contain carbon, like dioxins) compounds. Air-fall deposition is the process whereby pollutants in the atmosphere fall onto the ground surface. Note: Site conceptual models are part of the data quality objective (DQO) process. EPA (2006) defines a site conceptual model as follows: "The conceptual model is an important tool for organizing information about the current state of knowledge (about a site) and understanding of the problem...and how this relates to the project's goal." A site conceptual model summarizes key environmental aspects involving constituent release, transport, dispersion, transformation, and deposition. 3. What are Background Threshold Values (BTVs)? Background threshold values (BTVs) represent the true background concentrations of the target chemical and are established using statistical procedures on the data collected from the Chemical

2 Background Reference Areas. Each chemical with at least five detections, will have a separate BTV calculated for it, based on the range of detected concentrations and the statistical method employed (see discussion below). BTVs will be used (along with the sample lab reporting limits for those constituents with less than five detects for which no meaningful or defensible results could be statistically derived) in developing the Look-Up Table values. Analytical results from SSFL on-site soils that are less than the BTVs represent unimpacted locations and can be considered as coming from the background populations. Analytical results from SSFL on-site soils that exceed the BTV-derived Look-Up Table values potentially represent locations not belonging to the background population and may require further investigation and/or cleanup consistent with the Administrative Orders on Consent. 4. How are Background Threshold Values (BTVs) related to the range of detected concentrations for each constituent? Each constituent's detected results form a range; from lowest to highest concentration. The statistical evaluation used to develop the BTVs takes into account the entire range of detected results (possibly modified if any "outliers" are removed - see below). However, BTVs are selected to represent the upper end of the background analytes' ranges of concentrations in order to provide a balance between false-positive and false-negative results (see note below). This does not mean that BTVs will necessarily represent the highest detected concentration for a given analyte, although that is possible and, as derived from the Study s statistical analyses, did occur. Analytical results from soil samples collected at SSFL will be compared to the Look-Up Table values derived from the BTVs and sample RLs (see #7 below) to assess whether they exceed these values. If a concentration in a soil sample exceeds the respective Look-Up Table value, then it is deemed not to be within background and will require further analysis or action. Note: A "false positive" results in a concentration that is incorrectly selected as contamination when, in fact, it is within background. A false-positive result can lead to unnecessarily removing soil that has not been impacted. On the other hand, a "false-negative" result incorrectly selects impacted soil as being within background. This can result in contaminated soil ultimately being left in place. 5. What are "outliers?" By what method would DTSC determine outliers in the Study's data? In its "Technical Guide" (EPA 2007) to using the statistical-evaluation software "ProUCL," EPA defines outliers as: "Measurements (usually larger or smaller than the majority of the data values in a sample) that are not representative of the population from which they were drawn. The presence of outliers (high or low) distorts most statistics if used in any calculations." And, in its 2010 update, EPA (EPA 2010a) states: Decisions based upon distorted statistics can be incorrect, misleading, and expensive. It should be noted that the objective is to compute background statistics based upon the majority of the data set representing the dominant background population, and not to accommodate a few low probability outliers that may also be present in the background data set. In DTSC's 2008 guidance, "Strategies for Establishing and Using Background Estimates of Metals in Soil," outliers are described as: "...measurements that are extremely large or small relative to the rest of the data and are thought to misrepresent the population from which they are intended to be collected... These values may be a result of errors associated with the sample collection, laboratory analysis, transcription, or data entry. The values could also be a true reflection of the population being sampled (e.g., heterogeneous geologic conditions, evidence of contamination). 2

3 A conservative approach should be taken to identify outliers in the background data set because inclusion of these values may bias the population estimates derived from the data set. For example, mean (i.e., average) values are particularly sensitive to outliers... The background data set should be screened for outliers using multiple approaches...evaluating the reason for any extreme values, and using professional judgment to remove data points that are interpreted as outliers." To determine outliers, DTSC uses a process referred to as "exploratory data analysis" (EDA). In the DTSC guidance cited above, EDA is described as: "...an iterative process that uses several tools to evaluate data characteristics, make appropriate adjustments to the data set (e.g., adjust for censored [i.e., "non-detect"] values), and refine the data set (e.g., remove outliers). Prior to beginning EDA, all data should have been reviewed to ensure that it represents the target population...and that it is appropriate to include the data in the analysis..." For the background Study, EDA also involves using multiple lines of evidence to evaluate potential outliers in the background dataset. These lines of evidence include ensuring the data are valid (e.g., reviewing field and lab reports, data validation reports, etc.), visually examining graphical data representations (e.g., histograms, "box-and-whisker" plots, probability plots), and confirming outliers using statistical tests (e.g., Rosner's Test). When confirmed, outliers are removed from the dataset. The dataset is then evaluated statistically to derive the BTV(s). 6. Why was the DTSC study undertaken when a previous background study (MWH 2005) was conducted for SSFL? Prior to the present Study, DTSC received concerns from a number of community stakeholders that the previous background study (results reported in 2005) was suspect, primarily because the sample locations were situated close to SSFL such that the sample locations could have been impacted by operations conducted at SSFL. As a result, DTSC agreed to conduct the present (2011) Study, subsequently expanding the list of target analytes. The present Study includes metals, dioxin-furan compounds, perchlorate, pesticides and herbicides, semi-volatile organic compounds (including phthalates), formaldehyde, cyanide, ethanol and methanol. This comprehensive list addresses both naturally-occurring (e.g., metals) and anthropogenic ("man-made") constituents (e.g., pesticides and herbicides), with the latter potentially being present at the background Study sampling locations and SSFL due to natural processes (e.g., wildfires, air-fall deposition, etc.). 7. What are reporting limits (RLs)? In its 2010 (EPA 2010b) glossary of terms used in environmental measurement, EPA provides the following definitions for a RL: RL - "The minimum concentration that can be reported as a quantitated value for a target analyte in a sample following analysis. This defined concentration can be no lower than the concentration of the lowest calibration standard for that analyte, and can only be used if acceptable quality control criteria for the analyte at this concentration are met." In other words, RLs are the lowest amount of an analyte in a sample that can be quantitatively identified with acceptable precision and accuracy according to the Study's Quality Assurance Project Plan (see below) under stated analytical conditions. In order to take into account day-today fluctuations in instrument sensitivity, analyst performance, and other factors, the RL may be established at three times the method detection limit, or greater. 3

4 However, the definition above for RLs needs to be put into context. It is important to note that RLs are tied to method detection limits (MDLs); the latter defined in the U.S. Code of Federal Regulations, Title 40, Part 136, Appendix B as: "...the minimum concentration of a substance that can be measured and reported with 99% confidence that the analyte concentration is greater than zero and is determined from analysis of a sample in a given matrix (i.e., soil) containing the anlayte." Other than possible lab-related factors, such as those mentioned above (instruments, analysts, etc.), sample-related factors that may affect the RL values reported by a laboratory include sample contents, including organic material, other chemicals or constituents besides the analyte in question, and moisture. These may cause the RLs to be raised from three to five times the respective maximum detection limits, but higher values are common. 8. What is the Study's Quality Assurance Project Plan (QAPP)? EPA (2002b) defines a QAPP as: "...a written document that describes the quality assurance procedures, quality control specifications, and other technical activities that must be implemented to ensure that the results of the project or task to be performed will meet project specifications. Primary data collection, secondary data usage, and data processing...are described and documented in QA Project Plans." As is typical, the background Study's QAPP includes complete listings of the project's target analytes and their respective target RLs set as goals in the "ideal" sense; under conditions that are not affected by lab- and sample-related factors such as those indicated above. As is routine with other projects or studies involving laboratory analyses, the Study's QAPP was submitted to the labs, thus providing the set of requirements to be followed in meeting the Study's objectives, including communicating to DTSC those instances where requirements were not, or could not, be met and the corrective actions that were implemented accordingly. The Study s QAPP is available for review and/or downloading from DTSC s website: 9. What is the difference between the reporting limits (RLs) published by DTSC in its Quality Assurance Project Plan (QAPP) for the background Study and the RLs reported by the Study's laboratories that are associated with the actual concentrations detected by those labs? As indicated above for questions 7 and 8, the RLs published in the Study's QAPP are the target limits established as goals for the labs to accomplish, based on discussions with them in terms of what they and DTSC felt could be achieved given (1) the analytical methods involved, (2) modifications to the methods that were requested by DTSC, and (3) "ideal" conditions (i.e., "clean" soils). The reporting limits ultimately reported by the labs either met, were lower, or were higher than the QAPP RLs. The lab RLs that were lower than the QAPP RLs were typically achieved by increasing the amount of soil from a collected sample that was analyzed, whereas the lab RLs that were higher than the RLs identified in the QAPP were typically raised due to matrix interferences, such as those indicated above (e.g., organic material, moisture, etc.). 10. What roles do the Study's analytical results and associated laboratory reporting limits (RLs) play in developing the "look-up tables that are called for in the Administrative Orders on Consent (AOCs)? 4

5 The DOE and NASA AOCs state the following: "DTSC, in the course of overseeing and approving its chemical contaminant local background study, will determine local background levels and chemical detection limits (using methods that are consistent with EPA guidance on determining local background concentration values). Upon completion of the DTSC led chemical background study, a look-up table of the chemical cleanup levels will be prepared, which will include both local background concentrations as well as minimum detection limits for specific contaminants whose minimum detection limits exceed local background concentrations." The AOCs also state: "For chemical contaminants, detection limit means method reporting limit (or MRL), which is the lowest concentrations (sic) at which an analyte can be confidently detected in a sample and its concentration can be reported with a reasonable degree of accuracy and precision." And, "Detection limits (RLs) for specific contaminants exceed the local background concentration, in which case the cleanup goal shall be the detection limits for those specific contaminants." Plus: "...if...any constituent detected in a soil sample is above local background levels, step-outs will be taken to delineate the contamination and removed..." Thus, both the background threshold values (BTVs) derived statistically from the Study s laboratory analytical results and the lab-reported sample RLs are to be used in developing the Look-Up Table(s) for chemicals. 11. What values will be carried forward into the "Look-Up Table development process for constituents that were not detected by the Study's labs? In cases where a constituent was not detected, or for those analytes where no meaningful and defensible statistical assessment may be conducted, the sample reporting limit (RL) identified by the lab for the constituent will be considered in development of the Look-Up Table value. 12. Why and how did DTSC use the Rural Residential Risk-Based Screening Levels (RBSLs) that were published in the "Laboratory Selection Technical Memorandum" (URS 2011)? The rural residential RBSLs were used strictly in the lab-selection process to provide an initial basis for comparing the labs in terms of whether or not they were able to consistently achieve the reporting limits that DTSC was seeking for the background Study. Beyond that, the rural residential RBSLs play no further role in the Study. 13. How will DTSC determine if a detected constituent is naturally occurring or "man-made" ("anthropogenic")? By their very nature, certain constituents tested for in the background Study exist only as anthropogenic compounds; commonly produced by commercial-industrial manufacturing processes. These include the pesticides, herbicides, and phthalates. However, the other target analytes in the Study may be either similarly anthropogenic, or exist as bi-products of human activities, or exist by virtue of natural processes. The Study's site conceptual model (discussed above) and site-selection process for the Study s sampling areas postulate that the anthropogenic constituents found in the background Study s soils occur there as the result of the natural process of air-fall deposition. The other detected constituents that may be either anthropogenic, bi-products, or naturally-occurring are also 5

6 postulated to exist in soils where the Study was conducted due to natural processes, including wildfires, air-fall deposition, and/or soil formation. 14. How will DTSC incorporate data that may be residual from recent area wildfires into the background values? Would data (i.e., elevated concentrations of dioxins) be considered natural background for the Study's areas? All concentrations detected in the Study's samples are considered to be valid results and reflect either natural conditions or processes; whether they are from wildfires, air-fall deposition, or soil formation. Regardless of the type of constituent, whether an elevated analytical result is removed from further assessment or remains in the established background dataset and is carried through into the Look-Up Table development process, depends on the outlier determination discussed above in the answer to question 5. Thus, it is possible that elevated concentrations, such as for dioxins possibly resulting from wildfires, could be retained in the established background dataset. 15. For some of the Study's constituents, there may be so-called "outliers" (i.e., relatively low or high values) in the data. If, as DTSC contends, the CBRAs were not impacted by SSFL activities, then the resulting analytical data should reflect the natural ranges of constituents detected in the Study's samples. Why would such outliers be removed from the data to conduct the Study's statistical evaluations? As discussed above in the answer to question 5, outliers are determined through a series of steps and removed from the dataset because they are considered not to be representative of the overall population from which they were drawn and their presence in the dataset would distort most statistics and bias conclusions. An outlier is a valid result, may or may not be naturally occurring, and may be reflective of the conditions that exist at the location where the associated sample was collected. However, the outlier concentration is considered to be inconsistent with the data collected in the Study and retaining the concentration in the dataset would be both unreasonable and contrary to standard practice. Retaining the outlier would result in skewed statistics that may be either higher or lower depending on the outlier. 16. DTSC stated in its planning document(s) that the Study's analytical reporting limits (RLs) would be established as low as reasonably achievable, based on California Senate Bill 990 (SB990)-compliant RBSLs. In what way(s) are the reported background values SB990- compliant? As discussed above in in the answer to question 12, the Rural Residential-Risk-Based Screening Levels (RBSLs) were used in the lab-selection process and have no further role in the background Study. 17. Will the background threshold values (BTVs), to be derived using statistics for each constituent, be within the ranges of detected concentrations for those constituents? Depending on the statistical evaluation method that is used, it is possible that a resulting BTV could be a higher value than the maximum detected value for a given analyte. EPA considered and ultimately recommended (EPA 2011) that the "Upper Simultaneous Limit" at 95% confidence (USL95) statistic be used to derive the radiological background BTVs. EPA defines the USL95 as being: " based upon an established background dataset (that) represents that estimate of BTV such that all observations in the background dataset are less than or equal to USL95 with 95% (confidence)." 6

7 Thus, a USL95-derived BTV may be a higher value than the maximum detected concentration in order to satisfy the condition that all current and future "observations" are less than the USL95 with 95% confidence. DTSC is also assessing the USL95 as a possible statistic for deriving the background chemical BTVs (see below). 18. At what depth(s) were the Study's samples collected? Surface soil samples were collected between 0 and 0.5 feet below ground surface (bgs) and subsurface samples were collected, at random depths, between 2 and 10 feet bgs. 19. How will the results from soil sampling at SFFL be compared with the results from the Study? The Study results will be used in developing the Look-Up Table values, per the AOCs for the DOE and NASA portions of SSFL. The Look-Up Table values will become the clean-up values for DOE and NASA at SSFL. Chemical background results can also be used for the Boeing portion of the site when risk-based cleanup decisions are required. 20. What statistical methods will DTSC use to assess the Study's data and what process will DTSC use to make that decision? DTSC is considering several statistical methods that were also assessed by EPA for its radiological background study, including "Upper Percentile" at 95% confidence (UP95), "Upper Prediction Limit" at 95% confidence (UPL95), "Upper Tolerance Limit" at 95% coverage with 95% confidence (UTL95-95), and the USL95. However, DTSC is concerned that BTVs derived using the UP95 and UPL95 methods may not be suitable when comparisons are made using a large number of analytical results, based on: UP95 - The UP95 statistic does not take into account variability of future observations. When a large number of analytical results are to be compared, the use of the UP95 may lead to an unreasonably high number of false positives; site values declared to be exceeding background, when in fact, they are within background (see note above under question 4). UPL95 - Similarly, the UPL95 is useful when only a few known number of analytical results are to be compared against a UPL value. Using the UPL95 to compare many such results may also result in a relatively high number of false positives. However, both the UTL95-95 and USL95 appear to be reasonable choices for estimating BTVs, based on: UTL The UTL95-95 represents the statistic such that 95% of the results in the background population would be less than or equal to the UTL95-95 value (i.e., concentration) with 95% confidence. The UTL95-95 is designed to provide coverage for 95% of all potential analytical results (present and future) from the background population with 95% confidence. By definition, 5% of a large number of potential results from the background dataset are expected to exceed the UTL Use of the UTL95-95 is preferred when the number of future comparisons of laboratory analytical results is large or unknown. When the sample size is large (e.g., 500), UTL95-95 approaches the 95th percentile. USL95 - The USL95 is the statistic such that all potential laboratory analytical results (present and future) from the background population will be less than or equal to the USL95 value with 95% confidence (provides 100% coverage, with 95% confidence, for all present and future background 7

8 results). The USL95 may be an appropriate approach to estimating BTVs if there is little variability in the background dataset and there are no potential outliers. DTSC is initially calculating BTVs using the UP95, UPL95, UTL95-95 and USL95. For the reasons stated above, the use of UTL95-95 and USL95 are being further evaluated due to the implicit reduction in false positives while providing an adequate protection against false negatives (see note provided above under question 4). However, DTSC has not yet decided on the method it will use to derive the final BTVs. 21. Does DTSC intend to keep the Study's data separate, such as drainage and nondrainage or Chatsworth Formation and Santa Susana Formation, or combine the data? What affect would either decision have on the background threshold value (BTV) results? For purposes of the Study, the data is presented by formation and topography. For the Look-Up Table values, DTSC is evaluating whether, based on the Study's final statistical results, the data should be combined or kept separate. A significant challenge, also acknowledged by EPA, is that the soils within SSFL's developed areas, primarily within the Chatsworth Formation, have been disturbed; those soils having been mixed from different depths and/or mixed with fill taken from areas within the Santa Susana Formation. 22. Have "look-up tables been used before at other sites in California, or in the U.S.? A number of states, including California, and various federal agencies use what are referred to as "look up" tables, or similar listings that are intended to drive some form of action, including characterization and/or remediation. References: DTSC Strategies for Establishing and Using Background Estimates of Metals in Soil. Appendix B to Proven Technologies and Remedies Guidance Remediation of Metals in Soil. August. EPA. 2002a. Guidance for Comparing Background and Chemical Concentrations in Soil for CERCLA Sites. EPA/540/R-01/003. Office of Emergency and Remedial Response, Washington, D.C. September. EPA. 2002b. Guide for Quality Assurance Project Plans. Office of Environmental Information. Document EPA QA/G5. December. EPA Guidance on Systematic Planning Using the Data Quality Objective Process. Office of Environmental Information. Document EPA QA/G4. February. EPA ProUCL Version Technical Guide. EPA Research and Development. Document EPA/600/R-07/041. April. EPA. 2010a. ProUCL Version Technical Guide (Draft). EPA Research and Development. Document EPA/600/R-07/041. May. EPA. 2010b. Environmental Measurement, Glossary of Terms. Available at: January. 8

9 EPA Final Radiological Background Study Report, Santa Susana Field Laboratory, Ventura County, California. October. 9

10 MWH Final Soil Background Report, Santa Susana Field Laboratory, Ventura County, California. Appendix D. September. [in MWH Standardized Risk Assessment Methodology (SRAM) Work Plan, Santa Susana Field Laboratory, Ventura County, California. Revision 2 - Final. September.] URS Technical Memorandum. Laboratory Evaluation and Selection, Chemical Soil Background Study, Santa Susana Field Laboratory, Ventura County, California. July. 10