Evaluation of the Monitor / RED1 Statistical Inventory Reconciliation Method

Transcription

1 Evaluation of the Monitor / RED1 Statistical Inventory Reconciliation Method Final Report PREPARED FOR Leighton O Brien Pty. Ltd. December 13, 1995 (Revised August 24, 1999) Ken Wilcox Associates, Inc Valley Ridge Drive, Grain Valley, MO 64029, USA Voice (816) , Fax (816) info@kwaleak.com, Web

2 Evaluation of the Monitor / RED1 Statistical Inventory Reconciliation Method Final Report PREPARED FOR Leighton O Brien Pty. Ltd. Suite 13, 96 Camberwell Rd Hawthorn East, Victoria 3123 Australia December 13, 1995 (Revised August 24, 1999)

3 Preface Leighton O Brien Pty. Ltd. has developed the Monitor / RED1 Statistical Inventory Reconciliation Method. An independent, third-party evaluation of this SIR method was conducted by Ken Wilcox Associates, Inc. using test procedures which were taken from the EPA evaluation protocols "Standard Test Procedures for Evaluating Leak Detection Methods: Statistical Inventory Reconciliation Methods (EPA Document Number EPA/530/UST-90/007, June 1990)." Technical questions regarding this SIR method or this evaluation should be directed to Mr. Steve O Brien, Leighton O Brien Pty. Ltd., Victoria, Australia, at KEN WILCO ASSOCIATES, INC. H. Kendall Wilcox, Ph.D., President December 13, 1995 (Revised August 24, 1999) 1 1 Revisions consist of updated contact information and the addition the Leighton O Brien Pty., Ltd. international SIR brand name RED1. ii

4 Table of Contents Page Preface ii 1.0 Background Description of the Monitor / RED1 SIR Method Presentation of Inventory Data Description of Evaluation Methods Test Results and Discussion Conclusions 6 Attachment A. EPA Forms for the Monitor / RED1 SIR Method iii

5 Monitor / RED1 SIR Method 1.0 Background The Environmental Protection Agency's (EPA's) regulations for underground storage tanks require owners and operators to check for leaks on a routine basis using one of a number of detection methods (40CFR Part 280, Subpart D). To ensure the effectiveness of these methods, EPA set minimum performance standards for equipment used to comply with the regulations. All tank tightness test methods must be capable of detecting a 0.10 gallon per hour leak rate with a probability of detection of at least 0.95 and a probability of false alarm of no more than Similarly, all monthly monitoring procedures must be capable of detecting a 0.20 gallon per hour leak rate with a probability of detection of 0.95 and a probability of false alarm of In addition, the criteria stipulate that the procedure must be capable of detecting a leak from any portion of the tank system. It is up to tank owners and operators to select a method of leak detection that has been shown to meet the relevant performance standards. One mechanism for demonstrating that these performance goals have been reached is to conduct a performance evaluation using EPA's standard test procedures for leak detection equipment. The procedures for methods which involve a statistical analysis of daily inventory records taken by the tank owner/operator are described in the test protocol "Standard Test Procedures for Evaluating Leak Detection Methods: Statistical Inventory Reconciliation Methods (EPA Document Number EPA/530/UST - 90/007 Dated June 1990)." 2.0 Description of the Monitor / RED1 Statistical Inventory Reconciliation Method The Monitor / RED1 SIR Method is a means of detecting product losses from underground storage tanks by a detailed evaluation of daily inventory measurements maintained by the UST owner/operator. The purpose of the Monitor / RED1 SIR Method is to provide periodic leak detection testing of UST's to identify existing or emergent losses of product, should they occur, from these systems. The basic approach involves a systematic evaluation of the inventory observations obtained by the UST owner/operator who either gauges the tank manually or records data from an automatic tank gauging system. The observations consist of the amount of product dispensed daily, deliveries of product to the tank system, and gauge readings in either inches (centimeters) or gallons (liters), or both. In order to identify loss trends in an inventory record, the Monitor / RED1 SIR Method evaluates the cumulative differences between calculated book inventory of product and actual volumes of product in the tank over time, as determined by daily gauging. The statistical analyses consists of identifying the influences of various sources of error which have been introduced in the inventory data and eliminating the effects that they cause in the inventory record. In addition to detecting a consistent loss of product over time, other sources of error are detected. Among the various sources of error that the Monitor / RED1 SIR Method accounts for are: Page 1

6 Monitor / RED1 SIR Method Dispensing Meter Errors Calibration Errors Conversion Chart Miscalibration Vapor Loss Thermal Effects Delivery Errors Unexplained Losses or Gains Water Inflow and Outflow Dipstick Errors Transcription Errors Once identified, the effect of the influences of these sources of error are eliminated from the record to identify the underlying loss trend, if any, that may exist. Although these sources of error are identified and compensated for by the Monitor / RED1 SIR Method, the test protocol does not stipulate that all of these features of the analytical procedures be evaluated. An official description of the Monitor / RED1 SIR Method has been provided in Attachment A of this report on the EPA forms, Description - Statistical Inventory Reconciliation Method. 3.0 Presentation of Inventory Data The Monitor / RED1 SIR Method uses measurements recorded by the UST owner/operator during each day that the UST's are in active operation. Typically, one month of data is requested to be used for analysis. These measurements are provided to Leighton O Brien Pty. Ltd. either manually or on standard forms that are provided by Leighton O Brien Pty. Ltd. Regardless of whether the source of the data is manual gauging and associated measurements or output from ATG's, the essential data that must be provided to Leighton O Brien Pty. Ltd. includes: Dates of recording Daily gauge readings and/or conversion Quantity of product dispensed Quantity of product added (deliveries) Page 2

7 Monitor / RED1 SIR Method 4.0 Description of Evaluation Methods The evaluation of statistical inventory reconciliation methods such as the Monitor / RED1 SIR Method is described in the EPA protocol. This protocol specifies that Statistical Inventory Reconciliation methods can be classified as either quantitative or qualitative procedures, depending upon how results are reported. Quantitative procedures report results in terms of leak rate, if any, from the system. Qualitative procedures report results simply in terms of pass, fail or inconclusive. The protocol provides detailed test methods for evaluating each type of procedure. Since the Monitor / RED1 SIR Method reports results in terms of loss rate, those portions of the EPA protocol pertaining to quantitative statistical inventory reconciliation methods were used. Sections of this protocol which apply specifically to quantitative methods include Section b, Experimental Design for Quantitative Data and Section 7.2, Estimation of the SIR Method's Performance Parameters for Quantitative Data. Information describing each tank system that was used in the evaluation has been provided in Table 1. Inventory Data from 41 manifolded UST systems demonstrated to be tight were collected. The inventory data was provided for these tanks which ranged in size from 24,000 to 40,000 gallons and included two, three, and four tank systems. The tank systems used in the evaluation came from Fort Lauderdale, FL, Houston, T, and Memphis, TN. The data included records from a range of hot, mild and cold weather months. Simulated leaks were introduced into the inventory data by incrementing the appropriate volumes to each days stick reading after the stick values were converted to volumes using the strapping chart. The specific procedures for this are described in Section 6.2.4, Create Evaluation Data Base--Step 4, of the Evaluation Protocol. Inventory data that was prepared according to the EPA protocol was submitted to Leighton O Brien Pty. Ltd. for statistical analysis using their standard analysis algorithms. Three leak rates were introduced into the data: 0.05 gal/h, 0.10 gal/h, and 0.20 gal/h. Eight data sets were created for each of the three induced leak rates (0.05 gal/h, 0.10 gal/h, 0.20 gal/h). An additional 17 data sets were submitted without an induced leak rate. Nine of the data sets without leak rates were duplicates of data sets that had leak rates (Prepared according to the EPA protocol). The data sets used in the evaluation has been summarized in Table Test Results and Discussion The data that was analyzed by Leighton O Brien Pty. Ltd. is summarized in Table 1. The results of the statistical analysis are summarized in Table 2. The differences between the induced rate and the measured rates for each set of inventory data were used to calculate the standard deviation of the Monitor / RED1 SIR Method. The Monitor / RED1 SIR Method reports data which is not significantly different from zero as a zero leak rate. Page 3

8 Monitor / RED1 SIR Method Table 1. Test Data and Results for the Monitor / RED1 Evaluation Description No. of No. of Method Date of Date of Was the Was the Induced Estimated Est-Ind Inventory Inventory Nearest City Inventory Tank of Monthly Usable Non-Usable Used to Tightness Tightness SIR Result SIR Result Leak Leak Leak Test Start Date End Date the Tank is Recording Volume Manifolded Output Product Daily Daily Verify Tank Test Test Inconclusive? Not Analyzed? Rate Rate Rate No. (mmddyy) (mmddyy) Season Located Method (gallons) Systems (gal/h) Stored Records Records is Tight (1) (2) (Yes or No) (Yes or No) (gal/h) (gal/h) (gal/h) 1 07/01/93 07/31/93 hot Houston, T Stick 24, ,000 gal Unl Non-volumetric 10/29/93 10/31/94 No No /01/93 12/31/93 cold Houston, T Stick 24, ,000 gal Unl Non-volumetric 10/29/93 10/31/94 No No /01/94 07/31/94 Hot Houston, T Stick 24, ,000 gal Unl Non-volumetric 10/29/93 10/31/94 No No /01/94 10/31/94 Mild Houston, T Stick 24, ,000 gal Unl Non-volumetric 10/29/93 10/31/94 No No /01/94 11/30/94 Mild Houston, T Stick 24, ,000 gal Unl Non-volumetric 10/29/93 10/31/94 No No /01/94 12/31/94 Cold Houston, T Stick 24, ,000 gal Unl Non-volumetric 10/29/93 10/31/94 No No /01/95 01/31/95 Cold Houston, T Stick 24, ,000 gal Unl Non-volumetric 10/29/93 10/31/94 No No /01/93 08/31/93 hot Ft. Lauderdale, FL Stick 30, ,000 gal Unl Non-volumetric 11/01/93 10/08/94 No No /01/94 03/31/94 mild Ft. Lauderdale, FL Stick 30, ,000 gal Unl Non-volumetric 11/01/93 10/08/94 No No /01/94 07/31/94 Hot Ft. Lauderdale, FL Stick 30, ,000 gal Unl Non-volumetric 11/01/93 10/08/94 No No /01/94 08/31/94 Hot Ft. Lauderdale, FL Stick 30, ,000 gal Unl Non-volumetric 11/01/93 10/08/94 No No /01/94 09/30/94 Mild Ft. Lauderdale, FL Stick 30, ,000 gal Unl Non-volumetric 11/01/93 10/08/94 No No /01/94 10/31/94 Mild Ft. Lauderdale, FL Stick 30, ,000 gal Unl Non-volumetric 11/01/93 10/08/94 No No /01/93 11/30/93 mild Memphis, TN Stick 40, ,000 gal 8634 Unl Non-volumetric 10/24/93 10/22/94 No No /01/94 02/28/94 cold Memphis, TN Stick 40, ,000 gal Unl Non-volumetric 10/24/93 10/22/94 No No /01/94 04/30/94 mild Memphis, TN Stick 40, ,000 gal Unl Non-volumetric 10/24/93 10/22/94 No No /01/94 05/31/94 Mild Memphis, TN Stick 40, ,000 gal Unl Non-volumetric 10/24/93 10/22/94 No No /01/94 06/30/94 Hot Memphis, TN Stick 40, ,000 gal Unl Non-volumetric 10/24/93 10/22/94 No No /01/94 07/31/94 Hot Memphis, TN Stick 40, ,000 gal Unl Non-volumetric 10/24/93 10/22/94 No No /01/94 08/31/94 Hot Memphis, TN Stick 40, ,000 gal Unl Non-volumetric 10/24/93 10/22/94 No No /01/94 09/30/94 Mild Memphis, TN Stick 40, ,000 gal Unl Non-volumetric 10/24/93 10/22/94 No No /01/94 10/31/94 Mild Memphis, TN Stick 40, ,000 gal Unl Non-volumetric 10/24/93 10/22/94 No No /01/94 11/30/94 Mild Memphis, TN Stick 40, ,000 gal Unl Non-volumetric 10/24/93 10/22/94 No No /01/94 12/31/94 Cold Memphis, TN Stick 40, ,000 gal Unl Non-volumetric 10/24/93 10/22/94 No No /01/95 02/28/95 Cold Memphis, TN Stick 40, ,000 gal Unl Non-volumetric 10/24/93 10/22/94 No No /01/95 03/31/95 Mild Memphis, TN Stick 40, ,000 gal Unl Non-volumetric 10/24/93 10/22/94 No No /01/95 04/30/95 Mild Memphis, TN Stick 40, ,000 gal Unl Non-volumetric 10/24/93 10/22/94 No No /01/95 05/31/95 Mild Memphis, TN Stick 40, ,000 gal Unl Non-volumetric 10/24/93 10/22/94 No No /01/95 06/30/95 Hot Memphis, TN Stick 40, ,000 gal Unl Non-volumetric 10/24/93 10/22/94 No No /01/95 07/31/95 Hot Memphis, TN Stick 40, ,000 gal Unl Non-volumetric 10/24/93 10/22/94 No No /01/95 08/31/95 Hot Memphis, TN Stick 40, ,000 gal Unl Non-volumetric 10/24/93 10/22/94 No No /01/95 09/30/95 Mild Memphis, TN Stick 40, ,000 gal Unl Non-volumetric 10/24/93 10/22/94 No No /01/93 12/31/93 cold Houston, T Stick 24, ,000 gal Unl Non-volumetric 10/29/93 10/31/94 No No /01/95 01/31/95 Cold Houston, T Stick 24, ,000 gal Unl Non-volumetric 10/29/93 10/31/94 No No /01/94 03/31/94 mild Ft. Lauderdale, FL Stick 30, ,000 gal Unl Non-volumetric 11/01/93 10/08/94 No No /01/94 08/31/94 Hot Ft. Lauderdale, FL Stick 30, ,000 gal Unl Non-volumetric 11/01/93 10/08/94 No No /01/94 02/28/94 cold Memphis, TN Stick 40, ,000 gal Unl Non-volumetric 10/24/93 10/22/94 No No /01/94 04/30/94 mild Memphis, TN Stick 40, ,000 gal Unl Non-volumetric 10/24/93 10/22/94 No No /01/95 02/28/95 Cold Memphis, TN Stick 40, ,000 gal Unl Non-volumetric 10/24/93 10/22/94 No No /01/95 07/31/95 Hot Memphis, TN Stick 40, ,000 gal Unl Non-volumetric 10/24/93 10/22/94 No No /01/95 08/31/94 Hot Memphis, TN Stick 40, ,000 gal Unl Non-volumetric 10/24/93 10/22/94 No No Page 4

9 Table 2. Performance Parameters for the Monitor / RED1 SIR Method Monitor / RED1 SIR Method Parameter Value Basic Statistics Mean Squared Error gal 2 /h 2 Bias Negligible Standard Deviation gal/h Variance gal 2 /h 2 Threshold 0.05 gal/hr Probability of False Alarm for a Non-leaking Tank 0.4% Probability of Detection of a 0.20 gal/hr Leak 99.6% Optional Calculations Minimum Threshold for 5% P FA gal/h Minimum Leak Rate Detectable with 5% P FA and 95% P D gal/h Maximum Tank Size (Including Manifolded Systems) 60,000 gallons Page 5

10 Monitor / RED1 SIR Method With a 0.05 gal/hr threshold, an analysis of the complete set of 41 data records provides a probability of a false alarm (P FA ) of 0.4% for a non-leaking tank and a probability of detection (P D ) of 99.6% for a 0.10 gal/hr leak. The minimum threshold for which a leak can be declared with a 5% P FA is gal/h. The minimum leak which can be detected with a 95% P D using the threshold for a 5% false alarm rate (0.031 gal/h) is gal/h. Based on the size of the tank systems used in the evaluation, the maximum size tank system to which this method can be applied is 60,000 gallons. 6.0 Conclusions The following conclusions are based on the results of the testing described in this report. 1. The Monitor / RED1 SIR Method, for detecting leaks in underground storage tanks exceeds the EPA criteria for Annual Tank Tightness Testing for 0.10 gal/hr leaks. (P D > 95%, P FA <5%). 2. For a threshold of 0.05 gal/h, the P FA is 0.4% and the P D is 99.6%. 3. Based on the size tanks used in the evaluation, the method may be applied to tanks and manifolded systems with sizes up to 60,000 gallons. 4. There is no bias indicated in the results of the testing at the 5% level. Page 6

11 Attachment A EPA Forms for the Monitor / RED1 Statistical Inventory Reconciliation Method

12 Results of U.S. EPA Standard Evaluation Statistical Inventory Reconciliation Method This form tells whether the statistical inventory reconciliation (SIR) method described below complies with requirements of the federal underground storage tank regulation. The evaluation was conducted by the vendor of the SIR method or a consultant to the vendor according to the U.S. EPA's "Standard Test Procedure for Evaluation Leak Detection Methods: Statistical Inventory Reconciliation Methods." The full evaluation report also includes a form describing the method and a form summarizing the test data. Tank owners using this leak detection system should keep this form on file to prove compliance with the federal regulations. Tank owners should check with State and local agencies to make sure this form satisfies their requirements. Method Description Name Monitor / RED1 Version Vendor Leighton O Brien Pty. Ltd. Suite 13, 96 Camberwell Rd (street address) Hawthorn East Victoria 3123 Australia (city) (state) (zip) (country) (phone) Evaluation Results If applicable, vendor's threshold = 0.05 gallon per hour or vendor's criterion: This statistical inventory reconciliation method reports results on the following basis (check one): () quantitative results (leak rate reported) ( ) qualitative results (pass, fail, inconclusive) The test results are: Reported Results* Tight Leak Inconclusive Total Not Analyzed Analyzed Tight Actual Induced Leak Total * Table not applicable to quantitative systems. SIR Method -Results Form Page 1 of 3

13 SIR Method Monitor / RED1 Version Evaluation Results (continued) The proportions of inventory records reported inconclusive are: 0 % among tight tanks (see note below) 0 % among leaking tanks 0 % among all tanks The probability of false alarms, P(FA), based on the vendor's threshold, is 0.4 % For qualitative methods, a 95% confidence interval for P(FA) is from to %. The probability of detection, P(D) is 99.6 %. This is valid for a leak rate of (check one): () 0.10 gallon per hour ( ) 0.20 gallon per hour For qualitative methods, a 95% confidence interval for P(D) is from to %. Based on these results, the method () does ( ) does not meet the federal performance standards established by the U.S. Environmental Protection Agency of 0.20 gallon per hour at P(D) of 95% and P(FA) of 5%. Test Conditions During Evaluation The data evaluation set included data from tanks of the following sizes Tank Size (gallons) Number of Records <5,000 5,000-10,000 10,000-15,000 >15,000* Total # Records *Total volumes of manifolded systems ranged from 20,000 to 40,000 gallons. The tanks had various monthly throughputs. Percentile of Records (median) 75 Monthly throughput (gallons) 11,928 15,019 19,438 The data included 14 records during hot weather months. 17 records during mild weather months. 10 records during cold weather months. SIR Method -Results Form Page 2 of 3

14 SIR Method Monitor / RED1 Version Limitations on the Results The performance estimates above are only valid when: M The method has not been substantially changed. M The vendor's instructions for using the method are followed. M The manifolded tank system is no larger than 60,000 gallons. M The data records cover 30 days or more. M The method is based on single or manifolded tank systems. M Other limitations specified by the vendor or determined during testing: >Safety disclaimer: This test procedure only addresses the issue of the method's ability to detect leaks. It does not test data recording equipment for safety hazards. Certification of Results I certify that the statistical inventory reconciliation method was applied according to the vendor's instructions. I also certify that the evaluation was performed according to the standard EPA test procedure for statistical inventory reconciliation and that the results presented above are those obtained during the evaluation. H. Kendall Wilcox, President Ken Wilcox Associates, Inc. (printed name) (organization performing evaluation) (signature) Grain Valley, Missouri (city, state, zip) December 13, 1995 (Revised August 24, 1999) 1 (816) (date) (phone number) 1 Revisions consist of updated contact information and the addition the Leighton O Brien Pty., Ltd. international SIR brand name RED1. SIR Method -Results Form Page 3 of 3

15 Reporting Form For Test Results Statistical Inventory Reconciliation Method Method Name and Version: Monitor / RED1 Date: December 12, 1995 (Revised August 24, 1999) Submitted Results Reported by Vendor If Quantitative If Qualitative Induced Estimated Est.-Ind. Tank Tight? Record Leak Rate Leak Rate Leak Rate (Yes, No, or Vendor's Comments Code No. (gal/h) (gal/h) (gal/h) Inconclusive) SIR Method-Data Reporting Form Page 1 of 1

16 Description Statistical Inventory Reconciliation Method This section describes briefly the important aspects of the statistical inventory reconciliation (SIR) method. It is not intended to provide a thorough description of the principles behind the SIR method and associated computer software. General Information Method name: Monitor / RED1 If applicable: Version and revision number Date December 13, 1995 (Revised August 24, 1999) 1 Vendor Leighton O Brien Pty, Ltd. Vendor address and phone number, including area code: Suite 13, 96 Camberwell Rd., Hawthorn East Victoria 3123, Australia Contact Mr. Steve O Brien Data Requirements Does the method require use of a specified data form provided by the vendor? ( ) yes () no How are the inventory data recorded: ( ) manually, on provided forms () manually, no forms provided ( ) hand entered into a computer ( ) direct entry from ATGS () other Standard forms will be made available as an option to use. What is the required number of usable daily inventory records necessary to detect the indicated leak rate (gallon per hour) with 95% confidence? If the leak rate is 0.10, the number of daily readings is 26. If the leak rate is 0.20, the number of daily readings is. 1 Revisions consist of updated contact information and the addition the Leighton O Brien Pty., Ltd. international SIR brand name RED1. SIR Method - Description Page 1 of 3

17 Data Requirements (continued) What is the vendor's recommended number of daily records? ( ) 60 daily records ( ) 90 daily records () other, specify 26 daily records Does the method allow for closure of the station on one or more consecutive days per week? () yes ( ) no Does the method require meter calibration? () yes; specify how frequently per state requirements ( ) no (calibration recommended) Identification of Causes for Discrepancies Which of the following factors does the method consider? Check the appropriate categories. dispensing meter errors calibration errors conversion chart miscalibration vapor loss thermal effects others (list) Identify Only Compensate Not Considered Which of the following effects does the method identify and quantify? leak rate delivery errors unexplained losses or gains water inflow water outflow dipstick errors others (list) transcription errors Identify Only Quantify Not Considered SIR Method - Description Page 2 of 3

18 Reporting of Leak Status Is the leak status reported in terms of a leak rate (e.g., gal/h or gal/day)? () yes ( ) no ( ) if the answer to the above question is "No," how are the results reported? Explain What criterion does the method use to declare that a tank is leaking? Exceptions ( ) average daily discrepancy exceeds threshold of gal/h ( ) daily discrepancy relative to variability exceeds threshold of gal/h ( ) water level change exceeds threshold of inch () statistically significant continuous loss at the 0.05 level of significance ( ) other (specify) Are there any conditions under which the statistical inventory method is inadequate? () insufficient number of usable records ( ) irregular time intervals between dipstick readings () unacceptable daily variability of inventory records (if excessive) ( ) others (describe briefly) What elements in the record keeping are left to the discretion of the personnel on site? ( ) length of record keeping if beyond minimum requested ( ) others (describe briefly) () none If applicable, attach a copy of the inventory data collection form(s) as provided to the user by the vendor. SIR Method - Description Page 3 of 3