Measurement Uncertainty Estimates of Collocated CSN and IMPROVE Data for use in calculating a PM 2.5 light

Transcription

1 Measurement Uncertainty Estimates of Collocated CSN and IMPROVE Data for use in calculating a PM 2.5 light extinction indicator 11/2/ U.S. Environmental Protection Agency

2 Why? It s a good idea to understand what variability we may be dealing with in a potential new secondary NAAQS. We need to include measurement quality objectives for each pollutant in order to control data quality. It s probably time to review both the CSN and IMPROVE Quality Assurance Project Plans (QAPPs) 11/2/2011 U.S. Environmental Protection Agency 2

3 Calculating the PM 2.5 light extinction indicator PM2.5 βext = 3[AS]f(RH) + 3[AN]f(RH) + 4[OCM] + 1[FS] + 10[EC] Where: AS = ammonium sulfate, RH = relative humidity, AN = ammonium nitrate, OCM = organic carbon mass, FS = fine soil, and EC = elemental carbon are in units of µg/m 3 11/2/2011 U.S. Environmental Protection Agency 3

4 Two Measurement Uncertainty Approaches Discussed 1. Estimate variability of each parameter used in the LE calculation separately and then perform some additive error (propagation of error) assessment. 2. Estimate the LE value of the primary and the collocated sampler each day both samplers provide valid results. Estimate variability of the LE pairs and provide some statistically relevant estimate of the average variability or a distribution of variability. Approach #2 Chosen 11/2/2011 U.S. Environmental Protection Agency 4

5 Statistics Used - Variability 90 th percentile upper bound coefficient of variation (CV) 1) Estimate Relative Percent Difference d i = X i Y i ( X + Y ) i i / ) Estimate the precision upper bound statistic. CV ub, is a standard deviation on d i with a 90 percent upper confidence limit This is the current CFR Statistic performed for the collocated sites in the PM 2.5 network for the primary standard. EPA is planning to propose using the same statistics in the secondary standard. 11/2/2011 U.S. Environmental Protection Agency 5

6 Statistics Used - Bias The current CFR bias statistic simply estimates the percent difference between the two collocated values as follows: where, measured is the concentration indicated by the CSN light extinction values and audit is the IMPROVE light extinction values. We take the average of the percent differences of all collocated data pairs to estimate average bias for a site. 11/2/2011 U.S. Environmental Protection Agency 6

7 Three Data Sets Used Collocated CSN data for parameter level and LE evaluations, Collocated (modules) IMPROVE data for parameter level evaluations, and Collocated CSN and IMPROVE data for LE evaluations. Even though approach #2 was used to estimate variability at LE values, we did look variability at the parameter level 11/2/2011 U.S. Environmental Protection Agency 7

8 Sites Used for Variability Estimates CSN Collocated Sites CSN/IMPROVE Collocated Sites CSN/IMPROVE Collocation Site AQS Site ID Date of Carbon Conversion Number of paired values Birmingham, AL May 1, Phoenix, AZ October 1, Fresno, CA April 1, New York, NY May 1, Seattle, WA May 1, IMPROVE Collocated Module Sites Common Site Name Location Site Code Big Bend, NP TX BIBE1 Blue Mounds MN BLMO1 Everglades NP FL EVER1 Frostburg MD FRRE1 Gates of the Mountains MT GAMO1 Hercules-Glades MO HEGL1 Hoover CA HOOV1 Lassen Volcanic NP CA LAVO1 Medicine Lake MT MELA1 Mammoth Cave, NP KY MACA1 Mesa Verde CO MEVE1 Phoenix AZ PHOE1 Proctor Maple Research Facility VT PMRF1 Olympic WA OLYM1 Sac and Fox KS SAFO1 Saint Marks FL SAMA1 Sawtooth NF ID SAWE1 Seney MI SENE1 Trapper Creek AK TRCR1 11/2/2011 U.S. Environmental Protection Agency 8

9 Parameters Used in Evaluation AQS Parameter Code Parameter Aluminum Calcium Iron Titanium Silicon Nitrate Sulfate Organic Carbon * Elemental Carbon * * Parameter codes for carbon after the carbon conversion 11/2/2011 U.S. Environmental Protection Agency 9

10 Parameter Evaluation - CSN Variability Collocated CSN Data Upper Bound 90th Percentile CV From Values > MDL 11/2/2011 U.S. Environmental Protection Agency 10

11 Parameter Evaluation - IMPROVE Variability IMPROVE Data Upper Bound 90th Percentile CV From Values > MDL 11/2/2011 U.S. Environmental Protection Agency 11

12 Parameter CV Averages- By Network 11/2/2011 U.S. Environmental Protection Agency 12

13 Light Extinction Variability 90 Percentile Upper bound CV at 10 Percentile Concentration Breaks 11/2/2011 U.S. Environmental Protection Agency 13

14 LE Variability Around a Proposed NAAQS Illustrates variability where a number of CSN or IMPROVE sites had VI concentrations within the proposed NAAQS range a 5% CV would equate to a to +1.5 deciview variability around a NAAQS value 25 to 30 deciviews* * An example NAAQS value for illustrative purposes only 11/2/2011 U.S. Environmental Protection Agency 14

15 LE Relative Bias Goal- determine whether there is a systematic bias between networks relative to LE estimate. The true concentration is unknown. We picked the IMPROVE network to be truth. CSN bias slightly high compared to IMPROVE Site Average Bias (%) Birmingham, AL 2.1 Phoenix, AZ 6.1 Fresno, CA 0.54 NY, NY 3.9 Seattle, WA 3.7 Average-All sites /2/2011 U.S. Environmental Protection Agency 15

16 Summary Calculated Light Extinction (LE) CVs around the NAAQS value are in the range of 5% LE measurement uncertainty in current CSN and IMPROVE quality systems can be controlled to the current PM 2.5 precision data quality objective of 10% Although we do not have a true bias estimate both networks are producing comparable LE results with the CSN network producing slightly higher LE values CSN network is in the process of reviewing its QAPP and will look to determine if the current measurement quality objectives of the LE parameters need revision. Need to ensure that collocated CSN/IMPROVE samplers continue to operate at a reasonable number of sites to evaluate data comparability. May want to consider rotating collocated sites to improve data representativeness. 11/2/2011 U.S. Environmental Protection Agency 16