Guest Speaker: Bill Frietsche US EPA 1
April 7: QA Systems, EPA definitions, PQAOs and common sense Mike Papp April 14: Routine Quality Control and Data Management (1-pt QC, flow rate, and instrument stability checks) Travis Maki April 21: Audits Overview (NPAP, PEP, Annual PE, Flow Rate Audits) Jeremy Howe April 28: Calculating Bias and Precision and AQS reports Bill Frietsche May 5: 40 CFR 58 App. A- Gaseous Pollutants Glenn Gehring May 12: 40 CFR 58 App. A- Ozone Brenda Jarrell May 19: 40 CFR 58 App. A- PM filter and continuous methods Brandy Toft 2
the difference between your answer and the truth th Two components of total error (accuracy): Bias (jump) Precision (wiggle) 3
Some imprecision is unavoidable Sometimes up, sometimes down random Difference divided by best estimate of the truth truth is: For gas QC checks: known conc. For PM flow rate: audit (known) FR For PM2.5 collocated: their average 4
d i = the difference between a known value and Repeated measurement of the same thing your value (flow rate, conc., voltage, that should remain the Sequence of check k(date, time, same) check number) 5
Pairs of simultaneous measurements (collocated instruments) d i (difference between collocated values) date 6
Can be sudden or due to slow drift over time: d i = the difference between the true value (audit value) and the analyzer date 7
Includes both bias and precision At any one time, an audit value could be close to the analyzer value due to precision errors in both Or, audit value could be far from analyzer value due to precision errors adding Audits estimate Accuracy EPA uses audit results to verify that ongoing QC checks do represent total error 8
Auditor s see only one day, while your QC checks see overall big picture 9
1-pt QC checks no longer called precision checks, because the results are used (by YOU) to calculate both precision and to estimate total error Each check <= 7% is the CRITICAL criteria for each set of data since last passing check The audits basically verify your precision and bias that have been calculated all along from your QC checks (concentration and flow rate, for PM) 10
Method Pollutants Frequency MQO One-Point QC SO 2, NO 2, Every 2 O 3 : SO 2, NO 2, CO : O 3, CO Weeks Precision = Precision = 7% 10% Flow Rate Verification (QC check) Flow Rate Verification (QC check) Collocated Sampling PM 10 (lo- Vol) PM 2.5 PM 10 (high- Vol), TSP PM 10, TSP, PM 2.5 Once every 4 weeks Once per quarter 15% of Network Every 12 Days <= 4% of Standard <= 7% of Standard Precision CV= 10% (which means RPD =14%) *and total error, more on this later. 11
Estimates your overall system uncertainty, if you were perfectly calibrated Changes over time can point to operator error, lab error, poorly written procedures, equipment/standards going bad Changes can be fixed sooner rather than at audit EPA compares precision by site and agency 12
Useful to track precision (or bias) over time d i See this example (and you fill in your values) at: Another example in the DASC tool: 13
The DASC tool automatically writes your values from the columns into the chart: Measured value is your analyzer Audit value is the known d i You may want tto add dddates to the x-axis, or use my example if you want to set up control charts 14 in your program
In their annual QA reports, from your RP transactions in AQS: http://www.epa.gov/ttnamti1/qareport.html http://www.epa.gov/ttnamti1/anlqa.html html d i 15
Recommendations in redbook say: Critical criteria invalidate every hour that is not met Operational criteria i means something probably wrong, go check it Systematic criteria mean as a set (day? year?) data is not usable for NAAQS, but individual hours or more may be valid 16
(NO2 and SO2 are the same) 17
Method Pollutants t Frequenc MQO y Flow Rate PM 10 (lo- Once <= 4% of Standard Verification (QC check) Vol), PM 2.5 every 4 weeks Flow Rate PM 10 (high- Once per <= 7% of Standard Verification (QC check) Vol), TSP quarter 18
Method Pollutants t Frequency MQO Collocated Sampling PM 10, TSP, PM 2.5 15% of Network Every 12 Days Precision as CV < = 10% (meaning the relative percent diff must be less than 14% for conc > 3 ug/m3) X i Y i d i 100 ( X i Y i ) 2 19
Keep the bias component minimized by calibrating and verifying your equipment against a standard d Keep precision component low by consistency You can work to keep bias down, while precision is often out of your control below a certain limit EPA calculates from your RA transactions meas audit d 100 i audit 20
Method Pollutants Frequency MQO Annual Performance Evaluation (Audit) SO 2, NO 2, O 3, Once per <= 15% for each audit CO Year concentration OPERATIONAL Semi-Annual Flow PM 10, PM 2.5 Every 6 <= 4% of Standard Rate Audit Months OPERATIONAL Semi-Annual Flow Rate Audit PM 10 (high- Vol), TSP Every 6 Months <= 10% of Standard OPERATIONAL PM 2.5 PEP Program NPAP PM 2.5 SO 2, NO 2, O 3, CO Quarter Year (see QA Requireme nts.xls) Bias = 10% -- OPERATIONAL 21
Bias: systematic difference, or jump Precision: random error, or wiggle Accuracy = total error, a combination of both Simple temp bath illustrations at: http://itep68.itep.nau.edu/itep_downloads/q A101_Resources/AllDownloadableMovies/ 22
DASC (Data Assessment Statistical Calculator) l http://itep68.itep.nau.edu/itep_download s/qa101_resources/dasc%20epa%20prec / %20and%20Bias%20Calculator/ AQS: Data Quality Indicators Report (AMP255) 23
Minimize bias by regular calibrations both accuracy (total error) and precision are first estimated by d i on an ongoing basis, by you Audits verify total error estimates Course website: http://itep68.itep.nau.edu/itep_downloads/q p// p p / p_ /Q A101_Resources/ Our emails: Bill Frietsche: fi frietsche.bill@epamail.epa.gov il Melinda.ronca-battista@nau.edu Christopher.lee@nau.edu 24
Appendix A to Part 58 Table A-2 of App A AQS TTN web site AQSP&A Spreadsheet Discoverer AMTIC web site AQS Helpline AQS AMP255 Data Quality Indicator Report
Appendix A to Part 58 Regulations that define QA reporting requirements for criteria pollutants Define assessments for each criteria pollutant 1 Pt QC check for gases Annual performance evaluation for gases Flow rate verification for particulate matter Semi-annual flow rate audit for particulate t matter Collocated sampling requirements for particulate matter Pb audit strips for laboratory analysis QA Performance Evaluation Program for PM fine, PM coarse, and Pb Formerly called precision and accuracy data still use these terms on AQS transactions
This document available at: http://itep68.itep.nau.edu/itep ep. ep_downloads/qa101oads/q _ Resources/session%204- o prec%20&%20bias/ Table A-2 of Appendix A to Part 58--Minimum Data Assessment Requirements for SLAMS Sites Method Assessment method Coverage Minimum Parameters Frequency reported Automated Methods Response check at 1-Point QC for SO2, NO2, O3, CO concentration 0.01-0.1 0.1 Once per 2 Audit concentration 1 and measured Each analyzer ppm SO2, NO2, O3, and 1- weeks.. concentration 2. 10 ppm CO Annual performance evaluation for SO2, See section 3.2.2 of this Audit concentration 1 and measured Each analyzer Once per year NO2, O3, CO appendix concentration 2 for each level. Flow rate verification PM10 Check of sampler flow rate Each sampler Once every Audit flow rate and measured flow rate month indicated by the sampler. Flow rate verification PM2.5, Once every Audit flow rate and measured flow rate Check of sampler flow rate Each sampler PM10-2.5 month indicated by the sampler. Check of sampler flow rate Semi-annual flow rate audit Once every 6 Audit flow rate and measured flow rate using independent Each sampler PM10, PM2.5, PM10-2.5. months indicated by the sampler. standard. Collocated sampling PM2.5, PM10-2.5 Performance evaluation program PM2.5, PM10-2.5 Collocated sampling PM10, TSP, PM10-2.5, PM2.5, Pb-TSP, Pb-PM10. Flow rate verification PM10 (low Vol), PM10-2.5, PM2.5, Pb- PM10 Flow rate verification PM10 (High-Vol), TSP, Pb-TSP Semi-annual flow rate audit PM10, TSP, PM10-2.5, PM2.5, Pb-TSP, Pb-PM10. PM10 Pb audit strips Pb-TSP, Pb-PM10 Performance Evaluation Program PM2.5, PM10-2.5 Performance Evaluation Program Pb- TSP, Pb-PM10 Collocated samplers. 15% Every 12 days Collocated samplers. Collocated samplers. 15% Check of sampler flow rate. Check of sampler flow rate. Check of sampler flow rate using independent standard. d Check of analytical system with Pb audit strips. Collocated samplers. Collocated samplers. 1) 5 valid audits for primary QA orgs, with <= 5 sites. 2) 8 valid audits for primary QA orgs, with > 5 sites. 3) All samplers in 6 years Each sampler Each sampler Each sampler, all locations. Analytical. 1) 5 valid audits for primary QA orgs, with <= 5 sites. 2) 8 valid audits for primary QA orgs, with > 5 sites. 3) All samplers in 6 years 1) 1 valid audit and 4 collocated samples for primary QA orgs, with <=5 sites. 2) 2 valid audits and 6 collocated samples for primary QA orgs with > 5 sites. Over all 4 quarters Manual Methods Every 12 days PSD--every 6 days. Once every month Once every quarter Once every 6 months. Each quarter. Over all 4 quarters Over all 4 quarters Primary sampler concentration and duplicate sampler concentration. Primary sampler concentration and performance evaluation sampler concentration. AQS Trans Type RP Reqd Reporting to AQS Y RA Y RP Y RP N RA Y RP Y RP Y Primary sampler concentration and 3 RP (or RD) Y duplicate sampler concentration. Audit flow rate and measured flow rate indicated by the sampler. Audit flow rate and measured flow rate indicated by the sampler. Audit flow rate and measured flow rate indicated by the sampler Actual mass and audit mass for parameters: 14129 - Pb (TSP) LC FRM/FEM 85129 - Pb (TSP) LC Non-FRM/FEM Primary sampler concentration and performance evaluation sampler concentration. Primary sampler concentration and performance evaluation sampler concentration. Primary sampler concentration and duplicate sampler concentration. 1 Effective concentration for open path analyzers. 2 Corrected concentration, if applicable, for open path analyzers. 3 Precision data is system generated when raw data is submitted for both collocated monitors for same date-time, and monitor ids are populated on monitor collocation record in AQS. Gaseous Parameter Codes Particulate Parameter Codes RP N RP N RA Y RA Y RP Y RP Y
AQS TTN web site http://www.epa.gov/ttn/airs/airsaqs/ AQSP&A Spreadsheet http://www.epa.gov/ttn/airs/airsaqs/padata/ AQS Discoverer http://www.epa.gov/ttn/airs/airsaqs/aqsdiscover/ AMTIC web site http://www.epa.gov/ttn/amtic/ AQS Help Line EPACALLCENTER@epa.gov 1-866-411-4EPA (4372)
Example run of AMP255 shown onscreen
All QC data helps EPA balance your costs of QC with needed information to protect health Questions to AQS are welcomed EPA committed to improving user friendliness Course website: http://itep68.itep.nau.edu/itep_downloads/q A101_Resources/ Our emails: Bill Frietsche: frietsche.bill@epamail.epa.gov epa Melinda.ronca-battista@nau.edu Christopher.lee@nau.edu 31