Edgewater Environmental Research And Education Center
Why have GREAT Quality Assurance and Quality Control? It all started with the Drug Industry. Drug industry had none, gave drugs to humans with out testing Classic case: Thalidomide Babies Thalidomide Babies Drug was made in 1957 for pregnant women for morning sickness, everywhere in the world. It was not until the 62 that Frances Kelsey, M.D. stood up to the Pharm company and told FDA not to approve to drug. Eventually it was tested and found to be teratogenic.
QAM: Quality Management Plans QAPP: Quality Assurance Project Plans SOP: Standard Operating Procedures QA/QC: Quality Assurance Quality Control CFR: Code of Federal Regulations NIST: National Institute of Standards and Technology http://www.nist.gov/environment-climate-portal.cfm ANSI America National Standards Institute ASQC: American Society of Quality Control
Course Objectives After this training, you will be able to: Discuss what information is required in a QA Project Plan Describe your role and responsibility in the QA Project Plan development, review, and approval process
What is meant by environmental data? Information that describes environmental processes, locations or conditions, and health effects or consequences. Collected directly from measurements (primary data) Produced from models. Compiled from other sources (existing or secondary data).
What is a Quality System? A structured and documented management system which has a system in place for ensuring the quality of its work process, products and services. The QMP describes the policies, objectives, principles, organizational authority, responsibilities, accountability, and implementation plan of the organization
Sources: EPA Order 5360.1 A2 (the Order) EPA Order 5360 A1(the Quality Manual) ANSI/ASQC E-4-1994 Specifications and Guidelines for Quality Systems for Environmental Data Collection and Environmental Technology Programs
Regulations QA Documentation 48 CFR 46 Quality Management Plan (or equivalent) and a QA Project Plan (or equivalent) 40 CFR 30 Quality Management Plan and a QA Project Plan 40 CFR 31 Quality System must conform to the American National Standard (ANSI/ASQC-E-4, 1994). Submit a Quality Manual and a QA Project Plan 40 CFR 35 Comply with 40 CFR 31
What is a Quality Assurance Project Plan? A document that describes the technical and quality activities of an environmental data operations project that should be implemented to ensure that the results of the work performed will satisfy the data user s needs.
Why should I write a QA Project Plan? QA Project Plans are required for all EPA projects and tasks involving environmental data operations (generating or using). Non-EPA organizations wanting to use legally defensible data for litigation should write a QAPP. Organization that write and use a QAPP are taken seriously by governmental organizations.
QA Project Planning and QA Project Plan Preparation 1) What is your Statement of Work, i.e- what do you want to do? 2) Meet with your staff/volunteers and plan it out. 3) Example: you want to perform regular water quality monitoring 1) Why, where, parameters, ect
Project/QA Project Plan Development Scope Assemble a team Systematically plan Write the Plan Submit the Plan for review and approval Once approved, distribute the plan Start work, following the plan Revise when needed, get approval, then re-distribute the plan
Systematic Planning (e.g., DQO Process) Conduct Study/ Experiment Data Verification & Validation QA Project Plan Standard Operating Procedures Technical Assessments Data Quality Assessment Planning Implementation Assessment
Four Main Topics: A. Project Management 9 elements B. Data Generation/Acquisition 10 elements C. Assessment and Oversight 2 elements D. Data Validation and Usability
References Existing Documentation can be Referenced and/or Attached! qapp 15
Different Types of QA Project Plans Task-Specific QA Project Plans: those projects for which the environmental data operation is task, site or project specific. Generic or Programmatic QA Project Plans: those projects for environmental data operations conducted at multiple locations and/or over a long period of time. qapp 16
Level of Detail - the Graded Approach The level of detail in each QA Project Plan will vary according to the nature of the work being performed and the intended use of the data. high road low road qapp 17
QA Project Plan Review For EPA projects, the Project Manager and QA Manager, or designee, review the QA Project Plan. For extramural projects, the funded organization reviews the plan, as well as the EPA organization. qapp 18
Review to ensure that: The plan is accurate and complete, Sufficient data will be collected using appropriate measurement and data acquisition methods, Planned assessment procedures will be adequate to evaluate the project, and There is a process to identify any limitations on the data use. qapp 19
Reviewers should: Be timely in their review, Be consistent in their review, Have a process established to deal with revisions, and Have expertise in technical areas and recognize where they need advice. qapp 20
QA Project Plan Approval Is the responsibility of the EPA Project Manager and QA Manager, and Indicates that the QA Project Plan is accurate and complete, and that it provides sufficient detail to identify the project's technical and quality objectives. qapp 21
Conditional Approval May be given in situations where the project may be started while "non-critical deficiencies" are being resolved. qapp 22
QA Project Plan Implementation Data collection may not be started until the QA Project Plan has been reviewed, approved and signed. The QA Project Plan should be implemented as approved. qapp 23
Revisions - After Approval When changes are needed, the QA Project Plan should be revised and re-approved by the approving official(s) prior to the change being made. qapp 24
Regular Reviews QA Project Plans for multi-year projects should be reviewed annually and revised when necessary. qapp 25
A. Project Management qapp 26
1. Title & Approval Sheet qapp 27
2. Table of Contents qapp 28
Document Control Notation Project #/Name Revision No. Date Page of qapp 29
3. Distribution List List key personnel of all organizations who need a copy of the approved QA Project Plan and revisions qapp 30
4. Project/Task Organization Identify individuals and state their roles and responsibilities in the project Attach an organizational chart and indicate lines of communication (include data users and subcontractors) qapp 31
5. Problem Definition/Background Describe the underlying issue that led to this project, including background/ historical information indicating the need for this study. qapp 32
6. Project/Task Description Summarize work to be performed, products to be produced, and the schedule for implementation qapp 33
Resource Limitations Time People Materials qapp 34
7. Quality Objectives and Criteria State the quality objectives for the project and the performance criteria to achieve these objectives, i.e., the quality of data needed to support your decision - Decision quality is based on data/ information quality qapp 35
Data Quality Indicators Precision Accuracy/Bias Representativeness Comparability Completeness Sensitivity qapp 36
qapp 37
Representativeness Comparability Completeness qapp 38
Impact of Detection Limit and Contaminant Concentration Relative Positions of MDLs and CC Consequence MDL CC Non-Detects and Detects Useable MDL CC Possibility of False Positives and False Negatives CC MDL Non-Detects Not Useable Detects Useable Possibility of False Negatives lower concentration higher qapp 39
Detection Limits/Sensitivity C O N C E N T R A T I O N ------------------------------------------------ Action Level? - - - - - - - - - - - - - - - - - - - - - - - - - - - - MDL ------------------------------------------------- Action Level? 0 SAMPLES qapp 40
8. Special Training/Certification Identify any special training or certifications needed by personnel to conduct project activities State how this training will be provided, assured and documented qapp 41
9. Documents and Records Describe process to ensure project personnel have most current Plan Itemize information/records to be included in data report package Identify any other reports/documents to be produced State specifications for final disposition of records and documents qapp 42
B. Data Generation and Acquisition qapp 43
1. Sampling Process Design (Experimental Design) Describe the data generation or data collection design for the project: measurement parameters of interest, indicating those critical and those secondary to the study qapp 44
1. Sampling Process Design (Experimental Design) Describe the data generation or data collection design for the project: measurement parameters of interest, rationale for the sampling design qapp 45
Sampling Design Testing & Estimation - is the sampling matrix homogenous or heterogeneous? - do you care about average contamination levels, hot spots, or the proportion of site contaminated? - do you want composites samples or grabs? - are you looking for trends over time? - will there be a reference site? qapp 46
1. Sampling Process Design (Experimental Design) Describe the data generation or data collection design for the project: measurement parameters of interest, rationale for the sampling design, sampling locations and frequencies, types and numbers of samples needed, and sources of variability qapp 47
2. Sampling Methods Describe sample collection and preservation methods, and equipment preparation and decontamination procedures Indicate any performance specifications Identify sample equipment and support facilities needed Address what to do when things go wrong. qapp 48
3. Sample Handling and Custody Describe how samples are to be handled in the field, in the lab, and during transport, and who is responsible Detail how samples are to be transported so maximum holding time conditions are met Indicate sample identification scheme and chain of custody specifications qapp 49
4. Analytical Methods Identify all analytical methods and equipment needed for both field and lab List method performance standards, such as MDL, Precision, Accuracy, PQL, % recovery qapp 50
4. Analytical Methods (continued) Include method validation information for nonstandard methods State what to do when a failure occurs Describe sample disposal procedures Specify laboratory turnaround time qapp 51
5. Quality Control Identify QC activities for each sampling, analysis, or measurement technique State the frequency and control limits of each QC activity, and how corrective actions will be documented Describe procedures and formulas for calculating QC statistics qapp 52
Examples of Quality Control Samples Replicates Splits Blanks Reference Materials Matrix Spikes qapp 53
6. Instrument/Equipment Testing, Inspection, and Maintenance Identify equipment requiring periodic maintenance, the schedule for such, and availability of critical spare parts Describe inspection and acceptance testing to be performed, and how documented Identify how deficiencies will be resolved, and effectiveness determined and documented Identify who will do it qapp 54
7. Instrument/Equipment Calibration and Frequency Identify all equipment and instrumentation requiring calibration in field or lab Document calibration method and frequency, and test criteria and certified equipment and/or standards to be used Identify how deficiencies will be resolved Indicate how records will be maintained and traceable to the instrument qapp 55
8. Inspection/Acceptance for Supplies and Consumables Identify all supplies that may directly or indirectly affect the quality of the project Describe the inspection and acceptance testing criteria and procedures qapp 56
9. Non-Direct Measurements Identify data needed from computer databases, programs, literature files, and historical databases Describe the intended use of the data Cite acceptance criteria, and any limitations, for their usage qapp 57
10. Data Management Describe data management scheme from generation in the field to final use/storage Discuss procedures for record-keeping, document control system, and data storage/ retrieval on electronic media Discuss methods and equipment for detecting/correcting errors, and preventing data loss Identify who s responsible qapp 58
10. Data Management (continued) Describe procedures to demonstrate acceptability of hardware/software configuration Identify how applicable information resource management specifications are met (for EPA personnel) qapp 59
C. Assessment and Oversight qapp 60
Assessment Types Readiness Reviews Technical System Audits Surveillance Performance Evaluations/Proficiency Testing qapp 61
1. Assessments and Response Actions Describe each assessment needed, including frequency and type, information expected, and criteria for success Identify potential assessors and their authority Discuss how and to who findings will be qapp 62
2. Reports to Management Identify frequency and distribution of project QA status reports Identify preparer and recipients of these reports qapp 63
D. Data Validation and Usability qapp 64
Questions & Answers What is the difference between data review, verification, and validation? qapp 65
1. Data Review, Verification, and Validation Identify the project's criteria to review, verify, and validate the data qapp 66
2. Verification and Validation Methods Describe the process to be used for both data verification and validation, identifying any calculations needed Discuss how issues will be resolved and identify the individuals responsible Discuss how these results will be conveyed to the data users qapp 67
3. Reconciliation with User Requirements Describe how the results of the study will be evaluated to determine whether the project's objectives have been satisfied Outline proposed methods to analyze the data Describe how limitations in data use will be reported qapp 68
Summary qapp 69
Important Criteria to Consider for Quality Assured Data Anything that compromises data representativeness compromises data quality. Make room for independent checks along the way, and use them. Use methods that have been well established and/or well documented. qapp 70
Important Criteria to Consider for Quality Assured Data (continued) Most important: figure out what you need to know, and how well you need to know it. Then, plan how to get it, including all of the participants in the planning process. qapp 71
Remember There are four basic project groups that should be discussed in a QA Project Plan. These are: - management - data generation/acquisition - assessment/oversight - data validation and usability qapp 72
What is the value of having an approved QA Project Plan? qapp 73
Doing the Right Things the Right Way the First Time! qapp 74