Boston Government Services LLC

Transcription

1 G Boston Government Services LLC CORRECTIVE ACTION Revision 2 BGS Proprietary Information Notice This document and the information it contains is property of Boston Government Services, LLC (BGS). It shall not be reproduced, or its content otherwise made available to non-bgs parties without the express written consent of Boston Government Services, LLC. Prepared By: Approved By: Date Effective Date 3/31/16

2 Page 2 of 31 Revision History Rev. # Date By Type 1 Changes 0 T. Graben- Galyon N Original issue 0, mc 1 H. Boston mc Corrected typographical and formatting errors 1 R. Lambert M 2 2/19/16 R. Lambert M Revised to: - Address DOE EM Audit Team comments - Incorporate BGS-QA-20, Causal Analysis Updated to latest style template. Updated procedure references. Procedure has been revised to address CARs a corrective action (CA) #2 and CA #4. 1 M = major change, mc = minor change, N = new

3 Page 3 of 31 Table of Contents 1. PURPOSE SCOPE REFERENCES DISCUSSIONS AND OVERVIEW DEFINITIONS AND ACRONYMS RESPONSIBILITIES PROCEDURE RECORDS ATTACHMENTS...16 ATTACHMENT 1 Report Form, BGS-QF-16.1 EXAMPLE...17 ATTACHMENT 2 Issue Priority Level...20 ATTACHMENT 3 Guidelines for Developing Actions...22 ATTACHMENT 4 Extent of Condition (EOC) Evaluation...23 ATTACHMENT 5 Root Cause Analysis Process...25 ATTACHMENT 6 Apparent Cause Analysis Process...28 ATTACHMENT 7 Review and Approval of s Reports or Plans...29 ATTACHMENT 8 Effectiveness Review...30

4 Page 4 of PURPOSE This procedure outlines the process to identify and investigate issues and to develop, execute, and verify corrective actions put in place to preclude recurrence of issues. 2. SCOPE This procedure applies to activities performed or procured under Boston Government Services, LLC s (BGS) Quality Assurance (QA) Program. 3. REFERENCES 3.1. BGS-QAP-01, BGS Corporate Quality Assurance Plan (QAP) 3.2. BGS-MG-02, Continuous Improvements (formerly BGS-QA-14) 3.3. BGS-MG-03, Management Assessments (formerly BGS-QA-19) 3.4. BGS-QA-15, Nonconformances 3.5. BGS-QA-25, Independent Assessments 3.6. BGS-QA-26, Surveillance 3.7. BGS-RM-01, Quality Assurance Records (formerly BGS-QA-17) 3.8. DOE G , Guide 4. DISCUSSIONS AND OVERVIEW 4.1. The process detailed in this procedure is a critical component of BGS continuous improvement program This procedure is utilized by BGS management to identify, address, and correct issues ranging from audit/assessment findings and employee concerns to opportunities for improvement This procedure details a process to determine the significance of identified issues with the highest level of effort focused on significant conditions adverse to quality (SQAC) This corrective action programs serves to ensure identified issues are promptly corrected and actions are put in place to preclude recurrence. 5. DEFINITIONS AND ACRONYMS Apparent Cause The most probable cause(s) that explains why the event happened, that can reasonably be identified, that BGS management has the control to fix, and for which effective

5 Page 5 of 31 recommendations for corrective action(s) to remedy the problem can be generated, if necessary. Apparent Cause Analyst (ACA) Individual trained and assigned to guide the conduct of an apparent cause analysis. Assessment A review, evaluation, inspection, test, check, surveillance, or audit to determine and document whether items, processes, systems, or services meet specified requirements and perform effectively. (DOE O 414.1D) Audit A planned and documented activity performed to determine by investigation, examination, or evaluation of objective evidence the adequacy of and compliance with established procedures, instructions, drawings, and other applicable documents, and the effectiveness of implementation. (NQA ) Compensatory Actions Actions that are intended to offset the effects of an identified error or process defect prior to identifying the cause of the issue or implementing the preventive action. Compensatory actions are used to allow the overall process to continue until actions to correct the issue are completed. Condition Adverse to Quality (CAQ) An all-inclusive term used when a requirement has not been met and in reference to any of the following: failures, malfunctions, deficiencies, defective items, and non-conformances. Assessment/Audit findings identify CAQs. (NQA ) Contributing Cause Cause that made the event worse (exacerbated) or reduced the impact of the cause (mitigated). Causes that made the event worse need to be addressed as they represent an unwanted condition. Measures taken to rectify conditions adverse to quality and, where necessary, to preclude repetition. (NQA ) Plan (CAP) A plan that addresses collective actions, closure requirements, assignees, and schedule dates for resolving issues identified in internal or external audits/assessments.

6 Page 6 of 31 Report (CAR) A report that summarize the collective action(s), closure requirements, assignee(s), and schedule date(s) for resolving an issue. Effectiveness Review A review performed to determine if corrective actions have been effectively implemented to prevent recurrence. Extent of Condition (EOC) A review to determine how pervasive a problem is. This may include determining if the same problem exists in other equipment, organizations, systems, or programs than where it was initially identified. Finding An individual condition not meeting a committed requirement (e.g., contract, regulation, safety basis, QA program, authorization basis document, or procedure). Functional Manager (FM) Immediate Actions Is a person who has management authority over an organizational unit, such as department, within BGS. BGS identifies and assigns individuals to functional areas (e.g.; Accounting, Quality Assurance, Information Management, Business Development, etc.). Actions taken in a timely manner to isolate and control the issue. Actions implemented as soon as possible after identification for the purpose of mitigating or terminating the consequences. IR Independent Reviewer Issue Term used to refer to events, findings, adverse conditions, recommendations, suggestions, and opportunities for improvement. For example, any situation that may warrant management attention. Issue Owner (IO) Term to designate the BGS Personnel assigned responsibility to oversee evaluation of an issue(s), to plan corrective actions, and to bring the issue to closure. QA Quality Assurance

7 Page 7 of 31 QAP Quality Assurance Plan Preventive Action Measures taken to preclude repetition. For example, actions designed to prevent or reduce the likelihood of recurrence. Priority Level (P) Project Manager (PM) RCA System for classifying the significance of an issue for the purpose of establishing the appropriate level of investigation, analysis, and review of the issue. A person assigned to manage resources and activities to deliver products or services and is responsible for project performance (cost, schedule, and delivery of quality products) Root Cause Analysis Remedial Actions Actions taken to correct a specifically identified issue or to return a work product to a compliant state. Root Cause The base cause(s) of a condition or event leading to a deficiency that, when corrected, would prevent the recurrence of the problem. Significant Condition Adverse to Quality (SCAQ) A condition adverse to quality that, if left uncorrected, could have a serious effect on safety, operability, or credibility of BGS work. (NQA ). Subject Matter Expert (SME) An individual with demonstrated experience and documented knowledge of a program area or technical discipline. 6. RESPONSIBILITIES 6.1. BGS President Ensures the requirements of this procedure are established and resources are provided to ensure implementation Functional Managers (FM) FMs encourage others to identify and report near misses or actual events, adverse conditions, deficiencies, recommendations, suggestions, and opportunities for improvement using the Report (CAR) process at the earliest possible time before the issue becomes more significant.

8 Page 8 of The FM designates an issue owner (IO) for each identified issue In collaboration with the QA Manager, determines the significance of an issue and the level of effort of the corrective action Periodically reviews closed issues under their area of responsibility to determine the need for effectiveness reviews and coordinates such reviews, as necessary Provide timely and frequent feedback of the issue progress to the BGS Personnel member who identified the issue Project Managers (PM) PMs encourage others to identify and report near misses or actual events, adverse conditions, deficiencies, recommendations, suggestions, and opportunities for improvement using the CAR process at the earliest possible time before the issue becomes more significant The PM designates an issue owner (IO) for each identified issue In collaboration with the QA Manager, determines the significance of an issue and the level of effort of the corrective action Periodically reviews closed issues under their area of responsibility to determine the need for effectiveness reviews and coordinates such reviews, as necessary Provide timely and frequent feedback of the issue progress to the BGS Personnel member who identified the issue Issue Owner (IO) The IO is responsible for the preparation and generation of individual corrective action report(s) Identifies necessary immediate actions, conducts fact finding, investigation, and gathering of pertinent information related to the identified issue Establishes and implements corrective actions to prevent issue recurrence Quality Assurance Manager The QA Manager is responsible for development and maintenance of this procedure Ensures that the reported issues are tracked through the corrective action process and verifies closure of the CAR or CAP.

9 Page 9 of In collaboration with the FM/PM, determines the significance of an issue and the level of effort of the corrective action Maintains the CAR tracking Log Trends and report status of CARs on a regular basis to BGS Management BGS Personnel All BGS Personnel are responsible for identifying and processing issues, concerns, opportunities for improvement, and audit/assessment findings in accordance with this procedure. 7. PROCEDURE 7.1. Identifying Issues BGS staff uses the following methods to identify conditions adverse to quality (CAQ), issues or concerns: a) Nonconforming conditions identified and documented through the Nonconformance (NCR) process (BGS-QA-15, Nonconformance). b) Findings and observations identified and documented through BGS internal management assessment process (BGS-MG-03, Management Assessment). c) Findings and observations identified and documented through BGS Independent Assessment (internal and external) process (BGS-QA-25, Independent Assessment). d) Findings and observations identified and documented through BGS Surveillance process (BGS-QA-26, Surveillance). e) External assessments and audits. f) Management concerns. g) Opportunities for Improvements. h) Employee concern. i) Customer concerns/complaints Individual identifying an issue initiates a CAR by providing the following information by electronic mail to the Quality Manager, FM and/or PM: a) Date of Occurrence

10 Page 10 of 31 b) Description of condition or issue c) As applicable Reference Number (NCR, Assessment # or Finding #, etc.) Recommended corrective action Outside of BGS formal processes (assessments, NCR, etc.) examples of issues adverse to quality that result in the generation of a CAR are a) Deficiencies requiring stopping work. b) Deficiencies impacting worker s safety. c) Deficiencies left uncorrected could impact the environment or be in noncompliance with environmental regulations. d) Repetitive deficiencies. e) Deficiencies in which previous corrective action has been ineffective Issues adverse to quality, once identified, must be promptly brought to the attention of both the QA Manager and appropriate FM/PM Report Generation The FM or PM reviews, concurs with the issue and assigns an IO. Note: The FM/PM ensures that the individual selected as the IO possess technical expertise in the area being evaluated and is qualified to conduct analytical techniques for determining causal factors involved in the issue review and conducting a root cause analysis. If it is concluded that additional expertise in a technical area is required, the FM/PM can arrange for availability of that expertise The IO performs the following: a) Identifies necessary interim (immediate and compensatory) actions to correct the specific deficiencies. b) Enters issue information onto the corrective action form BGS-QAF-16.1, Report (Attachment 1) Part I, per form instructions. c) Performs additional research on the issue and provides further details on the CAR, as appropriate. d) Obtains a CAR number from the QA Manager. e) Forwards the CAR to the QA Manager for issue significance determination.

11 Page 11 of Issue Significance Determination The FM/PM and the QA Manager determine the issue priority Level (P1 P3) using Table 1 and guidance provided in Attachment 2, Issue Priority Level, and communicate this decision and associated actions to the IO. Issue Priority Level (P) BGS Internal Finding Level (Note 2 ) Table 1 Issue Priority Level System Requirement Summary CAQ or SCAQ Causal Analysis Corrective Action Effectiveness Review Extent of Condition Independently Verified P1 F1 SCAQ Root Cause Yes Yes Yes Yes P2 F2 CAQ Note 3 Yes Note 4 No Yes P3 Note 5 n/a No Yes No No No The IO updates the CAR form Part II, per form instructions Plan At the discretion of the QA Manager, external audit/assessment findings can be addressed and tracked through a CAP At the discretion of the QA Manager, multiple CARs can be assembled and addressed through a single CAP generated by the QA Manager, or designee Tracking The DCA or QA Manager maintains a CAR log. The log identifies as a minimum the following: a) CAR Number and Title b) IO and FM c) Date Identified and Date closed 2 See BGS-MG-03, Management Assessments, for definition of Finding Levels. 3 Apparent Cause Analysis is performed on a case by case basis (see section d) 4 An Effectiveness Review is performed on a case by case basis (see section ) 5 P3 level issues are assessment observations, opportunity for improvement, customer complaints, and employee concerns.

12 Page 12 of 31 d) Priority level, casual code The QA Manager tracks and maintains the status of all CARs and CAPs from submittal to closure The FM/PM or the QA Manager provides timely and frequent feedback of the issue progress to the BGS Personnel who identified the issue Issue Evaluation and Development The IO reviews all information to determine the facts of what actually occurred. Note: These facts are to be continuously validated and analyzed individually and together for relevance and accuracy. Any missing information or inconsistencies identified during analysis of the issue must be reviewed and followed up The IO ensures that information and physical evidence collected is verified, documented, and preserved Solicit input from both the individual who identified the issue and, as applicable, the organization management responsible for the activity where the issue was identified Based on the Issue Priority Level the IO, with support from the FM/PM and QA Manager as noted below, takes the following actions to address the issue as summarized in Table 1: a) For P1 and P2 issues, identify additional immediate and remedial actions to correct the specific deficiencies identified (see Attachment 3, Guidelines for Developing Actions). b) For a P1 issue, perform an extent of condition evaluation (see Attachment 4, Extent of Condition Evaluation for Guidance). c) For a P1 issue, perform a Root Cause Analysis (RCA) of the issue (see Attachment 5, Root Cause Analysis Process). d) For a P2 issue, the IO reviews the issue with the responsible FM/PM to determine if an Apparent Cause Analysis (ACA) needs to be performed. In determining when an ACA is necessary for a P2 issue, the responsible IO and FM/PM considers the following: The complexity of the P2 issue and significance of the corrective actions. Significance and impact of the P2 issue to the site/organization safety, mission operations, and security. Audits/assessments containing P2 issues with clear programmatic implications must include the

13 Page 13 of 31 performance of an ACA. e) If an ACA is to be performed complete per Attachment 6, Apparent Cause Analysis Process. f) For P1, P2, and P3 issues, develop corrective actions to correct the issue to prevent further occurrences (see Attachment 3, Guidelines for Developing Actions). g) The IO documents the actions needed and identifies a planned completion date for each actions. The IO updates the CAR Form Part III, per Form instructions. h) For P1 issue(s) the IO submits the corrective actions to the QA Manager prior to approval. The QA Manager performs and documents an evaluation of the proposed corrective actions to ensure the scope and range of the corrective actions will effectively address the underlining QA performance issue(s). This evaluation shall be documented and attached to the CAR. i) Update the CAR form; obtain FM/PM and QA Manager signature per Form instructions. Note: To maintain the independent QA review function, when the CAR is assigned to the QA functional area, the BGS President will perform the independent review and approval function, noted in this procedure, and sign for the QA Manager. j) IO distributes an updated copy of the CAR to the FM/PM and QA Managers The QA Manager updates the CAR Log as appropriate The IO, with management support, executes the corrective action and updates the CAR Form Part IV, per Form instructions The IO submits to the FM/PM and the QA Manager completed corrective action and analysis documentation supporting closure of the issues Issue Verification and Closure All corrective actions addressing P1 and P2 issues must be independently verified The QA Manager performs the independent verification or identifies an independent reviewer (IR) to verify completion of corrective actions when the IO has completed all corrective actions. Note: If the QA Manager is the IO or responsible FM for the issue, the BGS President will perform the independent verification or identifies an independent

14 Page 14 of 31 review to verify completion of corrective actions when the IO has completed all corrective actions If the QA Manager is the IO or responsible FM for the issue, the BGS President will perform as or identify an IR to verify completion of corrective actions The QA Manager or IR determines if the corrective actions for an issue were complete and effective in meeting the intent of the corrective action. The degree of closure verification required must be based on the severity of the deficiency, frequency of recurrence, and professional judgment. (See Attachment 7, Review and Approval of s.) If the corrective action(s) is determined sufficient, the QA Manager or IR approves and closes the corrective action If the corrective action(s) is determined not to be sufficient, the QA Manager or IR notifies the FM/PM and IO The IO corrects the deficiencies The steps above are repeated until all corrective actions for P1 and P2 issues have been verified and closed After successfully verifying actions for issue(s) the QA Manager issues notification to the FM/PM and IO that the CAR is closed. An is acceptable notification. If verification of effective completion of corrective action was documented in subsequent audit/assessment reports, separate correspondence is not required Effectiveness Review Corrective action effectiveness reviews will be conducted for CARs containing P1 issues, and some with P2 issues The responsible FM/PM ensures a CAR with P1 issues are subjected to effectiveness reviews For a CAR with a P2 issue, the responsible FM/PM reviews the CAR to determine if an effectiveness review needs to be performed. The FM/PM considers the following when determining if a effectiveness review is necessary: a) Number and complexity of issues and associated corrective actions in the CAR. b) Significance and impact of each issue and associated corrective actions to organization safety, mission, and security. A P2 issue with clear programmatic implications could be subject to effectiveness reviews.

15 Page 15 of Corrective action effectiveness reviews are conducted using the guidance of DOE G , Guide (sections 7.1 through 7.9, as appropriate) and Attachment G, Effectiveness Review. A corrective action effectiveness review can be conducted by an individual or by a team, but it must be conducted by assessment personnel using an approved plan. Therefore, it can be incorporated into an assessment (Internal and External) of a particular topic or facility that is performed after corrective actions from the previous assessment are completed Effectiveness review activities include one or more of the following as described in DOE G : a) Document reviews b) Performance indicators c) Work observations d) Facility tours e) Performance testing f) Personnel interviews The responsible FM/PM schedules effectiveness reviews after enough time has passed for the original problem to recur, or the corrective actions have been challenged by events. If conditions for implementing the corrective actions can only occur under certain circumstances (e.g., software verification processes) then the effectiveness review must be conducted to address effectiveness of the corrective actions under those circumstances Persons conducting effectiveness reviews document them in assessment reports per BGS-MG-03, Management Assessments Report Trending 8. RECORDS The QA Manager provides BGS Senior Managers with CAR trend data (Open, Closed, etc.) on a quarterly basis The QA Manager analyzes the information in the CAR log and reports to identify recurring conditions and potential adverse trends. This information is communicated to Senior Management via the Annual Trend Analysis Report noted in BGS-MG-02, Continuous Improvement Records generated during corrective action process are quality assurance records and will be maintained per BGS-RM-01, Quality Assurance Records. The Form and

16 Page 16 of 31 associated corrective action closure documentation detailed in this procedure is a QA Record Form BGS-QAF-16.1, Report, and attachments. 9. ATTACHMENTS 9.1. Attachment 1 Report Form, Example BGS-QAF Attachment 2 Issue Priority Levels 9.3. Attachment 3 Guidelines for Developing Actions 9.4. Attachment 4 Extent of Condition Evaluation 9.5. Attachment 5 Root Cause Analysis Process 9.6. Attachment 6 Apparent Cause Analysis Process 9.7. Attachment 7 Review and Approval of s Reports or Plans 9.8. Attachment 8 Effectiveness Review

17 Page 17 of 31 ATTACHMENT 1 Report Form, BGS-QF-16.1 EXAMPLE

18 Page 18 of 31 Report Form, BGS-QF-16.1 EXAMPLE (CONTINUED)

19 Page 19 of 31 Report Form, BGS-QF-16.1 EXAMPLE (CONTINUED)

20 Page 20 of 31 ATTACHMENT 2 Issue Priority Level The priority level system is designed to assist BGS Management and Personnel in determining the relative importance of issues based upon the professional judgment of the issue originator and with BGS management concurrence. The following criteria are guidelines for establishing priority levels: Priority Level 1 - An issue identifying a major event or systemic breakdown in safety, quality, or Integrated Safety Management. The adverse condition identified in the issue has impact on quality, worker health or safety, the public, the environment, facility operations, or regulatory compliance. P1 issue examples include conditions that could or do result in consequences such as a. Severe injury or death, permanent disability, and/or chronic or irreversible illness. b. Issuance of a stop work order. c. Delay of mission accomplishment for a period of four weeks or longer. d. A release to the environment exceeding reportable quantities under the Comprehensive Environmental Response, Compensation, and Liability Act. Priority Level 2 An issue requiring senior management attention to correct. The condition identified in the issue is a non-compliance with a requirement that could affect quality, worker health or safety, the public, the environment, or regulatory compliance. Management attention is required for activities such as procedure revisions, training, etc. P2 issues examples include conditions that could or do result in consequences such as a. Non-compliances with procedures or requirements affecting safety, quality, or environmental compliance that requires management attention to properly correct. If left uncorrected, the condition could result in defective work, personnel injury, adverse personnel health effects, a non-compliance to an environmental or other regulation, or damage to the reputation of Boston Government Services. b. Delay of mission accomplishment for a period of up to four weeks. c. Issues identified during external audits/assessment and inspections that do not meet the criteria for Priority Level 1. Priority Level 3 A minor issue identifying non-compliance with a procedure or requirement, in a process, program, system, or management structure. A minor issue is an isolated occurrence (one or two instances) with no impact on worker health and safety, the public, the environment, or regulatory compliance, and requires only remedial action. However, if left uncorrected, the condition could deteriorate into a condition meeting the criteria for a Priority Level 2 issue. Examples include: a. Non-compliances with procedures or requirements that do not require management

21 Page 21 of 31 ATTACHMENT 2 Issue Priority Level (CONTINUED) attention to properly correct and cannot lead directly to defective work, personnel injury, adverse personnel health effects, a reportable non-compliance to an environmental or other regulation. b. An issue characterized below the levels of P1 and P2 (guidance described above) that generally results from inspection of facilities, responses from responsible personnel, and review of pertinent records. P3 issues can also include employees or customer concerns and opportunities for improvement.

22 Page 22 of 31 ATTACHMENT 3 Guidelines for Developing Actions Types of Actions Interim (immediate/compensatory) Remedial Preventive Effectiveness review Developing Actions Developing actions includes the contribution, collaboration, and agreement of the personnel responsible for performing the actions. The actions meet the SMART criteria; that is, they are: Specific clear, concise, and in sufficient detail to allow personnel directly and indirectly involved to understand the activities to be conducted Measurable activities or mechanism available to verify completion and determine effectiveness of the completed actions Accountable specific responsibility for completing action is identified Relevant address the cause; reasonable and achievable within the ability of the organization to develop and implement Timely scheduled to be performed within a time period that corrects the identified issue before it worsens. Developed actions should determine (as appropriate): Appropriate action(s) is specified for each cause or there is an evaluation that no action is necessary. The actions are closeable and verifiable. The actions have been prioritized with consideration of the risk significance and regulatory compliance. A timeframe has been established for implementing and completing the actions. Quantitative or qualitative measures of success have been developed for determining the effectiveness of the actions to prevent or reduce the likelihood of recurrence (effectiveness review criteria). The state following the corrective actions is sustainable. For example, the activity performed consistently by all personnel in the future. This may require a process change to institute the action. The actions are necessary and unintended consequences have been considered. Words or phrases such as Determine why/what, Evaluate, Counsel, or Review are more closely associated with determining cause or extent of condition and indicate that the evaluation and development phase is not complete.

23 Page 23 of 31 ATTACHMENT 4 Extent of Condition (EOC) Evaluation An EOC evaluation contributes to more accurate identification of the underlying issue. A properly scoped, implemented, and documented EOC evaluation can help identify and correct problems before the problems become events. The EOC evaluation can save project resources and create a safer, better managed work environment. An EOC focuses on the actual condition and its existence in other places. The EOC evaluations are used to identify the broader implications of an issue the extent that is not found today tends to be found tomorrow in more expensive ways that raise questions about the organization s commitment to finding its own problems promptly. An EOC answers the question, What are the broader ramifications of the behavior or condition we are dealing with? Or, given what you know about this occurrence, what else would you expect to see? An EOC evaluation is generally performed after the issue has been through a degree of cause evaluation, but it may have some interdependency with the evaluation, resulting in changes to the cause factors. The EOC evaluation is documented as part of the evaluation: 1. Determine the extent of the condition/cause. 2. Enter the evaluation results or the applicable documentation number (e.g., root cause analysis number) into the extent field of the deficiency. 3. Explain how the results of the EOC will be addressed. 4. Clearly indicate that the entry is EOC (e.g., state Extent of Condition - After review of other ventilation systems, it was determined that this pressure transmitter is only found in the plant fire service water systems. ) Key questions to consider include Have I seen this before either at my work location or at other sites, from prior experience? If I am seeing it again, what are the factors resulting in the repetition? Is the management system deficient in some way since this circumstance occurred? How? Could other activities and facilities at the site be experiencing the same problem? To what extent does this problem have an impact or potential impact on the project or activity? Can this matter affect the ability of the project to conduct work safely and in compliance with requirements? Potential guidance regarding performance of EOC evaluations includes Treat EOC commensurate with the circumstances of the issue(s) being investigated.

24 Page 24 of 31 Extent of Condition Evaluation (CONTINUED) The issue owners determine the approach to EOC for their investigations in accordance with the requirements provided in this procedure. The description of the approach to EOC is incorporated into the CAR. The approach to EOC is based on the consequences and the factors of the cause evaluation. The results of the EOC investigation are reported and acted upon commensurate with their significance. Some questions for consideration in planning EOC evaluations are What are the adverse consequences of the event that prompted the EOC evaluation? Should we do more investigation of the causation of the already known extents of those consequences? Should we pursue the EOC of all of the important adverse conditions manifested by the investigation, or is it acceptable to restrict the EOC evaluation to those conditions that are included in the consequences? Should we pursue the EOC of all of the important cause factors manifested by the investigation, or is it acceptable to restrict the EOC evaluation to those cause factors designated as root causes? When we pursue the extent of the cause, do we consider all cause factors (both behaviors and conditions) or just conditions? When an adverse item is turned up in an EOC evaluation, should we look for its effects? When an adverse item is turned up in an EOC evaluation, should we look for the factors that resulted in it? Should we pursue indications of previously existing EOC or should we pursue only currently existing items? Should we pursue circumstances that would lead to future occurrences? To what extent should lessons learned be considered?

25 Page 25 of 31 ATTACHMENT 5 Root Cause Analysis Process The priority level defines the beginning level of evaluation and reporting requirements for the issue. For Priority Level (P) 1 issues, a root cause analysis (RCA) is performed. This is a methodical process to gather facts and applicable information concerning the issue or event and applying tools and techniques to organize and evaluate the information to determine the root or underlying cause(s). An RCA is best performed by a small dedicated team of three to five people. The team must have an individual qualified as an RCA team lead and generally includes appropriate SME. It is important to the success of the team to have a dedicated location to perform the analysis. It is important that management gives proper direction to the team in the form of a charter, especially if the team is being asked to cross organizational or project boundaries in the analysis. This direction includes the schedule for conducting a RCA and the problem statement that they will be evaluating, as well as any other direction. The development of the problem statement is key to a successful RCA. Typically, a RCA team should be chartered as soon as a P1 issue is identified. Often the team can begin to gather data within two to three days after a P1 issue is identified. The data/information gathering and evaluation typically will take up to two weeks of dedicated effort with another week for report writing, corrective action development, and management alignment. The RCA report is a defined work product that is often subject to intense internal and external review and scrutiny. As such, it should be closely reviewed by the RCA team and by BGS Management. The RCA reports are uploaded or attached to the associated CAR or issue in a CAP. The root cause report should include the following: Title Sheet The title sheet should provide a short description of the event, as well as lines for the appropriate approval signatures. Executive Summary The summary should be less than two pages. This is the Cliff Notes version of the report. A well-written summary should provide the information needed to make an informed decision about the depth and quality of the analysis. The summary should very briefly describe the event, present the statement of the problem that was evaluated, identify only the root or most significant causes, and include the corrective actions that address the causes from the summary. The first statement should state the basic facts: On <date, time> at <location, facility> while performing <activity, procedure, evolution> <event> occurred, resulting in <impact, significance, consequence>. The following information is also included in the Executive Summary: Problem statement (a clear statement of the issue of concern to be analyzed) The problem statement is not a restatement of the event, but rather the scope (hypothesis) of the analysis. Description of event.

26 Page 26 of 31 Root Cause Analysis Process (CONTINUED) Root cause(s). Corrective actions for Root Cause(s) (cause-action relationship). Table of Contents Event Description/Narrative The event narrative should be the detailed story of the event and should include information from the timeline, conditions that influenced actions and outcomes, responses and reactions, and the significance of the event. The narrative should include what happened and what did NOT Happen. A well-written root cause report tries to answer the questions a reader may have. Historical Review (Similar Occurrences) The historical or repeat event section should document any similar events found at the site of the issue. The corrective actions for these events should be evaluated to determine if they mitigated this event or if they should have prevented or merely reduced the probability for recurrence of this event. If the corrective actions failed to prevent, mitigate, or reduce the probability of this event, state why the corrective actions for those events failed to do so. Evaluation of Assessment Performance This section needs to look at defense barriers. Why was this event found at this point in time, and why was it NOT found through earlier, faster, and more reliable processes. Problem Evaluation This section should state what evaluation tools were used, why they were selected, and the results. Do not describe how a methodology or technique is performed (although the specific analysis details should be included as an attachment), but describe in detail what its logical outcome was with the logical connections. It should be clear to the reader how the facts of the issue or event translate to the conclusions: Root Cause(s) List the facts and conclusions that led to and support this root cause. State why addressing this cause prevents the event from recurring. Contributing Cause(s) List the facts and conclusions that led to and support this contributing cause. State why this is NOT a root cause (why correcting this cause would not prevent the event from recurring). Preventive Actions These are the actions that address the root and contributing causes and should be clearly linked. Remedial actions are not included in the RCA report as they are not intended to be preventive. They are documented and managed as part of the CAR. Extent of Condition/Generic ImplicationsState the results of the extent of condition (EOC) review. Given what you have learned about the occurrence/event, where else might you expect to see it? A properly scoped, implemented, and documented EOC

27 Page 27 of 31 Root Cause Analysis Process (Continued) evaluation can help identify and correct problems before the problems become events. The EOC evaluation can save project resources and create a safer, better managed work environment. Effectiveness Review Criteria BGS is expending time and effort to address this event. How will we know when we are done, and what will success look like? This is often an initial approach to establish effectiveness goals that may be enhanced when the effectiveness review plan is developed. Effectiveness review criteria provide a basis for determining whether the preventive actions, derived from the cause evaluation for an issue, once implemented, provide effective resolution of the issue. Other Information Attachments to the RCA report may include other information, such as List of documents reviewed, including identifying numbers, and other information reviewed during evaluation. List of personnel interviewed, if any. Charts, tables, and matrices resulting from the analyses, such as events and causal factors charts, barrier tables, and missed opportunity matrices. Photographs, sketches, and vendor information that enhances an understanding of the problem, the evaluation, the causes, or the solution. Information about the root cause team.

28 Page 28 of 31 ATTACHMENT 6 Apparent Cause Analysis Process Apparent cause analyses (ACA) usually involve one evaluator with documented causal analysis experience and one or more additional individuals involved in data gathering and evaluation. The total time for data gathering, evaluation, and report writing is generally from 10 to 20 hours, accomplished over a one to two week timeframe. Depending on the issue, managers may request the evaluator to go deeper (a mini root cause) to establish greater confidence in the cause identification. The ACA uses the same tools and techniques as a root cause analysis (RCA) but is based on readily available information. In contrast, a RCA will continue to dig for additional information until there is sufficient information to determine the cause with the level of confidence needed to assure prevention of recurrence. The apparent cause report contains much of the same information as the root cause report, but in less detail. Information to be covered includes: Executive Summary and Table of Contents: If the length of the report makes it appropriate. Problem statement. Event Description/Narrative: This is not the complete narrative of the root cause report. Problem Evaluation: Includes identification of the evaluation tools used and the results, statement of the apparent cause(s), and the extent of condition. Preventive Action(s): Includes cross reference to the cause the action addresses. Attachments: Attach documentation to support the conclusions of the analysis so that an individual can review the analysis and understand how and why the conclusions were reached. The Why Staircase is the simplest method of causal analysis and is intended for analyzing simple noncomplex problems (e.g., apparent cause analyses). However, other standardized methods for causal analysis may be used in place of or in addition to the Why Staircase. The following steps should be followed when performing the Why Staircase process: 1. Ask the question Why to determine the causal factors for the issue, following up each conclusion with another question Why until the analysis arrives at apparent causes that management has the control to fix and for which effective recommendations can be generated. 2. Keep asking Why until the answer to the question becomes less important than the answer to the previous question. This method not only helps the analyst to explain results, but also helps analysts detect flaws in logic.

29 Page 29 of 31 ATTACHMENT 7 Review and Approval of s Reports or Plans The purpose of verifying the completed actions and accepting, approving, or concurring with (reviewing) the collective set of corrective actions is to provide assurance the actions resolve the identified issue. Verification of CAR or CAP corrective actions include Was the individual action taken, as documented in the CAR or CAP, performed as stated? Was the intention of the action (in relation to the issue) appropriately implemented? For example, if a procedure was revised, do the changes have sufficient detail to ensure the issue was appropriately addressed? Are all the actions taken and documented in the CAR or CAP complete? This includes actions already taken (documented as appropriate to the action), and any actions added subsequently. Are the investigation, cause, and extent of condition appropriate to the identified issue? Was the collective and completed set of corrective actions appropriately completed? Did the collective and completed set of corrective actions satisfactorily resolve the identified issue? The objective evidence for each action should be reviewed. If the evidence is easily retrievable through the site records system, then reference to the document or other retrieval number is acceptable. If the evidence is not easily retrievable, then in addition to referring to the document number, an electronic copy of the document should be attached to the CAR or CAP.

30 Page 30 of 31 Development of Effectiveness Review Criteria ATTACHMENT 8 Effectiveness Review A reasonable amount of time should be provided prior to scheduling effectiveness reviews so as to allow enough time for the assigned reviewer to conclude that the corrective actions completed were effective in resolving the issues identified. In the same manner, allowing too much time before starting an effectiveness review after completion of corrective actions may not be useful either. A good rule of thumb is between six months and a year. Effectiveness review criteria provide a basis for determining whether the preventive actions, derived from the cause analysis for an issue, once implemented, provide effective resolution of the issue. Effectiveness review criteria are used to draw a conclusion regarding resolution of the issue. Effectiveness review criteria are established at the same time corrective actions are developed and are included in the root cause analysis report. They help identify what success looks like. An example includes During a management assessment, the quality assurance organization discovered that contrary to requirements, a significant number of completed surveillance reports did not have a cause code documented. Effectiveness criteria: 95 percent or more of the surveillance reports needed to document the cause code. Other effectiveness criteria could be established for the following: Compliance requirements have been met. Similar work has been performed without incident since corrective action implementation. Trending demonstrates that improvements beyond an established baseline have been achieved. Interviews indicate consistent understanding of process requirements. Methodology Identify the approaches used to conduct the effectiveness review. This can include a review of other CAPs for similar issues; reviews of performance (trending) analyses/metrics; reviews of incidents/occurrence reports/precursors due to the same causes or behaviors or resulting in similar consequences; interviews; work observations or facility tours; exercises (e.g., emergency exercises), or tests (e.g., crane lift tests); and reviews of recent assessments, as appropriate to the evaluation. Data and Analysis Present the data/information captured during the effectiveness review and provide a narrative

31 Page 31 of 31 describing the analysis of these data. Conclusions ATTACHMENT 8 Effectiveness Review (CONTINUED) State your conclusion regarding the effectiveness of preventive action(s). The conclusion is a professional judgment based on an analysis of the data compared to the effectiveness review criteria. Conclusions need to be focused on the scope of the original issue. If the effectiveness review team is able to determine the reason an issue was not resolved (e.g., corrective actions not implemented, inadequate cause evaluation), note this reason in the report.