Are You Getting the Most from Your Quality Engineer?

Transcription

1 Are You Getting the Most from Your Quality Engineer? Presented by Charles B Robinson Certified Lead Auditor / Senior Quality Engineer Quality Assurance & Risk Management Services, Inc.

2 This session is intended to provide an outline of how Quality Engineering (QE) should be adding value during the Life Cycle phases of a Project. Purpose This session will: Correlate the Life Cycle model with NASA Procedural Directives (NPD s), NASA Procedural Requirements (NPR s), AS9100, and the QE Disciplines Explain a QE s mission assurance contribution to processes and procedures during the project life cycle Provide a reference tool that QE s can use during the Life Cycle of a project. Provide an understanding of how a project has embraced the mission assurance aspects of AS9100D from a QE's perspective Software Disciplines are not addressed in this Session

3 Competency Wheel Supportive Systems Life Cycle AS9100 NASA Quality Engineers Mission Assurance Requirements NPD s & NPR s

4 1 Scope 4 Context of Organization No Requirements 2 Reference PLAN DO CHECK ACT 5 Leadership 3 Terms & Definitions 6 Planning 7 Support 8 Operation 9 Performance Evaluation 10 Improvement

5 Conduct Quality Evaluations & Data Analysis Implement Problem Reporting & Corrective Actions (PRACA) Define & Perform Inspection Testing & Acceptance Activities Perform Quality Assurance Management & Planning Quality Engineer Define & Implement Production, Material & Process Controls Assures Adequate Design & Development Controls Conduct Procurement Quality Engineering For more detail contact: Michael P. Kelly NASA Safety Center

6 NPR NASA Space Flight Program & Project Mgmt. Requirements Pre- Phase A Concept Studies Phase A Concept & Technology Development NASA Life-Cycle Phases Approval Formulation Phase B Preliminary Design & Technology Completion Implementation Project Life-Cycle Phases Phase C Final Design & Fabrication Key Decision Points Phase D System Assembly, Integration & Test Launch Launch Phase E Operations & Sustainment NPR NASA Systems Engineering Processes and Requirements Phase F Closeout MCR SRR MDR SDR PDR MCR Mission Concept Review SRR System Requirements Review MDR Mission Definition Review SDR System Definition Review PDR Preliminary Design Review CDR/PRR Critical Design Review / Production Readiness Review SIR System Integration Review CDR/ PRR SIR TRR SAR ORR FRR PLAR CERR PFAR DR TRR Test Readiness Review SAR System Acceptance Review ORR Operations Readiness Review FRR Flight Readiness Review PLAR Post Launch Assessment Review CERR Critical Events Readiness Review PFAR Post-Flight Assessment Review DR Decommissioning Review

7 SRR System Requirements Review To evaluate whether the functional and performance requirements defined for the system are responsive to the program's requirements on the project and represent achievable capabilities. The Quality Engineer understands the System Definition to assure their future activity is focused on mission success Understand initial requirements - End Items functional baseline Understand clarity of system definition - architectural level of design Create the SMA quality level requirements, e.g., develop the SMA Plan. Identify workmanship standards, critical technology, long lead items. NASA Advisories (NPR ) and GIDEP are addressed in the Safety & Mission Assurance Plan Communicate concerns (schedule, technology, long lead items, etc.) Life Cycle Phase A B C D E F AS9100 NPR (Sections) SRR MDR CDR PDR SIR TRR ORR FRR DR SDR PRR Risk Management Plan (G-3.3) P/2 B/1 U/1 6.1, Risk Mitigation Plan (F-7.0) I/2 U/2 U/2 U/2 U/2 U/2 U/ Safety & Mission Assurance Plan (G-3.2) P/1 B/1 U/1 U/1 U/1 6.0 Material Requirements Plan (NPR ) I/2 U/2 U/2 U/2 8.1, Configuration Management Plan (G-3.14) P/2 B/2 U/ Acquisition Plan (G-3.4) P/2 B/2 8.4 Technology Development Plan (G-3.5) P/2 B/ d I Initiate, P Preliminary, B Baseline, U Update, F- Final -&- 1 Develop, 2 Research & Review Requirements

8 Areas to consider to prepare for your involvement Environment Workmanship Mission Assurance Staff Ground Spacecraft Instruments Launch vehicle Duration of Mission Equipment Training Software Reliability Components Experienced? Safety Materials On Site Personnel? Mission Success University New Supplier Supplier History Length of Mission Spares GIDEP Mandatory Inspection Points LOD SOW Safety & Mission Assurance Plan Problem / Failure Reports from similar Projects Suppliers Material Documents

9 High Risk Electronic Parts [ERAI, Inc.] Commodity Information [SAM] General Supplier Information [DLIS] GIDEP Information [NANADARTS] NASA Advisories [NANADARTS, NARS] Supplier Assessment System (SAS) NPR , Customs & Border Protection C/F Parts Seizures [CBP] Audit Reports [NASA Audits] Supplier QMS Certifications [Audit Teams, Suppliers, OASIS] PQA Surveyed Supplier Info [GRC, JSC] Supplier Information Surveys [Supplier Provided] Debarred Suppliers [DLIS] Supplier Ratings [DCMA, Navy, SAS] SAS Administrator - jsc-sasadmin@mail.nasa.gov SAS Web site -

10 MDR (R) Mission Definition Review SDR (H) System Definition Review To evaluate the credibility and responsiveness of the proposed mission/system architecture to the program requirements and constraints, including available resources. To determine whether the maturity of the project's mission/system definition and associated plans are sufficient to begin Phase B. The Quality Engineer. Knowing the project's mission/system definition and associated plans, analysis the risks, identifies the resources needed for mission assurance, and starts a compliance matrix. As a component of the Verification & Validation Plan, Mandatory Inspection Points are identified by the QE. Identifies any concerns prior to beginning Phase B. Life Cycle Phase A B C D E F AS9100 NPR (Sections) SRR MDR CDR PDR SIR TRR ORR FRR DR SDR PRR Risk Management Plan (G-3.3) P2 B1 U1 6.1, Safety & Mission Assurance Plan (G-3.2) P1 B1 U1 U1 U/1 U/1 6.0 Verification & Validation Plan (G-3.9) P2 B2 U2 U , 8.4.3, m I Initiate, P Preliminary, B Baseline, U Update, F- Final -&- 1 Develop, 2 Research & Review Requirements

11 Baseline the Safety & Mission Assurance Plan (SMAP) QE Start GSFC 380 WI A Project and or Program Mission Assurance Requirements (MAR) Preparation Figure 1

12 PDR Preliminary Design Review To evaluate the completeness/consistency of the planning, technical, cost, and schedule baselines developed during Formulation. To assess compliance of the preliminary design with applicable requirements and to determine if the project is sufficiently mature to begin Phase C. The Quality Engineer Reviews the flow-down requirements to assure they are properly identified including workmanship standards, NASA Advisories/GIDEP, etc. Confirm the preliminary design is sufficiently mature to protect mission assurance, including schedule. Identify potential risk using FMEA as appropriate. Contribute to the initial Hazard Analysis Life Cycle Phase A B C D E F NPR (Sections) SRR MDR SDR AS9100 I Initiate, P Preliminary, B Baseline, U Update, F- Final -&- 1 Develop, 2 Research & Review Requirements PDR CDR PRR SIR TRR ORR FRR DR Mishap Preparedness and Contingency Plan (NPR ) P/1 U/1 B/1 U/ Safety Data Packages (NPR ) I/1 U/1 F/1 7.3, ELV Payload Safety Process Deliverables (NPR ) P/1 P/1 B/ Safety & Mission Assurance Plan (G-3.2) P/1 B/1 U/1 U/1 U/1 U/1 6.0

13 Failure Mode & Effect Analysis (FMEA) Screening Process NASA, National Space Transportation System (NSTS) 22206, Req ts. for Preparation & approval of FMEA & Critical Items List (CIL) Figure 3-2 Goddard Space Flight Center (GSFC)-FAP FMEA-Draft Functional, Design, Interface, Software, GSE, and Process FMEA instructions Figure 1-2 FMEA

14 Review the Corrective Action history of the department or supplier to identify weaknesses that should be mitigated. 18% 16% 14% 12% 10% 8% 6% 4% 2% 0% The above are the combined supplier audit results performed by a NASA Center from 2Q2015 to 1Q2016

15 CDR Critical Design Review PRR Production Readiness Review To evaluate the integrity of the project design and its ability to meet mission requirements with appropriate margins and acceptable risk within defined project constraints, including available resources. To determine if the design is appropriately mature to continue with the final design and fabrication phase. The Quality Engineer. Input Mandatory Inspections as appropriate Complete the Hazard Analysis and confirm controls are in place Confirm the Test Plan is in place Validate the requirements for integration are in place (if applicable) Life Cycle Phase A B C D E F AS9100 NPR (Sections) SRR MDR CDR PDR SIR TRR ORR FRR DR SDR PRR Integration of systems P/2 U/ I Initiate, P Preliminary, B Baseline, U Update, F- Final -&- 1 Develop, 2 Research & Review Requirements

16 Product Integration Process NPR , Figure C-7

17 SIR System Integration Review To evaluate the readiness of the project and associated supporting infrastructure to begin system Assembly Integration & Test (AI&T), evaluate whether the remaining project development can be completed within available resources, and determine if the project is sufficiently mature to begin Phase D. The Quality Engineer Review documentation prior to testing and/or integration to assure compliance. Verify compliance to the Test Plan Life Cycle Phase A B C D E F AS9100 NPR (Sections) SRR MDR CDR PDR SIR TRR ORR FRR DR SDR PRR Mission Operations Plan (G-3.10) P/2 B/2 U/2 6.2 Rank Safety Risk Management Process Documentation (H-3.24) B/ , 8.1 I Initiate, P Preliminary, B Baseline, U Update, F- Final -&- 1 Develop, 2 Research & Review Requirements

18 Develop a Checklist to assure the Mission Assurance areas are covered. Subject Sample of areas to review Final Design Change Notices clearly understood Safety Data Package Assure that mitigations are in place FMEA Analysis complete and mitigation plans in place Drawings Reviewed for the ability to manufacture and inspect Verification Plans Mandatory Inspection Points PQA Supplier flow-down appropriate QA Audit Audits complete and findings satisfied Hazard Analysis Created and hazards understood / mitigated Mission Assurance Requirements Risk Assessments addressed, As-built parts list approved Confirmed by QE

19 TRR Test Readiness Review Ensure that for each planned test or series of tests the test article (hardware / software), test facility, support personnel, and test procedures are ready for testing and data acquisition, reduction, and control. A TRR for each planned test or series of tests ensures that the test article (hardware/software), test facility, support personnel, and test procedures are ready for testing and data acquisition, reduction, and control. The Quality Engineer Review and/or contribute to the test procedures Identify Mandatory inspection points and assure they are incorporated into the test documents. Develop a plan to validate/witnessing test results (LOD for off-site, on-site SMA plan) NPR (Sections) NPR , Table G-10, TRR Entrance & Success Criteria Life Cycle Phase A B C D E F SRR MD R SDR PDR CDR PRR SIR TRR ORR FRR DR B/2 AS9100 I Initiate, P Preliminary, B Baseline, U Update, F- Final -&- 1 Develop, 2 Research & Review Requirements

20 A Sample of Things that need to be in place. Documented, Accepted, Configuration Control, & Communicated

21 ORR Operations Readiness Review To evaluate the readiness of the program, including its projects, ground systems, personnel, procedures, and user documentation. To operate the flight system and associated ground systems in compliance with program requirements and constraints during the operations phase. The Quality Engineer Confirm that all non-conformances have been addressed and closed. Verify that any mitigation plans have been satisfied. Validate the as built configuration. Confirm all GIDEP s or NASA Advisories have been addressed. Confirm the Safety Data Package is complete. Life Cycle Phase A B C D E F AS9100 NPR (Sections) SRR MDR CDR PDR SIR TRR ORR FRR DR SDR PRR Nonconforming Product I/1 U/1 U/1 U/1 8.4 Mitigation Plans P/2 U/2 U/2 U/ d As-built Configuration P/1 U/1 U/1 U/1 U/ , 8.3.6, I Initiate, P Preliminary, B Baseline, U Update, F- Final -&- 1 Develop, 2 Research & Review Requirements

22 An overview of GIDEP investigations throughout the Life-Cycle GIDEP ALERT, SAFE-ALERT, Problem Advisory, Agency Action Notice, and NASA Advisory Failure Experience Product Information Reliability Engineering Metrology & Advisories Safe Alerts Alerts Problem Advisories Agency Action Notices Lessons Learned Product Chang Notices Product Information DMSMS Reliability Methodology Failure Analysis Reliability Predictions Reliability Statistics Facilities Documents Computer Technology Process Specifications Test Reports Engineering Reports Management Reports Soldering Technologies Technical Manuals General Metrology Calibration Procedures

23 The Records Life-Cycle NPR , Figure 1 National Archives & Records Administration (NARA) NPR E - NASA Records Management Program Requirements, Appendix D: Agency Filing Scheme 5100 Procurement (Contracts) 5300 Reliability and Quality Assurance 5700 Safety and Mission Assurance

24

25 Presented by Charles B. Robinson Quality Assurance & Risk Management Services, Inc. / Navarro Engineering Solutions, 22 Highridge Drive American Canyon, CA / Crobinson@QAandRM.com Cell:

26

27

28