APPENDIX C Configuration Change Management Verification and Validation Procedures

Transcription

1 DCMA-INST 217 APPENDIX C Configuration Change Management Verification and Validation Procedures

2 Table of Contents C1. Introduction... C-1 C2. Implementation... C-1 C3. Pre-Inspection: Verification... C-1 C3.1. Pre-Inspection: Implementation.... C-1 C3.2. Pre-Inspection: Completeness Requirements.... C-2 C3.3. Pre-Inspection: Coordination Requirements.... C-3 C3.4. Pre-Inspection: Content Requirements.... C-4 C4. Post-Inspection: ECP Validation... C-5 C4.1. Post-Inspection: ECP Implementation.... C-5 C4.2. Post-Inspection: ECP Document Status (Class I).... C-6 C4.3. Post-Inspection: ECP Configuration Validation.... C-7 A

3 Record of Changes Table Version Date Description 00 11/30/2012 Initial Release 01 03/01/2013 Accomplished minor changes to text for clarity. Added Paragraph C3.4.4., Paragraph C2.2: revised paragraph in its entirety to comply with DCMA-INST 1201, Corrective Action Process. 02 2/15/16 Accomplished minor changes to text for clarity. Adds references to: SAE EIA 649-1, Configuration Management Requirements For Defense Contracts, Nov Adds terminology from SAE EIA B

4 C1. Introduction C1.1. This Appendix provides Configuration Change Management (CCM) document verification and validation procedures for CCM documents. This Appendix supplements the DCMA-INST 217, Configuration Change Management Instruction by adding verification and validation procedures. Verification and validation confirm that the CCM document is a valid configuration control document (verification) and that changes described in the document are incorporated properly into the product Configuration Item (CI) and the Configuration Document (CD) (validation). MIL-HDK-61A, Section 8, Configuration Verification and Audit, Reference (e), and SAE EIA-649-B, Section 5.5, Configuration Verification and Audit, Reference (f), discuss the overall verification and validation of systems. C1.2. Configuration Approval Authority (CAA) introduced by SAE EIA is a synonym for CCM Instruction, Chapter 2, Table 1, Configuration Change Authority (CCA). SAE EIA CAA is used within and should serve as a replacement for CCA. C2. Implementation C2.1. Use of Appendix C, CCM verification and validation procedures, is required by the CCM Instruction, Chapter 3, Procedures, and Appendix B, the Configuration Change Management Documentation Checklist. Appendix A, Configuration Change Management Concepts, Principles, and Guidance, defines principles, guidelines and best practices for each specific CCM document type. C2.2. A Corrective Action Request (CAR) is issued to address contract noncompliance discovered during Pre-Inspection: Verification, Paragraph C3, and Post-Inspection: ECP Validation, Paragraph C4. Refer to CCM Instruction, Chapter 3, Paragraph 3.2.5, for procedures for issuing CARs and for procedures for systemic noncompliance s to the Supplier s Configuration Management (CM) system. (SAE EIA CCM 649B/649-1 Principle CCM-1.) C2.3. DCMA Engineers performing CCM document verification and validation will comply with the Safety and Occupational Health requirements of the CCM Instruction, Chapter 3, Paragraph C3. Pre-Inspection: Verification C3.1. Pre-Inspection: Implementation. (See SAE EIA-649B/649-1 Principle CCM-1.) C A successful pre-inspection verification confirms that the document to be evaluated is a valid Configuration control document and is ready for CAA evaluation. MIL- HDBK-61A, Paragraph 9.2.5, Digital Data Transmittal, discusses the expected format of transmitted CM type documents. C-1

5 C Insufficient, incomplete or systemic CM issues pertaining to this preinspection verification should be discussed with the Supplier, and the CCM document should be returned to the Supplier for correction, clarification or cancellation. The DCMA Engineer should prepare a CAR, as appropriate, in accordance with Paragraph C2.2, to correct any contract noncompliance. (See SAE EIA-649B/649-1 Principle CCM-1.) C3.2. Pre-Inspection: Completeness Requirements. (See SAE EIA-649B/649-1 Principles CCM-2, CCM-3, CCM-4 and CCM-5.) C Upon CCM document receipt, the DCMA Engineer will: C Confirm that the format, structure, form number, and document title are in accordance with approved contract CM provisions, such as the Supplier s Configuration Management Plan (CMP), Contract Data Requirements List, or Statement of Work. MIL-HDBK-61A, Paragraph , ECP Preparation and Submittal, provides information on Acquirer CMPs. C Suppliers may have company or site level process instructions, others will have program specific instructions, and some will have a combination of both. Suppliers may also have CI instructions for a particular program design or acquisition production phase. Obtaining a complete set of all these applicable documents will be required to perform appropriate verification and validation. C Review the classification, justification codes, appropriate identifiers, and other field information attributes for missing or incomplete data. The Supplier s CCM classification rules should match or be synonymous to the Government s classification and guidelines identified in MIL-HDBK-61A, Paragraph 6.2.2, ECP Activity Guides. Appropriate or synonymous identifiers should be identified in related contractual CM documents, such as a CMP or similar document (Paragraph C ). (See MIL-HDBK-61A, Table 6-6, Activity Guide: ECP Content, Subsection: ECP Identification and Administrative Attributes. ). C Review the document for overall completeness. This includes reviewing to confirm that required sections are complete, and that previous corrections or markups are incorporated. A CCM best practice is verifying that empty form CCM document text blocks are acknowledged and recognized with editing marks, such as a hyphen, Not Applicable (NA), being grayed-out, etc., to confirm the document originator intends the text block to be empty. C Confirm that dates are sequential and not dated in the future, that they appear where required, and that their appearance, format and detail are satisfactory for a contractual technical document. C-2

6 C Confirm that the document addresses the appropriate contract number, that it is not confused with another contract or program, and that the document is configured and dedicated to the contract number(s) to which the Engineering Change Proposal (ECP) applies. (See MIL-HDBK-61A, Table 6-6, Activity Guide: ECP Content, Subsection: Contract Information. ) C Verify that the CCM document contains all the information needed to accomplish the engineering changes to the CD. Confirm that the CCM ECP document Technical Data Package (TDP) references valid documents. Review the document package enclosures or attachments for their existence, completeness, quality and clarity. A Notice of Revision document from the Supplier may help. (See MIL-HDBK-61A, Paragraph 6.4, Notice of Revision. ) C3.3. Pre-Inspection: Coordination Requirements. (See SAE EIA-649B/649-1 Principles CCM-6 and CCM-7.) C Upon CCM document receipt, the DCMA Engineer will: C Scan the Supplier department functions and signature blocks appearing in the document. C Verify that all Supplier signatures are complete and dated. Dated signatures should be organized in a normal workflow sequence and start at the lower functional Supplier program tiers or departments. The Government CAA should be the last blank (empty) signatory block in the document. C Verify that the CCM document has been properly coordinated and integrated across all impacted Supplier departments by verifying entries in the appropriate signatory blocks (engineering, quality, manufacturing, assembly, logistics, packaging, etc.). C The verification of appropriate Supplier signatory blocks (departments) is accomplished by reviewing contractual related CM documents, such as the CMP, and comparing them to the CCM document properties (form fields, text blocks). All the appropriate Supplier departments should be identified and listed. C Verification is also accomplished by confirming that all the signatory blocks required are identified and listed. Confirm with the Supplier or CCM document originator that all the required signatory blocks appear in the document. A CCM document TDP requiring packaging, quality review, or another Supplier s department handling, should have packaging, quality, or other listed as one of the signatory blocks. Confirmation can also be accomplished by evaluating the CCM document, making note of the actions required in the document, and noting whether that action had a signatory block listed. C-3

7 C Confirm that the document is being addressed in the normal or regular workflow process. Confirm that the format, document workflow, and sign-offs are equal to previous Supplier CCM documents of the same type. This type of verification helps to prevent unauthorized changes. C3.4. Pre-Inspection: Content Requirements. (See SAE EIA-649B/649-1 Principles CCM-2, CCM-3, CCM-4 and CCM-5.) following: C Upon CCM document receipt, the DCMA Engineer will perform the C Verify that the CCM document TDP contains all the technical information details required to justify disposition (views, figures, drawings, test data, technical analyses, performance data, or white paper). (See MIL-STD-31000, Standard Practice Technical Data Package, MIL-HDBK-61A, Paragraph , ECP Supporting Data, and Table 6-6, Activity Guide: ECP Content, Subsection: Description of Proposed Change. ) C Confirm that an adequate configuration control audit trail exists in the CCM document by verifying before and after conditions are provided in the CCM document or associated TDP. This verification process can be accomplished by capturing the specific before ( was ) condition. This procedure captures the previous configuration before any proposed revision to the CD or CI is accomplished. This practice will also assist in the validation process (Paragraph C4). C Verify that the CCM document description is sufficient in quantity and detail. Poor, unclear, ambiguous, or incomplete technical writing should be cause for rejection. Draft pen-and-ink mark-up or redlines to CDs in the TDP that accompanies the CCM document is acceptable, if the content is clear and provides definition. It is a best practice that TDPs of CCM documents include screen shots of CD or CI areas being changed, with the proposed changes identified. This practice will also assist in the validation process. C Confirm, as applicable, the document s effectivity as follows: C Verify that the effectivity is identified by quantity or range (retrofit/spares/future production) and the manner in which the effectivity will be incorporated (attrition or forced schedule). (See MIL-HDBK-61A, Paragraph , Effectivity, and Table 6-6, Activity Guide: ECP Content. ) C Verify that the same effectivity is addressed throughout the enclosed document TDP, as applicable. C Confirm that the appropriate acquisition or manufacturing production sequence is used and is being established, as applicable (design followed by C-4

8 production). A hardware CCM document would usually be in place before the software is developed. Product testing requires a product to be produced, and human engineering may need product integration to be done first. CI Process One is required to be accomplished and finished before beginning Process Two. C Confirm that the latest TDP CDs or CIs are being utilized to justify the proposed CCM document position. C Assess the product CI design producibility described in the CCM document, as applicable. Producibility is a measure of relative ease of manufacturing a product. The proposed product should be easily and economically fabricated, assembled, inspected, and tested with high quality on the first design or change. Producibility is often difficult to achieve because it may require performance or functionality sacrifices. A new manufacturing process for a design has increased producibility risk. C It is a best practice that the Supplier (CCM document originator) acknowledge and assess the CCM document classification, informing the DCMA Engineer by including a statement in the CCM document similar or equal to, This Class II ECP is in accordance with the CMP, contains no defined Configuration baseline changes and does not alter contract requirements or Government cost or property owned. C CCM documents should include a matrix of Reliability, Availability, and Maintainability assessment that provides an evaluation of cost, schedule, performance, risk, and life cycle metrics impact. (Search Acquisition Community Connection for the DoD Guide For Achieving Reliability, Availability, And Maintainability. ) C Verify that the Supplier has approved the proposed ECP. The ECP should include the Supplier s recommendation. The recommendation should be supported by the Supplier signatures recommending that particular condition. C Confirm the Notice of Revisions (NOR) (DD Form 1695, or similar form), if used, identifies the correct number of drawings scheduled for revisions. A NOR is used to document and define revisions to CDs which require revisions after ECP approval. Consult with the contract or Acquirer regarding the handling and submitting of NORs. C4. Post-Inspection: ECP Validation C4.1. Post-Inspection: ECP Implementation. C Post-Inspection ECP Validation can only be accomplished to approved, revised, and produced CDs and CIs resulting from an approved CCM document. These procedures confirm that the CCM document engineering changes are properly incorporated into the associated CDs for CI production. It is recommended that Post-Inspections ECP Validations C-5

9 precede any product CI signoff or approval by the Government. (See MIL-HDBK-61A, Paragraph 5.7, Engineering Release. ) C The DCMA Engineer will plan all CCM document Post-Inspection ECP Validation procedures in accordance with the DCMA-INST 207, Engineering Surveillance Instruction. C Insufficient, incomplete, or systemic CM issues pertaining to this Post- Inspection: ECP Validation should be discussed with the Supplier, and the previously approved CCM document identified for correction, clarification or cancellation. The DCMA Engineer should prepare a CAR, as appropriate, in accordance with Paragraph C2.2, to correct any contract noncompliance. C The DCMA Engineer will determine if risk resulting from validation testing indicates required or additional regression testing is needed to correct an issue. Regression testing is performed to validate that the modified CDs or CIs do not introduce additional issues or faults that did not exist prior to incorporating ECP change. If risk, cost, or lack of insight dictates that further regression testing may be required, the DCMA Engineer may refer this decision to the Government Program Office (Acquirer) where the risks and risk mitigation techniques are better understood or justified. C4.2. Post-Inspection: ECP Document Status (Class I). C These procedures apply to Class I ECPs. A similar procedure is accomplished to Class II ECPs in Procedure C The DCMA Engineer will: C Periodically, and as required, conduct an assessment of all open Class I ECPs not dispositioned. The Acquirer, Supplier, or DCMA Engineering Change Proposal (ECP)/Request for Variance (RFV) Tracking Log may help identify open Class I ECPs. Class I ECP. C Inquire with the Acquirer and request the status of each open C Offer assistance to satisfy the disposition requirements to closeout open ECPs. Note: Value Engineering Change Proposals (VECPs) have a 45-day Acquirer evaluation period. C The Acquirer should be informed of Class I ECPs not being dispositioned in a timely fashion. Timely disposition of CCM documents allow CDs and Configuration baselines to be updated quickly. Prompt updates are a best practice of appropriate CCM and CM configuration control. (See MIL-HDBK-61A, Paragraph 6.1, Configuration Control Activity, and Appendix D, Paragraph D.1, Scope. ) C-6

10 C4.3. Post-Inspection: ECP Configuration Validation. (See SAE EIA-649B/649-1 Principles CCM-8, CVA-1 and CVA-2.) C Audit the amount of time the Supplier takes to update CDs and product CIs based on CCM document approval. Updates to CDs and product CIs should occur in a timely fashion. Timely updates allow Configuration baselines to be updated quickly. Prompt updates is a CM best practice for appropriate CD Configuration control and prevents later engineering changes from being made to documentation pending an earlier revision. (Refer to the references in Paragraph C for the importance of timely updates.) C A CCM audit will be accomplished to CDs modified by CCM documents. The audit purpose is to verify the traceability (flow down) of approved CCM document TDP requirements into the referenced CDs for appropriate CI production. The CCM audit serves to validate that the CCM document TDP changes are incorporated into the CDs. The DCMA Engineer will perform or support the DCMA Quality Assurance (QA) Personnel in performing the following CCM audit tasks: C Participate in pre-production product examinations as necessary to confirm that the traceability of approved requirements addressed in the CCM document TDP are incorporated into the production CDs. C Select a CCM document and associated TDP for the CCM audit and obtain the associated CCM document CDs. The CCM audit should be performed on production CDs addressed by the CCM document TDP. C Verify the appropriate CD Data Version Control, and that the CCM document changes are incorporated correctly into the CD(s) by: (See MIL-HDBK-61A, Table 5-8, Activity Guide: Engineering Drawing and Associated Lists, and Paragraph 9.2.4, Data Version Control. ) indicator has incremented. is the CCM document. C Verifying that the applicable CD revision C Verifying that the CD revision indicator authority C Confirm that an adequate Configuration control audit trail exists by verifying that the CD revision has before conditions identified or cited. (See Paragraph C ) C Confirm verification of the CD revision by the appearance of Supplier signatures that approve or check the updated CD(s). These CD checkers verify that the individual who incorporated or revised the CD did it correctly. C-7

11 C Spot-check or review the revision to the CD for engineering change depiction accuracy. C Verify that the CCM document is accurately incorporated into the CI. The DCMA Engineer, in coordination with DCMA QA personnel, should reconcile the CCM document by performing a product CI examination after production or assembly, by inspection, and by observing or witnessing appropriate regression testing for adequacy and compliance. (See DCMA-INST 324, Product Evaluation - QA Instruction.) Confirm that the CCM document changes have been incorporated into the deliverable CI properly. The first item of production should be inspected for CCM document requirement incorporation similar to First Article Testing, depending on the change complexity. C Confirm compliance by reviewing the list of approved CDs for use in production. Verify that product CI configuration information and related support assets are updated with the CCM document and requirements. Documents such as the (1) CI production list of CDs, (2) Supplier process manufacturing instructions, (3) test equipment, (4) requirements for spares, (5) retrofit kits, (6) CI operating instructions (logistics), and (7) existing CI orders should be reviewed to verify that the changes identified in the CCM document and TDP are incorporated or addressed in the final product. C The DCMA Engineer will periodically audit the Supplier s Configuration Status Accounting (CSA) identifying effectivity and the installation status of approved CCM engineering changes impacting the CI product. The Supplier s CSA and release program should identify both engineering changes made to the current CIs and those engineering change configuration requirements that were superseded for the product CI being delivered. The Supplier s CSA should detail and log design, production, modification, retrofit, or maintenance changes occurring to the product CIs. The DCMA Engineer will periodically audit the Supplier s CSA for accurate Supplier process information for approved CCM documentation. (See MIL-HDBK-61A Paragraph 5.7, Engineering Release, and Table 7-1, Typical CSA Information Over The Acquisition Program Life Cycle. ) C Each approved CCM document change should be incorporated into all associated and affected CDs within its program CI set (lots, units, mission, design, series, type, model, or series). Released CDs for each CI, at the time of its formal acceptance, should be retained in the associated Supplier release records. This information is of particular importance in the case of warranties associated with the CI or its components. C-8