Product Assurance Audit Checklist

Transcription

1 GPQ-MAN-07 Product Assurance Audit Checklist GPQ-MAN-07 MSM-GP/27 APR 2001

2 I. General Product Assurance 1. Organisation and Staff 1.1 Does the contractor have a formally established PA organisation with the elements of Safety, Reliability, Maintainability, S/W PA, Quality Parts, Materials and Processes, Configuration Management? 1.2 Are the Product Assurance functions independent of Manufacturing, Engineering and other organisation? 1.3 Are there formulated objectives of the organisation? 1.4 Is management emphasis on prevention rather than detection of quality problems? 1.5 Are the responsibilities of supervisors and managers, etc. formally defined? 1.6 Does the person responsible for Product Assurance on a project have access to higher management for the focussing on and resolution of problems? 1.7 Are the PA functions staffed to the proper complement? 1.8 Does the Product Assurance organisation have sufficient facilities, equipment and tools for appropriate evaluation of materials and hardware quality? 2. Procedures 2.1 Does the contractor have a Product Assurance Manual or a set of procedures covering Product Assurance? 2.2 Do procedures identify who does what, when, how and where? 2.3 Are PA functions and responsibilities performed by other groups identified? 2.4 Do procedures identify information flow, forms and instructions for completing the forms? 2.5 Do procedures contain management approval signatures, dates of issue and revisions and change C = Compliant NC = Not Compliant Page 2 of 28

3 designators? 2.6 Is the distribution controlled and obsolete procedures removed from use? 2.7 Are procedures available at the appropriate workstations? 2.8 Are procedures kept updated? 2.9 Do detailed procedures exist for the performance of inspections and tests? 3. QA audits 3.1 Are systematic internal and external audits performed? 3.2 Are audits performed? a) According to an established plan? b) Using checklists? 3.3 Are internal audits conducted on an impartial basis by qualified personnel? 4. Receiving Inspection 4.1 Is the contractor's receiving inspection flow and activities documented? 4.2 Are facilities clean, proper lighting exist, and storage and handling practices followed? 4.3 Are all incoming supplies inspected and tested prior to use against the requirements of procurement documents? (appropriate drawings, specifications and technical instructions) 4.4 Are receiving inspections performed in accordance with established procedures and instructions? 4.5 Is the following verified? a) Identification / configuration? b) Supplier s documentation? 4.6 When supplies are accepted on the basis of certification or test report: a) Are test reports checked 100% against procurement C = Compliant NC = Not Compliant Page 3 of 28

4 documents requirements? b) Are reports and certification retained? 4.7 When supplies are accepted on the basis of certification or test report, are checks performed to confirm conformance? 4.8 Do receiving inspection records indicate date of receipt, inspections, tests, documents utilised, number accepted/rejected? 4.9 Is traceability maintained between materials and certificates, reports and receiving inspection documents? 4.10 Are there separated storage areas for materials waiting receiving inspection and materials already inspected and accepted? 5. Packaging and Handling Packaging 5.1 Are parts, assemblies and materials verified for cleanliness before packaging and properly packaged? 5.2 Are parts having critical or highly finished surfaces protected against damage? 5.3 Are leads of electrical parts protected from contamination which will effect solderability? 5.4 Are packaging operations covered by written procedures / instructions, which include at least the following: a) Selection of packaging materials? b) Inner protection? c) Mechanical protection? d) Cleanliness and contamination control? 5.5 Are packaging operations inspected and are items controlled before shipping for proper packaging, marking and accompanying documentation? C = Compliant NC = Not Compliant Page 4 of 28

5 Handling 5.6 Do handling procedures and instructions exist? 5.7 Are items leaving a controlled area or being transported between work/test areas carried in appropriate containers or fixtures to provide adequate protection from damage or contamination? 5.8 Do the operators have the proper tools to support the assembly while performing work/test on it? 6. Cleanliness and Contamination Control 6.1 Are there procedures available systematically covering cleanliness and contamination control aspects? 6.2 Are there specific instructions for: a) Entering and leaving clean rooms (personnel and equipment)? b) Cleaning of clean rooms, including: 1. Cleaning materials? 2. Cleaning methods? c) Cleaning of equipment to be entered in the clean rooms? d) Clean rooms maintenance? e) Control of clean room parameters? f) Actions in case of alarm including: 1. Stop work? 2. Shield critical items? 6.3 Are items tested and inspected for cleanliness such as (visual examinations, wipe tests, check for alkaline and acids) and are the appropriate methods used? 6.4 Are access control instructions posted, visitors sign in log properly dated and enforced as appropriate? 6.5 Are smock retention areas provided? 6.6 Are clean smocks available? 6.7 Are floor mats placed at all entrances? C = Compliant NC = Not Compliant Page 5 of 28

6 6.8 Are floor coatings peeling or generating dust particles avoided? 6.9 Is the shedding of sponge, mop of cloth material evident? 6.10 Are smocks buttoned and worn by all personnel including visitors? 6.11 Is there food, beverages or smoking? 6.12 Are personal belongings out of sight and stored properly? 6.13 Is the use of wooden containers with untreated interior and exterior surfaces or cardboard boxes minimised? 6.14 Are periodic particle counts performed to verify if within required limits? 6.15 Are gloves, caps, booties and masks appropriate to the area and worn by all personnel? 6.16 Is the integrity of the environment and contamination control maintained in relation to the surrounding areas or to material and mobile equipment entering the area? 6.17 Are other requirements such as: laminar airflow, vacuum shoe cleaners, air bath entrances, special waste disposals, etc. enforced? 6.18 Are clean room environment conditions, temperature, humidity, pressure controlled and recorded? 6.19 Is temperature maintained nominally at 20ºC+/-2ºC or meet a specific product specification? 6.20 Are positive pressure differentials maintained between clean room and surrounding area at 6mm Hg and between clean room and entrance lock at 2mm Hg with pressure monitors? 6.21 Is humidity maintained, preferably at 30-50% or as specified in a product specification and monitored with a hygrometer periodically calibrated? C = Compliant NC = Not Compliant Page 6 of 28

7 7. Manufacturing and Testing Manufacturing and Testing 7.1 Are manufacturing operations and inspections planned and described in manufacturing documents? (Shop travellers, work orders)? 7.2 Do shop travellers include: a) Revision status / date, release date? b) Item identification and configuration? c) Materials / parts to be used? 7.3 Are shop travellers identified / traceable to the contract no. / purchase order? 7.4 Are support documents to be used referenced? (e.g. drawings, work instructions)? 7.5 Are rework / repair plans used to document rework / repair manufacturing / inspections operations? Are they adequate? Are they archived? Manufacturing Control 7.6 Is shop traveller kept with product during manufacturing / inspection operations? 7.7 Are inspection, fabrication and assembly operations traceable to the individual performing the operation? 7.8 Are all articles inspected to applicable / controlled documents and are the documents available at the inspector s workstation? 7.9 Are unauthorised, marked-up, illegible or defaced drawings or work instructions being used? 7.10 Is the sequence of operations being followed as defined in the planning? 7.11 Are accepted and rejected items properly identified and tagged? 7.12 Are part lists complete for assembly work? 7.13 Are adequate inspection records generated and C = Compliant NC = Not Compliant Page 7 of 28

8 maintained and do they indicate the status of articles and inspections performed? 7.14 Are assembly operations performed in sequence with inspection operations? 7.15 Are torquing operations satisfactory? Calibrated tools? Test Documents 7.16 Does the contractor have standardised procedures and methods for the preparation, format, content, review and control of test procedures? 7.17 Are text procedures formally released and controlled with change and revision pages? 7.18 Do procedures include as applicable the following: a) Nomenclature and identification of the test article? b) Characteristics and design criteria to be inspected or tested, including values and tolerances for acceptance and rejection? c) Detailed steps and operations to be taken in sequence including verifications to be made before proceeding? d) Cross-reference of characteristics with measuring and/or NDT equipment to be used, specifying range and type? e) Details or instructions for operation of special data recording equipment of other automated test equipment? f) Layout and interconnection of test equipment and articles? g) Identification of hazardous situations or operations? h) Precaution to comply with established safety requirements, to ensure safety of personnel and to prevent damage or degradation of articles and measuring equipment? C = Compliant NC = Not Compliant Page 8 of 28

9 i) Environments and other conditions to be maintained? j) Constraints? k) Instructions for reporting non-conformance, anomalous occurrences or results? 7.19 Are test procedures formally approved by PA? Test Control 7.20 Are provisions made to protect personnel and items from physical damage and environmental hazards? 7.21 Is the test item inspected to determine evidence of damage or loose parts? 7.22 Are discrepancies or anomalies resolved prior to testing? 7.23 Are the following activities performed by PA as applicable: a) Witness each test and verify test performances? b) Observation? Perform surveillance or monitor periodically the test being performed as directed by explicit instructions? c) Data review? Review the procedure and recorded data at the conclusion of the test cycle verifying requirements and data entries are compatible? 7.24 Do all documents bear necessary approval signatures inspection stamps? 7.25 Is rework, repair or modification occurring to the testing properly documented? 7.26 Are test discrepancies, failures and non-conformance s reported, analysed and corrective action implemented? 8. Measurements and Test Equipment 8.1 Does the contractor have up-to-date written procedures defining his calibration system encompassing measuring and test equipment, standards, calibration C = Compliant NC = Not Compliant Page 9 of 28

10 and maintenance prescribed intervals and metrology laboratory environmental controls 8.2 Does the calibration system include all equipment used to verify the quality of the processes and products being manufactured? 8.3 Do calibration procedures exist for each equipment type and include: a) Equipment characteristics and identification? b) Location? c) Frequency of periodic checks? d) Acceptance criteria? 8.4 Is all measurement and test equipment subjected to a visual, dimensional and operational inspection upon receipt and at periodic intervals thereafter? 8.5 Are calibration labels used and do they include at least the following: a) Identifier? b) Due date for re-calibration? c) Control stamp? 8.6 Is the necessity for and/or frequency of periodic inspections based on objective evidence of the stability and continued accuracy of the equipment? Is there any interval exceeding 1 year of validity? 8.7 Is all measurement and test equipment checked against a standard of greater accuracy? 8.8 Are temperature, humidity, vibration and cleanliness controlled to the extent necessary to assure calibration and continued product measurements of the required accuracy? 8.9 Is it ensured that errors of measuring and test equipment do not exceed 10% of the highest tolerance of any items to be checked with that equipment? C = Compliant NC = Not Compliant Page 10 of 28

11 8.10 Are standards against which test and measurement equipment are periodically checked traceable to National or International Standards? 8.11 Are standards supported by certificates, report, variables data sheets attesting to the date, accuracy and conditions under which the results were obtained? 8.12 Is the calibration organisation staffed with sufficient qualified technical personnel? 8.13 Are tri-dimensional measuring machines and laboratory testing machines periodically calibrated by a qualified organisation? 9. Non-Conformance Control 9.1 Is a procedure available which describes the nonconformance system? 9.2 Are non-conformances initially reported on local records (discrepancy notes)? Who takes decision for further processing? Is PA involved? 9.3 Do NCR records reflect causes and responsibilities for the discrepancies and show corrective and preventive actions taken? 9.4 Is the internal organisation responsible for causing the discrepancy informed? And does it concur to identify preventive actions? 9.5 Are logs and records maintained of all NCR issued and MRB meetings? 9.6 Are there adequate provisions for segregation, removal and identification of non-conforming materials? Is the access to holding areas controlled? 9.7 Are NCRs identified on relevant shop travellers and quality records? 9.8 Are NCRs properly filled out and complete? 9.9 Are NCRs closed by PA after verification of successful C = Compliant NC = Not Compliant Page 11 of 28

12 remedial actions implementation? 9.10 Are preventive actions implementation tracked? (Correction of technical documents, modification of tools and work instruction?) 9.11 Are non-conformances classified according to applicable project rules? 9.12 Are the following dispositions applied? a) Use-as-is? b) Rework? c) Repair? d) Scrap? e) Return to supplier? 9.13 Are major non-conformances notified to ESA within 1 working day after the review by the local MRB? 9.14 Is a RFW issued if the non-conformance disposition affects the product configuration? 10 Process Control 10.1 Are standard / established processes classified in accordance to Project rules? 10.2 Is each process covered by a specification? 10.3 Do process specifications include QA provisions, methods for inspection and test, number of samples, accept / reject criteria? 10.4 Does the company maintain documented evidence of processes validation status? 10.5 Are changes to process implemented through established Configuration Control procedures? 10.6 Are the personnel who perform critical processes or evaluate process performance numerically adequate, trained and certified? 10.7 Are materials, equipment, computer systems and software and procedures involved in the performance of C = Compliant NC = Not Compliant Page 12 of 28

13 critical processes validated? 10.8 Are non-destructive-inspections (NDI) treated in the same way of critical processes? 10.9 Are representative NDI standards available for equipment calibration and to monitor equipment performance? Are NDI inspection records maintained? Are process equipment periodically checked (as applicable) for: a) Temperature uniformity? b) Integrity / precision of thermo-couple lines? c) Integrity of vacuum related features (lines, etc.)? d) End-to-end instruments calibration? e) Effectiveness of contamination control means? For each process performed are records maintained of main process parameters and of related laboratory samples tests results? Are there provisions in case of interruption of main power supply during operation? 11 Procurement Control 11.1 Does the contractor maintain a list of approved subcontractors and suppliers supported with qualitative and quantitative data? 11.2 Has formal assessment including periodic follow-up been accomplished on subcontractors and suppliers? 11.3 Does the contractor continually evaluate subcontractor / supplier s history / results and implement corrective actions when appropriate? 11.4 Are project peculiar requirements identified in order that quality personnel reviewing purchase documents have adequate project and technical data to perform the review? C = Compliant NC = Not Compliant Page 13 of 28

14 11.5 Does the contractor maintain and implement a system of effectively passing on contract / project quality requirements to the subcontractors and suppliers? 11.6 Do procurement documents contain Product Assurance provisions for the following as appropriate: a) Applicable ESA contractor PA specifications? b) Chemical and/or physical test results (raw mat.)? c) Accompanying specimen for incoming tests? d) Age control of limited life products? e) Characteristics to be inspected or tested? f) Requirements for inspection and test records? g) Processing of non-conforming materials? h) Requirements for source inspection and MIPs? i) Right of access and planned surveillance activities? j) Cleanliness and contamination control? k) Acceptance data package and delivery? l) Preservation, packaging and shipping? m) Critical process requirements? 11.7 Are purchase documents reviewed and approved by PA? 11.8 Does the contractor have a policy or rules for source inspection? 11.9 Are adequately trained personnel available for source inspection? Does the contractor maintain records of source activities, status deficiencies and action items? 12 Training and Certification 12.1 Are personnel performing critical processes and nondestructive testing trained and certified according to a documented training programme? 12.2 Are national or international training programmes and standards applied? C = Compliant NC = Not Compliant Page 14 of 28

15 12.3 Are NDT personnel trained and certified according to MIL-STD-410 or equivalent? 12.4 Is re-certification performed periodically according to an established programme or following extensive interruption of work or unsatisfactory performance? 12.5 Are quantitative tests used to verify the proficiency of the trained personnel? 12.6 Are records documenting personnel training maintained? 13 Stamp Control 13.1 Does a documented system exist for the usage of stamps? (inspection, fabrication, test, etc.) 13.2 Do records exist allowing traceability to the responsible individuals and defining the authority / limitations in use? 14 Design Control 14.1 Are there standards and procedures available for preparation and maintenance of engineering drawings and specifications? 14.2 Are there procedures for tracking and documentation of requirements verification? 14.3 Do QA personnel participate in internal design reviews to ensure that: a) Technical documents comply with procedures and standards? b) Required inspections and tests are appropriate and feasible? 14.4 Is QA participation to internal design reviews documented? 14.5 In support of formal design reviews with the customer does the PA function ensure that: a) Documents are complete and consistent? C = Compliant NC = Not Compliant Page 15 of 28

16 b) Analyses clearly identify the related baseline? c) Safety, reliability and maintainability are factored into the design? d) The design is producible, inspectable, testable? II. Reliability 1. Reliability 1.1 Does a system exist for ensuring that designers are aware of part derating criteria with which they are expected to comply? 1.2 Does a procedure exist describing the FMECA approach of the contractor? 1.3 Does the FMECA approach focus on: a. Identification and control of single point failure? b. Critical interfaces? c. Methods by which redundancy is implemented? d. Product design (e.g. layout of PCBs, trace width, pad sizes and other failure causes related to mechanical configuration?) 1.4 Is a functional FMECA performed at system (payload) level? 1.5 Does the FMECA include the identification of methods by which the existence of failure modes can be determined during operations? 1.6 Does the FMECA adequately reference basic design documentation such as specifications, design reports, drawings, circuit diagrams, part lists and operating procedures? III. Maintainability 1. Maintainability 1.1 Is the contractor aware of basic maintainability C = Compliant NC = Not Compliant Page 16 of 28

17 principles and techniques? 1.2 Is the contractor familiar with maintenance concepts, planning and supporting elements? 1.3 Do procedures exist to perform maintainability studies and maintainability predictions? 1.4 Is maintainability a design parameter monitored for compliance throughout the design process? IV. Safety 1. Safety 1.1 Are NASA and ESA safety related documents (specifications, standards, handbooks) available? 1.2 Has an adequate program of training, safety motivation and awareness been instituted? 1.3 Is the contractor able to support his subcontractors for safety related issues? 1.4 Are there adequate safety plans produced covering: a. Scope of safety effort? b. Emphasis on safety analyses and reviews? c. Hazard control procedures? d. Involvement of PA and safety personnel throughout design and development? e. Safety design criteria? f. Safety verification program? 1.5 Does a detailed procedure exist to establish and maintain a safety verification program? And does it allow for coordination of all interested contractor's functions so to provide prompt feedback to ESA? 1.6 Are safety related inspections and tests performed by or under direct surveillance of the PA and Safety function? 1.7 Does the PA and Safety function have review authority C = Compliant NC = Not Compliant Page 17 of 28

18 over all test plans with adequate lead time to identify and control associated hazards? 1.8 Is the generation of safety documents for a project well defined and scheduled? 1.9 Do the safety analyses make maximum use of FMECA results? V. Software PA 1. Organisational Structure 1.1 For each project involving software development, is there a designated software manager? 1.2 Does the project software manager report directly to the project manager? 1.3 Is there a software PA function for each project that involves software development? 1.4 Is there a software configuration control function for projects that involve software development? 1.5 Which responsibilities are assigned to the software PA function in the various phases of the software life cycle? 1.6 How is the independence of the software PA function assured? 1.7 Is the test organisation independent from the development? (unit, integration, system) 1.8 Who decides in case of conflict between different organisational units? 1.9 Is Product Assurance approval mandatory prior to software release? 2. Resources, personnel and training 2.1 Does each software developer have private computersupported workstation/terminal? 2.2 Are project managers familiar with software project C = Compliant NC = Not Compliant Page 18 of 28

19 management disciplines? 2.3 Is there a required software engineering training program for software developers? 3. Software engineering experience 3.1 What is the median number of years of applicable experience of software project managers? 3.2 What is the median number of years of applicable experience of software integration and test managers? 3.3 What is the median number of years of software development experience of the software staff? 3.4 What percentage of the software staff has at least one year of development experience with the design and implementation languages to be used? 3.5 Of those with experience, what is the median number of years of experience with those languages? 3.6 What is the median size, in source lines of code, of software development projects completed in the last five years? 3.7 What is the size of the smallest project? 3.8 What is the size of the largest project? 4. Documented standards and procedures 4.1 Does the software organisation use a standardised and documented software development process on each project? 4.2 Does the standard software development process documentation describe the use of tools and techniques? 4.3 Is a formal procedure used to assure periodic management review of the status of each software development project? 4.4 Is there a mechanism for assuring that software subcontractors, if any, follow a disciplined software C = Compliant NC = Not Compliant Page 19 of 28

20 development process? 4.5 Are standards used to specify the content of software documents (files/folders)? 4.6 Are inspections conducted during the software development process? 4.7 Are coding standards applied to the software development process? 4.8 Are standards applied to the preparation of unit/integration/system test cases? 4.9 Are code maintainability standards applied? 4.10 Are internal design review standards applied? 4.11 Are code inspection standards applied? 4.12 Are software classification rules applied? 4.13 Is a formal procedure applied to estimate software size? 4.14 Is a formal procedure applied to produce software development schedules? 4.15 Are formal procedures applied to estimating software development cost? 4.16 Is a mechanism used for ensuring that the software design teams are familiar with the software requirements? 4.17 Are man-machine interface standards applied to each appropriate software development project? 5. Process metrics 5.1 Are software staffing profiles maintained (actual staffing versus planned staffing)? 5.2 Are profiles of software size maintained for each software configuration item, over time? 5.3 Are statistics on software code and test errors gathered? 5.4 Are design/coding errors projected and compared to actual? C = Compliant NC = Not Compliant Page 20 of 28

21 5.5 Are profiles maintained of actual versus planned designed software units, over time? 5.6 Are profiles maintained of actual versus planned tested software units, over time? 5.7 Are profiles maintained of actual versus planned integrated software units, over time? 5.8 Are processor memory utilisation estimates versus actual size tracked? (sizing figures) 5.9 Are target processor throughput utilisation estimates versus actual used, tracked? (timing figures) 5.10 Are design and code review coverage measured and recorded? 5.11 Is test coverage measured and recorded for each phase of functional testing? 5.12 Are the action items resulting from software reviews tracked to closure? 5.13 Are software problem reports resulting from testing tracked to closure? 5.14 Are the action items resulting from design/code inspections tracked to closure? 5.15 Is test progress tracked for each deliverable software unit/component? 6. Data management and analysis 6.1 Has a managed and controlled process database been established to process metrics data across all projects? 6.2 Are the review data gathered during software reviews analysed? 6.3 Is the error data from code inspections and tests analysed to determine the likely distribution and characteristics of the remaining errors in the product? 6.4 Are analyses of errors conducted to determine their process related causes? C = Compliant NC = Not Compliant Page 21 of 28

22 6.5 Is a mechanism used for error cause analysis? 6.6 Are the error causes reviewed to determine the process changes required to prevent them? 6.7 Is a mechanism used for initiating error prevention action? 7. Process control 7.1 Does management have a mechanism for the regular review of the status of software development projects? 7.2 Is a mechanism used for periodically assessing the software engineering process and implementing improvements? 7.3 Is a mechanism used for identifying and resolving system engineering issues that could affect software? 7.4 Is a mechanism used for regular technical interchanges with the customer? 7.5 Is a mechanism used for ensuring compliance with the software engineering standards? 7.6 Is a mechanism used for ensuring traceability between the software requirements and top-level design? 7.7 Is a mechanism used for controlling changes to the software requirements? 7.8 Is configuration management implemented? (Baselines, Identification, Libraries, Release and version control) 7.9 Is a mechanism used for ensuring traceability between the software top-level and detailed designs? 7.10 Are internal software design reviews conducted? 7.11 Is a mechanism used for controlling changes to the software design? 7.12 Is a mechanism used for ensuring traceability between the software detailed design and code units / components? 7.13 Is a mechanism used for ensuring traceability between C = Compliant NC = Not Compliant Page 22 of 28

23 the software detailed design and code units/components? 7.14 Are software code reviews conducted? 7.15 Is a mechanism used for controlling changes to the code? (Who can make changes and under which circumstances?) 7.16 Is a mechanism used for configuration management of the software tools used in the development process? 7.17 Is a mechanism used for verifying that the samples examined by Software Product Assurance are truly representative of the work performed? 7.18 Is there a mechanism for assuring that regression testing is routinely performed? 7.19 Is there a mechanism for assuring the adequacy of regression testing? 7.20 Are formal test case reviews conducted? 7.21 Do the company's procedures describe tasks and responsibilities for different test types? 7.22 Is module (integration, system) level testing covered by these procedures? 7.23 Are test coverage measurements defined? 7.24 Is there an approach for formal qualification testing? 7.25 Are nonconformances properly documented, including dispositions and retesting? 8. Tools and Technology 8.1 Is automated configuration control and track change activity throughout the software development process? 8.2 Are computer tools used to assist in tracing software requirements to software design? 8.3 Are design methodologies used in program design? 8.4 Are computer tools used to assist in tracing the software design to the code? C = Compliant NC = Not Compliant Page 23 of 28

24 8.5 Which languages are used in software developments? 8.6 What is the percentage of software developments implemented in high-order language? 8.7 What is the percentage of software developments implemented in assembler language? 8.8 Are automated test input data generators used for testing? 8.9 Are computer tools used to measure test coverage? (tool for dynamic analyses) 8.10 Are computer tools used to track every required function and assure that it is tested/verified? 8.11 Are automated tools used to analyse the size and change activity in software components? 8.12 Are automated tools used to analyse software complexity? (tools for static analyses) 8.13 Are interactive source-level debuggers used? 8.14 Are computer tools used for tracking and reporting the status of the software in the software development library? 8.15 Are prototyping methods used in designing the critical elements of the man-machine interface? 8.16 Which other SDE tools are currently used? 9. Procurement of COTS 9.1 Is there a procedure for COTS procurement? 9.2 Is the supplier's capability or ensuring COTS maintenance taken into account when procuring software? 9.3 Are licensing arrangements taken into consideration when procuring software 9.4 What is the documentation policy for procured COTS to be used together with company's developed software? C = Compliant NC = Not Compliant Page 24 of 28

25 VI. Configuration Management 1. Organisation 1.1 Is a configuration Manager the single-point contact for all matters relating to CM? 1.2 Is there a clear statement of responsibilities and authority for the CM function? 1.3 Is the organisational placement of CM at appropriate level? 2. Subcontractor's CM 2.1 Do procedures provide for the application of CM requirements to subcontractors? 2.2 Does the change control system provide for input of subcontractor-initiated changes, processing and change authorisation? 3. Function of CM office 3.1 Does the CMO perform the following functions: a. Preparation and maintenance of CM procedures and plans? b. Release authority for engineering documents? c. Operation of the CCB? d. Support to design reviews? e. Processing and release of changes and waivers? 4. Configuration identification 4.1 Is the concept of Configuration Item used and implemented? (CI-specification-drawing list-design reviews-qualification) 4.2 Are criteria for selection of CIs provided? 4.3 Do procedures establish a standard identification system for drawings, specifications, hardware and S/W? 5. Configuration data management 5.1 Do procedures or manual provide instructions for C = Compliant NC = Not Compliant Page 25 of 28

26 format and content of drawings and specifications? 5.2 Do procedures or manuals establish methods for amendments and revisions to specifications and drawings? 5.3 Is the CMO function responsible for identification numbers assignment? 5.4 Are end-item serial numbers provided by CMO for configuration and change traceability? 5.5 Are there prescribed procedures for release of configuration documentation that provide, as a minimum, establishment of records and impound of originals? 5.6 Is there a defined approval procedure for configuration documentation prior to release? 5.7 Is release of configuration documentation by the applicable release system required prior to manufacture or procurement? 5.8 Do procedures provide for release of software documentation? 5.9 Is the S/W Library used as the release function and is there an established procedure for its operation? 5.10 Does the impound system provide adequate security to prevent unauthorised changes and provide disaster protection? 5.11 Do procedures provide for a S/W repository for permanent retention of delivered software? 5.12 Is CM applied to CAD/CAM software and software tools? 6. Configuration Control 6.1 Are there procedures for application of baseline identification and control? 6.2 Are baselines established at system, subsystem and C = Compliant NC = Not Compliant Page 26 of 28

27 equipment level? 7. Configuration Control Board 7.1 Does the system provide for a CCB with membership that will ensure a comprehensive impact analysis? 7.2 Is a Software CCB provided with appropriate membership to evaluate and disposition S/W changes? 7.3 Are clearly defined interfaces between the system CCB and S/W CCB hardware/software changes? 7.4 Is the CCB in charge of reviewing and controlling the design baselines? 7.5 Are CCB Directives issued? 8. Change Evaluation 8.1 Does the system provide for impact analysis prior to approval/disapproval? 8.2 Does change impact analysis include all aspects of engineering, manufacturing, logistics costs, schedule? 8.3 Are changes and implementation documents marked with the class? 8.4 Are changes and waivers submitted to the approval authority established by the contract, before implementation? 8.5 Are changes and waivers released by the same release system used for initial drawing and specification release? 8.6 Is the interchangeability status marked on the changes? 8.7 Is there a procedure for the re-identification of noninterchangeable items? 8.8 Does the system provide for the preparation and presentation of waiver? 8.9 Are waivers processed through the customer's CCB? 9. Configuration Status Accounting C = Compliant NC = Not Compliant Page 27 of 28

28 9.1 Does the CM system provide for the documentation of the 'as-designed' standard by means of a CIDL (Configuration Item Data List)? 9.2 Do procedures prescribe the function responsible for CIDL inputs? 9.3 Does the CIDL contain as a minimum the list of released: a. Drawings? b. Specification? c. Test procedures? d. EEE parts list? e. Material list? f. Processes list? 9.4 Does the system provide for the preparation of 'as-built' lists and establish the responsibility for preparation and data input? C = Compliant NC = Not Compliant Page 28 of 28