GUIDE FOR MANAGEMENT SYSTEM AUDITORS COMPILING STAGE 1 AND STAGE 2 AUDIT REPORTS

Transcription

1 GUIDE FOR MANAGEMENT SYSTEM AUDITORS COMPILING STAGE 1 AND STAGE 2 AUDIT REPORTS TABLE OF CONTENTS 1. TABLE OF CONTENTS GENERAL DEFINITIONS STAGE STAGE PERIODIC SURVEILLANCE / RECERTIFICATION AUDITS...10 Rev. 3 1/26

2 1. PURPOSE AND SCOPE This document is a practical guide for preparing stage 1 and stage 2 audit reports on Management Systems, as required by UNI EN ISO/IEC 17021: This instruction applies to the preparation of stage 1 and stage 2 audit reports for the certification of Quality Management Systems for conformity with the ISO 9001 standard and can be used as reference for the certification of correlated systems (ISO 13485, ISO 3834, etc.) and for management systems in general. 2. GENERAL The audit report is a fundamental aspect of the audit activity. The purpose of the audit report is to provide objective evidence of the verifications performed on an organisation's application of the Management system, as indicated in paragraph 9 of UNI CEI EN ISO/IEC 17021: 2011, to transmit the audit conclusions to the client and to provide sufficient elements and information to the person who must decide on whether an organisation may be certified in order to enable them to make the decision. The UNI CEI EN ISO/IEC 17021:2011 standard transposes many of the indications contained in the ISO standard relevant to the drawing up of the audit report, thus transforming them into requirements. The ISO/IEC TS 17022:2012 standard provides elements to draw up the third party audit report and even if it s a guide, and therefore not mandatory, it has been taken into account for the drafting of our audit reports. The audit report must, in any case, constitute added value to the audit activities and must therefore be usable by organisations subject to audit to improve their management system. The audit report (stage 1 and stage 2) is expected to include, as objective evidence of the verifications performed,, for each process, a description of what was examined by the auditing team and the relative results (ISO/IEC 17021: and ). The audit report must include a summary of the audit process, including the uncertainties and obstacles encountered that could decrease the reliability of the audit conclusions. For this purpose, the contents of the audit report should be identified while performing the on-site audit. Rev. 3 2/26

3 The UNI CEI EN ISO/IEC 17021:2011 standard foresees, moreover, that the report may supply opportunities for Management System improvement. Indications are given below on how to compile reports using the software application ASCESI (ASsistenza CErtificazione SIstemi) in order to meet the requirements of the reference standards and indicate the points verified and the audit results but which can, above all, give added value to audit activities and can thus be more useful to the organisations under audit as regards improving their Quality Management Systems. 3. DEFINITIONS In the text the following definitions/abbreviations are used QMS: Quality Management System Manual: Organisation s Quality Management System Manual Regulations: General Rules for Management System certification Rev. 3 3/26

4 4. STAGE Audit list This field must contain any observations made during the stage 1 audit for each point of the reference standard. It must also be indicated if the observation is critical, that is, whether it could halt the certification process if not eliminated before stage 2, by ticking the relative serious recommendation box. Rev. 3 4/26

5 EXAMPLE: Type of critical findings: Z4: RINA regulations; non-application (e.g.: QMS applied for less than three months) Z6: Manual: missing parts; description of the reasons for possible unacceptable exclusions; description of the processes and their interactions; field of application. Z7: List of procedures: no obligatory written procedures Z8: Chamber of Commerce registration: activity very different from the scope of certification Z9: No organisation chart (Quality assurance manager not defined) Z10: No site plan: (Only applies to stage 1 off-site; not relevant if the site is audited during stage 1 and if a judgement can be given) Z11: Management review not recorded Z12: No internal audit plan; internal audits not performed. No coverage of the entire system Z13: No list of applicable laws and regulations Z14: List of current sites or outsourced activities N.B: the points not applicable to stage 1 (Z1, 2, 3, 5) must be removed from the list on ASCESI. Rev. 3 5/26

6 4.2 Company & Representatives As usual, the name and company position of the interviewees must be entered. In this field, it s also necessary to tick the people who participated in the initial and final briefings. Moreover, it is necessary to tick the right-hand column, corresponding to the name of the person who will sign the report. Rev. 3 6/26

7 4.3 Production site Company data (scope, address, name) have already been entered by the Secretary, though it is always best to check them. If data are missing or incomplete, the RINA office secretary must be contacted to agree on the entry method. Rev. 3 7/26

8 4.4 Reference documents Reference must be made to the audited management manual and any other documents relative to the QMS. (e.g.: Quality Plan; Company job descriptions; etc.) - Any requirements of the standard rule excluded The admissibility of the exclusion of any points of the standard on the basis of verified documents must be checked and acceptance of the justification must be justified. Always compile this field: if there are no exclusions, write "INAPPLICABLE" or NO EXCLUSION. Admissible exclusions are only those relative to the requirements as per point 7 of ISO 9001: EXAMPLES: (A) The admissibility of the exclusion of the requirements as per point 7.3 of ISO 9001: 2008 has been verified as the company only makes products according to customer designs. (B) It has been verified that the final results of the production processes can be completely verified, for which reason we confirm that the exclusion of the requirements as per point of ISO 9001: 2008 is acceptable. Rev. 3 8/26

9 4.5 Evaluation - Verification that the management system has been implemented for at least three months: Indicate when the management system became operative (see, for example, the date of revision zero of the Manual) and indicate that all the audits and a management review of the QMS have been performed. Even if the Manual indicates a date prior to the stage 1 audit of at least 3 months (see Regulations), the QMS cannot be said to have been completely applied if the internal audits and a QMS review have not yet been performed. Moreover, this shortcoming becomes a critical observation (see point 4.1). EXAMPLE: (The auditing team has verified during the successful audit that the system has been applied for 6 months (three month condition respected). The management review was performed on 18/6/2008. A complete audit cycle was performed as per the 2008 audit plan of 21/2/2008. The audits covered all the requirements of the standard and all company processes. Internal audits were also performed on the most significant sites open at the moment the audit was performed. Moreover, all the procedures were issued in revision 0 on 1/2/2008, as was the quality manual. The indicators defined to assess the effectiveness and efficiency of the processes began to be monitored on 1/2/2008). - Assessment of site location (including any particular conditions) A brief description of the operating site must be given: whether it is a production facility, whether it is just offices, including any permanent or temporary operating sites and indicating what processes/activities are performed in the various departments. A judgement relative to the suitability of the sites should also be expressed. Rev. 3 9/26

10 It should also be indicated if there are any particular matters that can affect the QMS, such as special equipment (large capacity cranes; laboratories; archives; warehouses; particular means of transport; dry docks; paint shops, etc.). EXAMPLE: (The organisation's operating site comprises two sheds in which engineering processes and assembly operations are performed. A particular area, suitably separated from the production areas, has been set aside as a paint shop and is fitted with a suitable extraction system. Alongside, there is a room used to store incoming products with a zone for the delivery of finished products. The technical, administrative and management offices are located on the upper floor. No critical points were found and the site is considered suitable for the realisation of.. ; management, however, complains of a lack of space caused by the increase in activities during the last two years) - Information related to each document examined (including the Manual) A description must be given for each document examined. In particular: Manual: indicate how it is structured (by processes; points of the standard; etc.) and if it contains the company quality policy, a good description of the company processes and relative interactions, of the organisation with the company organisation chart, of the production method, indicating if any processes are outsourced, and if all the requirements of the applicable standards are covered. Procedures: if there are at least the procedures required by the applicable standards (e.g.: ISO 9001: 2008) and if there is a list of the same. QMS review: if it covers the requirements of applicable standards. List of applicable laws and standards: indicate whether it has been seen and if anything has been omitted. Organisation chart: if it reflects the corporate structure. Chamber of Commerce registration: indicate whether the activity for which certification is requested is present. Audit planning: if it covers all corporate processes. EXAMPLE: (The Quality Manual is structured according to the points of ISO 9001: 2008 and is in rev. 1 of 4/08/2010; it contains a description of company processes and relative interactions that seem adequate for the particular type of production; inputs, outputs and indicators to base improvement objectives on were identified for each process. The manual also contains a description of the company organisation, identifying the responsibilities and authorities of each function, and an organisation chart illustrating the various hierarchic and functional chains. A series of procedures was prepared, including those requested by ISO 9001: 2008, a list of which is contained in the manual. A list of the laws, standards and regulations applicable to the company product is available. The Chamber of Commerce certificate indicates the company scope and activities, including those for which certification is requested. The Quality System Review was performed on 10/09/2010 and includes all the issues required by the reference standard, including improvement objectives for the various company processes. The audit plan includes audits on all company processes; at the moment, the internal audit on the procurement and design processes has not been performed yet. This has been scheduled for next week.) Rev. 3 10/26

11 - Details of Stage 2 audit agreed with the client The established date and duration must be indicated for the stage 2 audit or, alternatively, the minimum time before the stage 2 audit. Furthermore, if it is possible to plan, indicate that the stage 2 audit plan has been prepared (see also 4.7). EXAMPLE (The stage 2 audit plan was defined during the audit. Two man days were defined, as established in the offer. The audit plan was agreed and delivered to the company. On the basis of the scope indicated in the manual, the number of employees declared by the organisation and the list of current operative sites, the audit was planned on two operative sites so as to confirm the scope of certification requested by the organisation) Rev. 3 11/26

12 4.6 Info sheet The judgement on the level of sufficiency of the QMS documentation and on the Quality System Management Review must be indicated in the relative spaces. - Review of each process defined by the Organization (Definition of objectives and programme, significant environmental aspects and performances, verification of applicable legislation) The fact that each company process has been evaluated at document level must be highlighted and a brief description of the findings for each process defined by the company must be given. A synthetic description must be made of the various main and supporting processes defined by the company. EXAMPLE 1: (Ex: A comment on each process defined by the organisation is indicated below. In general, the processes are sufficiently monitored using indicators that are checked with pre-established frequencies. If any indicators fail their objectives, the organisation implements corrective action. Rev. 3 12/26

13 - Management process: the organisation has defined a dashboard indicating all company indicators. In particular, the process is monitored using two indicators: Costs of quality/non quality and pro-capita turnover. - Quality management process: this process is managed by the quality manager. The documents are managed and distributed via the Internet. The process is monitored by the indicator: reissue of documents due to errors - Human resources management process: the process owner is the human resources manager. The process is monitored using the following indicators: staff motivation index, employee training hours, respect of training plans - Contract and tender management process: the process is well-structured and monitored using the following indicators: job turnover and margins, response times to offer requests, contracts/tenders out of offers made - Purchase process: supplier performance is monitored using the following indicators: respect of delivery times and quality of supplied product - Production process: the process owner is the Site manager. The process is monitored using: number of complaints, number of non-conformities, respect of delivery times, respect of work plans) EXAMPLE 2 The organisation's QMS is applied for the realisation of. The company has defined the main processes related to product realisation and support processes for QMS management. The audited system seems well/sufficiently implemented and shared (check interviews with staff). The managers and operative staff seem technically well-prepared and aware of their role inside the company and inside the QMS. - Sales process: The indicator system allows activities to be closely controlled. The data analysed reveal a trend to achieve annual objectives - Design process: the staff is technically well-prepared on design methods and techniques and understands the utility of the QMS tools (planning design and final results, reviews, audits, validation activities) - Purchase process: The products/services are purchased from long-standing suppliers whose validity has been tested over the years. The effectiveness of the system is proved by the reduction in the number of supplied product NC's. - Production process: by means of the competence of staff as concerns their products and the awareness of the importance of each single production stage for the subsequent activity, the process is effectively under control. The above is confirmed by the reduction in the number of complaints. - Assessment of the Organisation's Management System and of the client's level of understanding of the requirements of the standard An overall judgement on the QMS must be made, as regards both conformity with the standard and its suitability for the realisation of the company product, as well as reference to the organisation's understanding of the QMS. EXAMPLE: (The organisation's QMS appears to be well-structured at document level, suitable for the realisation of company products and compliant with the reference standard. The persons interviewed showed a good knowledge and awareness of the utility of the QMS) Rev. 3 13/26

14 4.7 Conclusions - Remark and supplementary notes Compile as usual with additional information, such as production with more than one shift, presence of observers, organisation's membership of national or international groups, etc.. IMPORTANT NOTE If possible, prepare the stage 2 audit plan (one of the outputs required by ISO 17021: 2011) at the end of stage 1 and give it to the customer. In the space, indicate that the stage 2 plan has been agreed with the customer. Alternatively, explain why this has not been done. If the execution of stage 2 has been planned consecutive to stage 1 and the stage 2 audit plan has already been given to the customer, indicate as a stage 1 output if the stage 2 audit plan has been confirmed or modified. Rev. 3 14/26

15 5. STAGE Audit list Record any type A, B or C observations deriving from the stage 2 audit for each point of the reference standard. When formulating the findings, take care to explain the contents, findings and reasons for the findings to the Technical Committee (always record objective evidence, the point of the reference standard and describe the finding in detail - See: Instruction to technicians: FORMULATION OF FINDINGS IN THE AUDIT REPORT ACCEPTANCE OF THE ORGANISATION S PROPOSALS ) It is necessary to record verification of all points of the standard ( Check All ). Rev. 3 15/26

16 5.2 Company & Representatives As usual, the interviewees must be entered. Also in this case, it s necessary to indicate the names of those who participated in the initial and final briefings and the name of the person who will sign the report (see 4.2). 5.3 Production site Company data (scope, address, name) have already been entered in the Main field by the Secretary, though it is always best to check them. In the field Extension reasons, it s necessary to explain the reasons for any extension (certification scope, site, etc.). It is recalled that a clarification or detail related to the scope of certification is not considered as an extension. In the field Any other activities checked, it is necessary to record activities checked which are not carried out at a specific site, together with the address (i.e. activities checked related to transport of waste, goods, on regular routes, etc.). The entries indicated in this field are transposed in the form Summary of activities/sites which summarises what has been checked in the three-year period. Rev. 3 16/26

17 In the Other sites field, enter the data related to the permanent sites (Were any permanent sites audited on the basis of a site sampling procedure? ) or to external activities ( Was the activity carried out outside the organisation audited (e.g. centres providing the service)? ) checked. In the case of multi-sites, it s necessary to select Yes and, using Add existing op. unit, select the sites sampled during the audit, already uploaded by the secretariat; if the sites are not available, they are to be uploaded through Add new op. unit (in the case of surveillance and recertification they should already be present as they have to appear on the certificate). In the case of organisations with several permanent sites but which don t come under the definition of multi-site, it s necessary to select N:A, but it is in any case necessary to select the sites subject to audit using Add existing op. unit to give evidence of the audited sites. In the case of non applicability, tick N.A. in any case. The entries indicated in these fields are transposed in the form Summary of activities/sites which summarises what has been checked in the three-year period. Rev. 3 17/26

18 5.4 Reference documents - Any requirements of the standard rule excluded Enter confirmation of the verification of the reasons why any exclusions as per applicable standards have been accepted. EXAMPLE: successfully checked as the company realises the works according to drawings, projects and documents provided by the customer successfully checked as the organisation can only sells products on behalf of third parties to which it commissions design activities f) - successfully checked as assistance is not a contractual requirement successfully checked as all the controls performed by the organisation are of the visual/documental type and therefore do not require the use of physical/mechanical measurements or write Following the audit and analysis of the processes and scope, we consider the exclusion of point. to be acceptable, as justified in the Manual - Any other activities that are checked on a documental basis It is necessary to record all documentary evidence collected in support of the activities indicated in the scope. The entries indicated in this field are transposed in the form Summary of activities/sites which summarises what has been checked in the three-year period. Rev. 3 18/26

19 5.5 Audit report - Closure of recommendations (C) found during previous audit Enter a comment on the implementation of any observations deriving from the stage 1 audit or from the previous surveillance or renewal audit. Rev. 3 19/26

20 5.6 Info sheet - Results of the audit for each process identified by the Organisation and assessed during the audit (evidence, strong points and positive elements) It must be indicated that the application of the QMS has been verified on all the processes indicated in the audit plan and defined by the organisation. For each audited process, make a brief comment on the assessed process, including significant objective evidence of what has been verified as, for instance, the identification number of the verified jobs, of the verified drawings and description and/or identification of the verified components, number and identification of the verified purchase orders, examined documents, reference to the various examined records, etc. It is very important to insert an explanation of the reasons that made it necessary to issue a Non Conformity. This is possible by describing well the findings that led to the issuing of the Non Conformity and explaining how that Non Conformity can constitute a danger for the application of the Quality Management System. Also give a brief global assessment of the QMS. Rev. 3 20/26

21 EXAMPLE 1 The auditing team considers the organisation's management system adequate and well-applied; in particular, the interviewees showed a high level of competence and awareness of the processes in which they are involved and a good awareness of the rules of the management system. (Or: we consider that the QMS is adequate for company production even though there is room for improvement, considering the recent application). During the audit, the following company processes, defined by the organisation and indicated on the audit plan, were verified: quality, design, procurement, staff, production, management. - Management process: company management is very involved in system management and improvement; every three months, it is given the progress of the indicators for each process. - Quality management process: the quality process is well structured. The process indicators are adequately monitored. The audit plans are respected. The internal audits revealed no serious non-conformities and identified methods for improving the QMS. Particular importance was given to the management of customer complaints which, furthermore, have significantly reduced during the last two years. - Production process: the production process is suitably controlled via the production order which follows the product up to final testing. The machine operators interviewed demonstrated technical competence and awareness of applicable basic statistical techniques. The efficiency and effectiveness of the production process are kept under constant control, partly through the percentage of internal waste. Staff showed mastery of production techniques. The production of material as per production order 755/08 was successfully audited. It was found, however, that component ZZZ, which proved to be non-conforming after machining, was not adequately identified as such, in disagreement with company procedures; considering the type of component this improper identification make a danger of involuntary utilization. Following this observation a non-conformity was issued. - Staff process: the process is well structured. Training plans are respected. Staff motivation and awareness assessed annually using special questionnaires. Suitable skills have been established for the various company functions and are assured by means of a training plan. The records of training to personnel of the new computer aid machines have been verified - Design: the interviewed staff showed competence as regards design methods and automotive core tools (Fmea, control plans, Ppap, etc.). The management of the design of the new reducer HRID 75 has been verified, in particular checking the record of verification, review and validation of the design. - Procurement: the organisation only uses suppliers with certified quality systems. Supplier performance is monitored by assessing delivery times and quality of the supplied product. The management of the order ORD 23/2008 relevant to some steel bars, to supplier PICO PALLINO has been verified. - Improvement opportunities Any QMS improvement opportunities must be indicated, even if not closely connected with the recommendations indicated in the report (Type-C observations). EXAMPLE: (Though the documentation management methods comply with the reference standard, it would be best to reduce the quantity of documents used in order to better align it to company requirements. It would be good to reassess the calibration criteria used for measuring tools in order to reduce calibration costs; rationalise and standardise components; check the calibration frequency of the measuring instruments bearing in mind the effective percentage of use; etc.). Rev. 3 21/26

22 - Degree of effectiveness of the internal audits/management reviews The reasons for the level of effectiveness of the internal audits and of the QMS review must be indicated; it is not sufficient to make an assessment such as good or can be improved EXAMPLE: (The internal audits were performed by suitably trained company staff according to a preestablished plan on all company processes and showed the quality system is applied well, even though performance criteria can be improved (see recommendation n 4). The 8 non-conformities found were discussed with the process managers who have already proposed, and in some cases implemented, the corrective action considered to be appropriate. The QMS management review was updated on 20/09/2008 and considers all the issues indicated in ISO 9001: 2008 and the analysis of all QMS aspects, including the analysis of the company improvement objectives for the current year and internal audit results). The review indicates the new improvement objectives established by management. Rev. 3 22/26

23 5.7 Conclusions - Remark and supplementary notes Compile as usual with additional information, such as production with more than one shift, presence of observers, organisation's membership of national or international groups, etc.. Rev. 3 23/26

24 6. PERIODIC SURVEILLANCE / RECERTIFICATION AUDITS 6.1 Report The layout of the stage 2 audit report must be used to prepare the surveillance and recertification reports. When filling in a report related to a periodic surveillance audit, it must be remembered that the organisation is already certified, therefore, any changes to the QMS, achievement or not of the improvement objectives and comparison with the results of previous audits are to be highlighted in particular, by filling in the fields as indicated in point 5 above. If no changes have been made, this is to be indicated. 6.2 Audit list It is necessary to record the closure of findings highlighted during the previous audit ( Correct All ). Rev. 3 24/26

25 6.3 Audit report - The audit team has checked the status of objectives and goals to be reached in the three-year period and the performance indicator trend This space appears and is to be filled in in the case of recertification audit only A comment must be made concerning the improvement of the QMS by the organisation in the three previous years. To do this, examine the progress of the process indicators and the achievement or not of the objectives established by company management, and the trend of these indicators. It is also necessary to verify the progress of the non-conformities (both internal and external) both as regards number and merit, any customer complaints, and anything else that can be an index of improvement. In this field it is necessary to include also the information and comments relative to possible claims received by RINA from third parties about the certified organization. EXAMPLE: (From the examination of the record of the Quality System Management Review it is possible to verify that the improvement objectives have always been achieved with the exception of those relative to the building of a new production line that, at the moment, has been delayed awaiting a general reorganization of the company. In particular, it has been verified that some indicators relative to the main processes give a positive trend of the improvement over time of performance. During the last three years, RINA received a complaint regarding the Company XXX relative to the delivery of incomplete materials; the correct management of this complaint by the company has been checked. The causes of this complaint have been analyzed by the company with the conclusions that, as this case was a single episodic case due to non perfect management of the dispatch, no corrective actions on the quality management system have been judged necessary.. During the last three years, no complaints or information have been received by RINA on the company or on their products. Rev. 3 25/26

26 6.4 Info sheet - Degree of implementation of the Organization's Management System, comparing it with the result of previous audits (when applicable) An assessment must be made concerning the improvement of the QMS over time. This can be derived from previous audit reports. It is therefore indispensable to check the previous RINA audit reports or those of any other certification bodies. EXAMPLE: (The reduction in the NCs denotes an improvement in the application of the system; some company procedures have been modified, etc.. The monitoring results of the objectives established for the various process indicators show an improvement with respect to the previous reports. The objective (OOO) for indicator xxx has not been achieved; the company has established and applied improvement activities.) 7. SPECIAL FIELDS There is a special field for sector EA28 where the data concerning the sites audited are to be recorded. The entries indicated in this field are transposed in the form Summary of activities/sites which summarises what has been checked in the three-year period. Rev. 3 26/26