EUROPEAN COMMISSION DIRECTORATE-GENERAL FOR HEALTH AND FOOD SAFETY

Transcription

1 Ref. Ares(2017) /03/2017 EUROPEAN COMMISSION DIRECTORATE-GENERAL FOR HEALTH AND FOOD SAFETY Health and food audits and analysis DG(SANTE) MR FINAL REPORT OF AN AUDIT CARRIED OUT IN AUSTRIA FROM 05 SEPTEMBER 2016 TO 14 SEPTEMBER 2016 IN ORDER TO EVALUATE THE SYSTEM PUT IN PLACE TO IMPLEMENT ARTICLE 4(6) OF REGULATION (EC) NO 882/2004 (NATIONAL AUDIT SYSTEM) In response to information provided by the competent authority, any factual error noted in the draft report has been corrected; any clarification appears in the form of a footnote.

2 Executive Summary This report describes the outcome of an audit carried out by the European Commission s Directorate-General for Health and Food Safety in Austria from 5 to 14 September The objective of the audit was to evaluate the systems put in place to implement Article 4(6) (on audits of competent authorities) of Regulation (EC) No 882/2004 of the European Parliament and of the Council, on official controls performed to ensure the verification of compliance with feed and food law, animal health and animal welfare rules. Overall the report concludes that Austria has put in place internal audit arrangements for the majority of controls falling under the scope of Regulation (EC) No 882/2004, with the exception of controls on the use of plant protection products and activities of the Federal Ministry of Health and Women directly related to official controls. For control activities on food safety and veterinary controls a national system has been established which involves all Länder. This system has a high level of acceptance by the auditees from the Länder. An internal audit system covering feed controls carried out by the Federal Office for Food Safety was at the time of the DG SANTE audit in an advanced stage of preparation. The Department 'Stabstelle' of the Federal Ministry of Health and Women is entrusted with the independent scrutiny of the audit systems in place for food safety and veterinary controls. At the time of the DG SANTE audit, the Stabstelle was still involved in the audit process and, therefore, was not completely independent from the audit process. Appropriate arrangements are largely in place for the implementation of the audit process and follow-up of audit findings. However, audit reports, in particular in the food sector, do not in many cases contain sufficient information, that would allow those who have not participated in the audit to understand the results of the audits and to identify system-wide strengths and weaknesses. The combination of system audits and sectoral audits gives assurance that planned arrangements are effectively implemented, although suitability' is not yet formally covered by the audit arrangements. A review of the audit process is carried out in the food sector, but is not well documented and does not provide sufficient guarantees that all relevant aspects are covered. For the veterinary sector there is no review of the audit process in place. Only very limited information has been made available to external stakeholders in relation to audit activities and results. This represents a lost opportunity to provide assurance to external stakeholders regarding the quality of official controls provided by the audit activity. The report makes recommendations, addressed to the relevant competent authorities, aimed at rectifying the identified shortcoming and further enhancing the control measures in place. I

3 Table of Contents 1 Introduction Objectives and scope Legal Basis Background Previous DG SANTE Audits Context: Article 4(6) of Regulation (EC) NO 882/ Methodology Findings and Conclusions Competent Authority Legal requirements Findings Responsibility for performance of official controls Responsibility for audits of official controls Audit arrangements Independence Legal requirements Findings Independent scrutiny Legal requirements Findings Auditor competence Legal requirements Findings Development of the programme of audits Legal requirements Findings Implementation of the audit process Legal requirements Findings Follow-up of audit recommendations Legal requirements Findings...20 II

4 5.2.7 Transparency Legal requirements Findings Challenges reported by the Competent Authority Overall Conclusions Closing Meeting Recommendations...24 III

5 ABBREVIATIONS AND DEFINITIONS USED IN THIS REPORT Abbreviation AG Audit AGES AG-QM ASVF BAES CA BMLFUW BMGF DG SANTE MANCP NAS Network PPP Stabstelle Explanation Working Group 'Audits' Austrian Agency for Health and Food Safety Working Group for Quality Management Audit system for issues related to veterinary controls and food safety Federal Office for Food Safety Competent Authority Federal Ministry of Agriculture, Forestry, Environment and Water Management Federal Ministry of Health and Women European Commission s Directorate General for Health and Food Safety Multi Annual National Control Plan (MIK Mehrjähriger Integrierter Kontrollplan) Network of Member States National Experts on National Audit Systems Plant Protection Products BMGF Department "Koordination MIK und AGES" IV

6 1 INTRODUCTION This audit took place in Austria from 5 to 14 September The audit team comprised two auditors from the European Commission s Directorate General for Health and Food Safety (DG SANTE) and a national expert from an EU Member State. The opening meeting was held on 5 September 2016 at the Federal Ministry of Health and Women (BMGF) with representatives from federal and provincial Competent Authorities (CA). At this meeting the audit team confirmed the objectives of, and itinerary for, the audit, and obtained additional information required for its satisfactory completion. 2 OBJECTIVES AND SCOPE The objective of the audit was to evaluate the system(s) put in place to implement Article 4(6), on audits of competent authorities, of Regulation (EC) No 882/2004 of the European Parliament and of the Council, on official controls performed to ensure the verification of compliance with feed and food law, animal health and animal welfare rules 1 (hereafter: Regulation (EC) No 882/2004). The criteria used for the evaluation are set out in Article 4(6) of Regulation (EC) No 882/2004: Competent authorities shall carry out internal audits or may have external audits carried out, and shall take appropriate measures in the light of their results, to ensure that they are achieving the objectives of this Regulation. These audits shall be subject to independent scrutiny and shall be carried out in a transparent manner. In addition, where applicable, the audit team took into account Commission Decision 2006/677/EC 2 setting out the guidelines laying down criteria for the conduct of audits under Regulation (EC) No 882/2004 (hereafter: Commission Decision 2006/677/EC). Where relevant, reference was made to Network Reference Documents produced by the Network of Member States National Experts on National Audit Systems (hereafter: the NAS Network), while recognising that they do not constitute an audit standard and are not legally binding. 3 LEGAL BASIS The audit was carried out under the general provisions of EU legislation and, in particular Article 45 of Regulation (EC) No 882/ Regulation (EC) No 882/2004 of the European Parliament and of the Council of 29 April 2004 on official controls performed to ensure the verification of compliance with feed and food law, animal health and animal welfare rules, Official Journal L 165, , pages 1 to 141, corrected and re-published in OJ L 191, , pages 1 to /677/EC: Commission Decision of 29 September 2006 setting out the guidelines laying down criteria for the conduct of audits under Regulation (EC) No 882/2004 of the European Parliament and of the Council on official controls to verify compliance with feed and food law, animal health and animal welfare rules. Official Journal L 278, , pp15 to 23. 1

7 Full EU legal references are provided in Annex 1. Legal acts quoted in this report refer, where applicable, to the latest amended version. 4 BACKGROUND 4.1 PREVIOUS DG SANTE AUDITS Detailed information on the structure and organisation of the Austrian competent authorities can be found in the Country Profile for Austria at: DG SANTE has carried out numerous inspections and audits in Austria, the reports of which can be found at: While the topic of the current audit has not been the specific objective of any previous audit, the subject has been considered within the scope of numerous sectoral audits. At the time of writing, there were no open recommendations to Austria in relation to the application of Article 4(6). 4.2 CONTEXT: ARTICLE 4(6) OF REGULATION (EC) NO 882/2004 The requirements laid down in Article 4(6) of regulation (EC) No 882/2004, that: Competent authorities shall carry out internal audits or may have external audits carried out, and shall take appropriate measures in the light of their results, to ensure that they are achieving the objectives of this Regulation. These audits shall be subject to independent scrutiny and shall be carried out in a transparent manner should be read together with the definition of Article 2(6) laid down in the same Regulation: Audit means a systematic and independent examination to determine whether activities and related results comply with planned arrangements and whether these arrangements are implemented effectively and are suitable to achieve objectives. Further guidance on certain aspects of the requirement and definition is provided in Commission Decision 2006/677/EC. In particular the guidelines in the Annex to this Decision provide information on aspects to be considered when ensuring that the audit process is systematic, transparent, independent and subject to independent scrutiny. In addition, guiding principles in relation to compliance with planned arrangements, effective implementation of arrangements and their suitability to achieve objectives are provided. Guidance is also provided in relation to audit reporting, follow-up of the audit outcome, audit review and dissemination of best practice, resources and auditor competence. 2

8 As reflected in the recitals of the decision: The guidelines are not binding but serve to provide useful guidance to the Member States in the implementation of Regulation (EC) No 882/2004. The NAS Network is a network of officials (auditors) from national competent authorities, responsible for the performance of audits of official control systems as provided for by Article 4(6) of Regulation (EC) No 882/2004. The Network meets regularly, under the chairmanship of, and facilitated by, DG SANTE to exchange experiences in implementing national audit systems on official control activities. During the course of these exchanges; discussions, workshops etc. good principles and practices are identified and agreed by the network. To enable dissemination of information the Network, working in plenary session and through sub-groups, facilitated by DG SANTE, consolidate agreed principles and good practices on specific topics into Network Reference Documents. In relation to NAS Network, at the time of this audit the NAS Network has produced the following Network Reference Documents: Audit Evidence - October 2015 Auditing Effectiveness of Official Control Systems - February Developing Objectives and Indicators April Independence and Independent Scrutiny February Risk Based Planning for Audits of Official Control Systems - February These documents may be used as reference documents; however, they do not constitute an audit standard and are not legally binding. 4.3 METHODOLOGY The evaluation process of the current DG SANTE audit consisted of: an initial desk study phase in which the relevant information already available in DG SANTE was collated and analysed. This included the Austrian Multi-Annual National Control Plan and associated Annual Reports, the Country Profile of Austria and sectoral audit reports; examination of certain documentation provided by the CA prior to the audit; and meetings with the CAs and auditors from the Länder responsible for carrying out audits under Article 4(6) and auditees. The DG SANTE audit team evaluated arrangements in place and verified their application through examination of a variety of evidence, including documentation of the audit programme development and its implementation. Observing the performance of individual auditors during audits was not included in the scope as the DG SANTE audit team considered that the effectiveness of an individual audit can be 3

9 better judged by DG SANTE s sectoral Units auditors in the context of relevant sectoral audits. The evaluation focussed particularly on those elements which the audit team considered essential to ensure the audit bodies can produce reliable audit results, with adequate coverage of official controls, to give assurance that the objectives of Regulation (EC) No 882/2004 are being met: Responsibilities for the implementation of Article 4(6); Status and reporting lines of auditing bodies/units; Arrangements for independent scrutiny; Procedures for the selection of auditors and management of auditor competence; Procedures for the development of audit programmes, with particular attention on how an adequate coverage of the audit/risk universe is ensured; Planning, conduct and reporting of audits, including the approach to auditing the suitability of arrangements in place for official controls to achieve the objectives of the Regulation; Follow-up of audit recommendations including the system in place for corrective action in cases where problems are identified during the audit activities; and How and to what extent transparency is ensured. In addition, the audit team gathered information on particular challenges faced by the CA when implementing Article 4(6). 5 FINDINGS AND CONCLUSIONS 5.1 COMPETENT AUTHORITY Legal requirements Article 4(6) of Regulation (EC) No 882/2004 states, that Competent authorities shall carry out internal audits or may have external audits carried out Article 2(6) of the same Regulation defines audit as a systematic and independent examination to determine whether activities and related results comply with planned arrangements and whether these arrangements are implemented effectively and are suitable to achieve objectives Findings Responsibility for performance of official controls 1. For full details of the division of all relevant competences between all involved Competent Authorities, see the country profile for Austria at: 4

10 2. In summary, official controls are organised as follows: The BMGF and the Federal Ministry of Agriculture, Forestry, Environment and Water Management (BMLFUW) are the main federal authorities in the areas of food and feed safety, animal health and plant health. The federal level is responsible for legislation on food and feed safety, animal health, animal welfare and plant health and coordinates the Multi-Annual National Control Plan (MANCP: MIK Mehrjähriger Integrierter Kontrollplan). Implementation of legislation falls either under the responsibility of the relevant competent Federal Ministers itself or the Länder. In the case of indirect administration, the federal level can issue instructions e.g. in the form of Decrees (e.g. animal health and food safety), which is not the case for direct administration (e.g. animal welfare). The Länder are in most sectors exclusively responsible for implementing legislation. The Länder are also responsible for the preparation of implementing provisions and the performance of controls on the use of PPPs. The responsibility of the federal level is limited to the preparation of the PPP framework law. There is no harmonised system for controls on the use of PPPs across the Länder. The relevant competent Federal Ministers are in charge of import controls, controls on the commercial production of feed and marketing of plant protection products. The competent body for providing scientific advice and laboratory services is the Austrian Agency for Health and Food Safety (AGES). The Federal Office for Food Safety (BAES) was established to enable the execution, as the authority of first instance, of laws for agricultural production such as for example for controls on the commercial production and marketing of seed and feedingstuffs and control on the marketing of PPPs. Controls are carried out by AGES staff which are permanently allocated to this task. At Länder level, control responsibilities are shared between the Veterinary Services, the Food Inspectorates and Plant Protection Services of the Länder Responsibility for audits of official controls Audit system for issues related to veterinary controls and food safety (ASVF) 3. For those areas where the national legislation entrusts the Länder with the responsibilities for controls, either through 'indirect' or 'direct' administrative functions, the Länder are also responsible for the organisation of audits, including food safety and veterinary controls. 4. In 2005 a nationwide audit system was established for food safety and in 2009 for all aspects of veterinary controls. 5

11 5. In 2010 the BMGF department "Koordination MIK und AGES" (Stabstelle) was entrusted with the independent scrutiny. At the time of the DG SANTE audit the Stabsstelle had the following tasks: Development and updating of the 'Orientierungsmatrix. This is a table which provides, based on expert opinions, risk-based guidance for the preparation of the annual audit programmes by the audit bodies. Monitoring of the audit programme. Elaboration of and keeping up-dating the audit handbook. Preparation of the annual audit evaluation report. 6. The annual audit programme, audit procedures and checklists are developed by audit specific working groups which are, according to the BMGF, the audit bodies. In the case of food safety the working group for quality management (AG-QM), which is chaired by a representative from one of the Länder, is the audit body and in the case of veterinary controls it is the steering group (Steuerungsgruppe), chaired by a representative from the BMGF. 7. The representatives of the Länder are informed about the audit process at dedicated meetings several times a year. The audit system, the audit programme and any changes including procedures and checklists are endorsed by the joint conference of the heads of the food safety and veterinary departments of the Länder, with the exception of the 'Orientierungsmatrix' which is only brought to the attention of the joint conference. 8. The steering group is assisted by technical working groups, in particular with regard to the elaboration of sector specific audit check lists. 9. The Stabstelle is supported by the working group 'audits' (AG Audit) which consists of two nominated experts of the Länder from each of the sectors (veterinary and food safety), representatives from the BMLFUW and the Austrian Agency for Health and Food Safety (AGES) and the chairpersons of the two audit working groups. 10. The audits are performed by a pool of auditors from the Länder. 11. Auditors from several Länder and representatives of CAs met in two Länder stated that the ASVF has a high level of acceptance and also mentioned the good cooperation between the Stabstelle and the Länder. Use of PPPs 12. The DG SANTE audit visited two Länder and noted that one of these Länder did not have an audit system for official controls on the use of PPPs in place at all. The other Land carried out internal audits in relation to processes defined within their QM system, which covered the registration of PPP users, but no other aspect of PPP controls. The federal level does not have an overview of the audit activities of the Länder with regard to controls on the use of PPPs. 6

12 Control areas at federal level 13. The activities of the BMGF, which for most control areas issues instructions and guidelines and monitors the implementation of controls, are not subject to any audit. 14. The BMGF performs annual audits at frontier posts with the aim of verifying that import controls are carried out in compliance with EU provisions. These types of audits follow detailed checklists covering the border control facilities, control procedures and controls. In 2014 an external audit by an expert from another MS was carried out, which followed the same checklists. Given that BMGF issues instructions also for import controls, there is a risk of self-review and in particular if the audits would cover aspects of 'suitability of the planned arrangements'. 15. For control areas for which the BAES is responsible, the AGES was entrusted with the organisation of internal audits. Until 2015 AGES performed internal audits in the context of the accreditation to ISO which covered these sectors. As a result of the revision of the ISO AGES could not renew the accreditation to this standard. This, together with changes concerning the organisation of BAES triggered the establishment of the new audit system. This new audit system was, at the time of the DG SANTE audit, not yet fully operational, but audit arrangements were in an advanced stage of development. It was planned to carry out the first audits in the second semester Conclusions 16. Responsibility for auditing official controls are clearly allocated, with two exceptions: Auditing arrangements are not defined for controls on the use of plant protection products falling under the scope of Regulation (EC) No. 882/2004. The activities of the BMGF which are directly related to official controls including the planning of controls, monitoring the implementation of controls and issuing instructions for controls are not subject to any audits. 17. An audit system for feed controls at federal level was in an advanced stage of preparation but had not yet been implemented. 5.2 AUDIT ARRANGEMENTS Independence Legal requirements Article 2(6) of Regulation (EC) No 882/2004 defines audit as a systematic and independent examination to determine whether activities and related results comply with planned arrangements and whether these arrangements are implemented effectively and are suitable to achieve objectives. 7

13 In section 5.3 of the Annex of Decision 2006/677/EC further guidance is provided to Member States on independence, including: Audit bodies should be free from any commercial, financial, hierarchical, political or other pressures that might affect their judgment or the outcome of the audit process. The audit system, audit body and auditors should be independent of the activity being audited and free from bias and conflicts of interest. Auditors should not audit areas or activities for which they have direct responsibility. All relevant competent authorities should introduce safeguards to ensure that responsibility and accountability for audit and control activities, such as the management and supervision of official control systems, are kept sufficiently distinct. In addition, the NAS Network Reference Document on Independence and Independent Scrutiny provides additional guidance to Member States on threats to independence and mitigating measures Findings 18. The two working groups (AG-QM and Steering Groups) have been mandated by the joint conference of the provincial heads of the veterinary and food safety services to act as audit bodies. 19. The audit documents including the audit handbook have been approved by the joint conference of the heads of the provincial veterinary and food safety services. The audit handbook defines the conditions that auditors have to fulfil, inter alia, independence of the auditors. According to the audit arrangements auditors are not allowed to perform audits in the Land where they are employed. 20. All auditors met showed a clear understanding of potential risks related to impartiality and conflicts of interests. They stated that the Austrian public service law provides guarantees regarding impartiality and conflict of interest. All auditors met confirmed that their management supports and respects their work as independent auditors for the ASVF. 21. The audit system operated by AGES in relation to controls carried out by BAES is based on a service level agreement and arrangements had been put in place to ensure the independence of the audit body and the auditors. 22. The AGES auditors are not involved in BAES control activities, although BAES is using staff employed by AGES for the performance of the controls. 23. The AGES is a private, state-owned company and BAES cannot give instructions to AGES. 24. In the case of the veterinary audits and AGES audits the auditors have to sign a declaration that they are free from interest of conflicts and are impartial. Although this is not based on legal provisions, it was considered by the steering group and AGES as a 8

14 useful instrument to remind auditors prior to the start of the audit to check whether there are potential threats regarding this matter. 25. In the case of audits of import controls at frontier posts, the auditor who is working at the BMGF, is also involved in the organisation of the import control system including drafting import control procedures and instructions. Although this has an advantage when auditing compliance with planned arrangements, there would be a risk of 'selfreview' when auditing the suitability of the planned arrangements. However, 'suitability' was not covered by the audit system for import controls at the time of the DG SANTE audit. Conclusions 26. Sufficient arrangements are in place to ensure independence of the audit bodies and the auditors. Auditors demonstrated a good understanding of threats to impartiality and of conflict of interests Independent scrutiny Legal requirements Article 4(6) of Regulation (EC) No 882/2004 requires that the audits shall be subject to independent scrutiny. In section 5.4 of the Annex of Decision 2006/677/EC further guidance is provided to Member States on the Independent Scrutiny of the Audit Process : In order to check whether it is achieving its objectives, the audit process should be subject to scrutiny by an independent person or body. Such independent person or body should have sufficient authority, expertise and resources to carry out this task effectively. The approaches to independent scrutiny may vary, depending on the activity or the competent authority. Where a body or a committee has been established with a view to independent scrutiny of the audit process, one or more independent persons should be members of such body or committee. Such independent persons should have access to the audit process and be empowered to contribute fully to it. Action should be taken to remedy any shortcomings identified in the audit process by the independent person or body Findings 27. The Stabstelle has been established outside the line management structures of the BMGF and is only responsible to the Head of the Section II of the BMGF (Legal Division). 28. The BMGF stated that the Stabstelle ensures together with BMGF experts (who participate as independent observers in audits of the ASVF) the independent scrutiny of the audit process. In this function the Stabstelle monitors the implementation of the audit plan, contributes to the revision of the audit instruction (e.g. audit handbook), 9

15 coordinates the 'Orientierungsmatrix' and prepares the annual evaluation report for the audit process which is part of the annual report of the MANCP. 29. BMGF, BMLFUW and representatives from the audit bodies stated that the Stabstelle, AG-QM and Steuerungsgruppe closely cooperate in order to identify areas for further improvement and gave some examples. However, the DG SANTE team noted that the results of the independent scrutiny were not systematically documented with the exception of the evaluation report, which is a two-page summary of the audit process and the main findings. However, this report does not provide an evaluation of the effectiveness and independence of the audit process and audit body or feed-back for continuous improvement. 30. The DG SANTE audit team noted that the BMGF experts who participate in audits (with a view to providing a contribution to independent scrutiny) may be involved in the issuance of instructions and may provide expert opinions during the audits. This was confirmed by some auditors. This may be of advantage when auditing compliance with planned arrangements and may facilitate the follow-up of audit recommendations by the BMGF experts. However, the DG SANTE team did not receive evidence that and how the participation of the BMGF experts in audits contributes to the independent scrutiny of the audit process. 31. The BMGF also stated that their understanding of Section 5.4 of Decision 2006/677/EC and in particular of 'Such independent persons should have access to the audit process and be empowered to contribute fully to it' was that the body responsible for the independent scrutiny was entitled to actively contribute to the audit process. The DG SANTE team noted that the CA's view was triggered by the German language version which uses the term 'mitwirken' which could be understood as 'being involved'. The BMBF stated that it was foreseen to further improve the national audit system and to take the NAS Network documents into consideration (see also Section ). 32. The AGES stated that for the audit system on BAES's control activities it was planned to establish the independent scrutiny at the BMLFUW. The BMLFUW cannot give instructions to AGES and is not involved in the audit process, with the exception of its participation in the working group audits. Conclusions 33. The Stabstelle is actively involved in the audit process and can therefore not be considered to be independent from the audit process. Also the BMGF experts which participate in audits as independent observers (with the aim to contribute to the independent scrutiny) cannot be considered to be independent from the control system as they may be involved in issuing instructions. This represents a lost opportunity to provide a truly independent assessment regarding the quality of official controls provided by the audit activity. 34. The process of the independent scrutiny and the outputs of it are not sufficiently 10

16 documented. In the absence of appropriate documentary evidence it could not be demonstrated that the independent scrutiny has contributed to the improvement of the audit process Auditor competence Legal requirements Articles 2(6) and 4(6) of Regulation (EC) No 882/2004 do not lay down specific requirements regarding the competence of auditors. Article 6 of the same Regulation requires that staff performing official controls receive, for their area of competence, appropriate training enabling them to undertake their duties competently and keep up-to-date in their area of competence and receive regular additional training as necessary Section 6.6 of the Annex of Commission Decision 2006/677 provides guidance on auditor competence: Auditor competence and selection criteria should be defined under the following headings: o generic knowledge and skills audit principles, procedures and techniques; management/organisational skills, o specific technical knowledge and skills, o personal attributes, o education, o work experience, o auditor training and experience. It is essential to put a mechanism in place to ensure auditors are consistent and their competencies are maintained. Competencies required by audit teams will vary depending on the area they are auditing within the control or supervision systems. As regards the technical knowledge and skills required by auditors, the training requirements for staff performing official controls (Chapter 1 of Annex II to Regulation (EC) No 882/2004) should also be considered Findings 35. The audit handbook provides the following selection criteria for auditors: Auditors have to: have technical expertise in their field including at least one year work experience as a food inspector or official veterinarian and participate in specific training courses for auditors organised by the BMGF. 36. The DG SANTE audit team received an overview of the initial and follow-up training courses organised in the past 10 years in order to develop and maintain the audit competence. The courses covered ISO standards, Regulation (EC) No 882/2004 related 11

17 issues, reporting and interview techniques. Some of the auditors had also participated in Better Training for Safer Food courses on auditing. 37. There are about 40 auditors listed for the veterinary audits and about 30 for food safety audits. Specific expertise of auditors is considered by the working groups when selecting the auditors. Where specific knowledge is required an expert may join the audit team, but according to the BMGF and auditors, the use of such experts is the exception. The auditors interviewed during the DG SANTE audit stated that in some cases the BMGF independent observer s expertise was used (see section 5.2.2). 38. Before being entrusted with a lead for an audit, new auditors are supposed to join as second or third auditor in order to allow them to familiarise with the audit process. This is not a specific requirement but common practice according to BMGF. 39. There is no arrangement in place for the evaluation of the performance of auditors and no feedback on the audit performance is requested from the auditees. 40. AGES stated that that the audit body will select auditors based on expertise. Participation in audit training courses is a prerequisite for new auditors and each auditor, before leading an audit, will have to participate in an audit as second auditor. An experienced auditor will have to accompany the first lead audit. Conclusions 41. Appropriate arrangements are in place to ensure that auditors develop and maintain audit competence. Additional technical expertise can be used when needed Development of the programme of audits Legal requirements Article 3(1) of Regulation (EC) No 882/2004 requires that: Member States shall ensure that official controls are carried out regularly, on a risk basis and with an appropriate frequency, so as to achieve the objectives of this Regulation. The definition laid down in Article 2(6) specifies, inter alia, that audits should be systematic. In section 5.1 of the Annex of Decision 2006/677/EC further guidance is provided to Member States on the Systematic Approach, including: A systematic approach should be applied to the planning, conduct, follow-up and management of audits. To that end, the audit process should: be the result of a transparent planning process identifying risk-based priorities in line with the competent authority s responsibilities under Regulation (EC) No 882/2004, 12

18 form part of an audit programme that ensures adequate coverage of all relevant areas of activity and all relevant competent authorities within the sectors covered by Regulation (EC) No 882/2004 at an appropriate risk-based frequency over a period not exceeding five years, be supported by documented audit procedures and records to ensure consistency between auditors and to demonstrate that a systematic approach is followed In addition: Where more than one audit programme is envisaged within a Member State, steps should be taken to ensure that such programmes are effectively coordinated, so as to ensure a seamless audit process across the relevant competent authorities. The audit programme(s) should also cover all relevant levels of the competent authority s hierarchy Findings 42. According to the audit handbook every year audits are to be carried out in three Länder in order to cover all nine Länder in a three year cycle. Each audit comprises sector audits including one or two witness audits (one or two topics) and one system audit. 43. The Stabstelle has developed a table called Orientierungsmatrix which defines the audit universe (control areas as defined in the Austrian MANCP). The matrix sets the control areas in relation to the following evaluation criteria: urgency (time elapsed since the last internal audit, time elapsed since the last DG SANTE audit/external audit); results of previous audits; results from the controls; changes (e.g. political developments, reorganisations, revised or new legislation). 44. Every year in September the Stabstelle submits the Orientierungsmatrix to all MANCP contact points in the BMGF and AGES, namely the technical Units, with a request to complete the table by giving score points from 0 to 2 for each control area/criteria combination. 45. The Stabstelle may flag those areas that have been over or under represented over time taking into account the audits carried out in previous years. This is done with the aim to ensure that in a 5 year cycle all topics mentioned in the audit universe are covered at least in one audit. 46. The Stabstelle compiles the information and the output of this process is discussed at a meeting with the MANCP contact points before it is discussed in November at the joint conference with the heads of the veterinary and food safety services. The representatives of the Länder may propose changes to the Orientierungsmatrix and the Stabstelle may take them into consideration. 13

19 47. The finalised table is submitted by the Stabsstelle to the AG-QM and the Steuerungsgruppe with a request to elaborate the annual audit plan. The working groups have limited flexibility as they can only deviate from the agreed Orientierungsmatrix in justified cases. 48. The annual audit plans are discussed at the meeting of the joint conference. 49. According to the audit procedures established by AGES all control areas for which BAES is responsible should be covered in a five year cycle. It is planned to revise the audit plan annually. Conclusions 50. Austria has arrangements in place for the establishment of the annual ASVF audit programmes applying a transparent, systematic risk-based approach. 51. In general the audit programmes for the ASVF achieve a good coverage of topics, administrations and Länder Implementation of the audit process Legal requirements Article 2(6) of Regulation (EC) No 882/2004 states that Audit means a systematic and independent examination to determine whether activities and related results comply with planned arrangements and whether these arrangements are implemented effectively and are suitable to achieve objectives. In section 5.1 of the Annex of Decision 2006/677/EC further guidance is provided to Member States on the Systematic Approach, including: A systematic approach should be applied to the planning, conduct, follow-up and management of audits. To that end, the audit process should: o o o o be supported by documented audit procedures and records to ensure consistency between auditors and to demonstrate that a systematic approach is followed, include procedures for generating audit findings, including the identification of evidence of compliance and noncompliance, as appropriate, and for preparing, approving and distributing audit reports, include procedures to review audit conclusions, in order to identify system-wide strengths and weaknesses in the control system, disseminate best practice and ensure the monitoring of corrective and preventive actions, be monitored and reviewed to ensure the audit programme's objectives have been met and to identify opportunities for improvement. Section 6.1 of the Annex of Commission Decision 2006/677 provides guidance on implementation of the Audit Process: 14

20 To comply with the requirements of Article 4(6) of Regulation (EC) No 882/2004, the audit system should cover the following three points set out in Article 2(6): o o o Verification of compliance with planned arrangements in order to provide assurances that official controls are carried out as intended and that any instructions or guidelines given to staff carrying out the controls are followed. This may largely be addressed by document review, but will also require on-site verification. The audit team will require good generic audit knowledge and skills to address this audit objective. Verification of the effective implementation of planned arrangements. In order to assess effectiveness, that is the extent to which planned results are achieved, on-site operational implementation must be included. This should include an assessment of the quality and consistency of the controls and should involve on-site audit activities. The audit team will require the relevant technical expertise in order to address this audit objective. The audit system should also seek to assess whether the planned arrangements are suitable to achieve the objectives of Regulation (EC) No 882/2004, and in particular the single integrated multi-annual national control plan. This should include assessing the suitability of official controls, with regard, for example, to their frequency and the methods applied, having regard to the structure of the production chain(s) and to production practices and volume. The audit team should have substantial knowledge and understanding of system auditing, together with relevant technical input to address this audit objective. In order to determine whether the planned arrangements are suitable to achieve the objectives set out in (c) above, the following should be considered: Audit criteria should include strategic objectives stemming from Regulations (EC) No 178/2002 and (EC) No 882/2004 (including the single integrated multi-annual national control plan) and national legislation. The primary focus of audits should be the control arrangements relating to the critical points for control in the production chain(s). The emphasis should be on assessing whether planned arrangements are capable of delivering sufficient guarantees on (a) the safety of the endproduct(s) and (b) compliance with other feed and food law requirements and with animal health and welfare rules. In order to achieve this, audit(s) should where possible extend beyond and across administrative boundaries. The Network Reference Documents on Auditing Effectiveness of Official Control Systems (February Version 1) provides additional guidance to Member States on how the effective implementation and suitability of official controls in achieving objectives may be evaluated. 15

21 Findings Documented Procedures 52. The audit hand book of the ASVF (updated in 2016) and the document Orientierungsmatrix provide a general framework for the planning and implementation of audits, the follow-up of audit recommendations and for the selection of the auditors. 53. Procedures for the implementation of audits are contained in the QM system for food safety (VA ) and the veterinary audit programmes contain general instructions for the audit performance and the audit planning. 54. The Steuerungsgruppe, with the support from technical working groups, has developed for the veterinary audits a common reporting format and detailed questionnaires for the system and sector audits. 55. The AG-QM has developed checklists for the system audits and sector audits. The completed checklists form the audit report. There are no specific instructions for the preparation and conduct of foods safety audits. 56. The AGES's QM-system contains SOPs for the newly established (but not yet implemented) audit system with regard to PPPs and feedingstuffs. 57. The import control sector has procedures for internal audits of frontier posts in place with detailed checklists. 58. The DG SANTE team reviewed several ASVF audit reports and received evidence of information exchanges between auditors and auditees (letters and mainly exchanges). It was demonstrated that documented procedures were applied Compliance with planned arrangements 59. The objective of the audits, as defined in the audit handbook, is the evaluation of the services entrusted with control tasks in the areas of food safety and veterinary controls. The BMGF and auditors met confirmed that the main purpose of the ASVF audits was to determine whether activities comply with planned arrangements. 60. The veterinary audits are prepared by sending a pre-audit questionnaire to the auditee. In the absence of specific instructions for food safety audits, most auditors met did not request system or sector specific information from the auditee, with the exception, in some cases, of information on the food business operators/sites to be visited. 61. The individual audits are carried out within a period of two days with one day dedicated to system audits and one day to sector audits. The auditors met stated that these time constraints put pressure on the audit team, but all auditors met considered that the time was sufficient to perform the task. This was confirmed by the auditees met. 62. The DG SANTE audit team checked documents, including audit reports and procedures and interviewed 11 auditors from several Länder and noted that the arrangements in 16

22 place are in general suitable to detect failure to comply with planned arrangements. However, the DG SANTE team noted that the food safety audit reports, which follow a checklist approach, contain limited information (and partly incomplete sentences) which makes it very difficult for anybody who has not participated in the audits to assess and understand these reports. 63. Audits carried out by the BMGF on import controls sufficiently cover compliance with planned arrangements Verification of the effective implementation of planned arrangements and their suitability to achieve objectives 64. The ASVF focusses largely on auditing compliance with planned arrangements, but all sector audits include on-site activities in order to assess the quality of controls. 65. The DG SANTE audit team noted that with some exceptions ASVF auditors did not have a clear understanding of the meaning of ' suitability of planned arrangements' as referred to in Article 2(6) of Regulation (EC) No 882/2004. Also training courses had not picked up the issue and the issue was not described in the audit procedures. The auditors met were largely not aware of the Network Reference Document 'Auditing Effectiveness of Official Control Systems'. 66. The BMGF stated that the current audit system focuses on compliance with planned arrangements and that it had been foreseen to improve the audit system by incorporating aspects of effectiveness and suitability'. 67. The AGES stated that the effectiveness of control arrangements would be considered during the process of finalising the audit arrangements and the implementation of the new audit system. The AGES was aware of the NAS Network documents Audit reporting 68. Arrangements for reporting are in place for ASVF audits. The audit handbook defines the general principles for the different stages of reporting (draft and final) and the responsibilities for reporting. The draft report has to be submitted for comments to the auditee, at the latest 20 days after the end of the audit with a 30 day deadline for comments. 69. The food safety area works solely with checklists and does not draw up additional reports. The checklists contain for each process/question to be checked a box which allows to enter also free text. The DG SANTE team noted that the entries are minimalistic and are in some cases hardly understandable for anybody who has not participated in the audit. 70. The veterinary area works with a common reporting format which provides a detailed description of the issues checked and the findings made. The reports contain overall conclusions, but these are in most cases only a summary of the individual findings without clearly distinguishing between individual findings and systemic issues. 17

23 71. There are no arrangements in place to check ASVF audit reports for consistency and quality. Auditors are responsible for their own reports Review of audit conclusions and dissemination of best practice 72. The Stabsstelle prepares every year an evaluation report which is mainly a summary of the audits carried out and the main findings. However, this report does not provide a systematic review of the system-wide strengths and weaknesses of the control system nor does it include best practice examples. 73. The audit handbook permits auditors to mention best practice examples but there is no obligation for this. The joint conference recommended identifying and collecting information on best practice examples during audits. The BMGF stated that the discussion on this issue is still ongoing and no decision had been taken at the time of the DG SANTE audit Monitoring and review of the audit process 74. The implementation of the audit process regarding the ASVF is monitored by the Stabsstelle and by the chairpersons of the working groups. This includes the performance of the audits as well as reporting. 75. The CA stated that the AG-QM has established a review process which is done at a seminar with all auditors where the audit process and the output of the audits are discussed. Comments and points for improvement are collected and discussed and eventually lead to a suggestion for the improvement of the audit process. The simplification of the reporting by using checklists was given as one example. The review process is not well documented. The BMGF stated that no report or comprehensive document with the outcome of the review process was issued, except the evaluation report. 76. A review process is not in place in the veterinary area. 77. An evaluation of the auditors is not carried out and no feedback is requested from the auditees, which could potentially provide information on the functioning of the audit system. 78. The BMGF stated that further improvements to the national audit system were planned, but that a comprehensive reform was preferred rather than frequently disrupt the audit system by too many changes. Therefore, BMGF intended to await the results of this audit. 79. The development and implementation of the audit programmes are documented and the DG SANTE team received evidence that the audit programmes have been implemented in past years as planned. 18