Model-based comparative assessment of the Australian and European hygiene monitoring programmes for meat production 1

Transcription

1 SCIENTIFIC REPORT OF EFSA Model-based comparative assessment of the Australian and European hygiene monitoring programmes for meat production 1 ABSTRACT European Food Safety Authority 2, 3 European Food Safety Authority (EFSA), Parma, Italy The main purpose of this work is to quantitatively compare the efficiency of the microbiological monitoring programmes at process level of Australia and Europe, using a model-based approach. Based on data collected from a literature review and from the Australian and the European control programmes in meat production lines, a probabilistic model was built to describe contamination levels of Aerobic Colony Count (ACC) / Total Viable Count (TVC) and E. coli / Enterobacteriaceae in cattle and sheep. This model accounted for the most relevant differences between the control programmes, as well as for the main sources of variability. This quantitative tool was then used to simulate possible outcomes over one year and to compare the time response (i.e. time needed to detect the first alert) of the system for different scenarios of contamination. The main conclusion was that, despite the two monitoring programmes being different in certain aspects, an equivalent level of hygiene control can be achieved by calculating and adjusting the necessary M-values (i.e. microbiological load cut-off). KEY WORDS Process hygiene criteria, Monte Carlo simulations, QMRA, total viable counts, hygiene monitoring programmes, cattle, sheep 1 On request of the European Commission, Question No EFSA-Q issued on 4 June Correspondence: amu@efsa.europa.eu 3 Acknowledgement: EFSA wishes to thank the members of the AMU Working Group Experts for Data and Modelling Issues for 'AQIS' BIOHAZ Working Group: Mary Howell and Kostantinos Koutsoumanis; the members of the BIOHAZ Working Group on the assessment of the comparison of the Australian monitoring programme for carcasses to requirements in Regulation (EC) N O 2073/2005 on microbiological criteria on foodstuffs: Mary Howell, Geraldine Duffy, Lieven De Zutter, Miguel Prieto Maradona, Kostas Koutsoumanis, Terence Roberts, Sava Buncic, for the preparation of this EFSA scientific output; Antonia Ricci for the peer-review and EFSA s staff members Gabriele Zancanaro, Billy Amzal and Pablo Romero Barrios for the support provided to this EFSA scientific output. Suggested citation: European Food Safety Authority; Model-based comparative assessment of the Australian and European hygiene monitoring programmes for meat production. [52 pp.]. doi: /j.efsa Available online: European Food Safety Authority,

2 SUMMARY Process Hygiene Criteria (PHC) for carcasses are laid down in Regulation (EC) No 2073/ and its amendments 5. The microbiological PHC indicate the acceptance functioning of the production process setting indicative contamination levels above which corrective actions are required in order to maintain the hygiene of the food processing in compliance with EU food law. The Australian Quarantine and Inspection Service (AQIS) have asked the European Commission to formally recognise the equivalence of the Australian monitoring programme to the European requirements. The letter from the AQIS contains a description of the Australian microbiological monitoring programme at process level: particular attention was given to those points where differences between the Australian and the European system exist. The main points of divergence between Europe and Australia, i.e. sampling frequency of carcasses, timing when samples are taken and calculation of test results, have been investigated and their impact on the assessment of the process hygiene has been evaluated using a model-based approach. Based on data collected from a literature review and from the Australian and the European control programmes in meat production lines, a probabilistic model was built to describe contamination levels of Aerobic Colony Count (ACC) / Total Viable Count (TVC) and E. coli / Enterobacteriaceae in cattle and sheep. This model accounted for the most relevant differences between the control programmes, as well as for the main sources of variability. This quantitative tool was then used to simulate possible outcomes over one year and to compare the time response (i.e. time needed to detect the first alert) of the system for different scenarios of contamination, allowing for a quantitative comparison between the efficiency of the Australian and European monitoring programmes. The main conclusion was that, despite the two monitoring programmes being different in certain aspects, an equivalent level of hygiene control can be achieved by calculating and adjusting the necessary M-values (i.e. microbiological load cut-off). For example, when comparing the two programmes as described in the respective regulations in force for ACC in cattle (EU M=5 for excision and Australia M=4.5 for swabbing), significantly different outcomes of the hygiene assessment, expressed as predicted days to corrective action (EU) or hygiene alert (Australia), were observed. This conclusion was robust with respect to the baseline level of contamination or number of slaughterhouses. This means the two programmes as described are different. Nevertheless, the extent of the above difference is influenced by several parameters, among which the sampling methodology (excision / swabbing) is one of the most important and requires an adequate and equivalent M-value that needs to be estimated. 4 5 OJ L , p1-26 OL L , p

3 TABLE OF CONTENTS Abstract... 1 Summary... 2 Table of contents... 3 Background as provided by the European Commission... 4 Terms of reference as provided by the European Commission... 4 Assessment Introduction Objectives Material and methods General principles of the model-based assessment Data availability and collection Data from the monitoring programs Data from the literature Data cleaning AU dataset UK dataset Literature dataset Final database AU dataset EU dataset (United Kingdom data) Literature dataset Model-based assessment methodology Data exploration analysis of the literature data Model fitting to national control programme data Simulation-based comparative assessment Results Results of the literature data exploration Results of the model fit to the data from the monitoring programmes Results for the Australian monitoring programme Results for the UK monitoring programme Results from the simulation-based comparative assessment Comparative assessment of the monitoring programmes in cattle Comparative assessment of the monitoring programmes in sheep Discussion Conclusions Appendices A. Appendix: List of available papers B. Appendix: Data Dictionary C. Appendix: Sampling Sites On Beef Carcasses D. Appendix: Microbiological criteria in force in europe and australia Abbreviations

4 BACKGROUND AS PROVIDED BY THE EUROPEAN COMMISSION Process Hygiene Criteria (PHC) for carcasses are laid down in Regulation (EC) No 2073/ and its amendments 7. The microbiological PHC indicate the acceptance functioning of the production process setting indicative contamination levels above which corrective actions are required in order to maintain the hygiene of process in compliance with EU food law. The Australian Quarantine and Inspection Service (AQIS) have asked the European Commission to formally recognise the equivalence of the Australian monitoring programme to the European requirements. The letter from the AQIS contains a description of the Australian microbiological monitoring programme at process level: particular attention was given to those points where differences between the Australian and the European system exist. The main differences in the Australian monitoring programme compared to European requirements in Regulation (EC) 2073/2005 are about: Sampling frequencies of carcasses: - EU: The food business operators of slaughterhouses or establishments producing minced meat, meat preparations or mechanically separated meat shall take samples for microbiological analysis at least once a weak. The day of sampling shall be changed each week to ensure that each day of the week is covered. - AUS: Sampling frequency is based on the amount of the production (e.g. 1 sample per 300 bovine carcasses) but the system does not specify the day of sampling nor a requirement to rotate day to ensure that all days of the working week are equally presented. Timing when samples are taken: - EU: After dressing but before chilling. - AUS: After a minimum of 12 hours chilling. Calculation of test results: - EU: Daily mean logs are calculated - AUS: Daily means are calculated TERMS OF REFERENCE AS PROVIDED BY THE EUROPEAN COMMISSION EFSA is asked to identify and describe the differences between European and Australian monitoring programmes and to evaluate the impact of these differences on the assessment of the process hygiene. 6 7 OJ L , p1-26 OL L , p

5 ASSESSMENT 1. Introduction To address the mandate from the European Commission recalled above and its main issues, the EFSA BIOHAZ Unit formed a dedicated Working Group to issue a Technical Report. In this context, the Working Group experts expressed the need to implement a model-based comparative assessment as part of their evaluation. This modelling work was therefore carried out by a Working Group of the EFSA Assessment Methodology Unit, under the supervision of the BIOHAZ Working Group. 2. Objectives To remain consistent with the main Technical Report issued the BIOHAZ Working Group, only meat production was addressed by this assessment. More specifically, cattle and sheep were the two species for which the data allowed the comparison. The three outcome variables of interest were: Total Viable Count (TVC) / Aerobic Colony Count (ACC), Enterobacteriaceae / E. coli, and Salmonella spp. The primary objective was to build a probabilistic model allowing for the prediction of future outcomes of the compared control programmes over one year, various under different scenarios of baseline level of contamination. Such a model: - should account for most relevant factors impacting on the contamination levels (chilling, sampling techniques, season, ); - should account for the main sources of variability (variations between slaughterhouses, over time, between carcasses,...); - should use all the available data within the timeframe of the assessment, from official control programmes as well as from the scientific literature where necessary. According to both Regulation EC 2073/2005 and AQIS meat notice 2003/6, following a failure in meeting the microbiological process hygiene criteria, the Food Business Operator has to implement an intervention to bring back the process to compliance. In order to limit the scope of this assessment and to avoid too uncertain or speculative outcomes it was decided not to model the impact of any intervention (restrictive or permissive). Therefore, the predicted parameters from the model would be valid from a baseline time (T 0 ) until the time of a first non-compliance, assuming that no changes were done on the sampling plan, e.g. reduction in sampling frequency following a series of compliant microbiological results. 3. Material and methods 3.1. General principles of the model-based assessment The approach is based on a statistical model, which integrates the most important sources of variability. The parameters to be used for the description of such variability were estimated from the available data (see Section 3.2). Once the variability is included, the model can then simulate more realistic and more specific outcomes from a monitoring programme if compared to a simplistic random independent sampling approach, as discussed in the BIOHAZ Technical Report 8 (see. Section of the BIOHAZ Technical Report). As an output, such a probabilistic approach can assess some quantitative measures of performance of a given monitoring system under certain scenarios to be defined. This model-based approach was 8 European Food Safety Authority; The assessment of the comparison of the Australian monitoring programme for carcasses to requirements in Regulation (EC) No 2073/2005 on microbiological criteria on foodstuffs. EFSA Journal 2010; 8(3):1452. [52 pp.]. doi: /j.efsa Available online: 5

6 implemented independently for cattle and sheep, which were the two species for which sufficient data could be collected in the timeframe of the assessment. The primary purpose of the model building exercise was to enable further simulations in the future giving the opportunity of a new assessment, e.g. following the collection of new updated data of from the Australian and European monitoring systems. In detail, once the relevant input data were collected (see Section 3.2 on the data used), the model building followed 3 consecutive steps: Data exploration: literature data from slaughterhouse contamination were collected, screened and explored using basic statistical approaches (i.e. ANOVAs, multiple linear regressions, correlation analyses, data visualization). The purposes of this step were: o o o to identify and possibly estimate the main sources of variability; to identify and possibly estimate the main influential or risk factors; to generate sensible model-assumptions, e.g. on the choice of the probabilistic distributions to be used. Model building and estimation: aimed at building a statistical model able to simulate future outcomes (i.e. up to date data) from a monitoring programmes. Such a model is basically a more refined and more specific version of the model used in the screening phase. Those refinements are based on the available data from the national monitoring programmes. Scenario definition and data simulations: definition of different scenarios under which the model, previously estimated, has to run. Scenarios definition included: o o o the number of abattoirs and their throughputs per week; a baseline level of prevalence (at time «zero»), with variations, e.g. across slaughterlines; some assumptions on parameter values which could not be estimated from the data coming from the monitoring programmes (e.g. sensitivity of the sampling techniques, chilling effect, etc.). Such parameters were evaluated from a meta-analysis of the literature as further explained in Section Based on the simulated data, several performance criteria could be compared such as the average time to reach a critical level (before action) or the expected distribution of contamination in positive batches. DATA Literature review UK and AU datasets scenarios MODELS ANOVA Descriptive model Predictive model OUTPUTS Influential factor Model Assumptions Parameter estimates Simulated predictions Under scenarios Figure 1: Overall workflow of the simulation-based assessment showing different data inputs, models and outputs involved. 6

7 The overall flow of this probabilistic simulation-based is illustrated by Figure Data availability and collection In order to ensure validity of the simulations, it was necessary to fit those simulations to sufficiently informative datasets, which basically corresponded to the individual sample tests results over at least one year, per country and per slaughter-line Data from the monitoring programs Both Australian and European data from national monitoring programmes were available. European data were only coming from UK. About the European data, it has to be pointed out that UK data were the only available allowing for an outcome fulfilling the deadline. Nevertheless, to avoid bringing some bias in the simulation model, a between-country variability has been estimated from the scientific literature via meta-analysis: this way, a degree of variability and uncertainty were included and the simulated data coming from the EU model, built this way, were not driven by the UK data. The Australian government, with three different official Notices (Meat Notices 2003/6; 2005/13; 2007/12) promoted an E. coli and Salmonella monitoring programme (ESAM) and established criteria for TVC counts on meat surface. According to these Meat Notices, in Australia, process hygiene criteria are required to be met for Total Viable Count (TVC), Escherichia coli and Salmonella. According to the European regulation EC 2073/2005 (as amended by the Commission Regulation 1441/ ) the micro-organisms which have to meet the Process Hygiene Criteria are Aerobic Colony Count (ACC), Enterobacteriaceae and Salmonella. Even though a different terminology has been used, it was agreed that, ACC and TVC can be considered as having the same meaning and value. Thus, when performing the literature search, both ACC and TVC data were collected and their related values, in order to obtain an exploitable and harmonised dataset, have been recorded as TVC. Nevertheless, for consistency with the BIOHAZ Technical Report, ACC will be used hereinafter to refer to both ACC and TVC. Concerning Enterobacteriaceae and E.coli, the latter being a subgroup of the former, data on contamination values were collected without any harmonisation rule. Regarding the Australian monitoring system, a database was provided by the Australian Quarantine and Inspection Service (AQIS). Individual sample test results from the official monitoring plan over 2008 were reported for various species and categories (cattle, sheep, lamb, goat, horses) with a total of 31,859 records. The outcome variables were Total Viable Count (TVC), Escherichia coli counts and Salmonella (presence/absence). Data were available from all the plants dedicated to export to EU (29 plants). In addition, the throughput was reported for each single plant and, within a plant, a detail on throughput per species / category. Regarding the European monitoring data, the UK Food Standard Agency (FSA) has an established database at where the results of Food Business Operators microbiological monitoring are recorded. The FSA provided randomly selected datasets from the database for Total Viable Count (TVC) and Enterobacteriaceae counts (Cattle: 5037 records; Sheep: 5191 records) covering excision and the range of swabbing methods and Salmonella (presence/absence outcome; Cattle: 397 records, Sheep: 475 records) by sponging, collected from 13 plants over 3 years. 9 OJ L 322; ; pagg

8 Data from the literature Model-based comparative assessment of the AU and EU monitoring programmes The main source of published papers was via the Internet. Other sources were based on the knowledge of the Working Group experts and the list of papers cited in the official letter from the Australian Government to the European Commission. The Internet search is first described here. Since a sufficient amount of data on Enterobacteriaceae and Escherichia coli was provided both by EU and Australia through the datasets on national control programmes, the web-search was focused mainly on Salmonella data. The first step of the web-search was to screen the available scientific peer-reviewed papers on this topic using the tool ISI tool 10 with the following search string: Topic=(salmonella AND carcass* AND swab AND (cattle OR bovine OR sheep* OR goat* OR horse* OR pig* OR pork*)) AND Year Published=( ) A first list of papers was then available: each abstract was individually checked and a list of authors that published on this argument was extracted. Then, a second search was performed using this author s list and the second resulting list of papers was used to integrate the outcome of the first search. Aside this Internet search, many papers were suggested and provided by the WG experts and targeted searches were also performed in specialized review papers and book chapters. A total of 69 papers were retrieved and collected (see Appendix A). Each study was individually checked first for its consistency and then for its relevance with respect to the wanted information. Peer-reviewed publications were selected for inclusion in a consolidated database on the following criteria: The study was published in an international peer-reviewed journal; The study reported contamination data on at least one of the selected microorganisms on at least one of the selected animal species as continuous variables (mean, standard deviation, median, SEM, SED, quantiles, sample sizes) or proportions of positive samples out of the sample size; The data were not already (fully or partially) used in previous studies. In the cases where data were used in more then one study, the one providing the most complete and detailed information was chosen (e.g. sample size, testing technique); The data did not refer to artificially inoculated meat samples; The data did not refer to sanitized carcasses (as allowed in the USA). Major covariates that might affect the bacterial contamination of the carcasses were also collected from the original publication, such as: season of sampling, geographical area, whether the plant, in which the study was carried out, processed one or more animal species, the throughput of the plant, washing and chilling procedures, tested carcass area, sampling and testing techniques Data cleaning In order to harmonize and validate the available datasets, in depth checking, data management and transformations were performed on each of them as described in the following paragraphs

9 AU dataset Model-based comparative assessment of the AU and EU monitoring programmes The mother database was sent in Excel format. The file was composed by 8 different spreadsheets: 1 spreadsheet per animal specie / category (i.e. Calf, Steer/Heifer, Cow/Bull, Sheep, Lamb, Goat skinoff, Horses), each containing data on ACC, E. coli and Salmonella test results and one more sheet on throughput data per export-plant. The Australian file was then processed and modified to make it exploitable through the following actions: selection and filtering of the animal species of interest; data aggregation in a unique comprehensive dataset. 0 (zero) values were reported for ACC and E. coli. As indicated in the Australian official Meat Notices, zero value has to be used when the test result is negative (no detected CFU), but it should not be considered as a true zero, but a value below the LOD. A specific treatment of those values was made at the analysis stage (see Section 3.5.2) UK dataset The data were sent to EFSA in Excel format. The file was composed by 15 different spreadsheets: 5 spreadsheets per animal species (i.e. Pigs, Cattle, and Sheep). Each spreadsheet reported data on ACC, Enterobacteriaceae and Salmonella test results (CFU counts) and referred to a single slaughter plant (total number of plants = 15, five plants per animal species). The file with the European data was then processed and modified to make it exploitable through the following actions: selection and filtering of the animal species of interest ; data aggregation in an unique comprehensive dataset. In the UK dataset, below-lod values were reported as half of the LOD Literature dataset Due to the huge heterogeneity between the different studies, the dataset built on data extracted from scientific papers required a significant number of validation steps and transformations. They are reported hereafter: Data formatting: o o Where the study did not report explicitly the season, this was assigned using the following conversion scheme: winter from January to March, spring from April to June, summer from July to September and autumn from October to December (i.e. Northern hemisphere months). A reference table was used to standardize the carcass sampling site (see Appendix C). In case the sites sampled in the study were not in the reference list, the nearest, with a similar microbiological behaviour, was chosen and recorded. Data reconstruction: When the sample size was not reported, an estimate was made on the available information, e.g. number of visits per plant, number of tested carcasses per visit, number of tested sites per carcass. An example is given in Table 1: 9

10 Table 1: Example of sample site per site re-calculated based on other available information available Available data Number of visits 3 Number of tested carcasses per visit 10 Number of tested sites per carcass 1 Sample size per site 30 Data transformation: In most of the cases, the number of microbiological tests giving a negative result, i.e. non detected result, is reported. In order to calculate the proportion of positive results, the following calculation was made: In case no Standard Deviation was reported, but the Standard Error of the Mean (SEM) or the Standard Error of the Difference between the Mean (SED), the Standard Deviation was calculated as: 3.4. Final database AU dataset The dataset for Australia in the end consisted on 31,525 records: 13,639 referring to cattle and 17,886 to sheep. Each record reports the test results on ACC, E. coli and Salmonella from a single carcass. Data comes from 27 plants visited by the inspection system between 2nd January 2008 and 31st December plants provide data only on cattle and 10 only sheep, while 8 plants provide data both on bovines and sheep. The samples were collected at different times during the day, covering the whole range of available hours EU dataset (United Kingdom data) The dataset for Europe in the end consisted on 10,228 records reporting the test results on ACC and Enterobacteriaceae from a single carcass and on 872 records reporting the Salmonella test results. Data came from 13 different slaughter-lines visited between the 3rd January 2006 and 27th April No information was available on the time of day at which the samples were collected. As it can be seen the timeframe of the available information were different for AU and EU, as for the former the results from the implementation of the monitoring programme referred just to year Thus, the implicit assumption underlying the model is that the data from Australia are somehow representative of the average trend of contamination for the considered bacteria. 10

11 Literature dataset Model-based comparative assessment of the AU and EU monitoring programmes The final number of studies included is 30 and the time span covers a period from 1976 to Differences affecting hygiene procedures and technology as well as microbiology have been occurred in this time spam: those differences were included in the model by specific fields (i.e. date of publication, type of slaughterhouse, microbiological techniques, etc). The data extraction exercise led to a total of 1048 records, each reporting a contamination value, either in terms of mean or of proportion positive. A summary of the collected data by type of microorganism and by animal specie is reported in Table 2 and Table 3. Table 2: Number of records collected by microorganism and animal species n records SPECIES BACTERIA cattle sheep ACC E. coli Enterobact Salmonella Total Table 3: Number of papers contributing to data by microorganism and animal specie (a study could contribute more than one time) n papers SPECIES BACTERIA cattle sheep ACC E. coli Enterobact Salmonella Total The largest set of data was for ACC. For this class of microorganisms, 671 records were on cattle and 119 on sheep. Fewer studies were available for E. coli and Enterobacteriaceae (131 and 77 records respectively) and just 8 papers contained information on Salmonella (50 records). Table 4: Number of records by season Observation Season n records spring 32 autumn 9 spring/autumn 8 summer 127 winter 18 na 854 Total

12 Table 5: Number of records by geographical area Geo Area n plants n records EU Oceania Asia na 12 Middle East 3 36 North America na Total 1048 Table 6: Number of records by accessory processing technique on the carcass before sampling (washing and chilling) Chilling Washing yes no na n records yes no na Table 7: Number of records by sampling technique Sampling Type n records swab 926 excision 72 na 50 Total 1048 Tables 4 to 7 describe the data availability on the main covariates reported in the dataset. Many other covariates are recorded (see the data dictionary in Appendix B), but were discarded from the analysis for various reasons (non-harmonized records, collinearity with other covariates, scarce number of records, minor importance, etc) Model-based assessment methodology In this section, the methodology used for the data analyses and for the simulations is described in order to enable reproducibility of results. As recalled in the previous sections, the model-based comparative assessment of monitoring programmes can be divided into three major steps: 1. Exploration of literature data; 2. Analysis of sample-level data from the Australian and the European monitoring programmes; 3. Simulation-based assessment of the two monitoring programmes under different scenarios. The specific purpose of each of these steps is recalled hereafter, before detailing the corresponding methods used. 12

13 Data exploration analysis of the literature data The purpose for exploration and meta-analysis of the literature data was two-fold: - To identify potential factors that may affect the contamination levels in cattle and sheep; - To evaluate the size of such effect as well as the variability components that could not be otherwise estimated based on the national programme data. Prior to any statistical analysis, some data management steps were performed to harmonize the format and units of measurement and to avoid rough sources of bias. To different Countries or different slaughter-lines that could be identified a specific identifier number was assigned. In those cases where more than one carcass-area was swabbed, the category many areas was assigned to the respective records, as an identified category. A (log-)linear mixed-effect model was used to describe the contamination levels. Since those levels were group averages from the published studies, some weighting or other meta-analytic techniques needed to be considered. In our case, study homogeneity had to be assumed because the convergence could not be met otherwise. As for the weighting, the study group size was used because it was reported for most records, unlike standard errors. One model was fitted separately for ACC, Enterobacteriaceae and for Salmonella, for each of the two species cattle and sheep. All models were fitted using SAS version 9.1, with proc MIXED. Given the objectives and the nature of the data, no standard statistical model selection method was formally implemented. Instead, factors were included in the model when the SAS procedure (REML algorithm) could properly converge with positive and definite Hessian matrix. In the very few cases where two model options were competing with each other, the AIC criterion was used to differentiate the best option. For example, in the case of Enterobacteriaceae in cattle, the survey ID was preferred to the country ID. These two covariates were obviously highly collinear. The two variables publication year and swabbed surface size were also tested but the results were reported just as indicative information. Indeed, as both variables could not be tested jointly together, they were tested one after the other. However, those variables were not included in the final model. Statistical estimates of such parameters have been put into brackets in the results tables. Given the wide range of swabbing sampling techniques used, and of the sampling areas (see data dictionary in Appendix B), those variables were processed as random effects. This between-sampling techniques variability was also important for the final purpose to simulate the between country variability within EU. No interaction factor was investigated in this assessment. No missing data imputation or treatment was performed except for the binary covariate such as washing or chilling variables. Those variables were coded as 0 or 1 whether the action of e.g. washing or chilling was made or not. In the case of missing data, it was the medium value 0.5 was assigned. More advanced methods such as multiple imputations or Bayesian approach could be used in a further assessment under the missing at random assumption. As always in such a literature review exercise, it is should be acknowledged that many sources of uncertainty and of confounding are attached to the estimates. More specifically in this case, many identifiers could be confounded with each other in the statistical analysis, such as country code with study code or with paper number. This aspect needs of course to be accounted for in the interpretation and in the use of such results. It was also a source of collinearity in the variables, which made sometimes impossible the joint analysis of all sources of variability. It was also the reason why a random-effect meta-analysis could not be performed (i.e. including a random study effect in the 13

14 analysis). However, this between-study variability was somewhat already included in the other random or fixed effects included (e.g. sampling technique, country, etc ) Model fitting to national control programme data The purpose of the analyses of data coming from official monitoring control programmes is to estimate the important parameters which characterize the respective monitoring systems, i.e major variability components, time and seasonal variations. Those parameters will then be re-used, including them in the model, to simulate realistic data in the final assessment. Sample-level data by slaughterlines, by sampling session over at least one year were necessary to meet the overall objective of this assessment. Prior to the statistical analysis, a few data management steps were performed in addition of those described in Section 3.3, in order to harmonize the format and to avoid rough sources of bias or misinterpretation. In particular: - all zero values from the Australian data were set to the LOD/2, in order to be consistent with the handling of the UK dataset. As it was not possible, within the timeframe of this assessment, to obtain the LOD values for each Australian abattoir, a common LOD value was imputed for all abattoirs based on the lowest values observed. Hence, the imputed values for cattle were log 10 (0.01) for ACC and log 10 (0.005) for E. coli. As for sheep, they were log 10 (0.05) for ACC and log 10 (0.08) for E. coli. These missing value imputations were especially sensitive for E. coli data for which more than half of the AU records were zeros; - the days were numbered from time 0 (corresponding to 1 st January 2006 in the UK dataset, and to 2 nd January 2008 in the AU dataset); - the seasons were determined based on the day. Winter for AU corresponded to summer in EU; - in reference Aqis06, it was assumed that all data were coming from the same country (for ACC and E. coli records in cattle); - in reference Aqis23, it was assumed that all data were coming from the same country (for Salmonella records in cattle). A (log-)linear mixed-effect model was used to describe the contamination levels of ACC and Enterobacteriaceae or E. coli. The analysis was done with SAS version 9 with proc MIXED. A logistic model was used for Salmonella fitted with a GEE approach (SAS proc GENMOD). One model was fitted separately for ACC, Enterobacteriaceae and for Salmonella, for each of the two species cattle and sheep. Various temporal correlation structures were tried to describe the time persistence of contamination within a given abattoir. Based on the AIC, the two best choices were seen as: - an auto-correlation structure (AR(1)) with daily increment for AU and weekly increment for UK); - a daily cluster (for AU) or weekly cluster (for EU) included by accounting for a random interaction line*day (resp. line*week for EU). This implies that measurements from a same week in a given abattoir are correlated with each other. The second option was seen more realistic from a biological point of view and was then chosen for all models. This option was also associated with the lowest AIC in most cases. 14

15 Simulation-based comparative assessment Model-based comparative assessment of the AU and EU monitoring programmes The purpose of this last step is to run a simulation model to generate realistic data over one year under different scenarios for the Australian and the European monitoring systems. The process hygiene criteria in force can then be applied on the simulated data. The microbiological criteria of the sampling plans in force in Europe and Australia are reported in Appendix D. Hence, based on repeated simulations, the distribution of the number of days required until the first alert can be evaluated and compared. The simulation model was similar to the model used to describe the national programme data, i.e. a log-linear mixed effect model. The fixed and random factors were included based on the exploration data analysis, data availability and biological relevance. The fixed factors included were: the swab vs. excision effect, the season effect, and a baseline level (intercept). The random factors included were: the slaughter-line, the week (EU) or day (AU) of sampling, the swabbing technique, and the chilling effect. The chilling effect was assumed to be specific to each abattoir, and randomly distributed in order to take into account substantial variations of the impact of chilling. As a matter of fact, its impact depends on many other factors or sampling conditions that are specific to the abattoir. The distribution of such chilling effects across abattoirs was defined, in the log scale, as a normal distribution with mean and standard error as estimated in the literature data analysis. Note that the chilling effect could not be estimated for E. coli in sheep. In this case, the same effect as for ACC in sheep was used. In order to describe the fact that the Australian programme controls E. coli as opposed to Enterobacteriaceae for EU, it was not necessary to make assumptions on the average E. coli proportion found in a sample of Enterobacteriaceae count. This ratio was first set to a fixed value as estimated in the national data, assuming that the same adjusted baseline level was present for Australia and UK. Subsequently, a distribution instead of a fixed value was assumed in order to analyze the sensitivity of results with respect to that assumption. The numbers of AU slaughterhouses used in the simulations were those communicated by the AU competent authorities for each species. The number of EU slaughterhouses could not be obtained precisely, hence were done with two plausible order of magnitude: n=100 and n=300 slaughter-lines per species. A fixed daily throughput was assumed per slaughter-line over the year (but the variability between lines and between years was accounted for). It was assumed to be normally distributed among lines, with mean and SD as follows: For AU: 1200 carcasses +/- SD=400 For EU: 1000 carcasses +/- SD=800 At least one daily sample per line was assumed for AU. No additional between-country variability was added to depict the variations across countries. As a matter of fact, this variability is at least partially included in the variability of the swab sampling technique used. Moreover, the major additional sources of between-country variability are likely to be: - differences in baseline levels: addressed by assuming various baseline levels in different scenarios; - differences in daily throughputs: accounted for by using a wide distribution of abattoir s throughputs; - differences in hygiene measures and actions: not addressed within our paradigm because the simulations stops at the first alert, hence before any actions are taken. 15

16 Finally, the respective microbiological criteria were implemented for each monitoring programme, in order to derive the average number of days before the first alert. This average was calculated over 1,000 simulations in each case considered. For the European programme, various values for M were assessed, with different sampling techniques (swab vs. excision). For the Australian programme, the moving window of n=15 samples within each slaughter-line was calculated as the number of days necessary to get at least 15 consecutive samples within a given abattoir. Since different abattoirs have different throughputs, the size (days) of this moving window was also different for each abattoir. 16

17 4. Results Results of the literature data exploration Depending on algorithm convergence and on the sparse data availability, the considered factors could not be all investigated for all microorganisms. Table 8 and Table 9 summarize the factors available in cattle and sheep, for ACC, Enterobacteriaceae and Salmonella, for which the statistical analysis could be performed. As expected, most of the important factors could be investigated for ACC, but very few, or none, for Salmonella. Table 8: Factors where enough data could be gathered to enable the statistical analysis, for ACC, E. coli / Enterobacteriaceae and Salmonella in cattle. The brackets indicate that the parameter was not included in the final model. Factor / effect ACC E. coli / Enterobact Salmonella Chilling Washing Surface size Publication year Between sampling techniques variability Between sampling area variability Between lines variability Between Countries variability Yes Yes (Yes) Yes Yes (Yes) (Yes) (Yes) (Yes) Yes Yes Yes Yes Yes Yes Yes Yes Table 9: Factors where enough data could be gathered to enable the statistical analysis, for ACC, E. coli / Enterobacteriaceae and Salmonella in sheep Factor / effect ACC E. coli / Enterobact Salmonella Chilling Washing Publication year Surface size Between sampling techniques variability Between sampling area variability Between lines variability Between Countries variability Yes Yes (Yes) (Yes) Yes Yes Yes (Yes) Yes Yes For each of the identified factors investigated, the results from the mixed-effect model fit are reported in Tables from 10 to 19. In brackets, the effect estimates are reported for information only, as 17

18 explained in the methods section: they usually correspond to estimates that are less reliable than the others. For this reason, they were excluded from the final model in which just the reliable estimates (not in brackets) were used. For ACC, the baseline level found was about 3 log 10 cfu for both cattle and sheep. As expected, the effect of washing always reduced the contamination level of about 0.5 log 10, although the significance was borderline for sheep (p=0.01). The effect of chilling was not consistent for cattle and sheep. It slightly reduced the contamination level in cattle (not significant, p=0.28), whereas it substantially and significantly increased contamination in sheep (+1.5 log with p<0.001). It has to be pointed out that the chilling effect was included in the model, as a random effect, according to the opinion of the WG experts. The reason was exactly this particular behaviour of the chilling effect, characterised by such a huge variability, which leads indeed to this high p-value. The variability components for ACC were consistent between cattle and sheep (15% to 25% for between-line variability and about 40% for between swabbing area variability). In cattle, the between country variability was the major source of variability (69%). For Enterobacteriaceae/ E. coli, the baseline levels were, as expected, much lower than ACC: below 1 log 10 for both cattle and sheep. The effect of washing was the same as for ACC, about 0.5 log 10 decrease for both species. Only between-swabbing techniques variability and between-sampling tool variability could be estimated. The CVs were relatively high (above 50%, up to 102%), but are likely to include other confounding variability sources such as between-country or between-study variability. For Salmonella, the contamination levels were too low to allow any analysis of effect (below 1% for cattle). The baseline level could not even be estimated for sheep. The other estimates are not reliable, as the fitting algorithm could not properly converge (Hessian not definite positive). Table 10: Statistical estimate (with standard error and p-value) of factors that may affect ACC levels in cattle Factor / effect Estimate (log 10 cfu/cm 2 ) Standard error p-value Adjusted baseline level Chilling Washing (Publication year) (-0.26 / 10yrs) (0.17 / 10yrs) (0.13) (Surface size) ( / cm 2 ) ( / cm 2 ) (0.68) 18

19 Table 11: Statistical estimate of various variability components that may affect ACC levels in cattle Variance component Between lines variability Between Countries variability Between sampling techniques variability Between sampling area variability CV estimate 25% 69% 10% 42% Table 12: Statistical estimate (with standard error and p-value) of factors that may affect Enterobacteriaceae/E. coli levels in cattle Factor / effect Estimate (log 10 cfu/cm 2 ) Standard error p-value Adjusted baseline level Chilling Washing (Publication year) (0.15 / 10yrs) (0.11/ 10yrs) (0.18) (Surface size) ( / cm 2 ) ( / cm 2 ) (0.13) Table 13: Statistical estimate of various variability components that may affect Enterobacteriaceae/E. coli levels in cattle Variance component Between sampling techniques variability Between sampling area variability CV estimate 102% 54% 19

20 Table 14: Statistical estimate (with standard error and p-value) of factors that may affect Salmonella levels in cattle Factor / effect Estimate Standard error p-value Adjusted baseline prevalence level 1% na (Chilling ) OR=0.04 na Table 15: Statistical estimate of various variability components that may affect Salmonella levels in cattle Variance component Between line variability Between Countries variability SD estimate 0.3% 1% Table 16: Statistical estimate (with standard error and p-value) of factors that may affect ACC levels in sheep Factor / effect Estimate (log 10 cfu/cm 2 ) Standard error p-value Adjusted baseline level Chilling <0.001 Washing (Publication year) (0.16 / 10 yrs) (0.06 / 10 yrs) (Surface size) ( / cm 2 ) ( / cm 2 ) Table 17: Statistical estimate of various variability components that may affect ACC levels in sheep Variance component CV estimate Between line variability 15% Between sampling area variability 42% 20

21 Table 18: Statistical estimate (with standard error and p-value) of factors that may affect Entreobacteriace/E. coli levels in sheep Factor / effect Estimate (log 10 cfu/cm 2 ) Standard error p-value Adjusted baseline level Washing (Surface size) / cm / cm Table 19: Statistical estimate of various variability components that may affect Enterobacteriaceae/E. coli levels in sheep Variance component Between swabbing technique variability Between sampling swabbing area variability CV estimate 10% 85% 21

22 Results of the model fit to the data from the monitoring programmes Results for the Australian monitoring programme Figure 2 and Figure 3 illustrate the daily-averaged Australian data per abattoir over one year period for respectively cattle and sheep. Just by visual inspection, between-line variability looks small compared to the within line variability. Figure 2: Daily averaged ACC levels plotted against days as recorded for cattle in the Australian data, for each slaughterhouse (one colour = one slaughterhouse) Based on the model choice procedure defined in the methods section, a log-linear mixed-effect model was chosen with slaughter-line and slaughter-line*day as random effects. A sample of the SAS proc mixed code is reported below: proc mixed data = DATASET; class line season day; model log_acc = season / s; random line day*line; run; In this example, the categorical fixed effects are the line, the season and the day; the numerical fixed effect was the season; the random effect were the line and the day-by-line. 22

23 Tables 20 to 29 report the results from the analysis of the Australian monitoring programme data. For cattle, a season effect was observed for E. coli (p=0.004) but not for ACC (p=0.6). When observed, the season effect was associated to maximal contamination in summer, and minimum contamination in spring. About ACC, a strong season effect was again observed in sheep (p<0.001) with maximal contamination in summer, and minimum contamination in spring, as for cattle. The between-line variability estimated was consistent with that observed in the literature (about 20%) except for ACC in sheep where it was substantially higher (CV=83%). The greatest source variability was generally observed for the between-carcass variations, ranging from CV=34% to 145% depending on the species and the microorganism. For Salmonella, no variability component could be estimated due to the very small number of positive samples. Only the average sample prevalence could be derived, ranging from 0.2% (sheep) to 0.4% (cattle). Table 20: Statistical estimate (with standard error) of ACC level in cattle for the Australian data Factor / effect Estimate (log 10 cfu/cm 2 ) Standard error Adjusted baseline level Table 21: Statistical estimate of various variability components that may affect ACC levels in cattle for the Australian data Variance component Between line variability Between week (within line) variability Between carcass variability CV estimate 16% 15% 145% 23

24 Table 22: Statistical estimate (with standard error) of E. coli level and the seasonal effect in cattle for the Australian data Factor / effect Estimate (log 10 cfu/cm 2 ) Standard error Adjusted baseline level Autumn Spring Summer Table 23: Statistical estimate of various variability components that may affect E. coli levels in cattle for the Australian data Variance component CV estimate Between line variability 22% Between week (within line) variability 9% Between carcass variability 34% Table 24: Statistical estimate (with 95%-CI) of Salmonella sample-prevalence in cattle for the Australian data Factor / effect Estimate 95%-CI Adjusted baseline prevalence level 0.4% 0.08%- 2.3% 24

25 Figure 3: Daily averaged ACC levels plotted against days as recorded for sheep in the Australian data, for each slaughterhouse (one colour = one slaughterhouse) Table 25: Statistical estimate (with standard error) of ACC level and the seasonal effect in sheep for the Australian data Factor / effect Estimate (log 10 cfu/cm 2 ) Standard error Adjusted baseline level Autumn Spring Summer

26 Table 26: Statistical estimate of various variability components that may affect ACC levels in sheep for the Australian data Variance component Between line variability Between week (within line) variability Between carcass variability CV estimate 83% 36% 102% Table 27: Statistical estimate (with standard error) of E. coli level and the seasonal effect in sheep for the Australian data Factor / effect Estimate (log 10 cfu/cm 2 ) Standard error Adjusted baseline level Autumn Spring Summer Table 28: Statistical estimate of various variability components that may affect E. coli levels in sheep for the Australian data Variance component Between line variability Between week (within line) variability Between carcass variability CV estimate 10% 8% 45% 26

27 Table 29: Statistical estimate (with standard error) of Salmonella level in sheep for the Australian data Factor / effect Estimate 95%-CI Adjusted baseline prevalence level 0.2% 0.1%- 0.4% Results for the UK monitoring programme Figure 4 and Figure 5 illustrate the weekly-averaged UK data per slaughter-line over a three-year period for respectively cattle and sheep. Even by visual inspection, between-line variability appears much higher than for Australia. It is only partly due to the fact that the sampling techniques used are not the same for all slaughter-lines, especially for cattle for which excision was also used in some abattoirs. Figure 4: Daily averaged ACC levels plotted against days as recorded for cattle in the UK data, for each slaughter-line (one colour = one slaughter-line) A similar model structure, with random slaughter-line, was favoured by the model selection, which confirms its structural relevance for the process described. Tables form 30 to 38 report the results from the analysis of the UK monitoring programme data. The variability associated with all test results were generally higher than those observed for Australian data, especially for between-line variability which range from 69% to 133% (3 to 4 times higher than for Australia). 27

28 For cattle, the season effect was significant for both ACC and Enterobacteriaceae (p<0.001), with maximal levels in winter and minimal levels in spring. The effect of sampling technique was also significant, mainly because data obtained by excision were significantly higher than those obtained by swabbing data (more than 2 log 10 above). No estimate could be derived for Salmonella due to absence of positive samples. For sheep, the season effect was again significant for both ACC and Enterobacteriaceae (p<0.001), with maximal levels in autumn and minimal levels in spring, which is a consistent finding across countries, species and microorganisms. The effect of sampling technique was now not significant, but this was because no excision data were reported for sheep. The same model structure was used as for cattle. Only the average level of sample prevalence could be derived for Salmonella (0.04%). Table 30: Statistical estimate (with standard error) of the baseline ACC level, the seasonal effect, and the sampling tool differences in cattle for the UK data Factor / effect Estimate (log 10 cfu/cm 2 ) Standard error Adjusted baseline level Excision Drumstick Drumstick DFI Autumn Spring Summer Table 31: Statistical estimate of various variability components that may affect ACC levels in cattle for the UK data Variance component CV estimate Between line variability 133% Between day (within line) variability 98% Between carcass variability 57% 28

29 Table 32: Statistical estimate (with standard error) of the baseline Enterobacteriaceae level, the seasonal effect, and the sampling tool differences in cattle for the UK data Factor / effect Estimate (log 10 cfu/cm 2 ) Standard error Adjusted baseline level Excision Drumstick Drumstick DFI Autumn Spring Summer Table 33: Statistical estimate of various variability components that may affect Enterobacteriaceae levels in cattle for the UK data Variance component CV estimate Between line variability 81% Between week (within line) variability 83% Between carcass variability 48% 29

30 Figure 5: Daily averaged ACC levels plotted against days as recorded for sheep in the UK data, for each slaughter-line (one colour = one slaughter-line) Table 34: Statistical estimate (with standard error) of the baseline ACC level and the seasonal effect in sheep for the UK data Factor / effect Estimate (log 10 cfu/cm 2 ) Standard error Adjusted baseline level Autumn Spring Summer