Proficiency Testing. Bryanne Shaw Biology Section Manager

Transcription

1 Proficiency Testing Bryanne Shaw Biology Section Manager

2 Proficiency Testing The evaluation of participant performance against pre-established criteria by means of interlaboratory comparisons.

3 ISO Requirement Section 5.9 Ensuring the Quality of Test and Calibration Results The laboratory shall have quality control procedures for monitoring the validity of tests and calibrations undertaken. The resulting data shall be recorded in such a way that trends are detectable and, where practicable, statistical techniques shall be applied to the reviewing of the results. This monitoring shall be planned and reviewed and may include, but not be limited to, the following:

4 ISO continued (a) regular use of Certified Reference Materials and/or internal quality control using secondary Reference Materials; (b) participation in interlaboratory comparison or proficiency-testing programmes; (c) replicate tests or calibrations using the same or different methods; (d) retesting or recalibration of retained items; (e) correlation of results for different characteristics of an item. Note: The selected methods should be appropriate for the type and volume of work undertaken.

5 Common Abbreviations AAFCO American Association of Feed Control Officials AB Accrediting Body ALACC AOAC Analytical Laboratory Accreditation Criteria Committee CRM Certified Reference Materials PT- Proficiency Testing RM Reference Material

6 Requirements in addition to ISO/IEC 17025:2005

7 ALACC Guidelines AOAC International Guidelines for Laboratories Performing Microbiological and Chemical Analysis of Food, Dietary Supplements, and Pharmaceuticals, An aid to interpretation of ISO/IEC 17025:2005. Recently revised in 2015 Includes accreditation requirements in addition to ISO/IEC 17025:2005

8 ALACC PT Requirements The lab shall have a documented PT plan for all test methods on the scope of accreditation. PT shall be performed following the normal working practices operated in the lab.

9 ALACC PT Requirements PTs shall be rotated among qualified analysts. When selecting an external scheme consideration should be given to using a scheme that is based on the requirements of ISO/IEC 17043:2010 and, when available, one that is accredited to this standard. Prior to accreditation the lab shall have successfully analyzed PT samples for each test, type of test/method, and/or technique for which they are seeking accreditation

10 ALACC PT Requirements When a relevant external scheme is not available, a lab can use alternative means of evaluation For Lab On-going Competency: the lab shall participate in at least 1 PT event annually for each test, type of test/method, and/or technique on the scope of accreditation

11 ALACC PT Requirements Alternative Means of Evaluation, in order of preference: Participate in a round-robin, interlaboratory test performed independently several times Testing of blinded well-characterized laboratory control samples Performance of a interlaboratory comparison with other accredited labs Performance of a comparison with another test method/technology Address the elements identified in ISO section 5.9, including the evaluation criteria of the data

12 ALACC PT Sample Acceptability The lab shall evaluate PT results when they are received. If using an external PT provider the lab shall use the provider s criteria to evaluate the results. If the provider does not issue acceptability criteria or if the lab is using one of the alternative means of evaluation they shall have procedures that define the acceptability of the results

13 ALACC Guidance for Evaluating PT Results The assigned value for the result can be established by one of the following options: Consensus of the majority of the labs The SPC ranges for blinded laboratory control samples Fortification value of prepared samples Assigned result from a previous round Results obtained from a group of 2 or more accredited labs for that analyte which have demonstrated proficiency in the past

14 AAFCO Guidelines AAFCO Quality Assurance Control Guidelines for Feed Laboratories, supplement to ISO/IEC 17025:2005, 2014 edition Includes accreditation requirements in addition to ISO/IEC 17025:2005

15 AAFCO PT Guidelines Labs are encouraged to participate in the AAFCO Check Sample Program because of its broad inclusion of analytes in feed matrices. The results from the AAFCO Check Sample Program are statistically analyzed, summarized and reported back to each participating lab.

16 Accrediting Bodies (AB) Requirements Accrediting bodies may have either guidance or policies in place defining this section of the standard. Check their websites. They are generally available to download for free. ABs A2LA AIHA ANAB-ACLASS L-A-B PJLA

17 Where to Start? Determine if there is a suitable PT available for: Analyte of Interest Matrix Fit-for-Purpose

18 Fit-for-Purpose Sufficiently reliable that any decision on it can be taken with confidence. Information can be found in Eurachem Guide The Fitness for Purpose of Analytical Methods and AOAC How to Meet ISO Requirements for Method Verification

19 Finding a PT Provider Check your AB s website for a list of PT providers AAFCO has an extensive list of PT providers Determine if the provider is accredited to ISO/IEC 17043:2010 (Requirements for PT providers) If there is an accredited provider the AB may strongly encourage the use of the provider. You and your customer may jointly agree that a CRM or non-accredited program is fine. Make sure it s well documented.

20 Example of a PT Providers Scope

21 Example of Reference Material Producer s Scope

22 Matrices Different PT providers offer different matrix/analyte combinations You do not need a PT for every matrix type you analyze But if, for example, your method is used exclusively for water samples a PT in a food matrix may not be appropriate. If practical, it is a good idea to change up your matrix/analyte combinations

23 AFFCO Resource for PT Providers Found on AAFCO.ORG\Laboratory Resources for ISO 17025:2005 Accreditation

24 Common Road Blocks If PT has to be shipped from outside the country and there are restrictions for importing the sample matrix. You can find the analyte of interest but not in a matrix or at the wrong concentration i.e., parts per million (ppm) and you need parts per billion (ppb) Issues with PT provider can t meet needs or not fit-for-purpose.

25 Documentation If you can t find a suitable PT or provider you must document the issue You decide what is suitable but you must be able to provide justification. Issues with the PT provider should be documented per ISO section 4.6 Purchasing Services and Supplies If you will not longer use the provider remove them from your approved supplier list

26 Use of Internal Checks These are an acceptable alternative when external PT / Check Samples are not commercially available or is determined to be unacceptable for your or your customer s analytical requirements.

27 Internal Checks 1) Use a Certified Reference Material (CRM) and/or internal quality control using secondary Reference Materials (RM) 2) Participate in a round robin. 3) Perform a comparison with another method. 4) Perform an inter-laboratory comparison. 5) Test previously run samples. 6) Correlation of results for different characteristics of an item. 7) Internal Check Samples

28 Inter-laboratory comparison The organization, performance and evaluation of calibration/test results for the same or similar item by two or more laboratories in accordance with pre-determined conditions.* *ANAB Guidance of Proficiency Testing/Inter-Laboratory Comparisons, 2015/06/17

29 Internal Check Sample Here are some examples of approaches you might take: Find a matrix with the analyte. Develop your own in-house using CRM/RM. Work with a CRM/RM company. Have another laboratory make it for you. Use a previously analyzed sample. Compare results for different characteristics of the sample.

30 MN Example Pesticide Analysis: We tried a couple of commercial PTs but we needed ppt and many of the commercial PTs are ppb/ppm. Also, the commercial PT compound lists did not adequately cover our lists. Went to Ultra Scientific to have 2 to 3 specialty mixes made each year covering each class of compounds. (expensive but meets our customer s needs). We can t cover all of the analytes in the mixes but our AB accepts using class of compound. We also review internal matrix spike recovery data especially for compounds that Ultra Scientific is not able to add to a mix because of stability issues.

31 MN Example Micro Analysis: Staph Enterotoxin It is a select agent and not available from accredited PT provider The lab is a member of FERN and every so often FERN offers a Staph Enterotoxin PT. FERN isn t an accredited PT provider but they meet the lab s and our customer s needs so they are deemed suitable. If FERN doesn t offer a PT the lab would prepare internal check samples

32 Re-Purpose PT/Check Samples a) Use as a laboratory control sample (LCS). b) Use for testing competency of analytical staff. c) Use for verification/validation of methods.

33