ISO 15189:2012 and Susceptibility Testing BSAC User Day 22 June 2016

Transcription

1 Martin Stearn, UKAS Senior Assessment Manager ISO 15189:2012 and Susceptibility Testing BSAC User Day 22 June 2016 UKAS June 2016

2 Objectives Discuss the various clauses within the ISO 15189:2012 Standard relevant to antimicrobial susceptibility testing Provide some clarity regarding UKAS assessment process Not to tell you what you need to do or how to do it 2

3 ISO 15189:2012 Requirements Discuss the various clauses within the ISO 15189:2012 Standard relevant to antimicrobial susceptibility testing 5.5 Examination processes 3

4 ISO 15189:2012 Management requirements 4.3 Document control 4.5 Examination by referral laboratories 4.6 External services and supplies 4.7 Advisory services 4.9 Identification and control of non-conformities 4.10 & 4.11 Corrective & preventive action 4.12 Continual improvement 4.13 Control of records 4.14 Evaluation and audits Periodic review of procedures Assessment of user feedback Staff suggestions Internal audit Risk management 4

5 ISO 15189:2012 Technical requirements Training Competence assessment Continuing education and professional development Storage facilities Equipment Acceptance testing Instructions for use Calibration and metrological traceability Maintenance and repair Incident reporting Equipment records 5

6 ISO 15189:2012 Technical requirements Reagents and consumables Reception and storage Acceptance testing Inventory management Instructions for use Adverse incident reporting Records Request form information 5.5 Examination procedures General Verification of examination procedures Validation of examination procedures Measurement uncertainty of measured quantity values 6

7 ISO 15189:2012 Technical requirements Documentation of examination procedures 5.6 Ensuring the quality of examination results Quality control (IQC) Interlaboratory comparisons (EQA) Comparability of examination results (procedures, equipment, sites as applicable) Review of results 5.8 Reporting of results Report attributes Report content 5.9 Release of results Automated selection and release of results Revised reports Information system management 7

8 Method selection Laboratories are free to use whatever method they select. ISO 15189:2012 does not prescribe any particular method or the use of a gold standard or the need to change to the latest method. Neither do UKAS, we ll assess your methods and your competence to use them. ISO 15189:2012 does specify the need to select methods which have been validated and verified to achieve the right result. 8

9 ISO 15189: Selection, verification and validation of examination processes General The laboratory shall select examination procedures which have been validated for their intended use. Shall = you must (only 321 shalls in the Standard!) Should = you shall unless you can justify not doing so (24) NOTE: Preferred procedures are those specified in the instructions for use of in vitro medical devices or those that have been published in established/authoritative textbooks, peer-reviewed texts or journals, or in international consensus standards or guidelines, or national or regional regulations. (Standard methods) 9

10 Validation Confirmation, through the provision of objective evidence, that the requirements for a specific intended use or application have been fulfilled. The method achieves the intended outcome because; Evidence A Evidence B Evidence C Evidence D The method works The zones of growth inhibition achieved by the AST method accurately demonstrate susceptibility or resistance for that organism 10

11 Validation (again) Confirmation, through the provision of objective evidence, that the requirements for a specific intended use or application have been fulfilled. The method achieves the intended outcome because; Evidence A; published in established/authoritative textbooks, peerreviewed texts or journals, Evidence B; international consensus standards or guidelines Evidence C; national or regional regulations. Evidence D: records of who has validated the procedure The method works The zones of growth inhibition achieved by the AST method accurately demonstrate susceptibility or resistance for that organism 11

12 ISO 15189: Selection, verification and validation of examination processes Validation of examination procedures The laboratory shall validate examination procedures derived from the following sources: a) Non-standard methods b) Laboratory designed or developed methods c) Standard methods used outside their intended scope Applying breakpoint criteria to related or even unrelated organisms a) Validated methods subsequently modified Alteration of the method in any way (e.g. incubation time/temperature, disc concentration, media, inoculum density) 12

13 Verification Confirmation, through provision of objective evidence, that specified requirements (as demonstrated by validation) have been fulfilled. We (the laboratory) can achieve the intended outcome because; Evidence X Evidence Y Evidence Z The method works in our hands The zones of growth inhibition we achieve by using the validated AST method accurately demonstrate the susceptibility or resistance of our isolates to which we have applied the AST method 13

14 Verification (again) Confirmation, through provision of objective evidence, that specified requirements have been fulfilled. We (the laboratory) can achieve the intended outcome because; Evidence X; performance characteristics (we can do) demonstrate the performance claims (what is expected) have been met in our lab Evidence Y; documented verification procedure Evidence Z; records of review & authorisation of the verification data 14 The method works in our hands The zones of growth inhibition we achieve by the AST method accurately demonstrate the susceptibility or resistance of the isolates to which the method has been applied

15 ISO 15189: Selection, verification and validation of examination processes Verification of examination procedures The laboratory shall obtain information from the manufacturer/method developer for confirming the performance characteristics (expected zone diameters, MICs) of the procedure. What are you aiming to achieve? What are the intended outcomes of the method being used? Appropriate control strains (ATCC/NCTC) subjected to the method should achieve the intended result (zone diameter, MIC) when we do the test. 15

16 ISO 15189: Selection, verification and validation of examination processes Verification of examination procedures Validated examination procedures used without modification shall be subject to independent verification by the laboratory before being introduced into routine use. Generally acceptable for retrospective verification of historical methods (inc. BSAC), e.g. demonstrable ability to meet performance claims through quality assurance. What evidence do you have to demonstrate new methods (inc. EUCAST) have been verified as fit for purpose before being applied to clinical isolates and informing clinical decisions. 16

17 ISO 15189: Selection, verification and validation of examination processes Verification of examination procedures The independent verification by the laboratory shall confirm, through obtaining objective evidence (in the form of performance characteristics) that the performance claims for the examination procedure have been met. The performance claims for the examination procedure confirmed during the verification process shall be those relevant to the intended use of the examination results. How can you show the assessors that you can accurately determine the susceptibility of your isolates when using the validated AST method? 17

18 ISO 15189: Selection, verification and validation of examination processes Verification of examination procedures The laboratory shall document the procedure used for the verification and record the results obtained. Show us your verification procedure. Is the technical approach to verification sufficient and suitable for the intended use of the method? Staff with the appropriate authority shall review the verification results and record the review. Who has reviewed the data and signed off the verification? Are they competent to do so? 18

19 Measurement uncertainty uncertainty (of measurement) parameter, associated with the result of a measurement, that characterizes the dispersion of the values that could reasonably be attributed to the measurand (VIM definition) VIM: International Vocabulary of Basic and General Terms in Metrology (ISO) Martin: How sure are you that the measurement is correct? 19

20 ISO 15189: Measurement uncertainty of measured quantity values The laboratory shall determine measurement uncertainty for each measurement procedure in the examination phase used to report measured quantity values on patients samples. Where examinations include a measurement step but do not report a measured quantity value, the laboratory should calculate the uncertainty of the measurement step where it has utility in assessing the reliability of the examination procedure or has influence on the reported result 20

21 Traceability Metrological Traceability Property of a measurement result whereby the result can be related to a reference through a documented unbroken chain of calibrations, each contributing to the measurement uncertainty (VIM) Applicable to: Calibration of callipers, rulers or whatever else is measuring zone diameters How sure are you of thee diameters? Can you demonstrate traceability? 21

22 ISO 15189: Equipment calibration and metrological traceability The laboratory shall have a documented procedure for the calibration of equipment that directly or indirectly affects examination results Where this is not possible or relevant, other means of providing confidence in the results shall be applied - use of certified reference materials 22

23 Documentation Entirely up to you Not applicable (e.g. MU) OK, so how have you come to that conclusion? Often see overarching/standalone procedures on MU, validation, verification & traceability/calibration OK, so how have you applied that to your AST method Need to consider how each applies to AST, do the work and document it 23

24 Initial assessment All of the above should be covered during the ISO 15189:2012 initial assessment Highly likely that findings of non-conformity will be raised in relation to AST Laboratory is expected to propose suitable improvement action at the assessment to address and clear the findings Evidence of improvement action needs to be submitted (12 weeks) for review by the Technical Assessor All findings must be cleared before grant of accreditation can be made 24

25 Schedule of Accreditation 25

26 Schedule of Accreditation 26

27 Extension to Scope (ETS) Proposed changes to your assessed and accredited scope need to be communicated to your UKAS Assessment Manager ETS process is required to add or change tests or methods on your published Schedule of Accreditation New AC6 required detailing the additions / changes plus supporting documentation, must include: SOPs Validation & verification data and summary Uncertainty of measurement estimation Measurement traceability chain 27

28 Extension to Scope (ETS) Many ETS applications run parallel to surveillance / reassessments Might be additional site time, often accommodated in existing effort If appropriate can be done by remote assessment, if so need: IQC performance data and summary EQA performance summary Training / competency records System suitability / evaluation summary Any other specific documentation 28

29 Extension to Scope (ETS) 29

30 Extension to Scope (ETS) UKAS has assessed your competence to use BSAC methodology Your susceptibility testing using BSAC methodology will be accredited UKAS has not assessed your competence to use EUCAST methodology Your susceptibility testing using EUCAST methodology will not be accredited until ETS for this has been granted 30

31 Extension to Scope (ETS) Inform UKAS early in the change control process UKAS can arrange to assess your competence to use EUCAST methodology through ETS process Grant of ETS before clinical use Your susceptibility testing using EUCAST methodology will be accredited on the day you go live 31

32 Thanks for listening Any questions? Martin Stearn, UKAS Senior Assessment Manager UKAS June 2016