Quality Management of the Total Examination Process. Elvar Theodorsson

Transcription

1 Quality Management of the Total Examination Process Elvar Theodorsson

2 2 The total testing chain consists of much more than the analytical phase The true value of laboratory results are expressed in their contribution to patient care Laboratory medicine needs to substantially increase its focus on the extraanalytical phases Result interpreted in full clinical context Results conveyed to clinician Interpretation in the laboratory Quality control Clinical response to result Postanalytic phase Clinical phase Analytic phase Measuring sample Test ordered Preanalytic phase Patient identification Calibration Patient preparation Taking sample Transporting sample Sample identification

3 3 Analytical phase = metrology % of errors Science most laboratory workers are comfortable with Can be dealt with by mathematics, statistics, logistics etc. Extraanalytical phase = humanities % of errors Common sense and the sciences of the humanities Management-, business and change sciences Bonini et al. Clinical Chemistry 2002; 48(5):

4 4 Internal- and external areas in laboratory work where methods from the humanities are needed Quality improvements in the extraanalytical phases Reduce pre- and postanalytical errors in co-operation with the clients of the laboratory Quality control and measurement uncertainty estimates using patient samples in order to reduce bias amongst several measurement systems likely to encounter samples from the patient over extended periods of time

5 5 ISO 9000 s definition of quality Degree to which a set of inherent characteristics fulfils requirement

6 6 The requirements of the users define quality Examples of requirements generally known to the users Short turnaround times Low prizes Low measurement uncertainty including minimal bias Examples of laboratory services usually not known to the users Laboratories can consolidate and thereby reduce cost and increase quality Very many analytes can be measured in a small volume of the same sample through aliquotting in the laboratory

7 Quality management, assurance and control Quality management Establishing and maintaining a climate of continuous improvements in order to deliver high-quality laboratory services to healthcare Quality assurance A component of quality management which provides confidence that quality requirements will be fulfilled These requirements are in all parts of the testing chain not only in the analytical phase Quality control Monitoring to indicate needed corrective responses

8 8 Quality management-, assurance- and control have their roots in the telephone industry

9 9 Western Electric Company 1905, Hawthorne Works, near to Chicago, employing persons Hawthorne effect - individuals modify their behaviour in response to their awareness of being observed Walter A. Shewhart

10 10 W. Edwards Deming Worked with Shewhart in the 1930:s at Western Electric Studied under Sir Ronald Fisher and Jerzy Neyman at University College, London in 1936 Book: Statistical Method from the Viewpoint of Quality Control, 1936 Worked in Japan from 1947 Deming orthogonal regression

11 11 Joseph M. Juran Western Electric, Hawthorn, 1924 Focused on management for quality Worked in Japan from 1954 Juran s Quality Control Handbook 1951 Pareto principle roughly 80% of the effects come from 20% of the causes

12 12 Levey-Jennings control chart The distance from the mean is measured in standard deviations (SD) Henry and Segalove control chart The relation to events, especially dates plotted on the X-axis

13 13 Westgard, de Verdier, Groth, Aronsson Westgard JO, Groth T, Aronsson T, Falk H, de Verdier CH (1977) Performance characteristics of rules for internal quality control: probabilities for false rejection and error detection. Clin Chem 23: Multirules Power function graphs

14 14 CLIA 88, RiliBÄK, ISO accreditation, IVD - directive Improvements in technical/metrological aspects of laboratory medicine IVD-directive, 1993 traceability and the responsibility of the manufacturers ISO extended focus on the extraanalytical phases Current trend Re-orientation of medical laboratories to closer collaboration amongst laboratories and specialties and also closer collaboration with their users to increase quality in the entire testing chain

15 15 Side effects of regulatory systems Obtained ISO accreditation/certification Accreditation a prerequisite for the business Fear of loosing accreditation Substantial changes introduce risks of tensions, lack of clarity and subsequent risks to the accreditation Resistance to change risks automation, consolidation, co-operation between disciplines of laboratory medicine etc.

16 16 The most important characteristic of manufacturing- and service industries continuously improving quality Focus on customer needs Good relations and frequent contact with customers Emphasis on innovation and developments rather than on documenting what has always been done

17 17 Perfect is the enemy of good Pescetti 1603, Voltaire 1770 or Perfect may be the enemy of good Is it possible that a portion of the about 3% of the total cost of laboratories that is currently spent on accreditation-related activities can be better spent on less documentation and more development? Better a diamond with a flaw than a pebble without Confucius

18 18 Major improvements in quality are likely to stem from methods from the humanities Qualitative methods Change management methods Tailored interventions/implementations Motivational factors, practice coaches and persistence Unfortunately there is no solid evidence as to the best methods to employ for the purpose of changing practices in healthcare

19 19 Total quality management Efforts to establish and maintain a climate of continuous improvements in the laboratory in order to deliver high-quality services to healthcare Customer needs define quality Continuous monitoring and systematic analysis of crucial work processes Continued improvements of the processes most important to healthcare The top leadership of the laboratory is responsible for the quality and quality improvement processed

20 20 Total quality management (TQM) Quality assurance must be implemented, managed and maintained by the leadership of the laboratory Procedures, processes and systems and not people represent the major obstacles to optimal quality Avoid treating the symptoms. Treat the disease instead with the intention to cure Every employee is responsible for hers/his part in the overall quality of the laboratory

21 21 Stairs should be cleaned starting at the top Dust gravitates downwards

22 22 Substantial quality improvements in laboratory medicine call for consolidation and integration Usually meets fierce resistance by most categories of the employees Dependent on political or administrative decisions high up in the decision hierarchy Politicians and administrators usually understand the need for consolidation and integration. We in the laboratories represent the most important obstacles

23 23 Diagnostic uncertainty

24 If different measurements result in different results for the same patient sample Physicians and patients will become confused Clinical guidelines will become less useful Suboptimal treatments and monitoring practices may be implemented

25 A bias of + 5 units means that healthy persons are diagnosed sick Decision limit Decision limit Healthy Healthy Sick Sick

26

27

28

29 29 Type of errors Systematic error Qualitative characteristics of errors Trueness Quantitative measures of errors Bias Menditto A. M. Patriarca, M. Magnusson B (2007). "Understanding the meaning of accuracy, trueness and precision." Accred Qual Assur 12: Total error Accuracy Measurement uncertainty Random error Precision Standard deviation

30 Quality management of the total examination process 30 Reduce pre- and postanalytical errors by teaching and learning from the clients of the laboratory

31 Quality management of the total examination process 31 Causes of variation and uncertainties in the testing chain Result interpreted in full clinical context Results conveyed to clinician Interpretation in the laboratory Clinical response to result Postanalytical phase Clinical phase Analytical phase Biological variation Preanalytical phase Test ordered Patient identification Patient preparation Taking sample Transporting sample Quality control Sample identification Measuring sample Calibration

32 32 Diagnostic uncertainty in laboratory medicine The variation/uncertainty a health worker needs to count in when using a result of laboratory analysis in the diagnosis and monitoring of patients Diagnostic uncertainty is a property of the measurement result itself and is influenced by several uncertainty components including Biological variation Preanalytical variation Analytical variation (including uncertainty in the calibration) Postanalytical variation

33 33 Preanalytical - sampling variation Standing, sitting or lying down Rest or physical activity Venous stasis Mixing of the sample with stabilizers/anticoagulants Transport of the sample to the laboratory

34 34 Biological variation 1956

35 35 Biological variation

36 36 Biological variation

37 37 Biological variation Explain its causes Inform about its size

38 Quality management of the total examination process Laboratory medicine exists in order to reduce diagnostic uncertainty U biological U preanalytical U analytical Patient Patient sample Measurement procedure 38 U postanalytical U diagnostic

39 39 GUM Evaluation of measurement data Guide to the expression of uncertainty in measurement

40 40 GUM incomprehensible or? Bayesian statistics instead of frequentist statistics Covariance matrices, partial derivatives, Taylor expansions The traditional GUM methods have been created to cater for all situations. We can commonly perform most of our measurement uncertainty calculations using much simpler methods including frequentist statistics

41 41 The TROLL book Handbook for Calculation of Measurement Uncertainty in Environmental Laboratories /content/article/27-technicalreports/environment/33-calculation-ofmeasurement-uncertainty.html

42 42 MUkit US/Services/Calibration_ser vices_and_contract_laborat ory/mukit Measurement_ Uncertainty_Kit

43 43 Measurement uncertiatinty using the Kragten method Ivo Leito, Youtube watch?v=qwqb08hzh-c

44 44 Monte Carlo simulation of diagnostic uncertainty Biological variation Preanalytical variation Analytical variation Postanalytical variation Diagnostic uncertainty 1. No mathematical function (output function) is needed to evaluate the diagnostic uncertainty 2. No assumptions about the input quantities is needed in addition to the assumption that they follow a Gaussian distribution 3. There is no need to calculate partial derivatives 4. It is unaffected by partial derivatives that vanish when estimating input quantities

45 45 Biological variation Resampling estimation of diagnostic uncertainty Preanalytical variation Analytical variation Postanalytical variation Diagnostic uncertainty 1. No mathematical function (output function) is needed to evaluate the diagnostic uncertainty 2. No assumptions about the input quantities is needed in addition to the assumption that they follow a Gaussian distribution 3. There is no need to calculate partial derivatives 4. It is unaffected by partial derivatives that vanish when estimating input quantities

46 46 Estimating bias Purchasing certified reference materials from companies or organizations of high metrological competence and comparing the stated concentration with the concentration your own methods shows Comparing the concentrations your method measured in natural samples with the concentrations a reference method measured in the same sample Participating in programs for external quality control. Most of these programs are based on consensus concentrations in modified control samples, but some few are based on comparison to reference methods. The latter are frequently preferable.

47 Two different models for external quality control / proficiency testing A Lab A B Lab A 47 Lab H Lab B Lab H Lab B Lab G ECQ Lab C Lab G ECQ Lab C Lab F Lab D Lab F Lab D Lab E Lab E

48 48 Advantages of split samples for quality control 1. The the material has optimal matrix properties (is commutable) 2. The material is available without cost for all laboratories accepting routine patient samples 3. There is general agreement that all measurement systems and reagents should optimally result in identical results when analyzing the same patient samples 4. The methods are optimal for identifying the measurement system(s) in the organization that contribute the largest part of the overall measurement uncertainty due to bias. Split sample methods are laborious in the absence of effective computerized systems, but convenient when properly implemented

49 49 Types of validation Single laboratory method validation is appropriate where the method is used for a specific purpose in a specific laboratory by personnel with the appropriate training. Full method validation includes, in addition to the procedures employed in single laboratory validation an interlaboratory study (collaborative study/ collaborative trial) with many measurement instruments several operators etc. The performance characteristics of the measurement method over extended periods of time are also studied in full method validation, including the effects of lot-to-lot variations etc. Full diagnostic method validation is establishing the diagnostic properties of the method e.g. in health and disease

50 50 Full method validation Samples from a patient are over time likely to be measured in different laboratories using different methods as the patient visits primary care and different levels of hospital care. In this case, full validation of all the methods a patient is likely to encounter is crucial, including efforts to minimize measurement uncertainty, in particular the effects of bias which usually is by far the major component contributing to measurement uncertainty. If this not feasible, samples from the same patient should be measured using the same measurement method and measurement instrument over time using reference interval established for that particular method.

51 51 Full diagnostic method validation Participants With disease Without disease Positive test True positives False positives (type I error) Negative test False negatives True negatives (type II error) Total with diease Total without diease [Sensitivity] [Specificity] Total positive [PPV] Total negative [NPV]

52 52 Full diagnostic method validation

53 Thank you