ASQ Measurement Quality Division

Transcription

1 World Metrology Day May 20, 2015 Riyadh (May 19), Jeddah (May 20), Dammam (May 21) The Global Impact of Metrology on Quality Dilip Shah E = mc 3 Solutions ASQ Measurement Quality Division

2 The International Year of the Light!

3 Abstract Precise measurements assure that products manufactured meet the specified requirements of the customers. It is assumed that the processes used to manufacture products make the appropriate measurements to deliver a consistent product for the customer.

4 Abstract The measurement process of making a product is transparent to the end user and is taken for granted. This presentation discusses the importance of specifying, verifying and assessing calibration service providers so that the global impact of the variability on making measurements in any environment or geographical region is minimized.

5 Learning Objectives Defining calibration requirements for measurement instruments. Identifying and selecting the right calibration provider that is globally recognized. Comprehending the calibration report from the calibration provider. Interpreting the role of mutual recognition arrangement in laboratory accreditation.

6 Introduction The customer outsources the same part to be made by two different suppliers in two different countries. One part is made by the supplier in one country that makes its measurements in imperial units and the other county makes its measurements in SI units.

7 Introduction Let the part be specified in inches: in. ± in. Converting the units in SI Units: mm ± mm

8 Introduction Rounding the units to 3 decimal places due to measuring device resolution: mm ± mm Converting back to Imperial Units of measure: in. ± in. (a 2.36 % error in tolerance)

9 Introduction As seen, this simple conversion exercise already creates many problems. The fit and finish of part tolerances adds more complexities. Lack of basic metrology (science of measurement) practices add more challenges.

10 Back to Basics Measurements require Metrological Traceability. Metrological Traceability is defined as Property of a measurement result whereby the result can be related to a reference through a documented unbroken chain of calibrations, each contributing to the measurement uncertainty.

11 Key Words in the Definition documented unbroken chain of calibrations each contributing to the measurement uncertainty.

12 Metrological Traceability Visualized BIPM (0.005) NATIONAL METROLOGY INSTITUTES (NIST) (0.011) REFERENCE METROLOGY LABORATORIES (0.02) WORKING METROLOGY LABORATORIES (0.05) GENERAL CALIBRATION (0.07) PROCESS MEASUREMENT (0.1) Measurement Uncertainty Data is cumulative from one level of calibration to another!

13 Who verifies Metrological Traceability? Independent Third Party Accreditation Bodies accredit calibration laboratories who calibrate measuring instruments. Accreditation Bodies accredit laboratories to ISO/IEC standard. Other standards such as ANSI Z540.3 are also comparable to ISO/IEC

14 What is accreditation? Independent evaluation of conformity assessment bodies against recognized standards to ensure their impartiality and competence to carry out specific activities, such as tests, calibrations, inspections and certifications.

15 Who Accredits the Accrediting Bodies? ILAC REGIONAL ACCREDITATION COOPERATION ACCREDITING BODIES ACCREDITED LABORATORIES Accrediting Bodies are assessed to ISO/IEC Standard

16 What is ILAC? International Laboratory Accreditation Cooperation Established in 1977 to promote communication among laboratory accreditation bodies of the world. Formalized as a cooperation in 1996 with 44 bodies signing a Memorandum of Understanding (MOU). On 2 November 2000, a mutual recognition arrangement (MRA) was signed, among those members which had successfully completed a peer evaluation. MRA was extended in October 2012 to include inspection body accreditation

17 What is ILAC? International Laboratory Accreditation Cooperation 89 Signatories (Full Members) to the Arrangement, representing approximately 67 economies. ILAC was incorporated in the Netherlands on 20 January Approx. 45,000 laboratories & over 7,800 inspection bodies have been accredited by the 90 ILAC Full Members & Associates.

18 The ILAC Mutual Recognition Arrangement ILAC EA ARAC APLAC IAAC AFRAC SADCA EA European Cooperation for Accreditation APLAC Asia Pacific Laboratory Accreditation Cooperation ILAC International Laboratory Accreditation Cooperation IAAC Inter-American Accreditation Cooperation SADCA Southern African Development Community Accreditation AFRAC African Regional Accreditation Cooperation ARAC Arab Accreditation Cooperation Unaffiliated Bodies Peer evaluated ABs who are not geographically located in one of the established regions

19 Coverage of the ILAC MRA (June 2013) ILAC MRA Signatories ILAC Associate Members ILAC Affiliate Members

20 Benefits of the ILAC MRA Eliminates the expense of government-administered accreditation and conformity assessment programs Increases the confidence in test results used to determine compliance with regulations Eliminates duplication in conformity assessment by reliance on recognized accreditation bodies Reduces costs of trade Encourages conformity assessment at the source of supply Upholds commitments of the WTO TBT Agreement

21 MRAs Between Accreditation Bodies The Fundamental Purpose - A laboratory or inspection body accredited by one partner has equivalent competence to a laboratory or inspection body accredited by the other partner(s).

22 How does this affect the Measurements? The question to ask is how does this affect the measurements an organization makes? Organizations use measuring instruments. Instruments require periodic calibration to verify accuracy and precision. Calibration requires Metrological Traceability.

23 Steps to Ensure Good Measurements Find an accredited laboratory for the organization s calibration requirements from the accrediting bodies web site.

24 Steps to Ensure Good Measurements Choose from the accrediting body s web site: Calibration Provider Review the Calibration Provider s Scope of Accreditation (SoA) Ensure the SoA covers the Parameter Range and Calibration and Measurement Capability (CMC) for instruments requiring calibration. A CMC is the laboratories calibration and measurement capability.

25 Scope of Accreditation

26 Steps to Ensure Good Measurements Specify EXACTLY what is required in the purchase requisition to the accredited laboratory. EXAMPLE BELOW: ISO/IEC Accredited Calibration with actual AS FOUND, AS LEFT and MEASUREMENT UNCERTAINTY data is required on all calibration certificates. Blanket statements of compliance and statements of Measurement Uncertainty shall not be accepted. Calibration interval should not be specified

27 Steps to Ensure Good Measurements Remember that without measurement uncertainty, one does not have the metrological traceability of measurements. Review the Calibration Report. Ensure that the requirements specified by the customer (you) are met upon receipt of calibration report.

28 Steps to Ensure Good Measurements Calibration Report should meet the requirements specified in ISO/IEC 17025:2005 Section 5.10 for format and content in addition to any specific customer requirements.

29

30 ISO/IEC 17025:2005 Section 5.10 requirements Test reports and calibration certificates Each test report or calibration certificate shall include at least the following information, unless the laboratory has valid reasons for not doing so: a) a title (e.g. Test Report or Calibration Certificate ); b) the name and address of the laboratory, and the location where the tests and/or calibrations were carried out, if different from the address of the laboratory; Calibration Report ABC Laboratory 123 Anystreet Anytown, Anystate U. S, A

31 ISO/IEC 17025:2005 Section 5.10 requirements c) unique identification of the test report or calibration certificate (such as the serial number), and on each page an identification in order to ensure that the page is recognized as a part of the test report or calibration certificate, and a clear identification of the end of the test report or calibration certificate; d) the name and address of the customer; e) identification of the method used; f) a description of, the condition of, and unambiguous identification of the item(s) tested or calibrated;

32 ISO/IEC 17025:2005 Section 5.10 requirements g) the date of receipt of the test or calibration item(s) where this is critical to the validity and application of the results, and the date(s) of performance of the test or calibration; h) reference to the sampling plan and procedures used by the laboratory or other bodies where these are relevant to the validity or application of the results; Date: Customer Name: Purchase Order No: Address: City: State: Zip Code: Telephone: Fax: Equipment I.D.: Other Description: Received Condition: Instrument found to be in Tolerance: Date Received: Procedure/Test Method (Including Revision): March 19, 2014 YES Serial Number: Date Calibrated: Calibration Location: Returned Condition: NO Report No:

33 ISO/IEC 17025:2005 Section 5.10 requirements i) the test or calibration results with, where appropriate, the units of measurement; j) the name(s), function(s) and signature(s) or equivalent identification of person(s) authorizing the test report or calibration certificate; k) where relevant, a statement to the effect that the results relate only to the items tested or calibrated. Measurement Data Range/Standards Used (Unit of Measurement) Parameter (e.g. Temp, RH, CO2) Set Point Indicated Value AS FOUND AS LEFT T.U.R. ABC LAB UUT Calibration Total UUT UUT Adjusted? Uncertainty Process Tolerance Resolution (k=2) Uncertainty (k=2) Measurement Uncertainty taken into consideration Temperature 4.0E+0 3.9E E E+0 NA 1.0E+0 1.0E E-3 1.2E

34 ISO/IEC 17025:2005 Section 5.10 requirements This is to certify that ABC Laboratories is accredited and that it s calibration system is in compliance with ANSI NCSL Z540-1, Z540.3 AND ISO/IEC17025:2005. Calibration of standards, reference standards and Intermediate standards in these calibration has been certified traceable to the SI Units of measurement through NIST or other NMI recognized under CIPM or has been derived from accepted values of natural physical constants. We Certify the equipment was calibrated using the above test equipment. Best Uncertainties represent expanded uncertainties expressed at approximately the 95% confidence level using a coverage factor of k=2. The results documented in this certificate relate only to the item(s) calibrated or tested. This certificate may not be reproduced, except in full, without the written approval of ABC Laboratories Boilerplate Language at the bottom of the Certificate

35 ISO/IEC 17025:2005 Section 5.10 requirements In addition to the requirements listed in , calibration certificates shall include the following, where necessary for the interpretation of calibration results: a) the conditions (e.g. environmental) under which the calibrations were made that have an influence on the measurement results; b) the uncertainty of measurement and/or a statement of compliance with an identified metrological specification or clauses thereof; c) evidence that the measurements are traceable (see note 2 in ).

36 ISO/IEC 17025:2005 Section 5.10 requirements The calibration certificate shall relate only to quantities and the results of functional tests. If a statement of compliance with a specification is made, this shall identify which clauses of the specification are met or not met. When a statement of compliance with a specification is made omitting the measurement results and associated uncertainties, the laboratory shall record those results and maintain them for possible future reference. When statements of compliance are made, the uncertainty of measurement shall be taken into account.

37 ISO/IEC 17025:2005 Section 5.10 requirements When statements of compliance are made, the uncertainty of measurement shall be taken into account % 2.14% % Guardband = Lower Tolerance Uncertainty (k=2) % % % 2.14% 0.13 % Nominal Value 0.13 % 2.14% % % % Guardband = Upper Tolerance Uncertainty (k=2) % 2.14% 0.13 % -3S LOWER -2S -1S Tolerance 0 +1S +2S +3S 68.26% 95.46% 99.73% Measurement Uncertainty -3S -2S -1S 0 +1S +2S +3S UPPER Tolerance 68.26% 95.46% 99.73% Measurement Uncertainty DEVICE UNDER TEST TOLERANCE

38 ISO/IEC 17025:2005 Section 5.10 requirements When an instrument for calibration has been adjusted or repaired, the calibration results before and after adjustment or repair, if available, shall be reported. Range/Standards Used (Unit of Measurement) AS FOUND AS LEFT Adjusted? Parameter (e.g. Temp, RH, CO2) Set Point Indicated Value Measurement Uncertainty taken into consideration Temperature 4.0E+0 3.9E E E+0 NA

39 ISO/IEC 17025:2005 Section 5.10 requirements A calibration certificate (or calibration label) shall not contain any recommendation on the calibration interval except where this has been agreed with the customer. This requirement may be superseded by legal regulations. Date Due /Calibration Interval: Note: Unless specified in the contract or per regulatory requirement; purchase order or by other means, calibration interval assignment and adjustment are the responsibility of the end user. It should be noted that many factors can cause the equipment to go out of calibration before the recommended interval.

40 ISO/IEC 17025:2005 Section 5.10 requirements Testing and calibration results obtained from subcontractors When the test report contains results of tests performed by subcontractors, these results shall be clearly identified. The subcontractor shall report the results in writing or electronically. When a calibration has been subcontracted, the laboratory performing the work shall issue the calibration certificate to the contracting laboratory.

41 ISO/IEC 17025:2005 Section 5.10 requirements Electronic transmission of results In the case of transmission of test or calibration results by telephone, telex, facsimile or other electronic or electromagnetic means, the requirements of this International Standard shall be met (see also 5.4.7).

42 ISO/IEC 17025:2005 Section 5.10 requirements Format of reports and certificates The format shall be designed to accommodate each type of test or calibration carried out and to minimize the possibility of misunderstanding or misuse.

43

44 ISO/IEC 17025:2005 Section 5.10 requirements Amendments to test reports and calibration certificates Material amendments to a test report or calibration certificate after issue shall be made only in the form of a further document, or data transfer, which includes the statement: Supplement to Test Report [or Calibration Certificate], serial number... [or as otherwise identified], or an equivalent form of wording. Such amendments shall meet all the requirements of this International Standard.

45 ISO/IEC 17025:2005 Section 5.10 requirements When it is necessary to issue a complete new test report or calibration certificate, this shall be uniquely identified and shall contain a reference to the original that it replaces Report No: abc Replaces Report No: xyz

46 ISO/IEC 17025:2005 Section 4.6 requirements The laboratory shall have a policy and procedure(s) for the selection and purchasing of services and supplies it uses that affect the quality of the tests and/or calibrations. Procedures shall exist for the purchase, reception and storage of reagents and laboratory consumable materials relevant for the tests and calibrations. Did you have a process for Purchasing that minimizes the risk?

47 ISO/IEC 17025:2005 Section 4.6 requirements The laboratory shall ensure that purchased supplies and reagents and consumable materials that affect the quality of tests and/or calibrations are not used until they have been inspected or otherwise verified as complying with standard specifications or requirements defined in the methods for the tests and/or calibrations concerned. These services and supplies used shall comply with specified requirements. Records of actions taken to check compliance shall be maintained. Did you get what you specified?

48 ISO/IEC 17025:2005 Section 4.6 requirements Purchasing documents for items affecting the quality of laboratory output shall contain data describing the services and supplies ordered. These purchasing documents shall be reviewed and approved for technical content prior to release. Did you specify the correct requirements and reviewed for technical content?

49 ISO/IEC 17025:2005 Section 4.6 requirements The laboratory shall evaluate suppliers of critical consumables, supplies and services which affect the quality of testing and calibration, and shall maintain records of these evaluations and list those approved. Is the laboratory evaluation process robust and minimizes risk of choosing an unqualified Clibration provider?

50 ISO/IEC 17025:2005 Section 4.6 requirements Specify on the Purchase Order/Requisition: As found As left data Measurement Uncertainty Data required Other: Review and record of purchasing documents before releasing them? Review and record of receipt of purchased service/item? Any out of tolerance issues upon receipt of calibration?

51 Beware of these statements or data on the calibration certificate!! The uncertainty of the measurement is larger than the tolerance of Unit Under Test (UUT). Standards used less accurate than Unit Under Test (UUT) All uncertainties are stated at 95% at k=2 (But no uncertainty data that is why you review certificates per ISO/IEC 17025:2005 Section 4.6) Traceable to NIST: NIST Report Number: xxxxxxx.yyyyy A single PASS statement NO DATA. As Left data worse than As found data (Then why did I send this unit in for calibration?)

52 Beware of these statements or data on the calibration certificate!! Data on certificate same as last year s data including last year s date (I could have photocopied last year s certificate and saved a lot of money ) Wrong serial number of Unit Under Test (UUT) on the certificate. Accreditation logo used on the certificate but the certificate number has expired or belongs to another laboratory (constitutes misrepresentation and fraud Report to the Accrediting Body).

53 In Summary Doing the right things minimizes the impact of measurements at the local level This includes: Proper Metrology (Good Measurement Science and Practices). Metrological Traceability that can be verified. Proper documentation. Global Recognition of measurements through ILAC MRA (Accredited Provider).

54 In Summary What is minimized locally helps propagate the effects of measurements on quality at each ascending stage to a global level. The measurement errors are minimized and provides confidence in minimizing the probability of false acceptance in the decision making process. Product Acceptance, Safety Margin, Food and Drug Safety, Aviation Safety.

55 Global ILAC BIPM REGIONAL COOPERATION REGIONAL COOPERATION Verifies Traceability ACCREDITING BODY Verifies Traceability Traceable to SI Units through NMI (NIST in USA) Verifies Traceability ACCREDITING BODY Verifies Traceability Accredited Laboratory Accredited Laboratory Accredited Laboratory Accredited Laboratory Calibrates Instruments Calibrates Instruments Calibrates Instruments Calibrates Instruments Process Measurements Process Measurements Process Measurements Process Measurements Process Measurements

56 Property of a measurement result whereby the result can be related to a reference through a documented unbroken chain of calibrations, each contributing to the measurement uncertainty GLOBAL International Bureau of Weights and Measures (BIPM) Defines the 7 SI Units of Measurement International Laboratory Accreditation Cooperation (ILAC) Defines Rules for Accreditation U (BIPM@95% C.I.) = 1.000e-9 REGIONAL REGIONAL ACCREDITATION COOPERATION Peer Evaluation of Accrediting Bodies per ISO/IEC National Measurement Institute (NIST in USA) Traceability to SI Units Traceability to SI Units U (NMI@95% C.I.) = 10.05e-9 Verifies Traceability Accredited Primary Laboratory, OEM U( PRIMARY C.I.) = 100.5e-9 Calibrates Standards ACCREDITING BODY Competence of Laboratories per ISO/IEC Verifies Traceability Accredited Laboratory Calibrates Standards U 9(ACCREDITED C.I.) = 510e-9 METROLOGY Verifies Traceability In-house calibration Laboratory U (CALIBRATION LAB@95% C.I.) = 1.12e-6 Calibrates Instruments Instruments used in Process Measurements U (INSTRUMENT@95% C.I.) = 1.64e-6 QUALITY

57 Thank you!

58