De-Mystifying Calibration Accreditation 15 Jan 2013

De-Mystifying Calibration Accreditation 15 Jan 2013 Ryan Fischer Laboratory Accreditation Bureau Chair, NCSLI 183 Accreditation Resources Committee Bob Stern Agilent Technologies, Inc. Co-chair, NCSLI 174 Standards Writing Committee

How fast do cars travel on average on Interstate Highways? Hint: Answer is not the posted speed limit. De-facto speed limit is what the police enforce.

How can end users know your instruments are tested properly according to ISO/IEC 17025, ANSI Z540.3, or even Z540-1? Hint: It s not just what a calibration lab claims. It s about what is enforced.

Calibration Modules 1 3 2 Why Calibrate What is it? Calibration Adjustments Who can do it? Measurement Uncertainty How accurate? 4 Traceability Why important? Nov 13 Dec 13 Standards Compliance Dec 13 Jan 15 Feb13 Which ones? Accreditation Why it matters? Z540.3 Compliance 5 How? 7 6

1 Why Calibrate? YouTube calibration videos www.youtube.com/agilentcalrepair 1 Why Calibrate http://youtu.be/hwsxbraxn_4 2 Adjustments http://youtu.be/iqslmygtoog 3 Accuracy/Uncertainty http://youtu.be/envl8byzmg0 4 Traceability http://youtu.be/-emq0kakiyq 5 Standards compliance http://youtu.be/8s2xx3c0_ds 6 Accreditation http://youtu.be/sh2wewublr8 7 ANSI Z540.3 10½ min

1 Why Calibrate? What are your calibration service expectations? When you send an instrument for calibration to an outside lab, what do you expect them to do?

1 Why Calibrate? What is calibration? 1. Characterize your device/instrument Measure the actual performance! 2. How Accurate? Measurement Uncertainty 3. Pass or Fail? Statement of Conformance Accreditation is the confirmation of competence and skills according to ISO/IEC 17025 by a (3 rd party) accreditation body. 4. Advise client if found out-of-tolerance. Adjust, if instructed, and repeat calibration. Formal definition at: http://www.bipm.org/utils/common/documents/jcgm/jcgm_200_2012.pdf

1 Why Calibrate? Attn: Engineers! Identify the key specs for your company - and check how your instrument performed. REVIEW Remember: The measurement report is physical evidence of work performed!

Accreditation Why it matters, even if you don t order an Accredited Cal Ryan Fischer Laboratory Accreditation Bureau Chair, NCSLI 183 Accreditation Resources Committee Bob Stern Agilent Technologies, Inc. Co-chair, NCSLI 174 Standards Writing Committee NCSL INTERNATIONAL SERVING THE WORLD OF MEASUREMENT SINCE 1961

What is accreditation? Per ISO/IEC 17000:2004 it is a third-party attestation related to a conformity assessment body conveying formal demonstration of its competence to carry out specific conformity assessment tasks covering: technical competence of staff validity and appropriateness of test methods traceability of measurements and calibrations to national standards suitability, calibration and maintenance of test equipment testing environment sampling, handling and transportation of test items quality assurance of test and calibration data NCSL INTERNATIONAL SERVING THE WORLD OF MEASUREMENT SINCE 1961 183 Accreditation Committee

To which standards will you comply? ISO/IEC 17025: 2005 ISO/IEC Guide 98-3:2008 ILAC-G8 ILAC-P14 ANSI/NCSL Z540-1-1994 rescinded 2007 General Requirements for the Competence of Testing and Calibration Laboratories Guide for Expression of Uncertainty of Measurements Guidelines on Assessment and Reporting of Compliance with Specification Policy for Uncertainty in Calibration ANSI/NCSL Z540.3-2006 ISO 9001:2008 Requirements for the calibration of Measuring and Test Equipment Quality management systems - requirements ILAC is the "International Laboratory Accreditation Cooperation" - a network of mutual recognition arrangements among accreditation bodies. NCSL INTERNATIONAL SERVING THE WORLD OF MEASUREMENT SINCE 1961 183 Accreditation Committee

ILAC-MRA (Mutual recognition arrangement) https://www.ilac.org/documents/mra%20signatories.pdf Each accreditation body signatory to the Arrangement agrees to abide by its terms and conditions and by the ILAC evaluation procedures and shall: Maintain conformance with the current version of ISO/IEC 17011, related ILAC guidance documents, and a few, but important, supplementary requirements, and Ensure that all accredited laboratories comply with ISO/IEC 17025 or ISO 15189 (for medical testing laboratories) and related ILAC policy and guidance documents. Note: ILAC is the "International Laboratory Accreditation Cooperation - a network of mutual recognition arrangements among accreditation bodies. NCSL INTERNATIONAL SERVING THE WORLD OF MEASUREMENT SINCE 1961 183 Accreditation Committee

Scope-of-accreditation How it works 1. The calibration lab tells the AB which parameters it seeks accreditation for 2. For each requested parameter the assessor then: i. Assess quality system ii. iii. iv. Spends time with the technicians assessing their competency/measurement technique to measure this particular parameter Assess the measurement uncertainty documentation Assess traceability path(s) to SI 1 units via a NMI 2 such as NIST, NPL, PTB, NMIJ, etc. 1. SI = International System of units 2. NMI = National institute of Metrology NCSL INTERNATIONAL SERVING THE WORLD OF MEASUREMENT SINCE 1961 183 Accreditation Committee

Agilent Service Center accreditation/ Roseville, CA

Compare cal lab quality Check for accreditation, right? Cal Lab ABC Cal Lab XYZ Is this all you need to know? Source : http://www.a2la.org/dirsearchnew/newsearch.cfm NCSL INTERNATIONAL SERVING THE WORLD OF MEASUREMENT SINCE 1961 183 Accreditation Committee

Scope-of-accreditation Cal labs XYZ vs. ABC I. Electrical DC/Low Frequency XYZ Lab ABC Lab DC Voltage Generate DC Voltage Measure DC Current Generate DC Current Measure Resistance Generate Resistance - Measure AC Current Generate AC Current - Measure AC Voltage - Generate AC Voltage - Measure AC Voltage Flatness - Generate AC Voltage Flatness - Measure Resistance - Generate Resistance - Generate - Shunts portion Capacitance - Generate Electrical Calibration of Thermocouple Indicators, Type J II. Electrical RF / Microwave Amplitude Modulation Measure 7 Frequency Modulation - Measure 8 Digital Modulation - Measure RF Absolute Power Measure 5 Tuned RF Power RF Absolute Power - Generate Power Sensor Calibration Factor 9 Thermal Noise Figure System Measure 9 Thermal Noise Measure Pulse - Generate Pulse - Measure CISPR Pulse Response Attenuation - Generate Attenuation - Measure Reflection S11 / S22 - Measure Reflection S12 / S21 - Measure Missing s means the lab is NOT accredited for these parameters! NCSL INTERNATIONAL SERVING THE WORLD OF MEASUREMENT SINCE 1961 183 Accreditation Committee

The risk to you of unaccredited parameters! Risk that your instrument is operating out-of-spec Upper specification A 95% expanded uncertainty interval B Nominal For The actual parameters instrument accuracy outside you the can CMC depend 1, on you is only have as good no idea as the [expanded] measurement uncertainty of your last calibration. what the sizes of the yellow(mu) & red(risk) bars are! Lower specification 1. CMC = Calibration Measurement Capability, see annex, https://www.ilac.org/documents/ilac_p14_12_2010.pdf NCSL INTERNATIONAL SERVING THE WORLD OF MEASUREMENT SINCE 1961 183 Accreditation Committee

Accredited Agilent service locations near you www.agilent.com/find/assist

Agilent uses 15 accreditation bodies worldwide Country Accreditation Body Name Accredited Calibration Facilities Australia NATA National Association of Testing Authorities Brazil ACLASS ANSI-ASQ National Accreditation Board Canada A2LA American Association for Laboratory Accreditation China CNAS China National Accreditation Service for Conformity Assessment France COFRAC Comité Français d'accréditation Germany UKAS United Kingdom Accreditation Service India NABL National Accreditation Board for Testing and Calibration Laboratories Israel ISRAC Israel Laboratory Accreditation Authority Japan JCSS Japan Calibration Service System of International Accreditation Korea KOLAS Korea Laboratory Accreditation Scheme Malaysia SAMM Skim Akreditasi Makmal Malaysia (Malaysian Department of Standards laboratory accreditation Mexico A2LA program) American Association for Laboratory Accreditation Singapore SAC Singapore Accreditation Council South Africa SANAS South African National Accreditation Service Spain ENAC Entidad Nacional de Acreditación Taiwan TAF Taiwan Accreditation Foundation United Kingdom UKAS United Kingdom Accreditation Service United States ACLASS or A2LA ANSI-ASQ National Accreditation Board or American Association for Laboratory Accreditation www.agilent.com/find/assist

Checklist for non-accredited Cals 1 1. Review the scope of accreditation for your cal supplier 2. Check to see that the scope covers the specified parameters of your instrument(s). 3. For any parameters your instrument requires not covered in the scope, ask the cal lab what is the expanded uncertainty for those performance tests. Your risk of being out-of-specification after cal is lowest when the expanded uncertainty is known and is significantly smaller than the instrument spec. 1. Note: this checklist already performed with Accredited Cals. NCSL INTERNATIONAL SERVING THE WORLD OF MEASUREMENT SINCE 1961 183 Accreditation Committee

Specific requirements to include an AB 1 symbol on a calibration certificate (a.k.a. Accredited Cal ) Americas ISO/IEC 17025:2005 ILAC-P8:12/2012 ILAC-MRA: Supplementary Requirements and Guidelines for the Use of Accreditation Symbols and for Claims of Accreditation Status by Accredited Laboratories & Inspection Bodies ILAC-P14 ILAC Policy for Uncertainty in Calibration ISO GUM measurement uncertainty for all measured points All measurements within scope-of-accreditation or annotated in report, if not. Europe, Japan, Far East Same as above plus Pass/Fail/Uncertain reporting per ILAC-G8 1. AB means Accreditation Body NCSL INTERNATIONAL SERVING THE WORLD OF MEASUREMENT SINCE 1961 183 Accreditation Committee

Guidelines on the Reporting of Compliance with Specification - ILAC-G8:03/2009 Source: www.ilac.org/documents/ilac_g8_03_2009.pdf NCSL INTERNATIONAL SERVING THE WORLD OF MEASUREMENT SINCE 1961 183 Accreditation Committee

ILAC-G8, paragraph 2.3 (c) non-compliance NCSL INTERNATIONAL SERVING THE WORLD OF MEASUREMENT SINCE 1961 183 Accreditation Committee

ILAC-G8, par. 2.3 Non-compliance In Case 2 It is not possible to state compliance using a 95% coverage probability for the expanded uncertainty although the measurement result is below the limit. NCSL INTERNATIONAL SERVING THE WORLD OF MEASUREMENT SINCE 1961 183 Accreditation Committee

5 Standards Compliance Agilent Wording: ILAC-G8, Case 2 Agilent Cal + Uncertainties Passed : P The measured values of the equipment were in specification at the points tested with measurement uncertainty not taken into account. A portion of the expanded measurement uncertainty intervals centered about one or more measured values exceeded specification. Consequently, compliance with specification cannot be declared based on the stated coverage probability. ISO/IEC 17025, Paragraph 5.10.4.2 ILAC-G8, Paragraph 2.3 ILAC-G8, Paragraph 3.2 Note: Wording reflects official standards text as closely as possible. P is P double-dagger

5 Standards Compliance Agilent Wording: ILAC-G8, Case 3 Agilent Cal + Uncertainties Failed : F One or more measured values of the equipment were out of specification at the points tested with measurement uncertainty not taken into account. A portion of the expanded measurement uncertainty intervals centered about one or more measured values were in specification. Consequently, non-compliance with specification cannot be declared based on the stated coverage probability. ISO/IEC 17025, Paragraph 5.10.4.2 ILAC-G8, Paragraph 2.3 ILAC-G8, Paragraph 3.2 Note: Wording reflects official standards text as closely as possible.

ILAC policy for uncertainty in calibration ILAC-P14 http://www.ilac.org/documents/ilac_p14_12_2010.pdf NCSL INTERNATIONAL SERVING THE WORLD OF MEASUREMENT SINCE 1961 183 Accreditation Committee

Excerpt: ILAC-P14, section 6 ILAC policy for uncertainty in calibration http://www.ilac.org/documents/ilac_p14_12_2010.pdf NCSL INTERNATIONAL SERVING THE WORLD OF MEASUREMENT SINCE 1961 183 Accreditation Committee

Sample report: Agilent Cal + uncertainties + Guardbanding

Accreditation Webcast summary Review the scope-of-accreditation for your cal supplier Essentially, all measurements performed at Agilent service centers or on-site are within the scope-of-accreditation Whether or not you request / order the specific deliverables that allow us to include the accreditation body symbol, Agilent always uses the same: (automated) test procedure test equipment measurement uncertainty

1 Why Calibrate? Please register for the free upcoming calibration Webcast: www.agilent.com/find/calibrationwebcasts w 3) 15 Jan 2013 De-Mystifying Calibration Accreditation 4) 13 Feb 2013 How to Comply with ANSI Z540.3 If you have Z540-1 requirements today, you should learn about Z540.3 On-Demand: 1) 8 Nov 2012 Calibration: Why it Matters to You and What it Should Include 2) 13 Dec 2012 Calibration Traceability & Standards Compliance

Questions and Answers