Definition of quality in statistics

Transcription

1 Definition of quality in statistics Sirkku Mertanen Statistics Finland, Helsinki, April 2015 ESTP course on Quality Management in Statistical Agencies Introductory course

2 Contents of the session Definition of quality in statistics Definition of quality Quality in statistics production Principles of GSBPM Generic statistical business process model April

3 1. Introduction What is good quality? Often used phrases: Conformance with requirements/standards Fit for purpose Zero defects Perspectives: Production Supply & marketing Customer - products in use - functions properly - control & monitoring - production for use - justifies the quality ISO definition for quality: Degree to which a set of inherent characteristic fulfills requirements April

4 2. Brief history of quality management Starting point: industrial revolution, introduction of mass production - uprise in early 20th century Manufacturing: the end to the old team work, machines and their users tiny parts in production Need to standardize the work process: Production time important in mass production Quality control and inspection Quality systems Early advocates: organization of work & organization theories Winslow Taylor and Henry Ford Later famous statisticians: Walter Shewhart and W. Edwards Deming April

5 2. Brief history of quality management Shewhart initiated Control charts Advocated the use of measurements Developed statistical theory to process control Deming had a crucial role in developing systematic production quality measurement Theory: Plan-Do-Check-Act cycle (Originally by Shewhart ); 14 points; Seven deadly diseases etc. Many applications in industry Eye on enterprise management April

6 Continuous Quality Improvement Plan-Do-Check-Act cycle April

7 3. Quality in statistics production Timeline 100 years: 1890s: Herman Hollerith invented the punch card tabulation machine while working at the US Census Bureau. It was a starting point for development in statistical computing like any industrial process 1920s and 1930s statistical quality control theories (Shewhart, Deming, Dodge, Roming ) April

8 3. Quality in statistics production s and 1940s: data process of censuses and sample surveys 1950 first UN recommendations: The Preparation of Sampling Survey Reports 1980s and 1990s: general awareness of quality Statistics Canada: Quality Guidelines (1985) Statistical agencies develop their own policies on quality US: Federal Committee On Measuring and Reporting the Quality of Survey Data Quality policies in International organisations: IMF, OECD April

9 3. Quality in statistics production s and 1990s: Europe: Regulations and agreements on quality reporting WG on Assessment of Quality in Statistics since 1998 LEG on Quality, And from 2000: Code of Practice, 2005 (slightly revised in 2011) Regulation on European Statistics No 223/2009 (to be revised in 2014) The Eurostat Quality Assurance Framework April

10 3. Quality in statistics production 4 Development in the European Statistical System (ESS): the mission, vision and values Code of Practise & European Statistical Law the ESS quality dimensions Standard quality indicators Quality assurance framework & tools Quality assessment plan April

11 3. Quality of European statistics: available tools and standards 11

12 21-24 April

13 4. Principles of GSBPM Generic Statistical Business Process Model The GSBPM provides a basis for statistical organizations to agree on standard terminology to develop statistical metadata systems and processes A flexible tool to describe and define the set of business processes needed to produce official statistics April

14 4. Principles of GSBPM Generic statistical business process model II The GSBPM applies to all activities undertaken by producers of official statistics, at both the national and international levels, which result in data outputs. It is independent of the data source, so it can be used for the description and quality assessment of processes based on surveys, censuses, administrative records, and other nonstatistical or mixed sources April

15 Describing a process v5.0 Archive (Phase 8, v4.0) has been incorporated into the over-arching process of data and metadata management April

16 GSBPM: Sub-processes with interest group contacts Quality Management / Metadata Management 1 Specify Needs 2 Design 3 Build 4 Collect 5 Process 6 Analyse 7 Disseminate 8 Evaluate 1.1 Identify needs 1.2 Consult & confirm need 1.3 Establish output objectives 1.4 Identify concepts 1.5 Check data availability 1.6 Prepare business case 2.1 Design outputs 2.2 Design variable descriptions 2.3 Design collection 2.4 Design frame & sample 2.5 Design processing & analysis 2.6 Design production systems & workflow 3.1 Build collection instrument 3.2 Build or enhance process components 3.3 Build or enhance dissemination components 3.4 Configure workflows 3.5 Test production system 3.6 Test statistical business process 3.7 Finalise production systems 4.1 Create frame & select sample 4.2 Set up collection 4.3 Run collection 4.4 Finalise collection 5.1 Integrate data 5.2 Classify & code 5.3 Review & validate 5.4 Edit & impute 5.5 Derive new variables & units 5.6 Calculate weights 5.7 Calculate aggregates 5.8 Finalise data files 6.1 Prepare draft outputs 6.2 Validate outputs 6.3 Interpret & explain outputs 6.4 Apply disclosure control 6.5 Finalise outputs Main 7.1 Update output systems 7.2 Produce dissemination products 7.3 Manage release of dissemination products 7.4 Promote dissemination products 7.5 Manage user queries Consult with users needs Consult with research Process owner 8.1 Gather evaluation inputs 8.2 Conduct evaluation 8.3 Agree an action plan 16

17 How to get started on GSBPM? 17

18 References Implementation of the Fundamental Principles of Official Statistics (UN, 2003) Declaration of Good Practices in Technical Cooperation in Statistics (UN, 1999) Data Quality Assessment Framework - A Factsheet, Statistics Department DQAF (2006, IMF) The Eurostat Quality Assurance Framework 2012 (Eurostat) The European Statistics Code of Practise (rev 2011) Statistical Data Quality in the UNECE (2010), Steven Vale April

19 References 2 Eurostat: Quality of statistics uction UNECE GSBPM: cussion+forum April