Standards and Guidelines. Volume 1. Quality in Statistics

Transcription

1 Standards and Volume 1 Quality in Statistics Quality Assurance & Audit Section Version 1.0 January 2006

2

3 TABLE OF CONTENTS 1 INTRODUCTION OFFICE POLICY QUALITY IN STATISTICS Defining Quality The ESS Quality Framework Quality Control and Auditing STATISTICAL QUALITY STANDARDS AND GUIDELINES Planning for Statistics Production Classifications and Standards Survey Frame Sampling Procedures Questionnaire Design Measures to Reduce Non-Response Data Editing Data Quality Evaluation (including Macro-Editing) Seasonal Adjustment Statistical Confidentiality Presentation and Dissemination Use of Administrative Data Documentation and Metadata Revisions to Published Data REFERENCES APPENDICES A. European Statistics Code of Practice (2005) B. UN Fundamental Principles of Official Statistics C. Recommendations of LEG on Quality Volume 1: Quality in Statistics 3

4 1 Introduction Good quality information is essential for informed public debate and decision-making. As the provider of official national statistics it is vital that the output of the Office is of the highest quality. Several National Statistical Institutes (NSIs) have advanced their reputations through the implementation of effective quality assurance procedures. The issue of quality assurance was addressed in the Deloitte & Touche Report to the National Statistics Board Review of Organisational Performance and Capability of the Central Statistics Office May The report recommended the establishment of quality assurance and internal audit functions. This recommendation was a High Level Goal in the Statement of Strategy and the Quality Assurance & Audit section was duly set up in This document, Quality in Statistics, mainly draws on guidelines developed by other NSIs 1,2 and adapted to fit the Office s statistical system. The document has a number of objectives and uses: To introduce and define what is meant by quality in the realm of official statistics (Section 3) To further awareness of the importance of quality in the work of the Office To help ensure adherence to principles to which the Office subscribes (see Section 2) To form the basis of the Office s quality assurance system To identify approaches to improve the overall quality of statistical outputs To set out in a clear and unambiguous way the high level rules under which the Office carries out, or aims to carry out, its work. These can be seen as goals to which all engaged in the work of the Office must aspire. Quality in Statistics represents a visible symbol of the Office s commitment to quality. Overall the purpose of the document is to be of assistance to those working in the Office and particularly those who are directly responsible for the data we produce. 2 Office Policy The Office subscribes to the European Statistics Code of Practice (see Appendix A) and the UN Fundamental Principles of Official Statistics (see Appendix B). Eurostat s increasing quality requirements are an ongoing consideration for the Office. Customer Service has been set as a key competency for all staff in the Office. Quality is at the core of customer service and is a challenge for all. This document is to be regarded as Office policy. It contains an agreed set of ways and means of ensuring that standards are met. It provides information on good practice and lists guidelines which should be followed in the work of the Office. Implementation of these guidelines, where relevant, is a requirement for each business area. However, it is accepted that the professional judgement by a relevant staff member in particular situations and circumstances may result in the guidelines not being fully implemented. For a summary of the Deloitte & Touche report s recommendations see: Corporate Documents\CSO\General information\1997 Deloitte & Touche Report on the CSO - Summary of Recommendations Made and Management Response Volume 1: Quality in Statistics 4

5 3 Quality in Statistics 3.1 Defining Quality Until fairly recently the quality of statistical output has traditionally been viewed in terms of accuracy. However, quality as employed in other activities has generally included broader interpretations. These tend to take in concepts such as fitness for use, meeting user need and customer satisfaction. Such concepts are clearly appropriate for official statistics and highlight the shortcomings of accuracy alone as a measure of overall quality. Quality initiatives in the European Statistical System (ESS) have reflected this need for a broader interpretation. The International Standards Office (ISO) approach is considered particularly appropriate, to both statistical products and statistical services. They define quality as the totality of features or characteristics of a product or service that bear on its ability to satisfy stated or implied needs of customers. Arising from the broader interpretations of quality is a need to define the elements that impact on quality and that can be used to characterise quality. There is no universally agreed list of the characteristics which define quality, however there is considerable overlap in the approaches adopted by various NSIs. A lot of effort has gone into standardising quality concepts within the ESS 3. In February 2005, the ESS Statistical Programme Committee (SPC) unanimously endorsed the European Statistics Code of Practice. The Code of Practice has the dual purpose of: - Improving trust and confidence in the independence, integrity and accountability of both National Statistical Authorities and Eurostat, and in the credibility and quality of the statistics they produce and disseminate - Promoting the application of best international statistical principles, methods and practices by all producers of European Statistics to enhance their quality. The code is a self-regulatory instrument consisting of 15 principles grouped into three sections addressing respectively the institutional environment, statistical processes and statistical outputs as follows: - European Statistics Code of Practice: 15 Principles Institutional Environment Statistical Processes Statistical Output 1. Professional Independence 7. Sound Methodology 11. Relevance 2. Mandate for Data Collection 8. Appropriate Statistical Procedures 12. Accuracy and Reliability 3. Adequacy of Resources 9. Non-Excessive Burden on Respondents 13. Timeliness and Punctuality 4. Quality Commitment 10. Cost Effectiveness 14. Coherence and Comparability 5. Statistical Confidentiality 15. Accessibility and Clarity 6. Impartiality and Objectivity A full explanation of the Code of Practice and indicators of good practice for each of the 15 Principles are given in Appendix A. The ESS comprises Eurostat and the statistical offices, ministries, agencies and central banks that collect official statistics in EU Member States, Iceland, Norway and Liechtenstein. Volume 1: Quality in Statistics 5

6 3.2 The ESS Quality Framework The principle risk for any NSI is that it should face a loss of credibility by users. Consequently, the last decade has seen considerable effort spent on assuring and evaluating the quality of the statistics produced. Much of the work has been undertaken co-operatively between NSIs in the ESS. In 1998 Quality of Statistics was the theme of the annual conference of Presidents and Directors-General of the European NSIs. This led in 1999 to the establishment of a Leadership Group (LEG) on Quality. The LEG was given a broad remit to: Establish a framework for considering quality issues Identify key elements to be considered Obtain information on the status of these elements in the ESS Demonstrate with examples how improvements in NSIs and in the ESS could be made Propose future actions for the ESS. The output of the LEG consisted of a summary report (see SMD\Quality\Summary Report from the Leadership Group (LEG) on Quality) containing 22 recommendations (see Appendix C). Several of the recommendations have been acted on and quality related requirements specified in Council Regulations now apply to certain statistics required by Eurostat. Such statistics include the Labour Force Survey, Short-term Business Statistics, Structural Business Statistics and Labour Costs Statistics All NSIs within the ESS have signed up to the LEG report, therefore the Office is committed to its recommendations. 3.3 Quality Control and Auditing Internal audit forms part of any quality control system. Quality Assurance & Audit Section will conduct audits to monitor the implementation of quality standards and to evaluate systems throughout the Office. Such audits might result in reports with recommendations to management with responsibility and authority for the matter in question. It is intended to issue, separately, standards and guidelines in relation to non-statistical (including financial) aspects of the Office s work. Auditing of internal financial systems will be undertaken. Requirements arising from the Report of the Working Group on the Accountability of Secretaries General and Accounting Officers (Mullarkey Report) will continue to be addressed. Volume 1: Quality in Statistics 6

7 4 Statistical Quality Standards and 4.1 Planning for Statistics Production The planning process for a new statistical activity or the redesign of an ongoing activity should include the definition of broad objectives, a targeted user population and the key questions or issues to which analysis will be directed. In order to translate this initial planning into actual production, objectives and uses should be stated precisely to help ensure that the new or redesigned activity will meet specific user requirements. The Office has set up a Project Office and has adopted project management as a core standard for the way the Office conducts its business. The principles of project management apply to all projects regardless of size. The document Project Management Guidance for the CSO (see Corporate Documents\Strategic Planning\Project Management\Documents\Project Management Guidance for the CSO) explains how project management is effectively applied in the CSO Focus analysis of user needs on finding the most cost-effective solution for both the short and long term Develop survey objectives in partnership with important users and stakeholders through, for example, Liaison Group meetings. Establish and maintain relationships with users in order to enhance the relevance of the information produced and as part of marketing products and services In determining the extent to which a survey will meet user needs, seek a reasonable trade-off between these needs and the budget, response burden and confidentiality considerations. Although the Office may have little discretion where a legal requirement is in place, in other cases look at alternative methodological approaches, frequencies, geographical details, etc. with a view to arriving at an optimum solution Review ongoing statistical activities at regular intervals. Statistical activities need to evolve, adapt and innovate to keep pace with the demands of the users they serve Apply project management to the statistical activity. State clearly the scope of the project and agree a detailed project plan including budgets and resource allocations. 4.2 Classifications and Standards The Classifications and Related Standards system (CARS) (see Corporate Documents\CSO\Home Pages\Classifications & Standards Homepage) is the Office's system for storing and accessing classifications. The CARS database is the Office's central repository for all classifications, concordances and coding indexes. The purpose of CARS is to: Use database technology to provide centralised classification storage, maintenance and access facilities for classification data that are used both in the development and processing of surveys and in the subsequent analysis and evaluation of the data Help reduce the time and resources required when developing surveys and to contribute to improved data quality by supporting the use of standard classifications Facilitate the comparison and analysis of data by storing concordances. Volume 1: Quality in Statistics 7

8 4.2.1 All classifications used in the Office must reside on CARS. The responsibility for ensuring this rests with the Office's business areas The official Office policy for CARS (see Corporate Documents\IT\Systems\Policy for Using CARS) must be followed in all matters relating to classifications. 4.3 Survey Frame A survey frame is any list or register that delimits, identifies, and allows access to the elements of the target population. The target population is the set of elements about which information is wanted and estimates are required. The extent to which a survey frame includes all the elements of the target population is referred to as coverage. Practical considerations may dictate that some units be excluded (e.g. companies with less than five employees, institutionalised individuals) from some frames. The survey frame should conform to the target population and contain minimal undercoverage and overcoverage (e.g. duplication). Frame creation, use, maintenance and monitoring should be implemented within operational and cost constraints. Characteristics of the frame units (e.g. classification, contact, address, size) should be of high quality because of their use in stratification, collection, follow-up, estimation, record linkage, quality assessment and analysis In designing business surveys, or in the redesign of existing ones, the Office s Business Register should be used to construct the appropriate survey frame Where possible, use the same frame for surveys with the same target population, to avoid inconsistencies and to reduce costs of frame maintenance and evaluation Incorporate procedures to eliminate duplication and to update for births, deaths, out-of-scope units and changes in characteristics Monitor the frame quality by periodically assessing its coverage For area frames, implement map checks to ensure clear and non-overlapping delineation of the geographic areas used in the sampling design (e.g. through field checks or the use of other map sources) For statistics production from administrative sources, determine and monitor coverage through contact with the source manager. Where influence on the frame is possible, negotiate required changes with the source manager Whenever necessary, adjust the statistical results or use supplementary data to offset coverage differences between the frame and the target population Include descriptions of the target population, frame and coverage in the survey documentation. Volume 1: Quality in Statistics 8

9 4.4 Sampling Procedures Sampling is the selection of a set of units from a survey frame. This set of units is referred to as the sample. The choice of sampling method has a direct impact on the data quality. It is influenced by many factors, including the desired level of precision of the information to be produced, the availability of appropriate frames, the availability of suitable stratification variables, the estimation methods that will be used and the available budgets. The intention is to gather useful information from the sampled units to allow inferences about the target population When determining sample size, take into account the required levels of precision needed for the survey estimates, the type of design and estimator to be used, the availability of auxiliary information, as well as both sampling factors (e.g. stratification) and non-sampling factors (e.g. non-response) For highly skewed populations, include in the survey a stratum of large units that will be sampled with certainty In determining sample allocation for stratified samples, account for expected rates of misclassification of units in the frame For periodic surveys that use designs in which the sample size grows as the population increases, develop a method to keep the sample size stable For periodic surveys, if efficient estimates of change are required or if response burden is a concern, use a rotation sampling scheme that replaces part of the sample in each period For periodic surveys develop procedures to monitor the quality of the sample design over time. Set up an update strategy for selective redesign of strata that have suffered serious deterioration. 4.5 Questionnaire Design A questionnaire is a set of questions designed to collect information from a respondent. A questionnaire may be interviewer-administered or respondent-completed, using paper methods of data collection or electronic modes of completion. Questionnaires play a central role in the data collection process. They have a major impact on data quality, respondent behaviour, interviewer performance and respondent relations. The design of questionnaires takes into account the statistical requirements of data users, administrative requirements of the survey organisation, and the requirements for data processing, as well as the nature and characteristics of the respondent population Questionnaires in periodic surveys should be evaluated regularly Use words and concepts that have the same meanings for both respondents and the questionnaire designers. In the case of business surveys, choose questions, time reference periods and response categories that are compatible with the respondent's record-keeping practices. Volume 1: Quality in Statistics 9

10 4.5.3 In the introduction to all questionnaires: Provide the title or subject of the survey Explain the purpose of the survey Request the respondent s co-operation Indicate the authority under which the survey is taken, and what confidentiality protection arrangements are in place Ensure that the value of providing information is made very clear to respondents. In addition, the importance of completing the questionnaire and how the survey data will be used must be highlighted The opening questions should be applicable to all respondents, be easy and interesting to complete, and establish that the respondent is a member of the target population Questionnaires that are to be administered in person or over the telephone must be made interviewer-friendly as well as respondent-friendly Ensure that the instructions to respondents and or interviewers are short, clear, and easy to find. Provide definitions at the beginning of the questionnaire or in specific questions, as required Ensure that time reference periods and units of response are clear to the respondent, specify include or exclude in the questions themselves and not in separate instructions Ensure that response categories are mutually exclusive and exhaustive Provide titles or headings for each section of the questionnaire, and include instructions and answer spaces that facilitate accurate answering of the questions. 4.6 Measures to Reduce Non-Response Non-response has two effects on results: one contributing to bias of estimates when non-respondents differ from respondents in the characteristics measured; the other contributing to a decrease in the accuracy of the survey estimates resulting from the smaller effective sample size. The degree to which response is pursued is subject to budget and time constraints and the risk of nonresponse bias. Adjustments are subsequently made to data to compensate for non-response (e.g. weighting adjustments or imputation). An effective respondent relations programme and a well-designed questionnaire are critical elements in maximising response Establish and maintain good relationships with respondents Ensure interviewers are fully trained in interviewing techniques etc When operational constraints permit, follow-up the non-respondents either as a complete enumeration or on a sub-sample basis Prioritise follow-up activities. For example, in business surveys, follow-up large or influential units first. Volume 1: Quality in Statistics 10

11 4.6.5 Record and monitor reasons for non-response (e.g. refusal, non-contact, temporarily absent, technical problem) Use an appropriate method of imputation to compensate for non-response. Only as a last resort should grossing factor adjustment be used Where applicable, ensure that those being surveyed are aware that the survey is statutory. 4.7 Data Editing The goals of editing are to: Provide the basis for future improvement of the survey vehicle Provide information about the quality of the survey data Tidy up the data. It may be that a disproportionate amount of resources is concentrated on the third objective of cleaning up the data. A danger is that learning from editing processes may play an undeserved, secondary role. While it is recognised that fatal errors (e.g. invalid or inconsistent entries) should be removed from the data sets in order to maintain credibility and to facilitate further automated data processing and analysis, caution should be exercised against the overuse of query edits (those pointing to questionable records that may potentially be in error) Ensure that all edits are internally consistent (i.e. not self-contradictory) Reapply edits to units to which corrections were made to ensure that no further errors were introduced directly or indirectly Perform edit checks for missing values, invalid values, etc. as quickly and as expediently as possible in the processing cycle Rationalise 'query' editing (i.e. checks for apparent errors or inconsistencies), and find an appropriate balance between error detection and cost Consider editing to be an integral part of the data collection process in its role of gathering intelligence about the process. Use editing to: Sharpen definitions Evaluate the quality of the data Identify non-sampling error sources Serve as a basis of future improvement of the whole survey process. 4.8 Data Quality Evaluation (including Macro-Editing) Data quality evaluation refers to the process of evaluating the final statistical output in the light of the original objective of the statistical activity, in terms of the data s accuracy or reliability. Such information allows users to make more informed interpretations of the survey results, and can be used by the Office to improve surveys. Volume 1: Quality in Statistics 11

12 Two general types of data quality evaluation can be distinguished: Macro editing or quality validation is the process of reviewing the data before official release to ensure that grossly erroneous data are not released, or to identify data of marginal quality. Sources of error studies generally provide quantitative information on specific sources of error in the data. While timeliness is still important, the results of these studies often are only available after the official release of the data Make planning of data quality evaluations part of the overall statistical process, as the information needed to conduct such evaluations often must be collected during the process itself Involve users of the results, whether they are external or internal, in setting the objectives for the data quality evaluation programme. Where circumstances permit, also involve them in the evaluation process itself The following macro editing and quality validation methods should be used: Checks of consistency with external sources of data, for example from other surveys or from previous instances of the same survey Internal consistency checks, for example calculation of ratios that are known to lie within certain bounds (sex ratios, average value of commodities, etc.) Unit-by-unit reviews of the largest contributors to aggregate estimates, typically the case in business surveys Debriefings with staff involved in the collection and processing of the data The following sources of error should be evaluated: Coverage errors, which consist of omissions, erroneous inclusions, and duplications in the frame used to conduct the survey Non-response errors, which occur when the survey fails to get a full response Measurement errors, which occur when the response received differs from the true value, and can be caused by the respondent, the interviewer, the questionnaire, the mode of collection, or the respondent s record-keeping system Processing errors, which can occur at the subsequent steps of data editing, coding, capture, imputation and tabulation Sampling errors, which occur when the results of the survey are based on a sample rather than the entire population. 4.9 Seasonal Adjustment Seasonal adjustment consists of estimating seasonal factors and applying them to a time series to remove the seasonal variations. These variations represent the composite effect of climatic and institutional factors that repeat with a certain regularity within the year. Many series are published in seasonally adjusted form to reveal the underlying trend movements and to help data analysis. Seasonally adjusted series comprise not only the trend but also the irregular component; consequently, they only give an approximate idea of the underlying trend movements. To eliminate the irregular component the seasonally adjusted series may be further smoothed and the Volume 1: Quality in Statistics 12

13 trend derived. In some cases the trend estimates may be considered for publication if the seasonally adjusted series is considered unsuitable. The Statistical Methods and Development (SMD) Seasonal Adjustment Policy document (see SMD\Quality\Corporate Seasonal Adjustment Policy) sets out the procedures to be followed for the seasonal adjustment of data in the Office All time series which exhibit evidence of seasonality and for which the underlying seasonality can be identified reliably should be seasonally adjusted Before publishing a seasonally adjusted time series for the first time, conduct a thorough seasonal adjustment analysis to assess if the seasonality is identifiable Follow the procedures set out in the SMD Seasonal Adjustment Policy document For seasonal adjustment, use concurrent seasonal factors. These are the factors obtained using all recent values of the series. This is will give rise to frequent, mostly small, revisions to seasonally adjusted series. A definite policy on the publication of the revisions should be used and clearly explained to users For aggregate series resulting from the addition or subtraction of component series, seasonally adjust only those component series that contain identifiable seasonality, and leave the others unadjusted. Seasonally adjust the aggregate series directly (i.e. do not use the aggregate of the component seasonally adjusted series). Inform users that small discrepancies will arise in the seasonally adjusted estimates of aggregate series Wherever seasonally adjusted figures pertaining to the same economic activity are published, co-ordinate the seasonal adjustment options applied by the areas involved Statistical Confidentiality The Statistics Act, 1993 (see Corporate Documents\CSO\Home Pages\CSO Policies Home Page) provides that all information collected by the Office can be used only for statistical purposes and that (subject to stated exceptions) any information which can be related to an identifiable person or undertaking cannot be disseminated, shown or communicated to any person or body. The requirements of the Statistics Act, 1993 are additional to: The obligations on the staff of the CSO under the Official Secrets Act, 1963 not to make unauthorised communications, directly or indirectly, about matters which come to their knowledge in the course of their official duties The obligations on the CSO under the Data Protection Act, 1988 regarding the handling of personal data The Office's Code of Practice on Confidentiality (see Corporate Documents\CSO\Home Pages\CSO Policies Home Page CSO Statistical Confidentiality Code of Practice) should be followed at all times. Volume 1: Quality in Statistics 13

14 4.11 Presentation and Dissemination Presentation and dissemination are among the final stages in the statistical production process. While presentation and dissemination procedures may not affect the accuracy of the statistical data, the perception of quality by the users may be strongly influenced. The forms of presentation and dissemination should facilitate access and interpretation by users. Tables should have clear headings, identifiable units and logical layout. Charts should convey a clear and accurate representation of the phenomenon being studied In official statistical releases and reports a suitable analysis highlighting the more important and interesting features of the published statistics should be provided. This should be based on the statistical results available and should never contain any political or similar judgements. In general the larger and more infrequent the release or report the more detailed the analysis should be for tables: Every table should be labelled to clearly identify the content The table layout must be clear and easy to follow All units of measurement should be displayed clearly Where a comparison between numbers is required, these should be listed in columns to facilitate reading Ensure all rounding to significant digits is mathematically correct Ensure that footnotes are clearly marked and that the text is clear and readable for charts: The chart title must explain what phenomenon is represented and the time periods covered. All axes must be clearly labelled and include the units of measurement. Legends, labels and tick marks should all be clear and readable. All elements on the chart should be identified. Any apparent discrepancy should be highlighted and explained The document Write Well, Write Clearly (see Corporate Documents\Dissemination \Information Section\Write Well, Write Clearly - CSO Usage and House Style (1996)) is the official policy on written content and must be followed for all releases and publications Use of Administrative Data The term administrative records refers to data collected for the purpose of carrying out various programmes, for example, income tax collection. Administrative records present a number of advantages. Since they already exist, costs of direct data collection and further burden on respondents are avoided. They are usually available for the complete universe, and hence, they are most of the time not constrained by sampling error limitations. Most importantly, they can be used in numerous ways in the production of statistical outputs. Examples of their uses include: The creation and maintenance of frames Volume 1: Quality in Statistics 14

15 The complete or partial (via record linkage) replacement of statistical collection Editing, imputation and weighting of data from statistical collection The evaluation of statistical outputs Actively investigate and assess all potential sources of administrative data. Bear in mind that even partially complete and/or partially inaccurate administrative data may still prove useful in areas such as the reduction of the response burden in surveys and the improvement of survey results Set up an edit and imputation procedure or a weight adjustment procedure to deal with nonresponse The confidentiality implications of the publication of information from administrative records must always be borne in mind. Although the Statistics Act, 1993 provides the CSO with the authority to access administrative records for statistical purposes, the use may not have been foreseen by the original suppliers of information Maintain continuing liaison with the provider of administrative records Understand and document concepts, definitions and procedures underlying the collection of the administrative data Implement continuous or periodic assessment of incoming data quality Documentation and Metadata Documentation refers to the collection of material that provides a description of the activity. It should include the concepts, definitions, metadata, methodology, and an outline of the production processes used. The Business Process Improvement Inquiry (BPI) (see Corporate Discussion\ITSIP\BPI\ Business Process Improvement Project Report) was a means of documenting, at a high level and in a standard way, the inputs, methodologies, processes and outputs for every survey in the Office Business areas must ensure that a Business Process Improvement (BPI) methodology questionnaire has been completed and is kept up to date for every survey or statistical process The metadata held on the Databank and, in future on the new Data Management System, must be reviewed and updated whenever a change is made to any part of the statistical activity Revisions to Published Data Revisions to published data can, and do, occur for many reasons. It is important that all users of statistics are at all times made fully aware of the revision policy relating to these statistics. Common sources of revisions to statistics are: The availability of additional information (e.g. late survey responses; a new period s data available for the calculation of seasonal factors) The receipt of amended information (e.g. as a result of the response to a query) Changes resulting from additional and more detailed data editing. Volume 1: Quality in Statistics 15

16 Occasionally revisions can arise from a change in methodology when it is deemed best to recompile previous period s data using the new methodology Produce, for users, a clear and concise data revisions policy for all published statistics When publishing data that is likely to be subsequently revised (e.g. preliminary estimates) always indicate this to the user Describe the main reasons why particular statistics are subject to revision For established statistical data series inform users of the frequency and size of past revisions. Volume 1: Quality in Statistics 16

17 References 1 Statistics Canada, October 1998, Statistics Canada Quality, 3 rd edition 2 Statistics Finland, 2002, Quality for Official Statistics 3 Eurostat, 2003, Assessment of quality in statistics, Methodological documents Definition of Quality in Statistics, Doc. Eurostat/A4/Quality/03/General/Definition Volume 1: Quality in Statistics 17

18 Appendices A. European Statistics Code of Practice (2005) Institutional Environment Institutional and organisational factors have a significant influence on the effectiveness and credibility of a statistical authority producing and disseminating European Statistics. The relevant issues are professional independence, mandate for data collection, adequacy of resources, quality commitment, statistical confidentiality, impartiality and objectivity. Principle 1: Professional Independence - The professional independence of statistical authorities from other policy, regulatory or administrative departments and bodies, as well as from private sector operators, ensures the credibility of European Statistics. The independence of the statistical authority from political and other external interference in producing and disseminating official statistics is specified in law. The head of the statistical authority has sufficiently high hierarchical standing to ensure senior level access to policy authorities and administrative public bodies. He/she should be of the highest professional calibre. The head of the statistical authority and, where appropriate, the heads of its statistical bodies have responsibility for ensuring that European Statistics are produced and disseminated in an independent manner. The head of the statistical authority and, where appropriate, the heads of its statistical bodies have the sole responsibility for deciding on statistical methods, standards and procedures, and on the content and timing of statistical releases. The statistical work programmes are published and periodic reports describe progress made. Statistical releases are clearly distinguished and issued separately from political/policy statements. The statistical authority, when appropriate, comments publicly on statistical issues, including criticisms and misuses of official statistics. Principle 2: Mandate for Data Collection - Statistical authorities must have a clear legal mandate to collect information for European statistical purposes. Administrations, enterprises and households, and the public at large may be compelled by law to allow access to or deliver data for European statistical purposes at the request of statistical authorities. The mandate to collect information for the production and dissemination of official statistics is specified in law. The statistical authority is allowed by national legislation to use administrative records for statistical purposes. On the basis of a legal act, the statistical authority may compel response to statistical surveys. Principle 3: Adequacy of Resources - The resources available to statistical authorities must be sufficient to meet European statistics requirements. Volume 1: Quality in Statistics 18

19 Staff, financial, and computing resources, adequate both in magnitude and in quality, are available to meet current European statistics needs. The scope, detail and cost of European statistics are commensurate with needs. Procedures exist to assess and justify demands for new European statistics against their cost. Procedures exist to assess the continuing need for all European statistics, to see if any can be discontinued or curtailed to free up resources. Principle 4: Quality Commitment - All ESS members commit themselves to work and co-operate according to the principles fixed in the Quality Declaration of the European Statistical System. Product quality is regularly monitored according to the ESS quality components. Processes are in place to monitor the quality of the collection, processing and dissemination of statistics. Processes are in place to deal with quality considerations, including trade-offs within quality, and to guide planning for existing and emerging surveys. Quality guidelines are documented and staff are well trained. These guidelines are spelled out in writing and made known to the public. There is a regular and thorough review of the key statistical outputs using external experts where appropriate. Principle 5: Statistical Confidentiality - The privacy of data providers (households, enterprises, administrations and other respondents), the confidentiality of the information they provide and its use only for statistical purposes must be absolutely guaranteed. Statistical confidentiality is guaranteed in law. Statistical authority staff sign legal confidentiality commitments on appointment. Substantial penalties are prescribed for any wilful breaches of statistical confidentiality. Instructions and guidelines are provided on the protection of statistical confidentiality in the production and dissemination processes. These guidelines are spelled out in writing and made known to the public. Physical and technological provisions are in place to protect the security and integrity of statistical databases. Strict protocols apply to external users accessing statistical microdata for research purposes. Principle 6: Impartiality and Objectivity - Statistical authorities must produce and disseminate European statistics respecting scientific independence and in an objective, professional and transparent manner in which all users are treated equitably. Statistics are compiled on an objective basis determined by statistical considerations. Choices of sources and statistical techniques are informed by statistical considerations. Errors discovered in published statistics are corrected at the earliest possible date and publicised. Information on the methods and procedures used by the statistical authority are publicly available. Statistical release dates and times are pre-announced. Volume 1: Quality in Statistics 19

20 All users have equal access to statistical releases at the same time and any privileged prerelease access to any outside user is limited, controlled and publicised. In the event that leaks occur, pre-release arrangements should be revised so as to ensure impartiality. Statistical releases and statements made in Press Conferences are objective and nonpartisan. Statistical Processes European and other international standards, guidelines and good practices must be fully observed in the processes used by the statistical authorities to organise, collect, process and disseminate official statistics. The credibility of the statistics is enhanced by a reputation for good management and efficiency. The relevant aspects are sound methodology, appropriate statistical procedures, nonexcessive burden on respondents and cost effectiveness. Principle 7: Sound Methodology - Sound methodology must underpin quality statistics. This requires adequate tools, procedures and expertise. The overall methodological framework of the statistical authority follows European and other international standards, guidelines, and good practices. Procedures are in place to ensure that standard concepts, definitions and classifications are consistently applied throughout the statistical authority. The business register and the frame for population surveys are regularly evaluated and adjusted if necessary in order to ensure high quality. Detailed concordance exists between national classifications and sectorisation systems and the corresponding European systems. Graduates in the relevant academic disciplines are recruited. Staff attend international relevant training courses and conferences, and liaise with statistician colleagues at international level in order to learn from the best and to improve their expertise. Co-operation with the scientific community to improve methodology is organised and external reviews assess the quality and effectiveness of the methods implemented and promote better tools, when feasible. Principle 8: Appropriate Statistical Procedures Appropriate statistical procedures, implemented from data collection to data validation, must underpin quality statistics. Where European statistics are based on administrative data, the definitions and concepts used for the administrative purpose must be a good approximation to those required for statistical purposes. In case of statistical surveys, questionnaires are systematically tested prior to the data collection. Survey designs, sample selections, and sample weights are well based and regularly reviewed, revised or updated as required. Field operations, data entry, and coding are routinely monitored and revised as required. Appropriate editing and imputation computer systems are used and regularly reviewed, revised or updated as required. Revisions follow standard, well-established and transparent procedures. Volume 1: Quality in Statistics 20

21 Principle 9: Non-Excessive Burden on Respondents - The reporting burden should be proportionate to the needs of the users and should not be excessive for respondents. The statistical authority monitors the response burden and sets targets for its reduction over time. The range and detail of European statistics demands is limited to what is absolutely necessary. The reporting burden is spread as widely as possible over survey populations through appropriate sampling techniques. The information sought from businesses is, as far as possible, readily available from their accounts and electronic means are used where possible to facilitate its return. Best estimates and approximations are accepted when exact details are not readily available. Administrative sources are used whenever possible to avoid duplicating requests for information. Data sharing within statistical authorities is generalised in order to avoid multiplication of surveys. Principle 10: Cost Effectiveness - Resources must be effectively used. Internal and independent external measures monitor the statistical authority s use of resources. Routine clerical operations (e.g. data capture, coding, validation) are automated to the extent possible. The productivity potential of information and communications technology is being optimised for data collection, processing and dissemination. Proactive efforts are being made to improve the statistical potential of administrative records and avoid costly direct surveys. Statistical Output Available statistics must meet users needs. Statistics comply with the European quality standards and serve the needs of European institutions, governments, research institutions, business concerns and the public generally. The important issues concern the extent to which the statistics are relevant, accurate and reliable, timely, coherent, comparable across regions and countries, and readily accessible by users. Principle 11: Relevance - European statistics must meet the needs of users. Processes are in place to consult users, monitor the relevance and practical utility of existing statistics in meeting their needs, and advise on their emerging needs and priorities. Priority needs are being met and reflected in the work programme. User satisfaction surveys are undertaken periodically. Principle 12: Accuracy and Reliability - European statistics must accurately and reliably portray reality. Source data, intermediate results and statistical outputs are assessed and validated. Sampling errors and non-sampling errors are measured and systematically documented according to the framework of the ESS quality components. Volume 1: Quality in Statistics 21

22 Studies and analyses of revisions are carried out routinely and used internally to inform statistical processes. Principle 13: Timeliness and Punctuality - European statistics must be disseminated in a timely and punctual manner. Timeliness meets the highest European and international dissemination standards. A standard daily time is set for the release of European statistics. Periodicity of European statistics takes into account user requirements as much as possible. Any divergence from the dissemination time schedule is publicised in advance, explained and a new release date set. Preliminary results of acceptable aggregate quality can be disseminated when considered useful. Principle 14: Coherence and Comparability - European statistics should be consistent internally, over time and comparable between regions and countries; it should be possible to combine and make joint use of related data from different sources. Statistics are internally coherent and consistent (e.g. arithmetic and accounting identities observed). Statistics are coherent or reconcilable over a reasonable period of time. Statistics are compiled on the basis of common standards with respect to scope, definitions, units and classifications in the different surveys and sources. Statistics from the different surveys and sources are compared and reconciled. Cross-national comparability of the data is ensured through periodical exchanges between the European Statistical System and other statistical systems; methodological studies are carried out in close co-operation between the Member States and Eurostat. Principle 15: Accessibility and Clarity European statistics should be presented in a clear and understandable form, disseminated in a suitable and convenient manner, available and accessible on an impartial basis with supporting metadata and guidance. Statistics are presented in a form that facilitates proper interpretation and meaningful comparisons. Dissemination services use modern information and communication technology and, if appropriate, traditional hard copy. Custom-designed analyses are provided when feasible and are made public. Access to microdata can be allowed for research purposes. This access is subject to strict protocols. Metadata are documented according to standardised metadata systems. Users are kept informed on the methodology of statistical processes and the quality of statistical outputs with respect to the ESS quality criteria. Volume 1: Quality in Statistics 22

23 B. UN Fundamental Principles of Official Statistics Adopted by The United Nations Statistical Commission, in its Special Session of April Principle 1. Official statistics provide an indispensable element in the information system of a society, serving the government, the economy and the public with data about the economic, demographic, social and environmental situation. To this end, official statistics that meet the test of practical utility are to be compiled and made available on an impartial basis by official statistical agencies to honour citizens entitlement to public information. Principle 2. To retain trust in official statistics, the statistical agencies need to decide according to strictly professional considerations, including scientific principles and professional ethics, on the methods and procedures for the collection, processing, storage and presentation of statistical data. Principle 3. To facilitate a correct interpretation of the data, the statistical agencies are to present information according to scientific standards on the sources, methods and procedures of the statistics. Principle 4. The statistical agencies are entitled to comment on erroneous interpretation and misuse of statistics. Principle 5. Data for statistical purposes may be drawn from all types of sources, be they statistical surveys or administrative records. Statistical agencies are to choose the source with regard to quality, timeliness, costs and the burden on respondents. Principle 6. Individual data collected by statistical agencies for statistical compilation, whether they refer to natural or legal persons, are to be strictly confidential and used exclusively for statistical purposes. Principle 7. The laws, regulations and measures under which the statistical systems operate are to be made public. Principle 8. Coordination among statistical agencies within countries is essential to achieve consistency and efficiency in the statistical system. Principle 9. The use by statistical agencies in each country of international concepts, classifications and methods promotes the consistency and efficiency of statistical systems at all official levels. Principle 10. Bilateral and multilateral co-operation in statistics contributes to the improvement of systems of official statistics in all countries. Volume 1: Quality in Statistics 23