NSS ASSESSMENT TOOLS (PARIS21 Study)

Transcription

1 NSS ASSESSMENT TOOLS (PARIS21 Study) Inter-regional Seminar on national statistical systems reviews in SIDS 1-3 August 2017, Nadi, Fiji Millicent Gay Tejada Regional Programme Coordinator, Asia-Pacific and SIDS

2 1 Why the study Outline 2 Typology of NSS Assessment Tools 3 Considerations for undertaking NSS assessment 2

3 1 - Why the Study Understand the various tools available to assess statistical capacity and/or current state of the NSS (and what aspects it reviews/assesses and why) Help developing countries find the appropriate tool for their NSS review/ assessment (intent of the review and where the results will be used for) Provide a Guide to NSS assessment (harmonised approach to assessing NSS in light of new developments in data) 3

4 Scope 15 different tools that assess/reviews national statistical systems (NSOs, statistical products, specific domain or area, etc.) The tools reviewed point to 3 distinct (but often overlapping) purposes: statistical capacity, data quality, and compliance to good practice. 4

5 2 - Typology of NSS Assessment Tools Type A national statistics plan/strategy or statistical advocacy for NSS modernization/transformation Type B statistical programme design and monitoring Type C global monitoring of statistical performance/ capacity Type D data quality assurance and compliance with codes of practice, norms, and standards 5

6 Type A national statistics plan/strategy or statistical advocacy for NSS modernization/transformation Tools intended to assess NSS performance and statistical capacity to address data users needs. Some tools utilise questionnaires intended to be completed by statistics stakeholders at the national level (e.g., NSO, Ministry of Planning or Finance, other stakeholders) as a form of self-assessment; in some cases, questionnaires are completed with the guidance or facilitation of external expert/consultant. These tools include: Statistical Evaluation and Progress Tool (STEP) NSDS Evaluation Tool Data4SDG Toolkit FAO Country Assessment Questionnaire and In-depth Country Assessment PARIS21 Peer Reviews Adapted Global Assessment 6

7 Type B statistical programme design and monitoring Tools to assess strengths and weaknesses of the NSS related to designing statistical programmes or projects that would address some identified issues/challenges Results is a mixture of factual and descriptive information, interpretation is somewhat subjective Tools can be administered by national authorities as part of a diagnostic exercise; questionnaire is administered by experts and analysed on behalf of development agencies The results of the assessments using these tools are not available in the public domain Example of Tools under this type: Snapshot Tool by EuropeAid/EUROSTAT Tool for Assessing Statistical Capacity (TASC) by the U.S. Census Bureau 7

8 Type C global monitoring of statistical performance/capacity Aimed at presenting global estimates on specific elements of statistical capacity of the NSS and the statistics they produced (but does not reflect institutional or organizational capacity) Country performance can be compared across time and space Information/data are collected from publicly available sources and are published with corresponding metadata and methodology to describe it; countries however are not involved in the assessment process as information are collected from Results are impartial, based on clear criteria, uses facts, scoring is not subjective These tools include: World Bank Statistical Capacity Indicator (SCI) Open Data Inventory (ODIN) by Open Data Watch 8

9 Type D data quality assurance and compliance with codes of practice, norms, and standards Use of tools are largely mandatory and intended to ensure compliance with standards (or with statistical codes of practice) set by external bodies Also used by countries to assess suitability of their NSS as part of the accession process to join a regional body Recommendations are likely to be acted upon by national authorities as noncompliance often has serious consequences or will prevent the country from joining a regional body Results are likely to be publicly available and stakeholders use them in making decisions Example of these tools include: IMF Data Quality Assessment Peer Reviews developed by UNECE and widely used by Eurostat OECD Peer Reviews African Statistical Development Indicators 9

10 STEP Tool (GAMSO) (Type A) NSDS Evaluation Tool (Type A) SNAPSHOT (ESS CoP) (Type B) TASC (DQAF & SCI) (Type B) FAO CAQ for GSARS (Type A) 1. NSS Vision 1. Information & Status of the NSO 2. Governance & Leadership of NSS 3. Manage Strategic Collaboration 4. Capability Management 5. Corporate Support Finances, Human Resources, IT, etc. 6. GSBPM (dissemination) 2. Resources and Expenditure 3. Demand Responsive 4. Dissemination and Use 1. Strategy for statistics (in national dev t ) 2. Organisation of the NSS (Legal, institutional and strategic foundations) 3. Demand Responsiveness Resources 4. Quality determinants 1. Institutional Capacity 2. Planning and Managing 1. Institutional Infrastructure 2. Resources 3. Mapping 3. Statistical Methods & Practices 4. Questionnaire Content and Testing 5. Data Quality 5. Relations with users 5. Sampling 6. Results & Performance 6. Field Operations 6. Key Indicators 7. Data Processing 8. Data Analysis and Evaluation 7. Output indicators 9. Data Dissemination 4. Availability of Core Data 10

11 3 - Considerations for undertaking NSS Assessment 1. Purpose/objective of the assessment Why undertake a review? Who is the target audience of the review? Who is to receive and act on the results? 2. Scope of the assessment Review of the whole NSS or only parts of it? Is there an issue or sector which requires special attention? 3. Managing the assessment Who will conduct the assessment? Which stakeholders to be consulted/key informants? Self-assessment or with support from an international expert, or an independent assessment by a development partner? 4. review/assessment process (include various options offered; pros and cons; costs and timing) How urgent is the assessment some are rapid others take several months Face-to-face meetings, consultations with decision makers and stakeholders 11

12 In the succeeding group work, consider 12

13 Statistical Laws Codes of Practice Funding Resources Management Business processes Dissemination Quality processes National regulatory environment Home user demand Freedom of press Rule of law Strength of economy Training Attitude Skills 13

14 PARIS21 Secretariat OECD/DCD 4 Quai du Point du Jour Boulogne-Billancourt, France contact@paris21.org twitter.com/contactparis21 facebook.com/contactparis 21 youtube.com/paris21oecd