MONITORING OF THE FIELDWORK IN THE CZECH EU-SILC DATA COLLECTION

Transcription

1 MONITORING OF THE FIELDWORK IN THE CZECH EU-SILC DATA COLLECTION Simona Měřinská, Jan Pospíchal Household Surveys Coordination Department Workshop on best practices for EU-SILC Prague, Czech Republic, September 2017 CZECH STATISTICAL OFFICE Na padesátém 81, Prague 10 czso.cz 1/20

2 ABOUT WHAT SHALL WE TALK? How does our process of data collection and network of interviewers look like? How do we transmit data into the centre and where is space for monitoring? What type of information do we include in final and period-to-period monitoring (current situation and experience with monitoring of fieldwork)? Our goal is to initiate discussion, to share experience with you and, perhaps, to start up a possible cooperation. CZECH STATISTICAL OFFICE Na padesátém 81, Prague 10 czso.cz 2/20

3 PROCESS OF DATA COLLECTION In CZSO we make standard electronic questionnaires in Blaise. Questionnaires are called (opened, saved, closed etc.) from our own proprietary case management system edomset. Used by interviewers, regional coordinators, national coordinator and main methodologists Simple redistribution of workload to interviewers and data gathering Overview of dwellings with outcomes from survey (simple entry into questionnaire, comfortable data editing, view on past un/successful contacts and questionnaires, overview of controls in questionnaires etc.) Overview of main information about work of interviewers (their activity) and progress of ongoing survey to coordinators Collected data and paradata logistics & data checking CZECH STATISTICAL OFFICE Na padesátém 81, Prague 10 czso.cz 3/20

4 EDOMSET WORKLOAD OF INTERVIEWER CZECH STATISTICAL OFFICE Na padesátém 81, Prague 10 czso.cz 4/20

5 THREE-LEVEL ORGANISATION OF ALL SURVEYS 1) CZSO HQ particular methodologists + coordination dept. entire survey preparation, questionnaire design, European regulations, definitions of target variables, harmonisation of key variables, testing in edomset programme, sampling design and frame, time schedules survey organisation on the national level 2) Regional coordinators in every region of CR (14) we have 1 coordinator for every task (survey) responsible for the distribution of workload, overseeing interviewers, also initial data checking and sending them to the central database 3) Interviewers fieldwork staff we have built up the network of universal interviewers they are employees of CZSO, sometimes external workers the key point is long-term capacity planning, a balance during the whole year, timing and coordination of all additional requirements CZECH STATISTICAL OFFICE Na padesátém 81, Prague 10 czso.cz 5/20

6 NETWORK OF INTERVIEWERS SILC 2017 Number of households In total 561 interviewers CZECH STATISTICAL OFFICE Na padesátém 81, Prague 10 czso.cz 6/20

7 DATA TRANSMISSION How does it work? Collected data are regularly uploaded via encrypted secure connection (VPN). Data are automatically processed and uploaded to the respective repositories (in ORACLE central database) Process of transmission of data for all users (interviewers and coordinators) is build as a component in edomset - very simple and interactive (information s about uploaded data) Also includes extensive paradata: recorded fieldwork contacts from edomset system Blaise audit trail questionnaire key-log data recorded GPS coordinates for interviews CZECH STATISTICAL OFFICE Na padesátém 81, Prague 10 czso.cz 7/20

8 DATA TRANSMISSION SCHEME CZSO server Interviewer s notebook Regional coordinator PC PC headquater National coordinator PC Converted Blaise database (in ORACLE) Oracle database Main methodologist CZECH STATISTICAL OFFICE Na padesátém 81, Prague 10 czso.cz 8/20

9 SPACE FOR MONITORING OF ONGOING SURVEY CZSO server Interviewer s notebook Regional coordinator PC PC headquater National coordinator PC LET SEE IT!? Converted Blaise database Oracle database Main methodologist CZECH STATISTICAL OFFICE Na padesátém 81, Prague 10 czso.cz 9/20

10 EXPERIENCE WITH MONITORING OF ONGOING SURVEY How? Interviewers have to upload collected data to central depository once every two weeks at the minimum (= as frequently as possible) Program to transmit not only metadata but also real collected data for main indicators to Oracle database Procedures and triggers to update previously saved data and complete them with new data Creation of views in PL/SQL Developer to show temporary results Dissemination of user friendly outcomes (tables in Excel) of temporary results to regional coordinators (also via central depository) and transmission of data necessary for methodologists CZECH STATISTICAL OFFICE Na padesátém 81, Prague 10 czso.cz 10/20

11 EXAMPLE OF DISTRIBUTED TABLE Households by region ongoing monitoring of EU-SILC NUTS Name of region Number of interviewers SAMPLE Finished Successful Unsuccessful Administrative waste Not finished / not yet retyped from PAPI New households traced and successfully interviewed Total % Total % Total % Total % Total % Total Hlavní město Praha % % % 81 7% % Středočeský kraj % % % 75 6% 107 8% Jihočeský kraj % % % 61 8% 23 3% Plzeňský kraj % % % 28 4% 15 2% Karlovarský kraj % % 66 24% 20 7% 0 0% Ústecký kraj % % % 39 7% % Liberecký kraj % % 76 17% 32 7% 18 4% 4 Královéhradecký 5200 kraj % % % 34 6% 13 2% Pardubický kraj % % 82 15% 38 7% 18 3% Kraj Vysočina % % 97 17% 40 7% 0 0% 7 Jihomoravský 6200 kraj % % % 71 6% 36 3% Olomoucký kraj % % % 51 8% 0 0% Zlínský kraj % % % 49 8% 15 2% 13 Moravskoslezský 8100 kraj CZECH STATISTICAL 87% OFFICE % Na 182 padesátém 15% 81, Prague 5% czso.cz 13% 11/2023 ČR % % % 682 6% % 125

12 EXPERIENCE WITH FINAL AND PERIOD-TO-PERIOD MONITORING What do we monitor? Number of dwellings according to the result of survey (response rate) Dwellings according to the mode of interviews (CAPI, CATI, retyped PAPI, number of unsuccessful contacts before the first successful contact, average number of contacts etc. Time of interviews dependence of the time of interviewing and success of interviews Length of interviews total average length of interview for a household according to the mode of interviewing and number of household members, average length of filling significant questions in the questionnaire (especially in repeated visits) CZECH STATISTICAL OFFICE Na padesátém 81, Prague 10 czso.cz 12/20

13 EXPERIENCE WITH FINAL AND PERIOD-TO-PERIOD MONITORING (2) What data do we use? Thanks to edomset we have detailed records of interviewers actions during data collection: number, types, system and real time of beginning and end of contacts with respondents (Access database uploaded along Blaise data to the central depository transmit to Oracle) Blaise audittrail txt file uploaded along Blaise data to the central depository transmit to Oracle How do we process them? edomset has functionality for national coordinator to show selected indicators in order to monitor suspected behaviour of interviewers work Several views in PL/SQL Developer that shows regional, period-to-period and interviewers differences CZECH STATISTICAL OFFICE Na padesátém 81, Prague 10 czso.cz 13/20

14 AUDITTRAIL OUTPUT IN ORACLE CZECH STATISTICAL OFFICE Na padesátém 81, Prague 10 czso.cz 14/20

15 EDOMSET FORM FOR ANALYTICAL PURPOSES CZECH STATISTICAL OFFICE Na padesátém 81, Prague 10 czso.cz 15/20

16 GPS POSITIONS ASSESSMENT we record the GPS position of the notebook when the interviewer declares CAPI (face-to-face) interview useful tool for checking interviewers where they are actually conducting the survey we monitor their systematic behaviour; in case of difference, we're dealing with this CZECH STATISTICAL OFFICE Na padesátém 81, Prague 10 czso.cz 16/20

17 TRACING MODULE FOR EU-SILC The specificity of EU-SILC is to trace respondents while they move We created a dedicated logistics module for tracing of respondents Enables standard CAPI interviews also for movers, including acrossregions mobility Monitoring of the tracing process CZECH STATISTICAL OFFICE Na padesátém 81, Prague 10 czso.cz 17/20

18 SILC TRACING DATA FLOWS Interviewer records the data about the move, and data move information file is sent to central repository by edomset module CENTRAL DATA REPOSITORY The concerned new region coordinator receives notification and data move information file, processes it in edomset and assigns the new address to one of their interviewers Regional coordinator receives notification about the move, and handles the case If the new known address is in another region, coordinator assigns new region NEW REGION INTO WHICH THE RESPONDENT MOVES Imp file with new case data with moved respondents REGION FROM WHERE THE MOVE IS TAKING PLACE CZECH STATISTICAL OFFICE Na padesátém 81, Prague 10 czso.cz 18/20

19 TRACING MODULE IN edomset CZECH STATISTICAL OFFICE Na padesátém 81, Prague 10 czso.cz 19/20

20 WHAT DO WE LACK AND WHAT IS OUR GOAL? we do have some prepared views in PL/SQL Developer that depicts fieldwork of interviewers but we do not disseminate all of them regularly in user friendly environment only several regular outcomes and ad-hoc analysis the biggest challenge for us now is to gain more from the available data infrastructure through the business intelligence / data mining tools we also have to further expand the existing set of indicators to evaluate the fieldwork of interviewers and regularly submit them to regional coordinators and main methodologists for comparison between periods CZECH STATISTICAL OFFICE Na padesátém 81, Prague 10 czso.cz 20/20

21 Thank you for your attention I would be thankful to listen to your experience! CZECH STATISTICAL OFFICE Na padesátém 81, Prague 10 czso.cz 21/20