Guidelines on Use of Electronic Data Collection in Censuses: Use of Handheld Devices and Internet

Transcription

1 1 Guidelines n Use f Electrnic Data Cllectin in Censuses: Use f Handheld Devices and Internet Meryem Demirci United Natins Statistics Divisin Experiences, gd practices and challenges fr the Natinal Statistical Systems f Latin American cuntries in the light f the 2020 Ppulatin and Husing Census rund UN ECLAC, Santiag, Chile, Nvember 27-29, 2017

2 Cntent f the Guidelines Decisin making prcess Cnsideratins fr Planning and Preparatins Electrnic Data Cllectin Applicatin Use f Handheld Devices fr data cllectin Use f Internet fr data cllectin

3 Decisin making prcess Steps fr making a decisin 1. Develping strategic bjectives 2. Identifying key factrs affecting decisin 3. Infrmatin required fr decisin making 4. Analysis f infrmatin (SWOT and Cst-benefit analysis) 5. Cmpnents f decisin making 6. Dcumentatin and evaluatin

4 Decisin making prcess Drivers fr adptin f electrnic data cllectin - Imprving census cverage and data quality - Disseminating timely census results - Reducing respnse burden - Ptential cst saving, cst-effectiveness Indirect drivers - Mdernizatin f business prcess f statistical peratins - Meeting public expectatins Pssible challenges - Systems infrastructure and building a institutinal capacity - High initial cst - Crdinatin f multi-mdes - Cnfidentiality and security f data Develp strategic bjectives based n natinal pririties

5 Data cllectin methds Face-t-face interview data cllectin Paper questinnaire with face-t-face interview (PAPI) Cmputer-assisted persnal interviewing (CAPI) Cmputer-assisted telephne interviewing (CATI) Self-interview data cllectin Paper questinnaire with self-enumeratin (PASI) Cmputer-assisted self-interviewing (CASI) Multimde data cllectin methds Paper-based data cllectin (PAPI/PASI) Electrnic data cllectin (CAPI/ CASI/ CATI) Cmbinatin f electrnic and paper-based data cllectin (PAPI and at least ne f CAPI/CASI/CATI r PASI and at least ne f CAPI/CASI/CATI)

6 Key factrs affecting decisin-general framewrk Institutinal factrs.experience in change management. High level and senir managerial supprt. Gvernment cmmitment and supprt. Technical skills and capacity. Previus experience in electrnic data cllectin. Stakehlder supprt. Apprpriate legislative framewrk Technlgical factrs ICT infrastructure. Internet accessibility (bth natinally and at the individual husehld level). Use f scial media. Cellular/mbile phne cverage.. Decisin n use f electrnic data cllectin Ecnmic factrs. Pressure fr cst reductin f census peratin. Resurce mbilisatin. Supprt frm Gvernment and ther stakehlders fr resurce allcatin Operatinal factrs. Time required fr - redesigning - capacity building - establishing IT infrastructure - prcurement. Hardware requirements. Human resurce needed. Assessment f csts, benefits, risks. Plans fr reuse Scial, cultural, envirnmental factrs. Public trust. Level f educatin. Transprtatin availability. Availability f electricity. Availability f bradband cnnectivity. Diversity - such as sci-ecnmic, gegraphic, etc.. Respnse rate

7 Infrmatin required fr decisin making a. Quantitative infrmatin Ppulatin size and density Literacy rate Prprtin f ppulatin having basic cmputer skills Number and prprtin f ppulatin wh can access t the Internet Number and percentage f husehlds with Internet cnnectin Prprtin f husehlds with electricity Cellular/mbile phne cverage Pattern f nn-respnse rates fr previus censuses and surveys By gegraphical area and rural/urban b. Qualitative infrmatin Hw ready is the statistical agency t adpt new technlgy fr enumeratin? Hw will Gvernment and main stakehlders (particularly data users) react t the use f new technlgy? Des existing legislatin allws fr a change in the methd f enumeratin, and, if nt, is a requisite change practicable? What is the public s reactin t adptin f Internet as the medium fr self-respnse? What are main expected challenges? What wuld be strategies t achieve these challenges? What are the risks in prcuring hardware and utsurced services?

8 Infrmatin required fr decisin making c. Estimate f Ttal Cst Ownership (TCO) Fixed csts and variable csts Three cmpnents f TCO: - Hardware/ sftware - Operatinal csts - Persnal csts Ttal csts f the multi-mde can be assessed by adding the cst estimate f each pssible methd Cmparisn with the csts f previus census An example f a template

9 Infrmatin required fr decisin making d. Benefits- imprving data quality, cverage, timeliness f delivery f utputs and future uses f same technlgy fr statistical peratins e. Risks and challenges- such as building a capacity, data transfer, resurce mbilizatin, hiring a qualified field staff, data security f. Timetable-sufficient time such as fr building a capacity, prcurement, system develpment, end-t-end testing (usually extensins t the timetable required fr decisin making and planning) g. Reuse f technlgy- ther censuses, husehld surveys, business registers, the use f hardware by ther rganizatins h. Best practices-practices f cuntries which have similar cnditins

10 Decisin making Analysis f infrmatin SWOT (Strengths, Weaknesses, Opprtunities, Threats) analysis: a better understanding f the balance between the benefits, risks and challenges in the intrductin f new technlgy Cst-benefit analysis: cmparisn f ttal csts with its benefits - If a multi-mde data cllectin methdlgy is an ptin, the cst estimates, benefits and risks shuld be determined fr each type f data cllectin methd under cnsideratin The cmpnents f decisin making: Data cllectin mde r mdes Type f devices and equipment Develpment f data cllectin applicatin (cmmercial, in-huse, utsurced, custmized) Understanding and balancing an investment in refreshing r updating existing systems What new skills are needed- based n existing skills

11 Electrnic Data Cllectin Applicatin Essential features f data cllectin applicatin Data quality cntrl-validatin between respnses Autmated skip patterns- imprve data quality, - Residency status - Occupancy status Benefitting mst with asking additinal questins fr clarificatin Pre-cding- selectin f apprpriate cdes frm drp-dwn menu What variables can be autmatically cded r cded during data prcessing (ccupatin/industry?) Validatin messages: Nn-respnse, invalid respnse, partial respnse, cnsistency verificatin, etc. Custmizing f questins- recalling a piece f infrmatin and inserting apprpriate place Cllectin f peratinal infrmatin dates, time and duratin f the interview and GPS crdinatr, etc. Multiple language capabilities

12 Electrnic Data Cllectin Applicatin Allws Case management- cllecting and updating status f individual enumeratin-persns/husing units and cntrlling the quality f recrds Design cnsideratin Matrix frmat: each questin appears nce and respnse ptins are repeated under the name f each persn in the husehld. This structure is mre apprpriate fr sectins f the questinnaire that are nt dependent n the respnses t previus questins Sequential frmat: all questins are asked abut ne persn at a time. This structure is mre apprpriate in sectins f the questins cntain mre autmated skip

13 Electrnic Data Cllectin Applicatin Testing prgramme shuld includes a series f tests fr data cllectin applicatin Scenari-based testing- using the Functinal testing-all aspects f the questinnaire design (texts, respnse ptins, missing values, instructins, errr messages Usability testing testing userfriendliness f applicatin input f subject matter specialists fr testing typical and cmmn prfiles, entry prblems, warning/errr messages Extreme-scenari testing- similar tests fr extreme cases such as large husehlds including several generatins, multiple ccupatins, multiple types f disabilities Pre-field and field tests t finalize data cllectin applicatin Pilt census fr testing whle prcesses

14 Data cllectin with electrnic devices Prcessr Battery Operating system Screen Strage capacity Outright purchase Sharing devices Renting BYOD Keybard Lcatin sensr Ruggedness Cnnectivity/ cmmunicatin Prtability

15 Use f Handheld Devices Data transfer Netwrking WiFi Cellular Bluetth Data transfer ptins: Online- Direct transfer frm devices t main server Partially nline-due t limitatin f cellular cverage/availability f Wifi, data transfer ccurs when the Internet is available Off-line Due t rare r never access t the Internet, data transfer ccurs between devices Hw the methd f data transfer affects data quality, pssibility f duplicatin, remving? Servers: Web server Clud based service

16 Use f Handheld Devices Security measures Measures fr securing data n devices User and device authenticatin - A device passwrd and/r ther authenticatin; Under the directin f administratr, remtely lck the device Data strage strngly encrypt stred data n bth built-in strage and remvable media strage General device use plicy Restrictins f particular applicatin t mbile device

17 Use f Handheld Devices Field rganizatin Central level Field structure Reginal/lcal crdinatr Reginal level Administrative crdinatr Census methdlgical crdinatr IT crdinatr and supprt team Lcal level Enumeratrs Supervisrs IT supervisrs

18 Use f Handheld Devices Field rganizatin Technical supprt t field staff IT supprt team- t reslve technical prblems, peratinal prblems related t devices, applicatins and data transfer encuntered by enumeratrs and supervisrs Call center prvide lgistical supprt, IT related supprt and methdlgical supprt t field staff and infrmatin t publics Supprt in cases f emergency cnsider GPS slutin t prvide immediate supprt t field staff in case f emergency

19 Data cllectin with Internet Requirements fr data cllectin via Internet High literacy rate High rate f access t cmputers and cmputer literacy High rate f access t Internet Increasing use f mbile devices Public trust and acceptance f the Internet fr fficial business Making reasnable assumptins abut take-up rates Availability and quality f address/building/dwelling register r list Legal authrity, especially in case f mving frm traditinal field enumeratin t nline data cllectin

20 Data cllectin with Internet Cnduct and cmmunicatin strategy - Hw the husehlds are invited t respnd by Internet - Hw many time they are cntacted - Whether the ptin f self-respnding by cmpleting a paper questinnaire is ffered at initial stage Offering the paper questinnaire fr self-enumeratin can be gruped int three categries: - Paper questinnaire sent t at initial cntact, where an Internet ptin is ffered including a unique Internet access cde - Paper questinnaires available n request-t maximize the verall respnse rate - Paper questinnaire sent during the reminder phase-sending nly invitatin letter prviding Internet access cde initially and then prviding a paper questinnaire with a reminder

21 Data cllectin with Internet Identificatin and authenticatin f respndents Unique identifiers t access the nline questinnaire is necessary fr data cnfidentiality and identificatin f respnding husehlds Different appraches fr sending unique identifiers

22 Data cllectin with Internet Develpment f data cllectin prtal Capacity f the infrastructure (such as servers, strage devices and cmmunicatin vehicles) shuld be decided based n ptential users The mst imprtant factrs are: - The average number f pages a respndent will see - The average time spent per page - The number f simultaneus users at the busiest time - The ttal number f respnses expected Supprt fr respndents Online supprt instructins and cnceptual help materials fr cmpleting the nline questinnaire; frequently asked questins; a help buttn; questinnaire guide Census Helpline (Call center)- in additin t questins regarding the census, the use f the Internet applicatin, the lgin prcess r ther technical difficulties, new Internet access cde, etc.

23 Management and Mnitring fr Electrnic Data Cllectin Real time mnitring is pssible Up-t-date infrmatin can be cllected fr mnitring and managing the prgress f the enumeratin Sharing perfrmance indicatrs, reprts and alerts amng field managers/staff - Key indicatrs Status f enumeratin Cmpleted, Refused, Fllw up, N cntact, Nnresidential Enumerated ppulatin Average number f ppulatin/husing unit per day Number f data transmissins per day by gegraphical area CAPI Number and percentage f devices that have t be renewed Cunts f submitted cmpleted questinnaire/incmplete questinnaires Cmparisns between actual return and planned returns Husing units enumerated by mde f data cllectin fr timely mnitring nn-respnse CASI

24 Multi-mde data cllectin- paper+electrnic Three factrs cncerned with data quality and assciated with mdeeffect include: Cverage Item nn-respnse Measurement differences Minimizing mde effect n data quality Optimize design - Sequential design- main data cllectin with additinal methd fr nn-respnse fllw up - Cncurrent design - the different methds are equally imprtant and respndents are given a chice Use a generalized design t make them insensitive t the effects f data cllectin methd with the risk f nt using a mde t its fullest ptential Cnduct empirical studies-understanding causes f mde effects

25 THANK YOU