FinScope Methodology

Transcription

1 FinScope Methodology

2 1. FinScope Surveys The FinScope survey is a research tool developed by FinMark Trust. It is a nationally representative survey of how people source their income, and how they manage their financial lives. In South Africa, FinScope is conducted annually since FinScope has now been conducted in 23 countries to benchmark the state of financial access and usage. It is used to better understand money matters, with an emphasis on the market needs and attitudes to both informal and formal financial offerings and usage. 2. Objectives of FinScope The FinScope surveys aim to measure and profile the levels of access to financial services by all adults in a country across income ranges and other demographics, and making this information available for use by key stakeholders such as policy-makers, regulators, and financial service providers. Information provided by the surveys help extend the reach of financial services in the country, as it provides an understanding of the adult population in terms of: To measure levels of financial inclusion (i.e. the proportion of the population using financial products and services formal and informal); To describe the landscape of access (the types of products and services used by financially included individuals); To identify the drivers of, and barriers to, the usage of financial products and services; and To stimulate evidence-based dialogue that will ultimately lead to effective public and private sector interventions in order to increase and deepen financial inclusion. 3. Sampling Methodology The sample size takes into consideration the FinScope methodology and the default sample requirement of adult individuals in order to determine the sample size and assess how the methodology will ensure reliable estimates are produced from the sample. Sampling considerations outlined for discussion include: 1. Minimum acceptable level for national reliable estimates at total level and associated margin of error 2. Most up-to-date reliable information at provincial/district and area type level to guide population estimates and design of survey of the adult population 3. Aggregated agreed domains (normally province or district) to base the sampling design on 1 P age

3 4. Minimum samples required per aggregated agreed domains 5. Available population information as per smallest demarcated census areas (Enumerator Areas of villages in the country) 6. Reliability of most recently available population data at village level The FinScope methodology requires representative probability sample drawn systematically using probability proportional to size sampling (PPS) method. A multi-stage sampling methodology is applied which entails selection of enumeration areas (EAs) from recent census or population estimates using PPS followed by the selection of households as well as the selection of one adult in the selected household using a Kish Grid. For a PPS sample to be drawn, household counts for each EA/village is required and will be used as the measure of size. Maps for all the EAs sampled should be provided to facilitate the field operations. The starting point is to ensure that the methodology is able to produce estimates that are comparable to known totals of relevant demographics. The consideration is made to ensure that the estimates from the survey cater for these relevant subpopulation indicators. a. Sample design specifications The survey population refers to a representative sample of the adult population in a country including both rich and poor, individuals residing in urban and rural areas, women and men. An adult in this case refers to the age an individual can open a bank account, which in most countries is 18 years and older. Anyone who is younger than 18 year of age at the time of the survey is excluded from the sample universe. Also excluded are those individuals who reside in national parks and game reserves or any other area agreed upon or recommended by the national statistics office. People living in institutionalised settings, such as students in dormitories and persons in prisons or nursing homes are also excluded. The FinScope sampling design is typically a three-stage sample with enumeration areas (EAs) villages in a country as primary sampling units (PSU), households as secondary sampling units (SSU) and individuals selected by Kish Grid from a list of eligible respondents at every selected household as tertiary sampling units (TSU). 1. Target population: Adult population of ages 18 years and older. In some countries this can be 16 years or 15 years in the case of Democratic Republic of Congo. 2. Sample representation levels: The reporting domains are normally discussed with the national statistics office of the country and depending on the budget, the reporting domains are mainly the following but can also consider lower levels: a. National b. Province 2 Page

4 c. District d. Urban/rural 3. Total recommended number of minimum households to be interviewed 4. Sample to be allocated proportionally to all province or districts depending on the agreed domains 5. Population frame: Recent census Enumeration Areas/villages 6. Stratification: The stratification levels are normal province/district and urban/rural levels. 7. Selection method: EAs / villages to be selected using Probability Proportional to Size (PPS) systematic sampling procedure 8. sample size per each EA / village in terms a. Urban b. Rural In some cases there can semi-urban level classification. 9. selection using systematic random sample 10. Selection of individual adult within the selected household respondent using a Kish Grid 3 P age

5 Figure 1: Diagram of generic FinScope sample design All EAs / villages per province/district All EAs will be stratified according to district and urban/rural Rural EAs / villages Urban EAs / villages Selected on probability proportional size (PPS) EA1 EA2 EA3 EA4 EA5 EA6 Random selection of about 6-10 households per EA / village Listing of all household members (aged 15/16/18 years and older) member member member member Selection of one individual per selected household using the Kish Grid Selected individual 4 P age

6 4. FinScope Survey sampling methodology implementation As indicated above, PSU selection is determined through a combination of stratification and PPS selection under the auspices of the national sampling authority. As far as secondary sample unit (SSU) selection goes, a reliable count of the number of households in each EA/village is used, either through up-to-date figures provided by the national sampling authority (in this case the Central Statistics Office) or by conducting a household listing survey. Within each selected EA/village, about six to ten households are selected systematically, selecting every k th household (systematic random selection). The sample interval is determined by the listing exercise. Finally in order to select the tertiary sampling unit (TSU), each and every household visited will have a full listing of all adults 18 years or older in the household who qualify to be interviewed. Within selected households, where there is more than one qualifying respondent, the Kish Grid is used to randomly select the one household member with whom to complete the interview (this individual will be the ultimate sampling unit). In order to maintain strict control over appropriate sampling implementation so as to ensure the basis of the sample respects the requirements for national representation, rules for substitution of households (SSUs) are provided and controlled in terms of the following: o If a selected person is unavailable, then up to three recalls at different times of the day and days of the week are made to maintain the integrity of the sample; o If a person is repeatedly unavailable or refuses participation, then very strict substitution rules are applied; o The substitute is also chosen using randomisation techniques; and o No substitution of individuals within a household is allowed. 5 P age

7 Data required to draw a FinScope sample FinScope Survey has specific requirements on how the samples are drawn to be representative nationally. The sampling frame is constructed from the EAs / villages that are demarcated from the most recent census. EAs / villages become the lowest geographic areas in which the sample of households is drawn. EAs / villages are clearly demarcated and the maps are produced for each EA / village. In order to draw a PPS sample, the methodology makes use of the census household counts. counts used as the measure of size for the smallest geographic unit are also used during stratification. counts also inform about how big the EAs / villages are for the sake of planning and also provide us with sampling rates for selected EAs / villages. Where there is no reliable census data available, FinScope methodology further requires listing of all the households in the smallest geographic unit in the frame i.e. EA / village. The listing will then be used as the frame from which the sample of households will be drawn. The total household count obtained during the listing process should be captured after data collection and the data will be used during weighting if there are any adjustments required. The research house should ensure that interviewers are properly trained and the questionnaire is properly piloted. a. Selection of households s are selected by systematic random sampling method. A full listing of households in the selected EAs/villages is carried out by the research house, which has to go round the EA/ village and compile a list of households. The random selection of the sample households is done, based on the full listing, which is also helpful in weighting for the FinScope survey. Step 1: Step 2: Step 3: Step 4: A list of all the households in the respective EA/village is compiled The total number of households is divided by sample size to get an interval A random number between 1 and the interval is generated Suppose the random number is three, the third household from the starting point is selected for the first interview. The next household is identified by adding the interval. In the case of multi-apartment buildings, the interviewer does the sampling clock-wise as per the agreed approach (normally using Kish grid also). 6 P age

8 b. Selection of Respondents A respondent aged 15/16/18 years or above is selected from each sample household by Kish Grid method. c. Rules for household substitution If a selected household member refuses or is not available, the household needs to be substituted. The enumerator will receive a substitution household that has been selected randomly from the supervisor or any member of fieldwork with authority to implement the substitution. d. Call-back policy Interviewing is face-to-face in the respondent s home. Only one member of the selected household is interviewed. After selection of the respondent, should the respondent be not at home, the interviewer makes two call-backs in order to find the respondent at home. If the respondent is not present at home on the interviewer s third visit (second call-back), the interviewer receives appropriate instructions from the field supervisor. The interviewers are not allowed to substitute, under any circumstance, the Kish Grid respondent with another member of the household. e. Conducting field interviews and quality control Field interviewers are grouped into small teams of manageable size. Each team comprises one supervisor and a few interviewers. The supervisors select sample households in an EA/village and has the interviewers do their work under his or her close supervision. The research house should implement the following quality control measures to ensure a high level of interviewer performance. At a minimum, quality control measures included verification of the: fact that the interview took place proper application of the sampling plan in selecting the households the approximate duration of the interview the proper administration of the various sections of the questionnaire interviewer's general adherence to professional standards Field log book: It is also recommend that a research house complete a field log book. Interviewers at all times should carry a field log in which they record relevant information on what happened in the 7 P age

9 field, such as contact and call-back details. The interviewer logs supply enough information for an independent observer to locate the selected household and to identify the respondent interviewed. The log book could be a physical book or electronic. f. Back-check control details The research house should assign a Quality Control (QC) team to monitor the interviews of the survey teams. The main purpose of the QC teams is to verify the field work process. The QC officers should have experience. Included in the verification of quality control are the following: the fact that the interview took place proper application of the sampling plan in selecting the respondent the approximate duration of the interview the proper administration of the various sections of the questionnaire interviewer s general adherence to professional standards 5. Data management Data management is a very important component of the FinScope methodology. Depending on whether CAPI or PAPI is used, data should not be compromised. For PAPI, each filled-out questionnaire should be thoroughly checked at all the stages. First, the field supervisor checks the questionnaires collected by the interviewers before they leave the EA/village where the interviews were conducted so that they could reconfirm answers given with the respondents, if necessary. Secondly, the editing team should also check again the questionnaires to ensure that they are properly completed. Each method of data processing (e.g. scanning) would afford different methods of quality assurance and those intended to be utilised would need to be tabled by tendering research houses. It is important that a database structure is used that is suited to the needs of the FinScope study, and that consistent data processing methodologies are employed across surveys and different countries. The most important issue, however, is the quality of the data input. Pre-coded capturing software is an important quality tool that is utilised to eradicate capturing errors and provide an efficient method of red flagging logical errors in questionnaire responses which can then be back-checked. These measures ensure the capturing of clean data to be fed into the analysis phase. It is thus crucial that logical checks are in place in the capturing program. If, for example, a respondent says that he/she is banked and later on in the questionnaire indicates that he/she cannot get a home loan because he/she does not have a bank account then the data capturing program should flag this as an error. 8 P age

10 Efficiency is achieved by setting up and testing the data capturing specification while the questionnaires are still in the field. A 20% back-check on each interviewer s work to ensure accuracy is mandatory and the minimum acceptable standard, either in-field check-back visits or telephonic check-backs for some of the interviews conducted in rural areas. Key questions to be back-checked should be identified. Suspect interviews will need to be repeated. In order to ensure high quality data processing, it is suggested that the framework in is used in Figure 2. Figure 1: Data processing framework a. Data Entry Controls To ensure accuracy in the handling of the data, the project management team used the following measures: Checking all the questionnaires before the data entry. Editors should be assigned to check all the questionnaires for completeness of the data coding mistakes, logical links, etc. They scrutinise every questionnaire from the first page to the last. For open ended questions, coding should be shared with FinMark Trust and approved. Entering double-entry data The data from the properly filled-out and coded questionnaires can be captured into CS Pro or any other data capturing software. Double entry method should be used in which all the questionnaires will be entered twice. Although the double-entry method doubles the workload of data entry, it is a very effective method of quality control in the data entry. Additional data entry control for Computer Assisted Personal Interviewing (CAPI) 9 P age

11 A fully approved paper questionnaire will be converted through scripting to a CAPI software. The key thing about scripting is to check the script to ensure all routing is done correctly. Another issue to control for interview flow within the questionnaire as different routing options can prompt the interview to get better answers. The data management of CAPI is made simpler through the elimination of data capturing. Data integrity is protected as the omission of possible data capturing errors are eliminated and there is a stricter control of data less likelihood of data manipulation. As in PAPI methodology, the research house should ensure that interviewers are properly trained. Figure 3 below shows the scripting of the questionnaire from a research house. Figure 2: Quality control of the scripting process The next critical step is to pilot the script to ensure overall quality of the survey instrument. The main objective of piloting the script are: Ascertain duration of interview Confirm accuracy of the translations Check the CAPI script flow Get a clear view of how the respondents understand and relate to the questions 10 Page

12 Ensure that both the interviewers as well as respondents understand the process of using showcards. 6. Role of Steering Committee (SC) Almost all FinScope surveys are hosted by government through the Ministry of Finance or Central Bank or any other institution delegated by the Ministry of Finance. FinScope surveys are nationally representative studies hence require the involvement and participation of the National Statistics Office of the country. The host ministry or institution will suggest who they would like to involve in the study and are also appointed as chair whose final sign-off represents approval of all other stakeholders. The stakeholders identified from the steering committee and usually consist, but not limited to, the following: a) Ministry of Finance b) Central Bank/ Reserve Bank c) Funder of the FinScope survey, e.g. UNCDF d) Bankers Association e) National Statistics Office f) FinMark Trust g) Ministry of SMEs h) Consumer representative body i) Other stakeholders suggested by host j) Local Project Coordinator k) Research House as the implementers of the survey 11 P age