AgResults Kenya On-Farm Storage Pilot Survey Statistician to provide assistance for the Randomized rural household survey Scope of Work (SOW) 1. Consultant Name: TBD 2. Period of Performance: TBD 3. Level of Effort: 4 days 4. Supervisor: Andrew Gathecha, Team Leader and Richard Ngunjiri, Technical Officer 5. Background: 5.1 Pilot Purpose: The Kenya On-Farm Storage Pilot project, managed by Agribusiness Systems International (ASI) as the Pilot Manager (PM), is a part of the larger $118 million AgResults Initiative and aims to stimulate improved food security in Kenya through widespread adoption of improved on-farm post-harvest grain storage devices to reduce smallholder farmer expenditures and post-harvest crop losses. The pilot seeks to promote high-impact agricultural innovations that promote Kenyan food security, health, and nutrition and benefit smallholder farmers. Using performance-based grant pull mechanisms, the initiative promotes the uptake of innovative on-farm storage devices to generate high-yield development impacts. The objectives of the pull mechanism are to: 1) Overcome market failures impeding agricultural innovations by offering results-based grants ( pull financing) to private actors introducing and marketing agricultural storage devices to 2) Test the effectiveness and efficiency of pull financing in comparison with traditional approaches for the promotion and use of innovative agricultural technologies. The pilot will be administered through performance-based grants awarded for sales (adjusted by storage capacity and useful life), sold by private companies (implementers) in Kenya s Rift Valley and Eastern regions. A total of US$7.75 million in performance-based grants is available to participating storage device providers in the following two categories: In the Rift Valley, a total of US$3.75 performance-based grants will be offered to the first five Implementers ($750,000 each) that meet a specified minimum threshold of storage devices to small holder farmers, as well as a proportional performance-based grant of US$1.0 million divided among participants, who meet the minimum threshold, for their total sales made over the three-year pilot timeframe. Page 1 of 6
In the Eastern region, a performance-based grant of US$3.0 million will be allocated proportionally to participating Implementers who meet the minimum threshold, for their respective contributions to total sales of larger grain borer (LGB) proof storage devices. 5.2 Role of Surveys in the Pilot: Essential to determining the overall impact of the Pilot and accurately distributing the performance-based grants is accurately tracking and verifying the number of sales to smallholder farmers. As the Pilot goal is to improve the economic welfare of smallholder farmers, which are defined by the Pilot as cultivating 5ha / 12.5 acres or less, we have to not only determine total sales by Implementers but total sales made to smallholder farmers. Implementers will report total sales, disaggregated by pilot county, to the Pilot Manager Team and an audit firm will be contracted to conduct regular audits throughout the pilot (anticipated to be quarterly) to ensure that reported total sales are accurate. In order to determine the proportion of each Implementer company s total sales that only went to smallholder farmers, two large market penetration household surveys will be conducted to determine the proportions purchased by smallholder farmers of each storage product. The proportions will then be applied to the reported total sales that each company has reported and that has been audited and confirmed. 1. Randomized Rural Household Survey: This survey will sample 5,600 in the 14 Pilot counties to determine farmer proportion of rural households and proportion of small farmers. This survey will inform the approach and size of the Mid- and End-term surveys as current census data only shows population of rural Households and not farmers or smallholder farmers. The assumption for this survey at this point in time is that 400 people per county will be surveyed. This assumption should be used for planning purposes at this time. This survey will be conducted 3-6 months before the first market penetration survey. 2. First Market Penetration Survey: Informed by the Rural Household Survey, the first market penetration survey will target non-commercial farmers. It is assumed large commercial farms will not buy the pilot s storage devices and so they will not be sampled. To achieve a 95% confidence level, the survey needs to find approximately 5-6,000 farmer end-users, which requires approaching 55,000 rural households. (Please note, household sample size will be specified following Randomized rural household survey.) Note: It is assumed that one firm will conduct surveys 1 and 2 in order to use data from the first survey to design the sample, and that the third survey may be done by a different firm. 3. Final Market Penetration Survey: Informed by the Rural Household and Mid-Term surveys, the End-Term survey will target small & medium farmers. It is assumed large commercial farms will not buy the pilot s storage devices and so they will not be sampled. During this round, to achieve a 95% confidence level, the survey needs to find approximately 6-7,000 farmer end-users, which requires approaching 65,000 households. (Please note, household sample size will be specified following Randomized Rural Household Survey.) 5.3 Randomized rural household survey: Some realities of the AgResults Kenya On-Farm Storage Pilot that limit direct verification of the devices that private sector companies sell to the specific population of interest (the smallholder farmers). First, companies are unable to disaggregate sales to specific consumer types. The private companies have a multilayered, informal and frequently undefined network of sales making it difficult to trace grain storage devices sold to SHF. Furthermore, private producers have a large volume of sales that extend to many other consumers beyond smallholder farmers. Page 2 of 6
Given these constraints, the AgResults Kenya Pilot will conduct household interviews of smallholder farmers to verify the purchase of the private sector companies' products. In order to do this, the Pilot will need to determine the proportion of smallholder farmers in the 14 counties. The costs of conducting a complete census of the 14 counties is beyond the scope of this program, not to mention very costly. Furthermore, current Kenyan National Bureau of Statistics (KNBS) census data does not provide the number of smallholder farmers as defined by the Pilot. Therefore the Pilot will conduct a Randomized Rural Household Survey 12-15 months after the inception of the program (sometime between May to August 2016) within the 14 counties to estimate the proportion of smallholder famers. 5.3.1 Sample Size The Randomized Rural Household Survey will use statistical sampling methods for selecting a representative number of households to visit within the 14 counties to establish the proportion of smallholder farmers residing in the region. The sample size is calculated using the most recent 2009 KNBS data. Given that the KNBS census data does not specify smallholder farmers, it will be assumed that the population numbers indicate that every household is a smallholder farmer. This gives us an estimated smallholder farmer population in the Eastern Region of 1.1 million and in the Central Rift region of 1.53 million. (Although it is important to note that this estimate is erring on the high side given that every household is assumed to be a smallholder farmer). However, the exact numbers are irrelevant at this stage. The Randomized Rural Household Survey is simply attempting to establish an order of magnitude of proportion of smallholder farmers for the planning and implementation of additional verification surveys that will be conducted later in the program. In other words, the Randomized Rural Household Survey will establish an estimate of the minimum expected number of buying smallholder farmers, which will assist the AgResults program to plan future verification activities in terms of survey scope and costs. The sampling framework was determined by employing the standard sample size equation below: n=((z^2 p q)/σ^2 ) D Where: Z = the corresponding standard score with a confidence level of 95%, it has a value of 1.96 p = is the occurrence level of the phenomenon under study and is equal to 0.5 where the occurrence level is not known q = is the absence of the phenomenon under consideration and is equal to 0.5 where the value is not known, q=(1-p) D = is the design effect, and the power calculations for the population size of the Eastern and Central Rift Regions estimates it at 1 σ^2 = is the selected probability of error (error margin) of the study at 5%, corresponding with 95% confidence level n = sample size Replacing the above values in the equation we get, n=(( 1.96 ^2 0.50 0.50))/ 0.05 ^2 ) 1 = 384.16 The sample size calculations gives us an n of 384.16. The number is rounded off to 400 respondents. The total number of respondents that will be targeted in the Randomized Rural Household Survey is 400 x 14 Counties = 5,600 respondents. In order to ensure the veracity of the results, it is absolutely imperative that the Randomized Rural Household Survey employ a cluster random sampling of enumeration areas (EA) and also simple random Page 3 of 6
sampling approach when selecting the 400 households to be interviewed in each EA. The random selection will be conducted at two stages: the first when selecting the cluster areas, and again for selection of households. At the household level, any selection of households that is not random, in other words, replacements households, must be verified before the household is eligible for inclusion in the Randomized Rural Household Survey. In other words, the primary survey firm must work with the PM to adopt a reasonable strategy for household replacement that does not involve convenience sampling (i.e. selecting a household because it is easy to make a contact there.) This again will reduce bias in the results. The Randomized Rural Household Survey will thus establish a proportion of smallholder farmers in each of the 14 counties, and this will be used for calculating the sample size for the mid-agresults Pilot and end- AgResults Pilot market share surveys that will be done to verify the number of improved on-farm storage products sales to small holder farmers. 5.3.2 Questionnaire The Randomized Rural Household Survey will be a short structured questionnaire of about 20 minutes or less and it will include questions on AgResults product purchases; although it should be factored in that product verification may take up to an additional 15 minutes for the enumerator to visually verify the product and take a photo. The questionnaire must be administered to the head of household, spouse, or a member of the household that is older than 18 years of age. It should be administered to the individual in charge of household farming, else another individual who works with the household farm if the first individual is not available. If the respondent indicates that the household does not cultivate land, or that the cultivated land is more than the program's definition for smallholder farmer, then the enumerator records the response, asks additional questions as guided in the skip patterns, before moving to the next household per the random selection methodology. Structured questionnaires will collect important household information, including the full name of person interviewed, contact information and number of members of household. Other questions will include asking farmers if they have purchased any of the AgResults Kenya Pilot company's storage products since May 2015, document which ones (both verbally and with photo verification), how many, price paid and location of purchases. Additionally GPS coordinates will be collected for all households visited, even those that are not identified as smallholder farmers to increase the robustness of the data for later verification activities. GPS coordinates should also be collected for product purchase points when possible. 5.3.3. Timing Planning for the Randomized Rural Household Survey should account for finalization of the sampling framework, methodologies for random selection and community entry, and data collection tools between the AgResults PM Team and the survey firms contracted for this assignment. This will take approximately 2 weeks. Timing will then need to account for survey firm staff training, which is about 1 week for supervisor training and 1-2 weeks for enumerator training (including piloting of the questionnaire for the Randomized Rural Household Survey). Following the pilot of the tools, this will result in the PM and survey firms incorporating any necessary changes (including improvements of the tools) to ensure successful data collection activities with minimized error. The timing of data collection activities will be greatly affected by how the 14 counties are clustered by the Primary Data Collection Firm to randomly select the 400 households. It is a reasonable assumption that an enumerator may cover about 8-10 households per day based on time variance between screening households out of the questionnaire and necessary activities for product verification. Thus with one survey team per county comprised of 4-5 enumerators and 1 field supervisor, the demographic should take 2-3 weeks (calculated based on a 6 day work week), generously allowing for call-backs, travel and other delays. Page 4 of 6
Data cleaning and finalization can be done during data collection given that the AgResults envisions all data will be collected electronically. However, additional time will be needed following field activities to ensure overall the database is cleaned and any discrepancies have been identified and addressed. Item Finalize Inception Plan Survey Team Training and Pilot Activities Data Collection Data Cleaning and Finalization 1 2 3 4 5 6 7 8 9 6. Objective of this Consultancy Assignment: The purpose of this assignment is for the Consultant (Survey Statistician) to provide technical guidance to the AgResults Kenya Pilot Manager (PM) Team on sample size calculations and eventual sampling protocols for the Randomized Rural Household Survey. Further the consultant will create the sampling framework for the Randomized Rural Household Survey and provide technical guidance on statistical tests for data validation. It will not be required for this person to be located in Kenya as they can provide technical support remotely. 7. Consultant Responsibilities/Tasks: The Consultant will be responsible for the following tasks: Pre-survey Scope of Work for the Primary Survey Firm: a) Review and provide comments/edits to Scope of Work (SOW) for Primary Survey Firm to ensure that it accurately reflects sampling proposed sampling framework Pre-survey Sampling Technical Guidance: a) Review existing proposed calculations (above) and provide technical guidance to create the final sample size calculations that will be used for the Randomized Rural Household Survey by the Primary Survey Firm (PSF) b) After review of existing population documentation and discussion of appropriate randomization efforts and feasible field procedures, provide technical guidance on: - the sampling framework to be used by the Primary Survey Firm (PSF) - list of randomly selected areas for Third Party Monitoring firm (TPM) activities - recommended replacement methodology and protocols for non-responsive households c) Document and present final sampling frameworks, strategies, and selected areas to PSF and TPM. Pre-survey Survey Tool Review: Page 5 of 6
a) Working closely with the Survey expert and PM team, review PSF proposed sampling and statistical strategies to ensure alignment with initial guidance provided previously (action 2 above). b) Working closely with the Survey expert and PM team, review proposed tools from the PSF and TPM to ensure alignment with initial guidance provided previously (action 2 above). Post-survey Survey Results Analysis: a) Provide guidance on recommended statistical tests for data validation to the Data Specialist b) Conduct final statistical review and analysis of survey results 8. Deliverables: Upon completion of the tasks the consultant will deliver: Task Pre-survey Scope of Work for the Primary Survey Firm Pre-survey Sampling Technical Guidance Pre-survey Survey Tool Review Post-survey Survey Results Analysis Deliverable Word document with suggested edits Email/Call exchange with PM team and Survey Expert to share edits and comments Submit final sampling frameworks, strategies, and selected areas to PSF and TPM Word document with suggested edits Email/Call exchange with Survey expert to share edits and comments Brief report summarizing analysis Estimated LOE (days) 0.5 day(s) 2 day(s) 0.5 day(s) 1 day 9. Qualifications: The consultant should have the following background and skills: At least 15 years experience providing statistical analysis, survey review, and data validation PhD in economics, statistics, social sciences or other relevant field Excellent quantitative and analytical background with a proven record of high quality analytical work, particularly related to agribusiness Extensive knowledge of household surveys, and other micro- and macro-economic data Ability to deliver high-quality policy relevant analytic and advisory products within tight deadlines and budgets Familiarity with Kenyan survey data is preferred 10. Mode of Application Interested persons are advised to submit their a cover letter, specifically addressing the qualification criteria outlined, and attaching a current CV. Applications to be emailed to smcdonagh@asintl.org by 5.00pm (East Africa Time) on 29 February, 2016. Page 6 of 6