Survey Solutions Supervisor Manual Computational Tools team, Development Research Group THE WORLD BANK GROUP 1
Table of Contents Overview of the survey workflow... 3 Why the survey workflow matters for your work... 5 Changing the Interface Language... 6 Mozilla Firefox... 7 Microsoft Internet Explorer... 9 The components of the supervisor software... 10 Reports... 10 Interviews... 11 Interviewers... 13 How to perform common supervisor actions... 14 Find a survey case... 14 By questionnaire... 14 By assigned team member... 15 By status... 17 Assign a survey case to an interviewer... 19 Case not yet assigned to an interviewer... 19 Case to be reassigned to another interviewer... 21 Review a survey case... 23 Navigate... 23 Add a flag... 24 Add comments... 25 Approve, Reject... 26 See the overall progress of data collection... 26 Surveys and statuses... 26 Team members and statuses... 27 Quantity... 29 Speed... 31 Change the password for an interviewer account... 34 Lock an interviewer account... 35 2
Manual for Field Supervisors Overview of the survey workflow To understand your role as a supervisor and how the use the supervisor software, you must first understand the survey workflow. The survey workflow can be summarized by Figure 1 below. Figure 1: Overview of the survey workflow Headquarters, designated by the building at the top of the figure, determines the households that need to be interviewed and their assignments across team supervisors. Team supervisors, denoted by the laptops in the middle of Figure 1, first receive these survey assignments (as depicted in Figure 2) and then allocate them to the members of their team (as depicted in Figure 3). Figure 2: Supervisors receive assignments Figure 3: Supervisors allocate assignments 3
Interviewers, represented by hands holding a tablet, receive the assignments from the team supervisors, collect data for those assignments, and send completed assignments back to supervisors for review (as depicted in Figure 4). Figure 4: Interviewers send completed questionnaires to supervisors Team supervisors, having received completed questionnaires, review these questionnaires to confirm that all questions are answered and that answers are accurate, coherent, and plausible. After reviewing these completed assignments, team supervisors either approve or reject them (as depicted in Figure 5). If a team supervisor approves a completed assignment received from an interviewer, the assignment is sent to headquarters, as represented by the paper with a checkmark in Figure 5. If a team supervisor rejects a completed assignment received from an interviewer, the assignment is returned to the interviewer initially responsible for completing it, as represented by the paper with an X in Figure 5. Receiving the rejected assignment, the interviewer must either correct it or provide explanatory notes on strange or implausible answers. When the assignments are corrected, the interviewer sends them back to the team supervisor for approval or rejection, a process that continues until the assignments are completed with the highest level of quality according to the team supervisor. Figure 5: Supervisors reject or approve questionnaires 4
Why the survey workflow matters for your work Your understanding the survey workflow is critical for at least two reasons. FIRST, team supervisors are first and strongest line of defense for survey quality. You MUST ensure that all survey assignments for your team are completed. You MUST review each survey assignment to ensure its quality. You can either approve or reject an assignment. This decision must be taken after a thorough review of each assignment. Headquarters will hold YOU accountable for how well you ensure the quality of data collected by your team. SECOND, team supervisors need to understand the status of EACH assignment in order to manage the work of your team and ensure that all assignments are completed, reviewed, and approved. As an assignment moves through the survey workflow, it takes on a different status, as illustrated in Figure 6. When an assignment sent from headquarters to the supervisor, that assignment is assigned to the supervisor (in the terminology of the software, SupervisorAssigned). When an assignment is allocated by the supervisor to a member of their team, that assignment is considered as assigned to an interviewer (in the terminology of the software, InterviewerAssigned). When an assignment is completed by an interviewer and sent to the supervisor, that assignment is marked as completed by the interviewer (in the terminology of the software, Completed). If the supervisor approves a completed assignment, that assignment is approved by the supervisor (in the terminology of the software, ApprovedBySupervisor). If the supervisor rejects the completed assignment, that assignment takes on the status of rejected by the supervisor (in the terminology of the software, RejectedBySupervisor) When headquarters rejects an assignment, that assignment is considered as rejected by headquarters (in the terminology of the software, RejectedByHeadquarters) When headquarters approves an assignment, that assignment is considered as approved by headquarters (in the terminology of the software, ApproveByHeadquarters) 5
Figure 6: Status of an assignment in each stage of the survey workflow Changing the Interface Language Survey Solutions interface for Headquarter and Supervisor users can be presented in different languages Russian, Chinese, or French. This article describes how to switch the interface to be presented in an alternative language. Recall that both HQ and Supervisor users work via a web browser. Switching the interface is done by telling the browser to request the content in a particular language. Note that this affects both Survey Solutions and other sites that are visited with the same browser. The remainder describes step-by-step instructions for the particular browsers you may be using. Other browsers likely to have similar functionality. Google Chrome The following instructions are based on Google Chrome version 50.0.2661.102. 1. Start Google Chrome browser; 2. Click on the drop down menu and select Settings item; 3. Click the Show advanced settings... link at the bottom; 4. Click Language and input settings button; 6
5. Click the Add button in the Languages list to add a language you need. 6. Drag and drop that language to the top of the list. 7. Click Done. 8. Close and restart the browser. Mozilla Firefox The following instructions are based on version 45.0.1 of Mozilla Firefox. 7
1. Start the Mozilla Firefox browser. 2. Click on the drop down menu and select Options item; 3. Select Content tab. 4. In the Languages section, click the Choose... button. 5. Select a new language you want to add from the drop-down menu and click the Add button. 6. Adjust the order of the languages to have the desired language at the top. 7. Click OK. 8. Close and restart the browser. 8
Microsoft Internet Explorer The following instructions are based on the version 10.0.9200.16750 of Microsoft Internet Explorer. 1. Start Internet Explorer. 2. Click the gear icon to access the drop-down menu. 3. Select Internet Options. 4. In the General tab click Languages button: 5. Click the Add... button to add the desired language. 9
6. Use the Move up and Move down buttons to change the order of the languages to have the desired language at the top of the list. 7. Click OK. 8. Close and restart the browser. If you are using a different browser the steps to take are likely very similar. See also the W3C Internationalization page for the additional information. The components of the supervisor software The supervisor software is a suite of connected tools for the supervisor to track the progress of their team (Reports), manage the quality of their work (Interviews), and manage their members of the team (Interviewers). The following section will provide an overview of each tool Reports, Interviews, and Teams and Roles and of how each tool works. Reports Reports tab provides four different types of reports: Survey and Statuses, Team members and Statuses, Quantity and Speed. For example, the Surveys and Statuses report (shown below), presents a summary of assignments by questionnaire. Details on each report type and ways to tailor each report are explained in the See the overall progress of data collection section. 10
Interviews The Interview panel constitutes the core of where supervisor does their work. It is a detailed list of survey assignments. To find this list, click on the Interviews tab at the top of the screen. The Interview interface consists of several pieces. FIRST (marked 1 in the image below) is the list of survey assignments, which captures location information (Featured Questions), which member is responsible (Responsible), when the assignment was last updated (Last Updated), whether it contains errors (Has Errors), and what status it has (Status). SECOND (2) is a set of filters for narrowing the list down to the questionnaire (Template), team member responsible (Responsible), and/or assignment status (Status). THIRD (3) is a tool for allocating one or several survey assignments to a member of the team, selected by clicking the tick box associated with the assignment. FOURTH (4) is a button that can be clicked in order to review and approve or reject a completed survey assignment. 11
The Interview panel allows the supervisor to (i) see the assignments allocated from Headquarters, (ii) allocate the assignments to his/her team members, (iii) review completed assignments, and (iv) reject or approve the assignments. For any completed assignment, the supervisor clicks on the green button associated with that assignment (depicted as 4 in the figure above). The supervisor software then opens an interface for reviewing, commenting, and approving or rejecting the assignment. The Interview review panel has several components (as displayed below). Filters (marked 1 in the image below) are in the left-hand of the screen. They allow the supervisor to navigate from one section of the questionnaire to another by clicking on a heading. Bolded headings represent modules. Indented headings represent groups of questions or rows of rosters. Additional filters (2) are on the top right-hand segment of the screen. They facilitate the display of questions that may be of interest to the supervisor. The All filter presents all questions. The commented and flag filters display those questions with comments or flags, respectively. The answered questions, as the name suggests, are all those that have an answer. The invalid filter displays those questions that failed a validation check (e.g., range, consistency with other responses, etc.). The supervisor s filter yields those special questions that only the supervisor sees and can answer. The enabled filter returns questions that are available to be answered based on the skip logic of the questionnaire and answers provided to key questions. 12
The center part of the screen (3) lists the questions and the answers of the content being reviewed. The panel immediately to the right (4) is where the supervisor reads comments from the interviewer and provides comments to the interviewer. The final components are the green approve button and red reject button. These important buttons determine whether the assignment is sent to headquarters for inclusion in the survey database (Approve) or returned to the interviewer for correction (Reject). Interviewers The final major component of the supervisor software is the Interviewer dashboard, opened through clicking on the Teams and Roles tab at the top of the screen. This component is responsible for the small parts of human resource management that the supervisor handles. Headquarters determines which human resources are in which team. Supervisor is relegated to the role of managing the user accounts for their team. To do so, the supervisor clicks in the blue Edit button associated with the team member (as depicted below). 13
How to perform common supervisor actions Find a survey case Use filters to narrow down your search by questionnaire, by team member, and by survey status. FIRST, click on the Interviews tab at the top of the screen (between the Reports and Interviewers tabs): By questionnaire To filter by questionnaire, move your cursor to the Filter panel at the left of the screen. Then, click on the drop-down menu below Templates. Select the questionnaire whose survey assignments you want to see. 14
The results will be a list of only those assignments for that particular questionnaire, regardless of who is responsible for them or what their status is. By assigned team member To filter by the team member responsible, click on the drop-down menu bellow Responsible and select the team member whose assignments you want to see. 15
This will yield a list of all the assignments for that team member. The supervisor is also one team member that can be selected. Selecting the supervisor s questions so will provide a list of all those assignments that have been sent to the supervisor by headquarters but not allocated to any other team members (as depicted below). 16
By status To filter by status, click on the drop-down menu below Status and select the status of interest. This will return the survey assignments that are in a given status. Of particular interest may be assignments whose status is SupervisorAssigned or Completed. The SupervisorAssigned cases still need be shared distributed to the team members. The Completed cases need to be reviewed and approved or rejected by the supervisor. An explanation of each survey status The possible statuses of an assignment are be explained by this diagram. 17
Any This will select assignments with any status that is the full list of assignments for the supervisor s team, including those that have been completed. SupervisorAssigned. Because all assignments come to the supervisor from headquarters, each new interview assignment starts its life assigned to the supervisor that is with a status of SupervisorAssigned. This filter may be useful, then, for supervisors to identify those assignments from headquarters that need to be assigned on to members of the supervisor s data collection team. InterviewerAssigned. Once a supervisor reassigns a new interview from the supervisor to an interviewer, that interview assignment s status changes from SupervisorAssigned to InterviewerAssigned. This filter provides the supervisor with a list of all of the interview assignment have been assigned to individual interviewers but have not yet been completed. RejectedBySupervisor. Supervisors play a strong role in quality control. They must either approve or reject each questionnaire that an interviewer completes. If the supervisor rejects an interview, that interview will have the status RejectedBySupervisor. If after review of the interview the supervisor rejects the interviewer s work then that case take on the status RejectedBySupervisor interviews are by default returned to the interviewer that completed them, but the supervisor could filter according by the RejectBySupervisor status in order to have the opportunity of reassigning a rejected questionnaire to another interviewer within the team. Completed. When an interviewer marks an interview as complete and synchronizes with the supervisor, the completed interview assignment will be marked with the status Completed. The 18
supervisor may find this filter particularly useful in efficiently identifying interviews that need to be reviewed and either approved or rejected. ApprovedBySupervisor. If after careful review the supervisor approves an interview assignment, the status becomes ApprovedBySupervisor. When the supervisor next synchronizes with the headquarters this assignment will be sent to headquarters. RejectedByHeadquarters. Headquarters plays a strong role in quality control. They must either approve or reject each questionnaire that a supervisor approves. If the headquarters rejects an interview, that interview will have the status RejectedByHeadquarters. ApprovedbyHeadquarters. When headquarters approves an assignment, that assignment is considered as ApprovedByHeadquarters. Assign a survey case to an interviewer To begin, click on the Interviews tab at the top of the screen (between the Reports and Interviewers tabs). The supervisor application will then show the full list of all assignments to the team. Next, use filters to find the cases to be assigned. Consider the following common scenario. Case not yet assigned to an interviewer Select the name of the survey from the Templates menu. Then select SupervisorAssigned from the status menu (as displayed below). 19
This will display all of the survey cases that have not yet been assigned to an interviewer for a particular survey or version of a survey (as displayed below). Next, click in the tick box to the left of the survey case(s) to be assigned to an interviewer. Then, click on the Assign interviewer and select an interviewer from the drop-down menu. This will assign the selected survey case(s) to the selected interviewer. The interviewer will see these new assignments on the dashboard of their tablet upon the next synchronization. 20
Case to be reassigned to another interviewer Use similar steps to reassign a survey case from one interviewer to another. This can be done in a few quick steps. First, use filters to find the case or cases that need to be reassigned. The most common is to reassign Rejected questionnaires or Complete questionnaires that are marked as complete but are not (e.g., one interviewer does as much of the interview as they can but another interviewer needs to continue the interview). Second, select the case(s) by clicking in their tick box associated with the case(s). Then, select a different interviewer from the drop-down Assign interviewer menu to be responsible for the interview. 21
While the steps for reassignment are simple, the supervisor needs to exercise extreme care in changing the assignment of an interview. There is no problem in reassigning assignments that were not picked up by their respective interviewers. However, if incomplete interviews are reassigned, there is a risk that two interviewers may submit different versions of the same interview that will need to be reconciled manually (e.g., one interviewer completes the interview, the other interviewer does not complete the interview but marks it as complete). The supervisor will know if any of the interviews being reassigned have been already received by the assignees, and will have to confirm reassignment of such interviews: This helps prevent the situations of unintentional duplication of assignments. Supervisors should reassign interviews already received only in cases where there is reasonable confidence that the interviewer has dropped out of the program and his device is unavailable. Reassigning the interview does not magically carry over unsynchronized data from 1 st interviewer to the second one. Rather it redirects the assignment as seen by the supervisor (empty in most cases until completed). If the first interviewer has opened a number of assignments and needs the second 22
interviewer to finish them, she should mark them as completed (even they are only partially filled out) and let the supervisor know to reject them to the second interviewer. In that case the data already collected to date will flow over to the second interviewer. Review a survey case First, click on the Interviews tab at the top of the screen. Then, click on the green icon next to the interview that you would like to review. To review the questionnaire, the supervisor will need to use several tools. Navigate To navigate through the questionnaire content, use two sets of filters. Those in the left-hand pane will allow you to navigate to various groups of questions. Clicking on bold-faced headings will bring you to the beginning of a module. 23
Clicking on non-bolded, indented headings will bring you to the beginning of groups of questions or rows of a roster. The rows of roster will follow the following format: Roster name: Roster row name. The supervisor can further navigate through the questionnaire by using the filters arrayed along the top of the screen: all, flagged, commented, answered, invalid, supervisor s, and enabled. The All filter presents all questions. The commented and flagged filters display those questions with comments or flags, respectively. The answered questions, as the name suggests, are all those that have an answer. The unanswered questions are those that do not have an answer. The invalid filter displays those questions that failed a validation check (e.g., range, consistency with other responses, etc.). The supervisor s filter yields those special questions that only the supervisor sees and can answer. The enabled filter returns questions that are available to be answered based on the skip logic of the questionnaire and answers provided to key questions. Add a flag The supervisor will have review large amounts of information in each questionnaire. Flags enable the supervisor to mark a particular question as a place to return later, perhaps when comparing the consistency of an answer in an early module of the questionnaire with an answer in a much later module of the questionnaire. The supervisor can see these flagged fields quickly, as explained above, by clicking on the flagged filter along the top of the questionnaire review screen. 24
How does the supervisor add a flag? Follow these simple steps. First, hover over the question where you want to leave a flag, moving your cursor to the left-hand side of the screen. Then, click on the flag icon. Once a flag has been added, the supervisor can click on the flagged filter to see the field or fields bearing a flag. Add comments The space on the right-hand side of screen is also where the supervisor can both read comments from the interviewer and provide comments to the interviewer. To leave a comment, hover over the question to comment and move your cursor to the right-hand side of the screen. Then, click on Comment text box. Next, type your comment into the field provided and press on the Add comment button. 25
Approve, Reject The culmination of reviewing the questionnaire is the supervisor s decision to approve or reject the questionnaire. To approve, click on the Approve button; to reject, the Reject button. Whatever button is pressed, a confirmation screen will appear with the opportunity to provide comments on the questionnaire as a whole. At this confirmation screen, click on the Approve or Reject button. But be very sure of your decision before proceeding. There is no way to un-approve or unreject a questionnaire unless it is rejected by Headquarters. See the overall progress of data collection Surveys and statuses First, click on the Reports tab at the top of the screen and select Surveys and Statuses from the dropdown menu. This will yield team-level summary of how many assignments are in each status, by questionnaire. 26
Using the filter in the left-hand portion of the screen, you can select this same summary for individual team members. To do this, click on the drop-down menu below Team members and select the member whose summary you want to see. Team members and statuses First, click on the Reports tab at the top of the screen and select Team members and Statuses from the drop-down menu. This report provides a summary of how many assignments are in each status, by team member (including the supervisor). 27
To triage the counts in the table further, select the questionnaire for which you want to see summary statistics using the drop-down menu under the Template rubric of the Filters pane on the left of the screen. This will yield a summary of progress for the particular survey chosen. 28
Quantity The Quantity report provides the following reports for each questionnaire template: Number of completed interviews: Provides the number of completed interviews for the supervisor s data collection team. Interviews that have been rejected and then completed are counted as an additional completed interview. Number of interview transactions by HQ: Provides the number of headquarters approvals and rejections for the supervisor s data collection team. 29
Number of interviews approved by HQ: Provides the number of headquarters approved interviews for the supervisor s data collection team. Number of interview transactions by supervisor: Provides the number of supervisor approvals and rejections on interviews collected for the supervisor s data collection team. 30
To use these report, first select Quantity from the Reports menu. Then, select the Type of report and the questionnaire template. Select a start date for the report (From). Next, define the time interval-- day, week, or month (Period), and the number of time intervals that you would like to display in the report (Period count). The maximum period count that can be defined is 12. Speed The Speed report provides the following reports for each questionnaire template: Average interview duration: Provides the average time difference between the first recorded answer on a tablet and when the complete button is selected. 31
Average supervisor processing time: Provides the average time difference between the moment the complete button is pressed and the supervisor s approval or rejection of that questionnaire. Average HQ processing time: Provides the average time difference between a supervisor approval of a questionnaire and a HQ approval or rejection of that questionnaire. 32
Average case assignment duration: Provides the average time difference between survey case assignment to an interviewer and the moment an interviewer sets the survey case as complete on a tablet. Average overall case processing time: Provides the average time difference between survey case assignment and survey case approval by HQ. 33
To use these reports, first select Speed from the Reports menu. Then, select the Type of report and the questionnaire template. Select a start date for the report (From). Next, define the time interval-- day, week, or month (Period), and the number of time intervals that you would like to display in the report (Period count). The maximum period count that can be defined is 12. Change the password for an interviewer account During the survey, an interviewer may forget his or her password. To address problem, the supervisor can change the interviewer s password. To start, click on the Teams and Roles tab at the top of the screen. Then, click on the Edit button associated with the interviewer whose password needs to be changed. 34
Next, compose a new password and confirm it. Then click on the Save changes button at the bottom of the screen. Lock an interviewer account An interviewer may leave the survey for a variety of reasons, from illness to poor job performance. If such a situation arises, you may need to lock the interviewer s account. But before locking the interviewer account, make sure that you have recovered all completely and partially complete interviews from the interviewer s tablet. When you are ready to clock the account, first click on the Teams and Roles tab at the top of the screen. Then click on Details button associated with the team member whose account you would like to lock. 35
Next, click in the box next to Is Locked. Finally, click on the Save changes button to lock the account. 36