The best times to call in a mobile phone survey

Transcription

1 International Journal of Market Research Vol. 57 Issue 4 The best times to call in a mobile phone survey Paula Vicente Instituto Universitário de Lisboa (ISCTE-IUL) Establishing contact with the sample units is an important part of the survey response process, and an efficient calling schedule is critical to achieve high response rates. The rapid increase in mobile phone ownership has triggered the interest of marketing researchers in the use of mobile phones for collecting survey data about consumers. Mobile phone surveys may favour establishing contact with sample units since the mobile phone is a personal device carried at all times, thus making the person permanently contactable. This paper aims to identify the best times to call in a mobile phone survey by investigating the influence of the day and time of the call on the likelihood of establishing contact and obtaining an interview. A three-level ranking of calling periods, based on call efficiency, is proposed. Outcomes also revealed that the level of efficiency of calling periods is not dissociated from respondents socio-demographic characteristics, namely in terms of age and region of residence. Introduction Minimising the effort taken to complete an interview in order to reduce costs and increase response rates without introducing bias is a key objective in any survey. This goal is entirely met when an interview is obtained from each sampled unit with a single call attempt. However, this is a utopian scenario for in reality interviewers may fail to get an interview from a sample unit either because they cannot establish contact with the unit, or because the unit refuses to cooperate or is incapable of participating in the survey (Groves et al. 2004, p. 170). Making contact with the sample units is the most critical challenge for a good response rate since it is relatively easy for a well-trained interviewer to get people s cooperation once contact has been made (e.g. Weeks et al. 1980; Brick et al. 1996). Received (in revised form): 27 February The Market Research Society DOI: /IJMR

2 The best times to call in a mobile phone survey In telephone surveys establishing contact would not be a problem if one knew in advance when people were at home. However, that information is generally unknown. Several studies have tried to predict people s at-home patterns in order to help define a call strategy that maximises the probability of contact. The investigation has focused mainly on the following issues: (1) discovering the best times to make first call attempts (e.g. Weeks et al. 1980; Hansen 2008) and (2) finding out the best pattern of non-contact scheduling so as to increase the likelihood of contact among cases that could not be contacted previously (e.g. Stokes & Greenberg 1990; Hansen 2008). In the context of telephone surveys, investigation has been conducted on the association between the timing of calls and the call outcome in order to identify the best times to make first call attempts. For the general household population, it was found that the best times to make first calls were Monday to Friday between 7 pm and 9 pm and weekends between 9 am and 7 pm, specifically Sundays between 4 pm and 9 pm, since the highest contact rates and highest interview rates are achieved in these periods (e.g. Weeks et al. 1987; Kulka & Weeks 1988; Lavrakas 1993; Ahmed & Kalsbeek 1998; Brick et al. 1996; Yuan et al. 2005). While discovering the best times to call is essential to increase the likelihood of making contact at the first call attempt, designing a strategy for non-contact scheduling is crucial to increase the likelihood of making contact on subsequent attempts (e.g. Stokes & Greenberg 1990; Dennis et al. 1999). Different aspects of non-contact scheduling must be defined, e.g. days and times to call back, the order in which calls are to be made to specific time periods, the time to wait between calls and the maximum number of call attempts. At any point in the surveying period, there is limited but well-defined information about each unreached telephone number in the sample. This information, which includes the time and outcomes of unsuccessful calls to the number, is an important input for scheduling callbacks. Approaches to non-contact scheduling may include conditioning delivery of a number to a call at a specific time based on outcomes of prior contact attempts or assigning priority scores for all numbers based on call histories each time the scheduler runs (Hansen 2008). With the steady increase in the use of mobile phones to conduct survey research (Couper 2011), it is important that survey researchers get to know this new survey mode in order to take the best from it. In the context of mobile phone surveys, little attention has yet been given to the subject of best calling times. Although Yuan et al. (2005) analysed contact rates and interview rates across time periods in a mobile phone survey, the main 556

3 International Journal of Market Research Vol. 57 Issue 4 purpose was to make a comparison with a fixed phone survey in order to disclose mode differences, as opposed to identifying best periods to call in the mobile phone survey. Vicente and Reis (2009) identified best times to call in a mobile phone survey, but based solely on the interview rate; non-contact rate was not analysed. One particular aspect of mobile phones that may favour the likelihood of establishing contact is the fact that the mobile phone is a personal device carried at all times, thus making the user contactable anywhere, any day of the week and time of day. Respondents who are typically difficult to reach at home may now be reached more easily thanks to mobile phones. But to what extent is this feature advantageous for survey research? Are all calling periods equally good for making contact and obtaining cooperation? The purpose of this study is to explore the association between the time period of the first calls and their outcome in a mobile phone survey and, if evidence of covariance is found, to identify the best calling periods for achieving a high contact rate and a high interview rate. Specifically, the study addresses the following questions. a) How does the day and time of the call affect the likelihood of establishing contact and obtaining an interview? b) Are calling periods equally good for establishing contact and obtaining an interview? c) To what extent is the call efficiency of a specific calling period associated with respondents characteristics? This paper aims to offer academic researchers and survey practitioners guidance on how to design an efficient first call schedule. Differences between calling times will have implications for the allocation of workloads, time and staff resources by survey agencies. Data and methods Data used in this study were collected by means of a mobile phone survey using a national RDD sample of mobile phone numbers. The survey was conducted in a CATI environment between 3 and 23 May 2012 by the marketing research company Marktest. It covered the general Portuguese population of users of mobile phones aged 15 years or older. Sample selection was not list-assisted as there is no database of mobile phone numbers that can be used as a sampling frame. The sample was stratified 557

4 The best times to call in a mobile phone survey by service operator and, in each stratum, the numbers to be called were created by a generator of seven-digit random numbers, not all of which were necessarily attributed to someone. A total of 11,472 numbers were dialled, 9,662 of which were eligible numbers. Up to 15 call attempts were made to eligible numbers. Calling times were assigned to the mobile phone numbers to be dialled using the company s usual calling protocol for a study of this size. Based on ESOMAR guidelines for mobile phone research, Marktest adopts the same calling hours as for fixed-line phone surveys (ESOMAR 2011). As such, mobile phone numbers were not randomly assigned to each available time period because it is not efficient to make contacts in every time period and on every day of the week. The calling times were spread over the 5 pm until 10 pm period on weekdays, and the 10 am to 2 pm period at weekends. The study therefore assumed the profile of a field experiment using an after only design employing standard procedures utilised by the marketing research company. A standard approach when analysing call outcomes in RDD surveys is to classify all call attempts into a predefined number of time periods. There is no standard classification that fits all surveys (for different alternatives, see Brick et al. 1996; Dennis et al. 1999; Yuan et al. 2005). Ultimately, the categorisation is very dependent on the calling protocol of the survey organisation. The calling periods in this study were collapsed into 19 categories: from 5 pm until 7 pm, from 7 pm until 9 pm, and from 9 pm until 10 pm (all weekdays), from 10 am until 12 noon and from 12 noon until 2 pm (weekends). All call attempts were classified by call outcomes as follows. Completed interview: when a completed questionnaire was obtained. Refusal: when the interviewer established a conversation with the mobile phone user but he/she declined to cooperate with the survey. Contact/non-contact: when the interviewer was able to talk with the mobile phone user (aged 15 or older), irrespective of whether he/she actually completed the interview. Non-contact comprises cases in which no communication was established with the mobile phone user, i.e. ring with no answer, voic , call rejection, or busy. Non-eligible: when the interviewer determined that the mobile phone number was not working, not attributed or disconnected. Someone outside of the survey s scope i.e. the person answering the mobile phone call was under the age of 15 years of age was also considered as non-eligible. 558

5 International Journal of Market Research Vol. 57 Issue 4 The analysis starts with a description of the call outcomes. Afterwards the relation between calling times and calling outcomes is investigated. Since the structure of the data cannot be identified with a randomised design, logistic regression models can be used to examine the relationship between the timing assigned to the numbers and the outcomes of the calls. As such, logistic regression is used to estimate the effect of the day and time of calls on (1) the probability of establishing contact and (2) the probability of obtaining a completed interview. In a third step, a clustering analysis is performed to define a ranking-typology for the 19 calling periods according to contact rate and interview rate. Finally, an assessment is made of whether there is an association between ranking-typology of calling periods and respondents characteristics. Descriptive analysis of calling outcomes Table 1 presents some statistics on the number of call attempts and the number of call attempts to first contact so that the overall level of effort taken to establish contact with sampled mobile phone users can be gauged. The number of call attempts ranged from 1 to 15, either considering all dialled numbers (n = 11,472) or only the eligible numbers (n = 9,662). However the third quartile equals three calls, which means that 75% of the numbers dialled/eligible were called a maximum of three times. Both distributions of the number of call attempts are therefore strongly positively skewed. Taking all dialled numbers, an average of 2.46 call attempts were made per mobile phone number; when restricting the analysis to eligible numbers the average number of call attempts is The distribution of the number of calls to first contact reveals that among the numbers that could be contacted (n = 3,127) 50% were contacted with Table 1 Statistics on the number of call attempts and number of call attempts to first contact Number of call attempts 1 Number of call attempts 2 Number of call attempts to first contact 3 Mean st quartile nd quartile rd quartile Minimum Maximum Notes: 1 Based on all dialled numbers (n = 11,472); 2 based on eligible numbers (n = 9,662); 3 based on numbers that could be contacted (n = 3,127) 559

6 The best times to call in a mobile phone survey a single contact (second quartile = 1 call) and that two calls were enough to establish contact with 75% of the mobile phone numbers. On average, 1.70 calls were required to make the first contact with mobile phone users. Table 2 presents the distribution of the outcomes of first call attempts, considering both the total numbers dialled and the eligible numbers. Approximately 16% of the total numbers dialled were considered non-eligible. The majority of non-eligible numbers were either not attributed or disconnected (13.5% of total numbers dialled). As the analysis focuses entirely on the ability to reach active mobile phone numbers, numbers determined to be non-eligible are excluded from the subsequent analyses. Among eligible numbers, the percentage of completed interviews was 9.7% 139 respondents ended the interview before the final question. There were 1,012 refusals, corresponding to 10.5% of the eligible cases. The percentage of cases coded as contact reached 21.6% (11.1% %) and the overall non-contact rate was 78.4%. The main components of non-contacts were voic (38.9%) and unanswered ring (31.2%). Table 3 presents the distribution of the first calls by the time of day and day of week considering the collapsing of the days into weekdays vs weekends. Most of the first call attempts were made in the weekday afternoon and evening periods (28.9% and 31.6%, respectively). The calls made at weekends corresponded to 20.6% of the overall calls. Table 2 Detailed results of first call attempts Result after first call attempt n % % eligible Interview 1, Completed Uncompleted Refusal 1, Non-contact 7, Unanswered ring 3, Voic 3, Call rejection Busy Non-eligible 1, Out-of-the-scope Not working Non-attributed or disconnected 1, Total numbers dialled or eligible 11, ,

7 International Journal of Market Research Vol. 57 Issue 4 Table 3 Distribution of the first call attempts by time of day and day of week Time of calls Weekday Weekend 10 am 12 noon 8.6% (830) 12 noon 2 pm 12.0% (1,160) 5 pm 7 pm 28.9% (2,790) 7 pm 9 pm 31.6% (3,051) 9 pm 10 pm 19.0% (1,831) Note: Total n = 9,662 Determinants of contacts and interviews A binary logistic regression was performed to investigate the determinants of the probability of establishing contact with the mobile phone user, taking as dependent variable the call outcome coded as 1 contact and 0 non contact. A similar analysis is made to estimate the probability of obtaining a completed interview; in this case the dependent variable is coded as 1 completed interview and 0 other outcome. The predictor variables in both models are the time of call (five categories) and the day of call (seven categories). Both main effects and interaction effects are explored. The estimated odds ratio is also reported for each parameter. An odds ratio greater than one indicates that the likelihood of making a contact/interview is higher in the corresponding category than in the reference category; an odds ratio lower than one indicates that the likelihood of making a contact/interview is lower in the corresponding category than in the reference category. Table 4 shows the estimated parameters of the binary logistic regression model for predicting the likelihood of contact. Only the categories of the predictor variables with a significant effect are presented. There are significant effects from the day (p 0.001) and from the day*time interaction (p 0.05). The main effect of time was not significant, which means that the likelihood of contact was not affected by the time of the call. The likelihood of establishing contact is approximately twice as high on Saturdays (odds ratio 1:2.138), Fridays (odds ratio 1:2.170) and Mondays (odds ratio 1:1.910) than on Wednesdays (the reference category) (all p 0.001). The likelihood of establishing contact between 5 pm and 7 pm is also higher on Thursdays than on Wednesdays (odds ratio 1:1.785) (p 0.001). Table 5 shows the estimated parameters of the binary logistic regression model for predicting the likelihood of obtaining an interview. 561

8 The best times to call in a mobile phone survey Table 4 Estimates of the binary logistic regression model for the likelihood of contact Predictor Contact 1 Odds ratio Day (ref: Wednesday) Monday *** Thursday * Friday *** Saturday *** Interactions Thursday*5pm 7pm *** Constant Notes: 1 Estimation using Forward Wald method; categories with non-significant effect omitted; * p 0.05; *** p Table 5 Estimates of the binary logistic regression model for the likelihood of completed interview Predictor Interview 1 Odds ratio Interactions Sunday*10 am 12 noon * Monday*7 pm 9 pm *** Tuesday*5 pm 7 pm 0.499** Thursday*5 pm 7 pm *** Thursday*7 pm 9 pm *** Friday*5 pm 7 pm *** Friday*7 pm 9 pm *** Saturday*10 am 12 noon *** Saturday*12 noon 2 pm ** Constant Notes: 1 Estimation using Forward Wald method; categories with non-significant effect omitted; *p 0.05; **p 0.01, ***p Only day*time interactions have a significant effect on the probability of obtaining a complete interview (p 0.001). The main effects of day or time were not significant. There are several calling periods that are favourable to the likelihood of getting a complete interview (p 0.001), namely the 5 pm to 7 pm period on Fridays and Thursdays, and the 7 pm to 9 pm period on Mondays (odds ratio above 2). 562

9 International Journal of Market Research Vol. 57 Issue 4 Ranking of calling periods So far it has been found that the probability of making contact or obtaining a completed interview is affected by the day of the call, and/ or by the day*time interaction. In this section, the efficiency of calling periods is investigated by considering contact and interview outcomes simultaneously. A calling period that achieves both a high contact rate and a high interview rate is considered to be the most efficient. Calling periods are ranked by conducting a cluster analysis on the k = 19 calling periods. A cluster analysis allows the identification of groups of calling periods with strong internal homogeneity in terms of contact rate and interview rate. The resulting classification leads to a ranked-typology of calling periods in terms of calling efficiency. The contact rate and the interview rate are computed, respectively, as follows: contact rate k number of contactsk = 100% eligible numbers k interview rate k number of interviewsk = 100% number of contacts The contact rate in the k calling period is measured by the percentage of calls whose outcomes was coded as contact over the total number of eligible mobile phone numbers in that period. The interview rate in the k calling period is computed as the percentage of completed interviews over the mobile phone numbers for which the calling outcome was coded as contact in that period. The cluster analysis was first carried out using a hierarchical approach. Squared Euclidean Distance was used as a dissimilarity measure and the Ward method as the criteria to merge the clusters (Arabie et al. 1996). The dendrogram (not presented) revealed three groups. Since no single method can be considered the best to perform a cluster analysis, a non-hierarchical approach k-means was then adopted. The results of both approaches were then compared in order to make a final decision on the groups to form. A strong association between the outcomes of the two approaches was found (contingency coefficient = 0.771, p 0.001), which indicates that the cluster analysis consistently revealed the segmentation structure of the data. Table 6 presents the range and the mean of both the contact rate and the interview rate for each of the three groups (hierarchical Ward method) formed. k 563

10 The best times to call in a mobile phone survey Table 6 Range and mean of contact and interview rate by group (%) Group 1 Best Group 2 Medium Group 3 Worst Contact rate a,c Range Mean Interview rate b,d Range Mean Notes: Significant differences between groups with: a p and b p 0.01; c Group 1/Group 2 were not significantly different (p > 0.1); d Group 2/Group 3 were not significantly different (p > 0.1) A Kruskall-Wallis test was performed to assess whether or not the contact rate and interview rate of the three groups differ (Gibbons 1993). Significant differences were found both in terms of contact rate (p 0.001) and interview rate (p 0.01). Dunnett s C Post-Hoc test was computed to assess the differences between each pair of groups. 1 Group 3 is different from Group 1 and Group 2 in terms of contact rate (p 0.05). As for interview rate, significant differences were found between Group 1 and Group 2, and between Group 1 and Group 3 (p 0.05). The groups can be described as follows. Group 1 has the highest mean contact rate (mean = 28.3%) and the highest mean interview rate (mean = 53.4%). The contact rates of Group 2 and Group 1 are similar (mean of 26.9% and 28.3%, respectively) but Group 2 has lower interview rate (mean = 41.7%). Group 3 has the lowest contact rates (mean = 15.6%); its interview rate (mean = 40.6%) is similar to Group 2 but smaller than Group 1. As such, and in terms of efficiency of call attempts, Group 1 can be rated as the best, Group 3 the worst and Group 2 as medium. Table 7 is a graphical summary of the classification of calling periods revealed by the cluster analysis. Calling periods classified as 3-worst are Tuesdays and Wednesdays in all time periods, Thursdays 9 pm 10 pm and Sundays 12 am 2 pm. The following periods have a 2-medium level of efficiency: Sundays 10 am 12 am, Mondays 5 pm 7 pm and 9 pm 10 pm, Fridays 7 pm 10 pm and Saturdays 10 am 2 pm. Finally, Mondays 7 pm 9 pm, Thursdays 5 pm 9 pm and Fridays 5 pm 7 pm are rated as 1-best. 1 Although the groups independence assumption for computing both the Kruskall-Wallis and the Post-Hoc test is not fulfilled, its output is presented to determine how big the difference must be between groups in order to be noticed. 564

11 International Journal of Market Research Vol. 57 Issue 4 Table 7 Ranking of calling periods according to call efficiency Sun Mon Tue Wed Thu Fri Sat 10 am 12 noon noon 2 pm pm 7 pm pm 9 pm pm 10 pm Note: 1 = best, 2 = medium, 3 = worst Socio-demographics of respondents by calling period So far it has been found that calling periods vary in terms of calls efficiency, but it can also be hypothesised that calling periods may be different regarding the type of person interviewed. In this section an evaluation is made of whether there is an association between the typology of calling periods and respondents socio-demographic characteristics. The socio-demographic profile of the respondents interviewed in each of the three types of calling period was analysed by sex, age, educational level, occupation, marital status, social class and region of residence. Chi-square tests of independence were performed to test whether there is an association between the ranking of calling periods and the socio-demographic profile of the respondents. Seven chi-square tests were run with the demographic variables. Only region of residence (chi-square = 81.09; p 0.001) and age (chi-square = 41.64; p 0.001) were found to be associated with the calling period typology: this means that the region of residence and age of the mobile phone users interviewed changes across the various calling periods of the interviews. Table 8 presents the percentage distribution of region of residence and age of the mobile phone users interviewed in each ranked-calling period. The main findings can be described as follows. The respondents living in the M.A.L. are strongly represented in the best periods (40.1%), while the respondents living in the M.A.P. are strongly represented in the worst periods (46.1%). In the remaining regions the majority of the respondents were interviewed in the medium periods. The year and year age groups are strongly represented in the worst periods (41.9% and 41.3%, respectively), while 565

12 The best times to call in a mobile phone survey Table 8 Respondents age and region of residence by ranked calling periods (%) Characteristics Best Medium Worst Region of residence Metropolitan Area of Lisbon (M.A.L.) Metropolitan Area of Porto (M.A.P.) North Centre South Archipelagos of Madeira and Azores Age Lines sum up to 100% respondents aged 65 or older are the less represented in the worst periods (17.8%) and among all age groups is the one with higher percentage of respondents in the best periods (38.1%). Concluding remarks and implications for survey design This paper has explored the association between the day and time of calls and calling outcomes in the context of a mobile phone survey, and classified calling periods according to their efficiency in terms of contact rate and interview rate. The main findings can be summarised as follows. Establishing contact with mobile phone users was a greater challenge than obtaining a completed interview. In the total 19 calling periods analysed, the contact rate ranged between 13.0% and 33.1%, while the interview rate ranged between 33.9% and 60.1%. These results are to some extent in line with Yuan et al. (2005), who found a contact rate of up to 33.4% and an interview rate up to 72%. The interview rate in fixed phone surveys also tends to be higher than the contact rate (e.g. Brick et al. 1996; Dennis et al 1999; Yuan et al. 2005). The probability of establishing contact was influenced by the day of the call, but not by the time. The likelihood of making contact was found to 566

13 International Journal of Market Research Vol. 57 Issue 4 be higher on Saturdays, Fridays and Mondays, and lower on Tuesdays, Wednesdays and Sundays. The day*time interaction had a significant effect on the probability of obtaining a complete interview. The day or time alone did not significantly affect the likelihood of obtaining an interview. The most favourable days/ times to obtain an interview were Thursdays and Fridays from 5 pm to 9 pm, Saturdays from 10 am to 2 pm, and Mondays from 7 pm to 9 pm. This is somewhat different from fixed phone surveys in which weekday evenings are a good time to obtain interviews, but not the afternoons (e.g. Brick et al. 1996; Ahmed & Kalsbeek 1998). Calling periods were classified according to a three-level typology: Best, Medium and Worst. The periods that yielded the highest contact rate and highest interview rate were considered the best: Mondays 7 pm 9 pm, Thursdays 5 pm 9 pm and Fridays 5 pm 7 pm. Tuesdays and Wednesdays from 5 pm until 10 pm were found to be bad calling periods. The mean contact rate in the worst periods was approximately 13% (on average) lower than that of the best periods; although the interview rate rose to over 40% in the worst periods, they were considered worst because the interviewer was able to establish contact with only 16% (on average) of the mobile numbers called. There is an association between the profile of the respondents and the calling periods in which they were interviewed. The younger (15 24 and 35 44) tend to be in the worst periods; high percentages of respondents living in the heavily urbanised areas of Lisbon and Porto are found in the best periods. These results have clear implications for survey practice. They inform the design of efficient call scheduling so as to increase response rates at first call attempt and potentially reduce non-response bias. By identifying the most favourable calling periods for establishing contact and obtaining an interview, as well as the most likely demographic profile of mobile phone user in each calling period, the findings allow survey agencies to allocate staff, workloads and time resources more efficiently. The modest contact rates observed (ranging from 13% to 33%) suggest that people are not always available on the mobile phone. Although users always have their mobile phones with them, this does not mean they are contactable at any time. The majority of non-contacts were voic cases. A call is directed to voic after 7 ring tones or when the mobile phone is turned off; the near 40% of cases that were coded as voic reveal that turning off the mobile phone or not noticing a call is quite 567

14 The best times to call in a mobile phone survey common among mobile phone users. A voic rate of similar magnitude (37.6%) was also found in the research by Roy and Vanheuverzwyn (2002). Nothing can be done from the survey research perspective to overcome voic cases. However, it is reasonable to assume that the likelihood of getting voic cases might vary across days and/or times of the day, probably influenced by work or leisure activities of the mobile phone user. If an association can be made between times of the call and the voic outcome, the schedule of calls can accomodate that information which will lead to greater efficiency. Further research into mobile phone usage patterns would allow a better understanding of when and how mobile phone users use their mobile phone so that a link can be made between usage patterns and users contactability. The study highlights the importance of scheduling calls in order to maximise the likelihood of finding people on their mobile phone since, once the contact is achieved, it is likely to get people s cooperation. This is confirmed by the fact that the interview rates were higher than the contact rates in all three of the ranking calling periods. However, interview rates in the medium and worst periods were low approximately 40% on average. These results should be taken into consideration when allocating interviewers to calling periods by choosing the most experienced interviewers who can easily make people change their minds about refusing to cooperate for the most difficult periods. The outcome for the 9 pm 10 pm period was poor. It was classified as worst on Tuesdays, Wednesdays and Thursdays, and as medium on Mondays and Fridays. This is probably linked to mobile phone usage patterns. Mobile phone users probably consider they are entitled to some privacy and relaxation after a certain time, or may turn off the mobile phone because they do not expect to be called after certain times. However, this was a period favourable to interview younger people. Survey agencies should give careful thought as to whether it is worth scheduling first calls in these periods by weighing up costs and time allocation and sample composition. Most survey operations have household surveys as the large part of their work mix. Operating at full capacity is therefore restricted to times when people are most likely to be at home (i.e. evenings and weekends). Call centres usually only work 30 hours a week at full capacity (three hours each week night and all weekend) (Kelly et al. 2008). When dealing with mobile phones surveys, operations should also be concentrated on the days and times of the day when users are most likely to be available on the mobile phone. It is particularly important to maximise the efficiency of call times and days in the case of projects with short fieldwork periods. 568

15 International Journal of Market Research Vol. 57 Issue 4 It is important to note that mobile phone numbers in this study were not randomly assigned to all the times and days available for calling; moreover, the effect of time was conditioned by the fact that the calls were made across a four-hour slot at weekends and across a four-hour slot on weekdays. The results might be different had a larger time period been considered. Additionally, the calls were scheduled from 10 am to 10 pm, in line with common practice in Portuguese market research companies; this time schedule is different from that adopted by North America countries or northern Europe countries 9 am to 9 pm or 9.30 pm (e.g. Dennis et al. 1999; Roy & Vanheuverzwyn 2006; Yuan et al. 2005). Nevertheless, the findings remain valuable to researchers as they reveal that the link between the time/day of calls and the outcome of mobile phone calls cannot be ignored. The results highlight the importance of understanding the effect of survey design factors on calling outcomes in order to improve researchers strategy for mobile phone studies. The classification of calling times obtained from contact rates and interview rates can serve as input information for dealing with the challenge of defining effective non-contacts scheduling strategies and may ultimately be used for adjustments at the data analysis stage. Further research is recommended to explore these options. Acknowledgements This work has the financial support of Fundação para a Ciência e Tecnologia through the PTDC/EGE-GES/116934/2010 project. References Ahmed, W. & Kalsbeek, W. (1998) An analysis of telephone call history data from the behavioral risk factor surveillance system. Proceedings of the Survey Research Section, American Statistical Association, pp Arabie, P., Hubert, L. & De Soete, G. (1996) Clustering and Classification. Singapore: World Scientific. Brick, J., Allen, B., Cunningham, P. & Maklan, D. (1996) Outcomes of a calling protocol in a telephone survey. Proceedings of the Survey Research Section, American Statistical Association, pp Couper, M. (2011) The future of modes of data collection. Public Opinion Quarterly, 75, pp Dennis, J., Saulsberry, C., Battaglia, M., Rodén, A., Hoaglin, D., Frankel, M., Mathiowetz, N., Smith, P. & Wright, R. (1999) Analysis of calling patterns in a large random-digit dialing surveys: the national immunization survey. Available online at: nis/data_collection/dennis1999b.pdf (accessed 25 September 2012). ESOMAR (2011) ESOMAR guideline for conducting mobile market research and other resources. Available online at: (accessed 25 September 2012). 569

16 The best times to call in a mobile phone survey Gibbons, J. (1993) Nonparametric Statistics. California: Sage University Press. Groves, R., Fowler, F. Jr, Couper, M., Lepkowski, J., Singer, E. & Tourangeau, R. (2004) Survey Methodology. New York: Wiley. Hansen, S. (2008) CATI sample management systems, in Lepkowski, J., Tucker, C., Brick, J., de Leeuw, E., Japec, L., Lavrakas, P., Link, M. & Sangster, R. (eds) Advances in Telephone Survey Methodology. New Jersey: Wiley, pp Kelly, J., Link, M., Petty, J., Hobson, K. & Cagney, P. (2008) Establishing a new survey research call center, in Lepkowski, J., Tucker, C., Brick, J., de Leeuw, E., Japec, L., Lavrakas, P., Link, M. & Sangster, R. (eds) Advances in Telephone Survey Methodology. New Jersey: Wiley, pp Kulka, R. & Weeks, M. (1988) Toward the development of optimal calling protocols for telephone surveys: a conditional probabilities approach. Journal of Official Statistics, 4, 4, pp Lavrakas, P. (1993) Telephone Survey Methods: Sampling, Selection and Supervision, 2nd edn. Newbury Park, CA: Sage Publications. Roy, G. & Vanheuverzwyn, A. (2002) Mobile phone in sample surveys. Paper presented at the International Conference on Intelligent Computing, Copenhagen, Denmark. Stokes, S. & Greenberg, S. (1990) A priority system to improve callback success in telephone surveys. Proceedings of the Section on Survey Research Methods of the American Statistical Association. Vicente, P. & Reis, E. (2009) Telephone surveys using mobile phones: an analysis of response rates, survey procedures and respondents characteristics. Australasian Journal of Market and Social Research, 17, pp Weeks, M., Kulka, R. & Pierson, S. (1987) Optimal call scheduling for a telephone survey. Public Opinion Quarterly, 51, pp Weeks, M., Jones, B., Folsom, R. & Benrud, C. (1980) Optimal times to contact sample households. Public Opinion Quarterly, 44, pp Yuan, A., Allen, B., Brick, M., Dipko, S., Presser, S., Tucker, C., Han, D., Burns, L. & Galesic, M. (2005) Surveying households on cell phones results and lessons. Paper presented at the Annual Conference of the American Association of Public Opinion Research, Miami Beach, Florida. About the author Paula Vicente is Assistant Professor of Statistics and Data Analysis at Instituto Universitário de Lisboa (ISCTE-IUL), Department of Quantitative Methods for Management and Economics, Portugal. She is also a researcher at Business Research Unit Research Methods Group (BRU-IUL). Her current research interests are focused on mobile phone surveys, and use and analysis of paradata. Address correspondence to: Paula Vicente, ISCTE-IUL, Av. Forças Armadas, Lisboa, Portugal. paula.vicente@iscte.pt 570