Bing Pan, Ph.D. College of Charleston, USA

Transcription

1 Bing Pan, Ph.D. College of Charleston, USA

2 State_Province list s subscribers are state tourism offices and tourism researchers.

3 Sprenet list is for for recreation and tourism scholars in the U.S.

4 Online survey An increasingly popular research method Faster, easier, more flexible, cost saving (Dillman 2007; Tierney 2000) Problems: low response rates; questionable representativeness; many times biased results (Hwang and Fesenmaier 2004; Pan 2010)

5 Online survey: various forms and definitions Advantage of online surveys Cost saving (Schleyer and Forrest 2000, Sheehan and Hoy 1999) More interactive survey content and richer format (Dillman 2007; Schaefer and Dillman 1998) More responsiveness to open-ended questions (Schaefer and Dillman 1998) and sensitive inquires (Link and Mokdad 2005; Wang et al. 2005) Faster turn-around time (Dillman 2007; Hwang and Fesenmaier 2004) Online survey response rate Response rate: large variation (6%-75%), lower response rate, and decreasing (Best et al. 2001; Leece et al. 2004; Sheehan and McMillan 1999; Sheehan 2001) Means to increase response rate: contacts (pre-, post-, and personalized), incentives, survey sender, timing (Sheehan and McMillan 1999; Cook, Heath, and Thompson 2000; Deutskens et al. 2004, Pan 2010)

6 Suggestions from literature on response rate Pre-contact, reminder, personalized solicitation messages, official letterhead, sponsorship/ endorsement, incentives, etc. Response rate and result bias: mixed/intertwined Identifying brand sponsorship in advertising studies will decrease response rate but bloat conversion rate (Woodside and Dubelaar 2003)

7 Bias of online survey result Representativeness: the young and better educated (Best et al. 2001; Ross et al. 2003) Further coverage bias: self-selecting nature of online respondents (Hwang and Fesenmaier 2004; Pan 2010) Missing data and lower data consistency (Cole, 2005) Technological uncertainty (Pan 2010) Scarce research has tested the intertwined effects of response bias and result bias using the experimental research design

8 When to send out online surveys to obtain maximum results? What factors influence the online response rate? How to increase response rate in online surveys? How to limit the bias and increase the validity of results?

9 Pan, B. (2010). Online travel surveys and response patterns. Journal of Travel Research, 49(1):

10 Research Questions Does the response speed of online travel surveys follow consistent patterns? How fast are online surveys returned by most respondents? How long should a researcher wait to collect sufficient responses? How many surveys in total should a researcher expect given the numbers of responses from the first few days?

11

12

13

14

15

16

17

18 Conclusions The individuals tendency to respond to the surveys as soon as they receive them. There are peak periods in the daytime and early night when the respondents tended to return the majority of the surveys. The early morning hours in the respondents local time seem to be the best time for sending out solicitations, so that the s will be opened and checked with minimal delay in the respondents inboxes. The exponential growth model is the appropriate parsimonious model. One could also predict the final number of returned surveys with a relatively small error rate (less than 10%) by using the first 7 or 8 days of response data.

19 Pan, B., Woodside, A., & Meng, F. (in press). How contextual cues impact response and conversion rates of online surveys. To appear in Journal of Travel Research.

20 The study aims to test the effects of contextual cues on response rates and survey results of online market studies with an experimental design Specific contextual cues in study Researcher s identity Sponsorship of the study Incentive type

21 A between-subjects, fixed effect factorial study with three factors (Campbell, Stanley, and Gage 1969) Two different researchers Research A from a regional university in City C Researcher B from a national university Two different sponsors CVB of City C International academy of tourism and hospitality Two different incentives One unnamed hotel in City C One unspecified hotel from one of the 8 Southeast cities (as competitors to City C)

22 2x2x2 full factorial design with eight different cells Table 1. Three Factors and Eight Cells Factors First Condition Abbreviations Second Condition Abbreviations Researcher Researcher A in a City Researcher A Researcher B in a national Researcher B C University university Sponsor City C Convention and Visitors Bureau CVB Sponsor An International Academy of Tourism and Hospitality Academy Sponsor Incentive A Hotel in City C City C Incentive A Hotel in one of eight cities in Southeast United States SE Incentive Condition Cell 1 Cell 2 Cell 3 Cell 4 Cell 5 Cell 6 Cell 7 Cell 8 Details Researcher A, CVB Sponsor, City C Incentive Researcher A, CVB Sponsor, SE Incentive Researcher A, Academy Sponsor, City C Incentive Researcher A, Academy Sponsor, SE Incentive Researcher B, CVB Sponsor, City C Incentive Researcher B, CVB Sponsor, SE Incentive Researcher B, Academy Sponsor, City C Incentive Researcher B, Academy Sponsor, SE Incentive

23 Sampling frame: website visitors who opted to receive alerts from City C s CVB website 17,415 (one in five) active addresses were randomly selected Equally and randomly assigned to the 8 cells (2,177 s per cell) 8 different solicitation s were generated with three variation: Researcher, Sponsor, and Incentive Two waves of solicitations on Sept 7 and 9, 2010, and the third wave after two months (1,600 addresses in total, 200/cell) The continuum-of-resistance model (Filion 1976)

24 Dear [First Name], I am conducting a survey on visitor behavior and satisfaction to eight cities in the Southeast of the United States (Asheville, NC, Atlanta, GA, Charleston, SC, Hilton Head, SC, Myrtle Beach, SC, Savannah, GA, St. Augustine, FL, and Wilmington, NC). Please participate by completing the survey even if you did not visit any of the eight cities in If you live in one of the eight cities, please do participate in the study as well. The study is sponsored by the [Sponsor]. By responding to the survey, you are helping to increase knowledge about traveler behavior and travel-related services. The survey takes at most 7-8 minutes to finish. Complete the survey and you are invited to enter to win a three night hotel stay at one selected tourist destination in [Incentive] (you do not have to enter to win the hotel stay to complete the survey). This survey is voluntary and all responses are anonymous, confidential and with no address coded with answers. Please click on the following link to go to the survey directly. If the link does not work, please copy and paste the URL in your browser. Thank you for participating in the study! Please do write to me if you have a question about the study. Thank you very much for your participation. Sincerely, [Researcher]

25 1,257 completed surveys collected one month after the first two waves Overall response rate: 7.2% Cell response rate: 5.9% - 9.0% 71 returned surveys one month after the 3 rd wave Table 3. Response Rates in Eight Cells Incentive Averag Researcher City C SE e Researcher B Sponsor Academy 7.4% 7.3% 7.4% CVB 9.0% 7.9% 8.5% Average 8.2% 7.6% 7.9% Researcher A Sponsor Academy 5.9% 6.3% 6.1% CVB 7.3% 6.7% 7.0% Average 6.6% 6.5% 6.5%

26 Multiple Chi-square tests were used to examine the relationship between the three factors (Researcher, Sponsor, and Incentive) and response rate Researcher and Sponsor both have significant impact on the response rate, but not the Incentive type Table 4. Chi Square Tests of Response Rate Value Significance Incentive Sponsor Researcher

27 Logistic regression analysis is further used to test if one responds from the three factors No interaction effects between those three factors Researcher B from the national university can garner 19.2% more responses than Researcher A from the regional university City C s CVB as the sponsor organization generates 22.3% response increase comparing to the national academy sponsor Combining the two factors of Researcher and Sponsor, the response rate could increase up to 41.5%

28 City C Incentive SE Incentive City C Incentive SE Incentive Research A Researcher A Researcher B Researcher B

29 A logistic regression on the prediction of being one of past visitors to City C on the three factors does not yield any significant result at 0.1 level. Thus, the past visitation rate is not related with the three contextual cues Table 5. Testing Visitation Rates Using Logistic Regression Have Ever Visited City C Visited City C in Previous Six Months Visited City C in the Most Recent Trip Value Significance Value Significance Value Significance Incentive Sponsor Researcher

30 Chi-square tests or t-test/anova were used to test the demographic/behavior features under the influence of the three factors: The type of incentive had relationship with the respond-willingness of those who had the different length of stay When the incentive award is the hotel stay in City C, the respondents are likely to be those who stayed shorter time in their most recent trip Table 6. Incentive Type and the Length of Stay Incentive Mean* N City C Incentive SE Incentive Total * t=4.401, p=0.043

31 Mentioning City C in any of the three factors influences the ratio of married vs. nonmarried (single/divorced/widowed) respondents who are willing to answer the survey More non-married respondents fill out the survey as City C is mentioned more often in the solicitation s Table 7. The Mention of City C and Marital Status of Respondents Times City C Mentioned Total Non-Married Married Total * Chi-square =8.893, p =

32 The response rate could be low in online survey study (7.2% in this study) Researchers and sponsors may influence the response rates Destination mentioned in the message could affect the type of visitors responding to the survey and connect to behavioral characteristics of the respondents

33 Researcher identity Name of the researcher University affiliation Sponsorship Local vs. national organization Industry vs. academic organization Recommendations: Credible researcher with formal titles and affiliation Suitable sponsor / endorsement Refrain from revealing the target destination s information in solicitation message and as incentive

34