Website Usability & Effectiveness October 2014
Background As Visit California shifted its paid media creative and execution for 2014, the organization began to transition its owned marketing sources, including the online presence at www.visitcalifornia.com. With the launch of Dream 365 in traditional and experiential marketing, the state s website had to keep pace with what was being promoted through these various sources. Just as Visit California ties influence and economic impact to its paid marketing through advertising effectiveness research, Strategic Marketing & Research Insights LLC has developed a methodology to determined the influence of owned media sources, most notably destination marketing organization s web presence. This research, conducted over the course of the year, is able to evaluate satisfaction with the site, overlap with paid media sources and influence on travel and spending. The specific goals of this research include:» Understanding users of the site in terms of where they are in the travel decision-making process and their awareness of paid media;» Evaluating site activity and satisfaction with their experience on www.visitcalifornia.com;» Measuring the impact of the website on generating travel; and» Calculating the total influence of www.visitcalifornia.com on visitor spending in the state. 2
Methodology While SMARInsights has conducted similar evaluations for www.visitcalifornia.com, previous processes have relied on an intercept survey on the site. This self-selection of participation by users can result in biases in the data. These respondents are willing to share an e-mail address by completing this intercept so it is possible they could be more likely to visit or are different demographically from other users. Advances in technology have allowed for understanding who users of the site are without requiring an intercept survey. A digital tag was placed on the site that allowed sample vendors to know when someone on their panel had visited the site. Follow-up surveys can then be sent to these panelists, eliminating the potential self-selection bias. For this evaluation, both capture and tagging were used to gather information about users. The intercept was available on the site for a full year while the tag was placed in February 2014. Given travel began to be evaluated in June 2014, users to the site between June 2013 and May 2014 are considered for this Capture Tagged evaluation. During that time, there were more than 3 million 5,694 12,986 unique users to the site, but by applying the bounce rate or Intercept Tagged those that only hit the home page and then navigated away 98 197 there were 1.1 million users who had experiences on the site that Usability and could have resulted in influenced travel or spending. Awareness Previous evaluations have used the number of unique users to the site combined with those captured via intercept. Therefore, 105 492 comparisons to previous years are based on these users. Travel Travel However, moving forward, the non-bounce users who were tagged as having been to the site will provide a more accurate picture of the site s influence. Usability and Awareness 3
USABILITY & SATISFACTION 4
Satisfaction Though navigation and functionality remained the same, the site was refreshed to reflect the Dream 365 campaign. While this move improved users overall reaction to the site, it was considered less helpful. Destination marketing organizations, and especially states, have long struggled with what they are supposed to be. Many are positioned as task-oriented planning sites, offering thousands of listings of lodging properties and attractions. There is no shortage of ways to plan a trip with brand and attraction sites, online travel agents and the emergence of shared economy resources. SMARInsights has found consumers want state DMO sites to be inspirational resources, providing stunning and iconic visuals that can motivate them to consider a trip and provide ideas on places to go and things to do. Given this, it s ok to have a state site that is considered less helpful. There are innumerable resources available to consumers that might be considered more helpful to trip planning but less motivating. 42% 27% Overall Website Reaction 4% 2% 6% 4% 39% 37% 51% 56% 2012 2014 Excellent Good Fair Poor Helpfulness of Site 1% 2% 3% 5% 3% 1% 12% 12% 24% 34% 48% 46% 38% 2009 2012 2014 Extremely helpful (5) 4 3 2 Not helpful at all (1) 5
Purpose One of the concerns of the intercept process is that those willing to complete the survey are different from other users. Those who are willing to share personal information via the capture survey are more likely to already have a trip to California planned. SMARInsights has found in general consumers are not likely to provide this information on DMO sites because they are afraid they will end up on a newsletter mailing list. Those doing so via the capture survey are willing to provide that information as they already have a trip being planned and likely believe that even if they were to end up on a mailing list, it could be relevant to their trip. Given the differences between these groups, site evaluations and impacts will be broken out separately throughout this report. Of those who are already planning a trip, there remains room for the site to influence where they go and what they do. Planning Process 2012 2014 Captured 2014 Tagged 65% 64% 51% 35% 36% 49% Already Planning Considering Pre-determined Trip Details 51% 50% 47% 43% 34% 37% 30% 26% 27% 26% 19% 14% Duration of trip When to travel Where to stay Primary activities 2012 2014 Captured 2014 Tagged 6
Usage As Visit California moves to launch a new site, the focus should be on why users are accessing the site. Most often, this is for ideas about attractions and activities. Though many DMO sites have focused on building a monstrous database of listings, this can be in the form images and lists rather than searchable trip planners. And while a fair amount of users will do other things on the site, there are countless other resources available for some of these, especially accommodations and dining. Visit California should focus on inspiring consumers to consider a trip and rely on other resources to fill in the necessary details. Local attractions Activities Highway maps State parks Beaches Calendar of events Accommodations Outdoor recreation Request information Dining Historical sites Museums Fairs & festivals Reason for Accessing Site 22% 32% 30% 36% 27% 23% 29% 27% 32% 30% 63% 56% 53% 48% 55% 50% 51% 48% 51% 59% 51% 47% 56% 50% 49% 47% 42% 45% 46% 45% 44% 37% 38% 36% 40% 57% 86% 79% 74% 2012 2014 Captured 2014 Tagged 7
Communication Attributes With consumers more positive about the site overall, it is useful to understand what the site is communicating differently. Again, there were only minor changes from the last evaluation, primarily to reflect the Dream 365 effort. But these changes focused on those things consumers most want to see in a state DMO site it made significant improvements in communicating the state was scenic and beautiful. Website Communicates California... 2012 2014 Captured 2014 Tagged Change from 2012 to 2014 Capture Is scenic and beautiful 61% 68% 60% 7% Is a place with lots to see and do 36% 37% 46% 1% Has excellent beaches 26% 22% 27% -4% Is culturally diverse 23% 19% 15% -4% Has a lot of family activities and theme parks 22% 18% 24% -4% Has a variety of outdoor activities - hiking/climbing/biking 17% 14% 12% -3% Has many quaint small towns 14% 14% 13% 0% Has friendly people 11% 13% 9% 2% A place where food and wine are part of the culture/lifestyle 8% 12% 9% 4% Is unlike anyplace else 11% 12% 6% 1% Is fun 13% 11% 8% -2% Offers vacationers the best life has to offer all in one place 13% 11% 7% -2% Has interesting historical sites 4% 10% 10% 6% Is a place where people have a unique approach to life 12% 8% 7% -4% Has large cities with exciting urban experiences 10% 6% 16% -4% Provides a good value for the money 9% 5% 8% -4% Is a good place for snowsporting 1% 1% 1% 0% 8
Evaluation Though only three of the individual attributes the site is evaluated on saw improvements, consumers had a more positive reaction than in previous years, an indication that these are the three that matter to the overall experience. With the inclusion of the Dream 365 materials, much of which is video, the site saw noticeable gains for presenting a good impression of the state and having a good balance of content. Top-Performing Attributes It presents a good impression of California It has a good balance of video, pictures, and text It is easy to use It helps me feel confident a trip would be a success It clearly communicates California's main qualities Homepage helps me understand what the site has to offer It shows experiences and places that I am interested in It is organized in a sensible manner 88% 92% 83% 82% 87% 79% 86% 87% 81% 86% 85% 75% 85% 84% 78% 85% 82% 79% 84% 82% 79% 85% 81% 74% 2012 2014 Captured 2014 Tagged 9
Evaluation Though the site s mid-performing attributes saw some slight declines, these are not impacting overall satisfaction with the site. Through qualitative evaluations of DMO sites, SMARInsights has found that consumers do think links to other sites and attractions are important. So while future incarnations of the Visit California site will not be listing driven, it should offer ways for consumers to directly connect with partner attractions. Mid-Performing Attributes It has the right variety of information Describes specific attractions & events Helps me choose attractions/events/activities It has good links to other websites/attractions Helps me tailor a trip to my specific interests Maps on the site are useful Is detailed enough for my questions The website's suggestions and tips are helpful 83% 80% 80% 81% 80% 78% 87% 80% 79% 82% 78% 73% 82% 77% 75% 81% 76% 76% 74% 76% 74% 81% 72% 74% 2012 2014 Captured 2014 Tagged 10
Evaluation The bottom-performing attributes saw the biggest declines. Again, this is not necessarily cause for concern as they are less important than inspiring and motivating consumers. As the site has evolved, many of these attributes have become less relevant. Future evaluations should focus on those features of the site truly available at www.visitcalifornia.com. Bottom-Performing Attributes It is easy to forward website information to others Helps me tailor a trip according to my time Visitors' opinions on the website are helpful Helps me choose restaurants It helps me save money Helps me choose accommodations Helps me plan getting to and from California It is easy to get feedback from a website representative Helps me tailor a trip according to my budget It is worthwhile purchasing/reserving at this website 78% 68% 72% 74% 67% 72% 67% 65% 62% 65% 61% 58% 60% 60% 50% 69% 58% 58% 68% 58% 56% 62% 57% 58% 72% 55% 55% 63% 52% 53% 2012 2014 Captured 2014 Tagged 11
Exposure As seen in the Advertising Effectiveness process, with the launch of the Dream 365 campaign, awareness rose slightly from previous years. Those acquired through the intercept process had significantly more awareness of the digital components of the campaign. Not only are those whose visits to the site were tagged digitally considered to be less biased, but they also more closely reflect general travelers in their awareness of the Visit California marketing. Those willing to give their contact information via the capture process are highly engaged through both paid media and their travel intentions. Advertising Awareness 71% 69% 70% 60% 52% 49% 44% 36% 29% 24% 20% 83% 77% 70% 65% TV Digital Dream 365 Total 2012 Web Users 2014 Captured 2014 Tagged 2014 Spring Domestic 12
ECONOMIC IMPACT 13
Trip Spending A significant amount of the impact of the website comes from its ability to influence spending. As seen previously, many visitors to the site have already made the decision to visit California or have been exposed to other Visit California marketing. Therefore use of the site cannot take credit for all of their spending. In order to understand how the economic impact calculations are determined, it is necessary to understand from where the baseline spending comes. In the 2013 Advertising Effectiveness ROI, overall visitor spending was reported at $1,790. However, of those visitors from that survey who indicated they did not use the website, the spending was $1,497, which will be considered the base level of spending against which the web effectiveness is measured. All 2013 Visitors 2013 Visitors (Non-Web Users) Category Lodging/Accommodations $653 $558 Meals/Food/Groceries $435 $352 Attractions $190 $156 Shopping $206 $164 Entertainment $39 $41 Transportation within California $216 $195 Other $51 $33 Total $1,790 $1,497 14
Trip Spending Among those using the website, spending is higher among those captured via the intercept process. This is primarily due to who is being tagged and captured. Of the tagged audience, 39% are California residents; of those intercepted, only 17%. In their site statistics, Visit California sees 43% of all U.S. visits coming from residents. Those coming from farther away stay longer and spend more, resulting in the captured respondents higher trip spending. Category 2014 Capture 2014 Tagged Lodging/Accommodations $653 $572 Meals/Food/Groceries $408 $347 Attractions $379 $216 Shopping $330 $184 Entertainment $102 $51 Transportation within California $391 $208 Other $105 $44 Total $2,367 $1,622 18% of visitors In-State 53% of visitors In-State 15
Website Marketing Integration There are various ways usage of the website can influence consumers. It is important however, that those whose visitation were already accounted for in the Advertising Effectiveness process not be double counted. Their spending has already been attributed to awareness of the paid media. However, any additional spending above those who didn t use the site can be attributed to site usage. As well, those who were already planning a trip to the state before their usage of www.visitcalifornia.com have indicated their decision to visit was not influenced by the site. So only any additional spending beyond those who did not use the site is considered influenced. But the final group of those who were not aware and were not already planning a trip can have the entire trip and associated spending attributable to usage of the site. Therefore, the following examines the impact of the site on three groups:» Website visitors who were not already planning to visit California and were not aware of ads. All trips and spending from this group are considered influenced by usage of the website.» Website visitors who were not aware of the advertising but were already planning to visit. The influence of this group will be on the incremental spending attributable to the website.» Website visitors who were aware of the paid advertising. Again, the influence here is the additional spending the website is able to generate over other trips to California. 16
Potential Overstatement However, given some of the differences highlighted among the captured and digitally tagged audiences, SMARInsights believes the influence of the site is overstated when the captured audience is evaluated. It was a necessary process given this is how the evaluation has been conducted in the past. However, the tagged users likely tell a more accurate story on the site s influence. Therefore the following lays out each of the potential impact categories by audience:» Captured Users Overall Travel Influenced Trips by unaware/not already planning Influenced Spending by unaware/already planning Influenced Spending by aware» Tagged Users Overall Travel Influenced Trips by unaware/not already planning Influenced Spending by unaware/already planning Influenced Spending by aware The captured audience provides a comparison to past performance, but the tagged users will be considered the baseline for future evaluations. 17
Captured Users Overall Travel For comparison to previous years, the following calculations are based on only those captured through the intercept process. Future evaluations will be based on those digitally tagged given their similar geography to overall site users. With steady increases in usage and similar levels of travel, overall spending by those accessing the website has continued to rise. Captured 2010 Website Visitors 2011/12 Website Visitors 2013/14 Website Visitors Total U.S. unique users 1,874,671 2,321,179 3,180,004 Travel rate 44% 64% 63% Influenced travelers 828,343 1,478,016 1,997,436 Avg. trip expenditures $1,964 $2,034 $2,367 Revenue from web visitors $1.6 B $3.0B $4.7B 18
Influenced Trips Captured Those who were not aware of the ads and were not already planning a trip can have the entirety of their travel and subsequent spending attributable to the campaign. With more web visits and less awareness, more users have the ability to be influenced. Captured Unaware/Considering 2010 Website Visitors 2011/12 Website Visitors 2013/14 Website Visitors Total U.S. unique users 1,874,671 2,321,179 3,180,004 % Not aware of ads 10% 17% 23% Non ad aware website visitors 196,394 394,600 731,401 % not already planning a trip 18% 15% 37% Potential influence 35,708 58,795 272,929 Travel rate 33% 33% 24% Influenced travelers 11,784 19,598 66,463 Avg. trip expenditures $1,964 $2,034 $2,367 Spending attributed to the web $23.1 M $39.9 M $157.3 M 19
Influenced Spending Captured For the audience who were unaware but indicated they were already planning their trip, there was more follow-through than in the past, resulting in more trips for which any additional spending beyond that by other California visitors who did not use the site. Given the level of spending by intercepted users and those who visited in 2013 but did not use the site, usage of www.visitcalifornia.com generated an additional $870 on each of these types of trips. Captured Unaware/Already Planning 2010 Website Visitors 2011/12 Website Visitors 2013/14 Website Visitors Total U.S. unique users 1,874,671 2,321,179 3,180,004 % Not aware of ads 10% 17% 23% Non ad aware website visitors 196,394 394,600 731,401 % already planning a trip 82% 64% 63% Potential influence 160,686 250,966 458,472 Rate of travel 48% 33% 76% Trips for which spending level can be influenced 77,129 82,819 346,469 Spending by website visitors $1,964 $2,034 $2,367 Spending by non-website visitors $1,315 $1,409 $1,497 Spending per trip attributable to the website $649 $625 $870 Incremental spending attributable to the website $50.1 M $51.8 M $301.3 M $2,367 $1,497 $870 20
Influenced Spending Captured Though awareness was down, the number of unique users was up. So though the rate of travel among this group was similar, influenced spending is applied across more trips. For those who were aware of the Visit California marketing, usage of the website was still able to generate an additional $1.4 billion in visitor spending. Captured Aware 2010 Website Visitors 2011/12 Website Visitors 2013/14 Website Visitors Total U.S. unique users 1,874,671 2,321,179 3,180,004 Awareness 89% 83% 77% Aware households 1,678,277 1,926,578.57 2,448,603 Rate of travel by aware 37% 65% 66% Aware website users who visit 624,890 1,248,888 1,608,303 Spending by website users $1,806 $2,034 $2,367 Spending by CA visitors $1,315 $1,409 $1,497 Incremental spending $491 $625 $870 Influenced spending $307.1 M $781.0 M $1.4 B 21
Total Influence Captured Given each of these categories of influence, usage of www.visitcalifornia.com was more than double that of the previous evaluation, with the biggest gains among those who were aware of the advertising. Captured Total influence 2010 Website Visitors 2011/12 Website Visitors 2013/14 Website Visitors Influenced travel Unaware/considering $23.1 M 39.9 M $148.3 M Influenced spending -- Unaware/already planning $50.1 M 51.8 M $283.9 M Influenced spending -- Aware $307.1 M 781.0 M $1.4 B Total influence $380.3 M 872.7 M $1.9 B 22
Tagged Users Overall Travel In addition to the tagged audience being more representative of www.visitcalifornia.com users, this evaluation takes into account the bounce rate on the site, or those that hit only the home page and navigated away from the site. So rather than the nearly 3.2 million unique users, these calculations use the non-bounced unique users, or 1.2 million users. Tagged 2013/14 Website Visitors Total U.S. unique, non-bounce users 1,189,951 Travel rate 50% Influenced travelers 594,080 Avg. trip expenditures $1,622 Revenue from web visitors $963.6 M 23
Influenced Trips Tagged Because the tagged sample is so robust, there are enough interviews to break out the spending for each type of audience: unaware/considering; unaware/already planning; and aware. Again, for those who were unaware and not already planning a trip, all of the visitor spending is attributable to the web. Therefore, this small audience brought in more than $61 million in visitor spending. Tagged Unaware/Considering 2013/14 Website Visitors Total U.S. unique users 1,189,951 % Not aware of ads 35% Non ad aware website visitors 413,204 % not already planning a trip 34% Potential influence 141,925 Travel rate 41% Influenced travelers 58,536 Trip expenditures by unaware/considering $1,051 Spending attributed to the web $61,520,923 24
Influenced Spending Tagged For those who were unaware but already planning their trip, any additional spending beyond that by those who did not use the site is considered influenced. Here, there was only $2 more in spending by those who used the site but were already planning. Given this small increment, there is minimal influenced spending among this audience. Tagged Unaware/Already Planning 2013/14 Website Visitors Total U.S. unique users 1,189,951 % Not aware of ads 35% Non ad-aware website visitors 413,204 % already planning a trip 66% Potential influence 271,278 Rate of travel 66% Trips for which spending level can be influenced 178,101 Spending by unaware/already planning website visitors $1,499 Spending by non-website visitors $1,497 Spending per trip attributable to the website $2 Incremental spending attributable to the website $356,202 $1,499 $1,497 $2 25
Influenced Spending Tagged Finally, those who were aware had their visitor spending attributable to Visit California activities though the Advertising Effectiveness research measuring the influence of paid marketing on travel. However, there is still room for influence of the website. This group spent $251 more than those who did not use the web for additional visitor spending of $97 million. Tagged Aware 2013/14 Website Visitors Total U.S. unique users 1,189,951 Awareness 65% Aware households 776748 Rate of travel by aware 50% Aware website users who visit 386,046 Spending by aware website users $1,738 Spending by CA visitors $1,487 Incremental spending $251 Influenced spending $97,038,128 $1,738 $1,497 $251 26
Total Influence Tagged Given these three sources of influence, the site can be credited with nearly $159 million in visitor spending. And while this is far smaller than the influence calculated similarly to the past processes, this is likely a far more accurate measure. The tagged audience looks more similar to www.visitcalifornia.com s overall users, and by removing bounced visits the total gets at a much more reliable figure. While the captured totals can point directionally to the site having more influence than in the past, this tagged total should be considered the base against what future performance will be evaluated. Tagged Influence 2013/14 Website Visitors Influenced travel Unaware/Considering $61,520,923 Influenced spending Unaware/Already Planning $356,202 Influenced spending Aware $97,038,128 Total influence $158,915,253 27
Appendix Questionnaires 28
Website Intercept 29
Website Usability & Satisfaction 30
Website Usability & Satisfaction 31
Website Usability & Satisfaction 32
Website Awareness & Travel 33
Website Awareness & Travel 34
Website Awareness & Travel 35
Website Awareness & Travel 36
Website Awareness & Travel 37