The Modern Web Analytic

Transcription

1 The Modern Web Analytic Snapshot of an emerging discipline Prepared for: Prepared by: The Web Analytics Association WAA Research Committee The Dihedral Group, LLC Glenn Andersen

2 S1.1 Methodology Matters The Dihedral Group, LLC (TDG) was selected to conduct community research for the Web Analytics Association. This research is designed to better understand and personalize the current diversity of Web Analytic professionals worldwide. This segmentation research primarily focuses on Web Analytic specific skills/competencies, work responsibilities, compensation/benefits, demographics, organizational awareness, and attitudinal perspectives via work-life satisfaction. Insights were gained through quantitative statistical analysis, gathered from responses to an online survey instrument fielded in Q1 of The analysis was further augmented with a qualitative discovery phase which included in-depth interviews and secondary research. A significant amount of perspective into the mind of the modern Web Analytic also was achieved from the study of white papers, blog posts and vendor positioning propositions via subject/topic selection, writing styles, and the reverse engineering of marketing strategies. This study will be comprised of this global section (S1) followed by these three sections: S2 Web Analytic Profiles: Two Models, eight clusters, one profession. S3 Web Analytics Salaries: Compensation Profiles & Data Slices S4 Web Analytic Mobility (to be released) <global> research abstract

3 focused S1.2 discovery instrument design The survey instrument was designed by the WAA Research committee with a clear focus on understanding the work tasks, functions, and professional mobility of this community. Upon feedback gained from an initial online beta test, accommodation for best practices in online research and from previous experiences gained across other professional communities, TDG helped revise the design of the instrument towards some universal aspects of the modern professional mind-set, such as satisfaction, professional backgrounds, and levels of professional sophistication (beyond years of experience). sample, modes & reach Given the overall youth of this emerging profession, delivering a truly random sample of the population with 100% representation was not possible. That said, a non-probability sample was employed, based on a random sample of the WAA list of members, interested non-member Web analytic professionals, and oncemembers. A purposive sample also was drawn from blogs, discussion groups, and other relevant points of contact. Links to the survey were directly posted or shared, and participants could utilize TDG s viral marketing engine (tell-a-friend technology) to encourage other Web Analytic professionals to participate. This is know as snowball sampling. Identity capture (regardless of mode or reach) was used only for the sake of facilitating single-person participation. confidentiality Identity-related information was systematically severed from the respondents participation records. All research was conducted in strict accordance with the ICC/ESOMAR guidelines for ethical research. data validation Upon close of the survey, TDG ensured the integrity of the data using a series of proprietary tests designed for these types of projects. These tests range from simple automated rules that flag inconsistent responses for further analysis (e.g. professional experience > age less 18) to more complex response-bias issues (e.g. speed of completion < 2 std. deviations from central tendency AND 75% or more of question matrix responses = same value). Compensation-validation measures range from cross-referencing the findings to TDG s extensive database of 25,000 similar professional compensation matches to firsthand conversations with similarly aligned organizations. In a couple of cases, HR recruiters were consulted to discuss the plausibility of particular compensation datapoints. Less than 25 cases were eliminated from the analysis due to participation fraud. statistical analysis Upon completion of the data-validation phase, statistical models were built, clustering algorithms (k-means) employed, multivariate regressions run, and profiling conducted. The following pages further highlight some of these models for additional consideration when consuming the findings.

4 focused S1.3 discovery qualitative response coding data removal Many other responses were provided to the survey question sets. TDG coded these responses and, wherever critical mass warranted, included these other responses in the aggregated statistical analysis. Participants classified as students, part-time temporary, and those with less than 25% of their time dedicated to Web Analytics (WA) were automatically filtered out of the analysis. Examples of the variable coding process include professional backgrounds, position titles/levels, and organizational benefits. This process uses linguistic patternrecognition technology to initially categorize the responses, followed by an extensive human analysis of these findings to check quality control. The linguistic libraries have been trained on many other data sets and professional communities, and therefore, the systematic coding achieved an accuracy rate of 95%. geographic market Participants were asked to identify the country from which they primarily work. U.S. and Canadian respondents were asked to provide their state/province as well. All participants were asked to provide their nearest city and the zip/postal code of their place of work. This data was used to assess comparable compensation data points based on the company size, years of employee experience, and cost of living within the market. TDG mapped the real survey compensation data to the predicted compensation data and flagged cases outside of a single standard deviation. These cases were reviewed by humans and qualified using the aforementioned validation process. Unlike other methodologies that crop compensation responses automatically above the 90% threshold, TDG included these responses whenever they were deemed valid. Using the median (not mean) as the measure of central tendency ensures that these data points do not skew the aggregated findings. definitions Throughout the analysis, particular terms are used pertaining to reading compensation data, understanding accuracy, and interpreting the findings. The following key definitions are provided for those unfamiliar with the terms. Quartiles (25th percentile, median, 75th percentile) refer to the cut-points whereby the data is divided into four groups. This technique is used for banding numeric scale variables, which is particularly useful for understanding salary ranges and compensation distributions. In compensation studies, the median represents the person-in-the-middle where half of the other participants are making more and half are making less. Total Compensation is a measure built from summing a participant s base salary, yearly bonus values, stock options exercised within the year, and side work moonlighting as a WA consultant.

5 focused S1.4 discovery banding Scale/interval variables were cleaned using the automated outlier removal process. These numeric variables were then banded/binned together using either of the following two approaches. 1. Equal distribution using two, three, or four cut-points creating fairly even segments of 33%, 25%, or 20% respectively. 2. Developing the central tendency and banding based on one or two standard deviations away from this peak. In some cases (such as the number of posted WA positions) fractional responses were rounded to the nearest whole number. Though a discussion about fractional hires vs. full-time employees is an interesting discussion within an emerging profession, it is outside the scope of this research and pertained to less than 5% of all respondents on average. age # of Web Sites managed +/- 1 std. dev. 1. < cut-points of 25% 1. <= 2 sites sites sites or more sites years with company 3 cut-points of 25% 1. 1 year or less years years 4. 6 or more years years in current role 3 cut-points of 25% 1. 1 year or less 2. 2 years 3. 3 years 4. 4 or more years # of full-time WA professionals on team 2 cut-points of 33% 1. <=1 person people 3. 5 or more on team overall years of experience 3 cut-points of 25% 1. <=8 years years years or more years years as a professional delivering online experiences 3 cut-points of 25% 1. <=4 years years years or more years web analytic specific experience 3 cut-points of 25% 1. <= 2 years 2. 3 years years 4. 6 or more years

6 focused S1.5 discovery industries The list of industries was developed using the U.S. government s Bureau of Labor Statistics NAICS typology. Some sectors were not particularly heavily represented with WA professionals (e.g. natural resources, agriculture, mining). As expected, other sectors, such as Information Services, were heavily represented. TDG inserted sub-families where particularly relevant (i.e. ad agencies, market research). Due to the significant volume of responses in these areas, the Web Analytic Salaries (S3) section reports on these sub-families equally alongside the larger category of Professional, Scientific, and Technical Services. Underscoring the rapid ascension of the field, the Bureau of Labor and Statistics currently does not classify Internet publishing (only non-internet publishing) in any way except as an other information related industry. Thirteen responses were fielded as other information before TDG decided to include a separate category for Internet Broadcasting / Publishing specifically. This category ended up with 7.8% of the overall respondents which may have actually been as high as 10% if these 13 other responses were relevant to this sub-family. This was the only adjustment made to the instrument while live and, as a result, is mentioned here in full disclosure. industry representation The graph below represents the overall distribution of participants by industry. Note that advertising agencies accounted for 11.2% of the responses for Professional, Scientific and Technical Services. Information Professional, Scientific, and Technical Services Retail Trade Finance and Insurance Manufacturing All Others Arts, Entertainment, and Recreation Health Care and Social Assistance Transportation and Wharehousing Educational Services Accomodation and Food Services 0% 6% 12% 18% 24% 30%

7 focused S1.6 discovery single person participation To ensure the survey could not be stacked or overloaded with skewed data, the following protocol was put in place. 1. Invited participants had a unique access key that expired upon completion of the survey. 2. Participants outside of this list could access the survey upon entry of their . While no method is ever foolproof, this approach in conjunction with a rigorous data cleaning process serves as an effective deterrent. s and IP addresses were reconciled and where duplicate s existed, only one was counted. Cases were flagged for further analysis where IP addresses matched AND the participation was greater than 80%. compensation data Compensation data points of professionals that spend less than 50% of their average work week on Web Analytics were originally going to be screened from further analysis. The differences between this group and a larger sample of those spending more than 25% of their time on Web Analytics was found to be statistically insignificant across many measures. For this reason, the larger population was used. variable transformations In addition to the previously discussed banding, this project utilized predictive modeling to train certain variables with missing data. One particular case is the percentage of a participant s time spent conducting Web Analytics. Responses were missing from 137 of the 517 valid cases. TDG scored the data back to the database using an extremely high accuracy threshold. Subsequent controlled tests revealed very little difference between these groups. Note the differences between groups on the graph to the right. The following three pages are included to further highlight some additional weighted and factorial models used in this study. <$60k $60k-$75k $75k-$95k $95k-$120k $120k+ 0% 50% 100% Predicted variable Percentage of time dedicated to Web Analytics relative to 2007 Total Comp <$60k $60k-$75k $75k-$95k $95k-$120k $120k % 50-75% 75-90% 90%+ Original variable Percentage of time dedicated to Web Analytics relative to 2007 Total Comp 0% 50% 100%

8 organizational size model S1.7 The organizational size model uses the following parameters to define the organizational size levels: (Responses falling between conditions- e.g. 900 employees with organizational revenues at $17 million- were flagged for further consideration and manual adjustment based on all available insight.) * Very small / consultancy = fewer than 10 employees AND less than $1 million in organizational revenue OR budget * Small = employees AND $1 million to $10 million in organizational revenue OR budget * Medium = employees AND $10 million to $500 million in organizational revenue OR budget Very Large Large Organizational Size Model: * Large = employees AND $100 million to $1 billion in organizational revenue OR budget Medium Small * Very Large = employees AND $1 billion or more in organizational revenue OR budget Very Small Number of Cases 1 = 100

9 task level & diversity models S1.8 40% 34% Task Level Model 30% 20% 21% 23% 14% 10% 0% 6% 2% Level 1 Level 2 Level 3 Level 4 Level 5 Level 6 The task level model analyzes both the actual number of different tasks a professional is involved in as well as the role this participant plays in executing these tasks within their organization. The continuum ranges from Level 1, where a professional is immersed in all 16 of the 16 specified Web Analytic tasks in a highly strategic capacity, to Level 6, where professionals are involved in fewer than 9 of the 16 tasks-- and typically only in a production capacity. Task Diversity Model 16/16 Tasks 15/16 Tasks 15% 18% 13-14/16 Tasks 10-12/16 Tasks Less than 9/16 Tasks 20% 25% 23% 0% 10% 20% 30% 40% The task diversity model focuses on a broader slice of analysis than the task level model does. Here, only the actual task diversity is measured regardless of the role in which it is performed.

10 professional sophistication models S1.9 =(involvement + longevity + external recognition + intellectual engagement) Where: involvement = Average of the following weighted variables (real & predicted levels of Web Analytic expertise) longevity = Average of the following weighted variables (professional experience + Web Analytics experience + online experience) external recognition = Average of the following weighted variables (actual compensation reward + coded position title) intellectual engagement = Average of the following weighted variables (level of education + professional certification + alternative training and knowledge acquisition) Sophistication, as defined, is intended to make [something] more complex or inclusive. In this frame, this particular model is designed to go beyond age, education, and experience among other traditional modeling drivers to develop a more realistic segmentation of the modern Web Analytic professional. The resulting analysis now accounts for the intellectual engagement a participant has applied toward his/her Web Analytic skill-sets, the daily levels of Web Analytic involvement, the strategic leadership in the space of Web Analytics as well as the external recognition (in both compensation and title) that others attribute to the participating respondent. Every professional has a unique story-- this model bands these stories into four levels of sophistication. They are: Emerging (n=100; 23.5%), Evolved (n=96; 22.6%), Sophisticated (n=113; 26.6%), and Highly Sophisticated (n=116; 27.3%).

11 participation statistics S : The Dihedral Group, LLC selected to conduct an investigation designed to profile the Web Analytic community and deliver compensation data relative to these profiles : Survey in the field at 4 PM EST. Adjusted sample of Weekly metrics: Open rate- 16.1% Interested rate- 33.1% Conversion rate- 63.5% Overall response rate- 3.9% Opt-outs : Reminder distributed at 11 PM EST. Weekly metrics: Open rate- 18.1% Interested rate- 28% Conversion rate- 71.4% Overall response rate- 3.9% Opt-outs : Reminder distributed at 12 PM EST : Reminder distributed 12 PM EST. Weekly metrics: Open rate- 11.9% Interested rate- 31.4% Conversion rate- 57.9% Overall response rate- 4.2% Opt-outs : Participation closed. 517 completed surveys. 425 valid cases used in segmentation modeling : Sweepstakes drawn and awarded online. Clickthrough Conversion Open Undeliverable Initial (I) I+6 days I+12 days >> I+15 days

12 project frequencies S1.11 Percent of Time Devoted To Web Analytics: Interest in Web Analytics: 27% 22% 2% 18% 22% 30% 57% 6% 17% 25% to 49% 50% to 74% 75% to 90% 90% to 100% Gender: Agency Consultant or Industry Analyst Web Analytics Supplier Web Analytics User Other WAA Membership Status: 34% 19% 8% 66% 73% Male Female Age Groups Banded: Yes No Joining in 2008 Organizational Web Analytics Expertise: 24% 24% 19% 15% 25% 28% 36% 30% Younger than or Older Basic Intermediate Mature / Experienced Leading Edge

13 regional S1.12 reach CANADA- 10.1% Alberta- 11.5%, British Columbia- 25%, Nova Scotia- 1.9%, Ontario- 38.5%, Quebec- 23.1% Slice Total- 100% EUROPE- 16.3% Belgium-5% Denmark-6% Finland-2% France-10% Germany-10% Hungary-1% Italy-4% Luxembourg-1% Netherlands-10% Norway-2% Poland-1% Portugal-1% Spain-4% Sweden-10% Switzerland-2% Turkey-1% United Kingdom & Scotland- 31% OUTSIDE N. AMERICA & EUROPE- 6.3% Argentina- 18% Australia- 45% Brazil- 36% Chile- 9% China- 10% India- 20% Israel- 9% Japan- 10% New Zealand- 15% Singapore- 9% South Africa- 18% *Sub-regions sum to 100% relative to their Macro-region. UNITED STATES- 67.4% 37/50 states + District of Columbia Northeast & Mid Atlantic- 21.1% (ME, NH, VT, CT, MA, RI, NY, NJ, PA) South Atlantic- 20.7% (FL, GA, SC, NC, VA, DE, MD, WV) Central- 27.1% (MN, SD, ND, WI, MI, IA, IL, IN, OH, MO, NE, KS, AK, TX, MS, AL, KY, TN) West- 31.1% (CA, AK, HI, OR, WA, ID, NV, MT, CO, WY, NM, AZ) 17% 6% 12% 20% 14% 14% 18% US: Northeaset & Mid Atlantic US: Central US: Southern Atlantic US: West Europe Outside N. America and Europe Canada </end global>