With the global leader in sampling and data services Putting Social Media Data in a Probability Frame Context

Transcription

1 With the global leader in sampling and data services Putting Social Media Data in a Probability Frame Context Lisa Drew, Two42Solutions Mohammad Hamid, Two42 Solutions Edward Paul Johnson, SSI 2014 Survey Sampling International 1

2 While not Everyone, Most Adults Use Social Media Adult Population on Social Media* 70% 60% 58% 50% 40% 30% 20% 19% 23% 22% 21% 10% 0% Facebook Twitter LinkedIn Pinterest Instagram *Pew Research Center Internet Project Survey Sampling International 2

3 Demographic Composition is Broadening 2014 Survey Sampling International 3

4 People Use Social Media in Everyday Life Frequency of Use Among Site Owners 100% 90% 80% 70% 13% 17% 40% 27% 54% 62% 60% 24% 50% 40% 30% 20% 10% 0% 24% 70% 29% 49% 25% 36% 17% 13% Facebook Twitter Instagram Pintrest LinkedIn Daily Weekly Less Often *Pew Research Center Internet Project Survey Sampling International 4

5 Lots of Data for Market Researchers to Use 2014 Survey Sampling International 5

6 Value add of Social Media for Market Researchers Increase Participation Less Obtrusive Than Active Surveying Increase Data Quality Can Eliminate Recall Error 2014 Survey Sampling International 6

7 Legal & Representivity Obstacles to Using Data Two Legal Challenges Privacy Concerns Data Ownership Representivity Challenges Those not on Social Media Differential Participation Rates 2014 Survey Sampling International 7

8 How We Address The Legal Concern Legal Concerns Privacy-policy-compliant Double Opt-In Methodology 2014 Survey Sampling International 8

9 How Does Double Opt-In Work? 1. Agree over the phone or web to participate 2. Send them an social media opt-in invite 3. Fully disclose what you are doing What you are collecting How long you are collecting How the data will be used Any privacy protection 4. Obtain legal right to use data Critical for researchers to understand how they can & cannot use data 2014 Survey Sampling International 9

10 How We Address The Representivity Concern First, we utilized traditional sample frame methodology from RDD Survey based on a voter file Then, We applied Our Nearest Neighbor Algorithm Allows researchers to maintain sample frame from RDD, while capturing social media data for nearly 50% of survey respondents. Works by plotting all possible neighbors (source: voter file) into a 2-dimensional Euclidean Vector, where y-axis is determined by psychographic characteristics and the x-axis is determined by demographic characteristics. The formula we use to calculate the 2014 Survey Sampling International 10

11 How does NN Work? Random Sample of Population* Accounts for Lack of social data 4 Final Result Social Media Population* Social Media Matched Population* Survey Sampling International -3 *Simulated Data, not Real Data

12 Have you tried this in real life? Yes Michigan Governors Race Collected 2,000 survey responses > RDD Dialing Dual Landline and Cell phone frame (using voter file) > Two Waves (September & October 2014) > 7 minute survey > Moderate weighting with design effect of 1.13 Asked them to participate in social media component > Retrieve relevant social media fields for opt-ins and append to survey file > For those with no social media account or didn t opt-in, we looked for nearest neighbors 2014 Survey Sampling International 12

13 100% 90% Pop Composition Comparison After NN, We get 14% N=138 80% N=760 38% 70% 60% 50% 80% N=800 40% 30% N= % After NN, We get 20% 10% 0% N=66 3% RDD Composition Opted-In Didn't Opt-In No Social Media Account After Opt-in, We get 5% N=49 Final Social Composition No Social Media Account Didn't Opt-In Opted-In To get N=2,000 RDD completes, we processed 118,000 voter file records. To get N=987 Social Completes AND maintain the sample frame, we processed 14,341 voter file records 2014 Survey Sampling International 13

14 Nearest Neighbor Types 2 nd Degree Neighbor 1 st Degree Neighbor 59% 36% 1 st Degree = Closest Neighbor, 2 nd Degree = Second Closest neighbor, based on demos & psychos. Results in social data for nearly 50% of survey population Nearest Neighbor Approach Resulted in almost 2,000% more social media data Direct Match 5% Final Social Composition N = 987 (direct + 1 st & 2 nd Degree) 2nd Degree 1st Degree Direct But how did the matched sample compare to the original 2,000 survey respondents? 2014 Survey Sampling International 14

15 Matched Sample Skews a Little Younger Demographics of Matched to Unmatched Social Data 80% 70% 60% 50% 40% 30% 20% 10% 0% Blue Collar Suburbs Northern Subs / Thumb University/Industrial Belt Central & West North and UP City of Detroit Male Female Caucasian African American Hispanic Other Region Gender Ethnicity Age Not Matched Matched 2014 Survey Sampling International 15

16 No Political Skew in the Matched Sample 70.0% Political Differences Between Matched and Unmatched Sample 60.0% 50.0% 40.0% 30.0% 20.0% 10.0% 0.0% Extremely Likely Very Likely Somewhat Likely Not at all Likely Rick Snyder (R) Lean Rick Snyder (R) Lean Mark Schaeur (D) Mark Schaeur (D) Other Strong Rep Moderate Rep Lean Rep Independent Lean Dem Moderate Dem Strong Dem Liberal Conservative Niether DK Refused Vote Likelihood Vote Candidate Party Ideology Not Matched Matched 2014 Survey Sampling International 16

17 New Insights Gained from Social Append Who will you vote for?* % That Like Candidate FB Page Who will you vote for?* % Mentioned Candidate on Facebook Snyder/Lean 25% Snyder/Lean 9% Schauer/Lea n 14% Schauer/Lean 13% Social Append allows us to analyze how strongly voters felt about their candidate Social append allows us to analyze differences between active & passive support * Indicates Survey Responses 2014 Survey Sampling International 17

18 New Insights Gained from Social Data (continued) Who will you vote for?* Top Issue* % That Expressed Issue Opinion on Facebook Snyder/Lean 17% Schauer/Lean Economy 6% Snyder/Lean 8% Schauer/Lean Healthcare 9% Snyder/Lean 1% Schauer/Lean Education 5% Social Append allows us to analyze Social Activity by Issue * Indicates Survey Responses 2014 Survey Sampling International 18

19 Conclusion Lots of Social Media Data Out There Holds great promise for the future of research Important Legal and Representivity Barriers to Overcome Doing Nearest Neighbor Matching Can Help Social Media Data Can Generate New Insights Ability to determine consistency, or lack thereof between survey & social media data. Blending Attitudinal & Social Data Results in Richer Individual Profiles Need to do Further Research and Develop Product Before Ready 2014 Survey Sampling International 19