12/10/2012. Disney Marketing ROI Case Study DMA Conference. Defining A Marketing ROI Solution. Presentation Agenda

Transcription

1 Disney Marketing ROI Case Study DMA Conference Presented by Disney & SAS October 2012 Defining A Marketing ROI Solution Reach the right audience Maximize Return on investment for marketing spend Through the right channel At the right time With the right frequency At the right price Stand-alone studies often fail to achieve long-term success trying to implement a project instead of a process! Presentation Agenda Introduction Disney Management Science & Integration SAS The Science Behind Marketing ROI Case Study Overview Project Goals & Organization Data Management Science Integration Tool Development Lessons Learned Questions & Answers 1

2 Disney Management Science and Integration 4 employees employees Consulting support for analytics, data and reporting needs Technology integration for reporting and data tools Development and management of decision science tools SAS Company Overview SAS is the largest independent software vendor in the world 2011 & 2010 Fortune Magazine: #1 Place to Work 2011 Revenue: $2.73 billion SAS reinvests ~25% of annual revenue into R&D 90 of top 100 companies on FORTUNE Global 500 use SAS SAS Annual Revenue Science Behind Marketing ROI ing Measurement Marketing Effort For Each Channel (spend, impressions, etc.) vs. Response Variable (sales, leads, etc.) Sales TV Radio More effective Less effective Marketing Spend 2

3 Science Behind Marketing ROI The Right Selecting the right modeling approach is critical for success! Regression / Series R 2 = 97% Sales (t) = +0.7 * Sales (t-1) -0.2 * Price * TV * Online + Heavy weight on lagged sales; sales not responsive to price & media changes Better for FORECASTING Econometric / Panel R 2 = 67% Sales (t) = +0.2 * Sales (t-1) -1.0 * Price +0.1 * TV * Online + Less weight on lagged sales; price & media elasticities more reasonable Better for MEASUREMENT Science Behind Marketing ROI Measurement Analysts pay careful attention to data considerations and choice of models to robustly fit the data for measurement Impressions by Media Type Saturation Curves Cable Goodwill Impressions Print Radio Spend Input Output Science Behind Marketing ROI Optimization Planners leverage model output and their insights to adjust and optimize marketing plans per business constraints Impressions by Media Type Saturation Curves Cable Goodwill Impressions Print Optimal Radio Media Mix Radio Cable Spend Print Spend Radio Impressions Optimal Flighting Spend 3

4 Case Study Overview A television network is seeking decision science support to improve return on investment for the marketing of primetime television shows How effective is our current marketing spend? Which shows should get more marketing dollars? Which channels are the most effective? Most efficient? Based on current practices, where are we over-saturated? Case Study Challenges Previous attempts to answer these questions have yielded valuable insights, but have not created sustained changes Avoid the temptation to answer all questions with a single model Ensure inputs into the solution are readily available and cost effective Avoid bundling decisions that are controlled by separate teams Limited data availability prevents the network from getting accurate measures of performance for marketing efforts Data is warehoused in multiple systems, with few connection points Impression-level data is extremely difficult to capture, with actualized data existing in combinations of spreadsheets, s, and faxes Given the state of the data, common reports can take days to generate Disney and SAS Partnership Project Management 15% 15% Data Management 30% 15% Science Integration 30% 30% Tool Development 25% 40% 4

5 Project line Established a separate timeline for each work stream, inclusive of milestone and reports out to key stakeholders Data Collection Overview Data collection ultimately took four times longer than originally planned, due in large part to data quality issues Identified over 30 potential data sources and almost 250 variables Data sources ranged from databases, spreadsheets, s, and faxes Established weekly meetings with key stakeholders and implemented dashboards to review data collection progress Placed an analyst in the media agency office for four weeks to speed data collection and improve understanding of the data Data collection is never really over continue to find errors or missed opportunities even months later! Data Collection Challenges database changed 17 times during a 1-year span, most often due to missing data or data collection errors Bad circulation estimate for Entertainment Weekly Misclassified OOH support as Events Nielsen P3 vs. C3 Magazine Cume based on all publications instead of purchased Duplication from SQL Errors Week 53 Issue 5

6 Data Visualization Showing clients the relationship between impressions and costs helped to identify likely errors in the data (e.g., misclassification of spending) Data Visualization (cont.) Exploring flights enabled us to recognize the need to model certain media types differently than others 15% 70% 15% Data Transformation Often necessary to transform the data for measurement variables in our models to avoid creating misleading insights or recommendations Episode Air Date Sunday Monday Tuesday Wednesday Thursday Friday Saturday Promos in Calendar Week S S M S M T S M T W S M T W R S M T W R F S M T W R F S M T W R F S T W R F S W R F S R F S F S S Promos in Past 7 Days S S M S M T S M T W S M T W R S M T W R F S M T W R F S Transform to a full week 6

7 Data Handoff to Science Key milestone was the go/no-go decision on beginning the development of the measurement model NIELSEN Program Name Air Date Start Duration Program Type Program Rating Lead-in Rating Competition PROMOS & MARKETING On-Air Promos TRPs, Seconds, # of Spots Digital Impressions & Clicks Cinema Impressions, Seconds Per Spot National Cable TRPs Newspaper Impressions & Circulation Magazine Total & Weekly Impressions Spot Cable TRPs & Impressions Spot Radio TRPs OOH Impressions AWARENESS Survey Respondents Aware Respondents %Aware Unaided & Aided Intent to View Top Box, Top 2 Box, Non Committed, Bottom Box Data Handoff to Science (cont.) Future iterations of the model will incorporate new data that is either unavailable right now or represents a higher level of complexity MISSING DATA Network Radio Synergy Cable Synergy Online s & Newsletters Public Relations Affiliate Promotions MISSING COMPLEXITY On-Air Promos Day-of-Week, Promo Length Nielsen Reach, Share, HUT, PUT Print Size, Placement, Inserts National Cable Channel, # of Spots, Promo Length Spot Cable & Radio # of Spots, Seconds of Promo OOH # of Units, Size, Media Form Digital Size, Placement, Pillar Social Media Facebook, Twitter, Blog Mentions DATA RECONCILIATION On-Air Promos Digital Impressions MODEL EXPANSION Geo-Panel Data Local Market and Marketing On-Air Promo Precision Minute-by-Minute Efficiency Costs for Marketing & Promotions Science Integration Science Data Integration between the team managing data collection and model development is critical to the success of the project When it doesn t work well each revision of the data model would delay the science timeline by 3 weeks! Science Tool Critical to integrate science team with tool developers to ensure alignment with the expected input and outputs of the models 7

8 Overview of Planning & Optimization Tool The tool is designed to become self-sustaining to support updates to the measurement model and to allow media plan comparisons Historical Data (one time) Data Measurement Adjustments Actualized Media Plans Optimization Goals & Constraints Agency Media Plans Approved Media Plans Recommended Media Plans Optimization Goals Objective is to maximize total ratings for the premiere episodes of all shows within a marketing campaign portfolio Provide recommended spending by channel for each show/week combination Allow users to input constraints on total spending by show/channel/week Define spend thresholds that reflect minimum purchase amounts for each channel Compare optimal recommendations against manually created plans Critical to understand relationship between spend and impressions; some channels have a significant delay between purchase and delivery! Evaluating Media Plans Ability to compare different plans by measuring the number of new households generated for each incremental unit of spend Recommended Plan: (balanced by optimization) Week Cable Radio Print Outdoor Cinema t= 5 20 N/A t = 4 20 N/A t = 3 20 N/A t = t = t = Media Agency Plan: Week Cable Radio Print Outdoor Cinema (incremental opportunities) t = 5 70 N/A t = N/A t = N/A t = t = t =

9 Key Lessons Learned Creating Clear Requirements Designing a Structured QA Process & Team Having a Test Environment Shadow Implementation Questions and Answers? 9