Predictive Conversion Modeling Lifting Web Analytics to the next level Superweek, Hungary February 1, 2017 @mertanen & @ronluhtanen
Our approach to data-driven marketing a global focus It s our people that make the difference EMEA Offices 1200 employees globally Regional Centres of Excellence alongside agency embedded experts NA Offices MENA Offices APAC Offices
Using machine learning for explaining and predicting user behavior from web analytics data. 3
Universal Analytics made the platform more open.
Tag Management was a game changer! 5
Insights based on aggregated data. 6
Advanced methods limited to different regression models. 7
Traditional Web Analytics is dying...
What are the problems with traditional Web Analytics? 9 Time consuming Not cost effective Human brains are not able to work with large amount of complex data Outputs depends too much on the analyst Insights are too simple Predicting in a very rough level
Web Analytics Data Science
Easy & fast implementation for the Modeling.
Setup for the Modeling 12 Tag website features and elements like never before, more is more in this case! Collect session ID Save browser ID Think about User ID Adform cookie ID or similar
This means we see every interaction that each user has during each visit. The granularity of the data greatly increases the possible model selection as well as the accuracy of the models.
Extreme Gradient Boosting
About the modeling 15 Leaf Cover: 13,25 Gain: 1,85965 The bulk of the modeling is done by Extreme Gradient Boosting odor=none Cover: 1628,25 Gain: 4000,53 stalk-root=rooted Cover: 788,852 Gain: 832,545 < -9,53674e-07 < -9,53674e-07 spore-print-color = green Cover: 703,75 Gain: 198,174 stalk-root=club Cover: 924,5 Gain: 1158,21 Leaf Cover: 20,4624 Gain: -6,23624 odor=none Cover: 768,39 Gain: 569,725 < -9,53674e-07 < -9,53674e-07 < -9,53674e-07 Leaf Cover: 690,5 Gain: -1,94071 Leaf Cover: 112,5 Gain: -1,70044 Leaf Cover: 812 Gain: 1,7128 Leaf Cover: 309,453 Gain: -0,96853 Leaf Cover: 458,937 Gain: 0,784718 The method is a decision tree based algorithm Gradient Boosting can handle regresssion as well as multiclass classification We have great flexibility with selecting the KPIs that we want to model and predict, without having to change the core modeling algorithm
About the modeling 16 Incredibly accurate, hard to overfit and very fast Ability to extract complicated non-linear relationships from very varied data The Algorithm uses only the relevant data from all the data that is available to it Huge improvement over some other regression models that break if they are fed with irrelevant data https://github.com/dmlc/xgboost
Outputs from the Predictive Conversion Modeling 17
18 Outputs from Predictive Conversion Modeling Generally the output of the analysis is a predictive model that gives a predictions for the measurement we are modeling against. The predictions can be used by themselves or further analysis can be done on the model to further explain the dependencies in the user interactions. The model will be available for digital marketers and analysts. Following are 4 example uses for the modeling.
Data-to-output in Predictive Conversion Model application 19 Input Output Machine learning based predictive modelling Profiling by clustering customers based on on-site behavior Retargeting based on predicted responses Enhanced Web Analytics data Conversion optimization Twinning to expand reach to the most prospective customer profiles
Output Application 1: Enhanced Retargeting 20 The predictions can be used in more effective retargeting. Instead of bombarding all the past site visitors with advertisements we can target the advertisements based on the specific interactions as well as the likelihood of having converted. For instance we can create a rule that targets people who have over 20% probability of purchase and have visited the promotion page of a specific product. Recipe IF >20% Probability of purchase Visited product page THEN Target advertisement to specific people Trigger Action
Output Application 2: Clustering and Profiling 21 The modeling process can also be used in acquiring valuable information on the behavioral differences of the users. Uncovering certain dependencies in their interactions allows the marketers to design (and later automate) their marketing messages differently and more effectively to each of their visitor groups (segments). On-site behavior Off-site behavior Person A WEB BEHAVIOR Person B Has visited booking page twice Has visited promotion page three times Likes gambling sites Buys clothes online Visits homepage regularly Has read product description page for three minutes Reads gardening blogs Watches regularly movie trailers online
Output Application 3: Conversion Optimization 22 The machine learning models can help in conversion optimization. We are not restricted with just A/B testing, but instead we can create rules that change the site in order to maximize the likelihood of purchase or conversion of each and every user. By leveraging the trained model we can direct the user towards the interactions that are most effective in increasing the likelihood of conversion. WEBSITE CONTENT RULES Activated rule Not actived rule
Output Application 4: Twinning 23 Once we have identified the most beneficial behavioral patterns, we can use the cookie data of the most prospective visitors in order to build larger target groups out of similar web users. The groups can then be used in programmatic buying of advertisements. BUYING RULES for different target groups
How to target marketing so that it maximizes users likelihood to convert? 25
Case Tallink Silja 26 Finland s largest shipyard builds and operates cruise ships Operates in a very competitive online environment High maturity with online optimization and data-driven marketing Large portion of sales through online Annually 9 mil. Passengers * 945 mil. Turnover *
The model 27 Accuracy 98% Sensitivity 99% Specificity 75% ROC Curve Very accurate predictions for nonconverting visitors Possibility to adjust prediction treshold for different actions
Clustering using on-site behavioral data 28 Previously possible only to create custom segments Now clustering using unsupervised machine learning over 240 dimensions 8,8 Mean Conversion % - Indexed 8,4 9,5 Four distinct behavioral groups Heavy users Intermediate users Reactivated Just visiting 1
Exploring differences time spent on site 29 Mean Session Duration - Indexed Mean Duration from past Session* - Indexed 27 21 10 0,5 1 2,4 1,6 1 *Calculated as a cumulative sum with 50% daily decay
Not limited to averages 30 Session Duration Heavy User Session Duration Just Visting 4,3 3 315 250 216 280 240 213 1,5 0,28 1 No convertion Converted No convertion Converted
Proportional differences depending on source 31 Proportion of visitors from Direct Proportion of visitors from Display *Calculated as a cumulative sum with 50% daily decay
Feature Importance 32
Gaining insight from a complex model 33 Partial dependencies Change inputs Observe outputs Automate Can be applied to advertisement messages, channels, or on-site elements Possible to use smart optimization algorithms to identify actions that maximize conversion probability https://github.com/fmfn/bayesianoptimization
Discount campaign s effect on mean propability for conversion 34 Heavy Users Intermediate Users Reactivated Just visiting
Future development with the Predictive Conversion Modeling
Future Development Streams 36 Input Output Semantic data Organic clustering based on offand on-site data ID s from ad serving platforms Basic Web Analytics data Enchanced Web Analytics data Machine learning leveraged analytics and real time predictive modelling Immediate onsite adaptations based on off-site data AI driven marketing: test and modify content based on predicted behaviours Retarget to increase conversion percentage Client s Customer Data Automated optimization of online advertising spending
How this is changing our work? 37 Spend less time on manual analysis No more headache with complex data and pressure for outputs Think more about the business questions The model will do the counting and give answers with a high confidelity level You will interpret results for the business and edit the model for more in-depth analysis You are able to enable analysts with tools previously available only to data scientists Shift the focus from simple metrics to the actual business objects Set up automatically optimizing feedback loops in order to continiously increase conversion rates
Executive Summary
Executive Summary 39 No more time consuming, labor heavy and expensive manual analysis Enable analysts with machine learning Fast to implement and quick to show results Ask another question Continiously improve marketing efficiency and ROI Get real competitive edge with analytics Petri Mertanen Director, Digital Analytics petri.mertanen@annalect.com +358 400 792 616 Ron Luhtanen Analyst, Data Science ron.luhtanen@annalect.com +358 50 431 8166
Q&A Petri Mertanen Director, Digital Analytics petri.mertanen@annalect.com +358 400 792 616 Ron Luhtanen Analyst, Data Science ron.luhtanen@annalect.com +358 50 431 8166 Annalect Finland www.annalect.fi info.finland@annalect.fi @annalect_fi Annalect Finland is a part of Omnicom Media Group.