EXPERIENCES ON INCREMENTAL RESPONSE MODELLING

Transcription

1 EXPERIENCES ON INCREMENTAL RESPONSE MODELLING SAS User Forum Finland Helsinki May 24th 2017 Jaakko Riihimäki Senior Data Analyst, Customer Insights & Analysis Telia Finland Oyj

2 OUTLINE Telia Company Incremental Response Modelling in Marketing Optimising Outbound Marketing Campaign References

3 TELIA COMPANY

4 TELIA COMPANY PROVIDES COMMUNICATION SERVICES HELPING MILLIONS OF PEOPLE TO BE CONNECTED AND COMMUNICATE, DO BUSINESS AND BE ENTERTAINED. BY DOING THAT WE FULFILL OUR PURPOSE TO BRING THE WORLD CLOSER ON THE CUSTOMER S TERMS

5 TELIA COMPANY IS THE LEADING NEW GENERATION OPERATOR IN THE NORDICS AND BALTICS FOCUS ON NORDICS & BALTICS SEK BILLION 84.2 NET SALES 25.8 EBITDA 15 CAPEX 21,000 EMPLOYEES December figures refer to continuing operations, i.e. the group excluding the former segment region Eurasia

6 INCREMENTA L RESPONSE MODELLING IN MARKETING

7 EXAMPLE: MARKETING ACTION FOR PRODUCT UPDATE Marketing action Probability model: P(update X, y) Targeted marketing action with the model Given observations from a marketing action, build a conditional probability model for an update. Response variable: update / no update P(update X, y) X = explanatory variables, y = response variable For example, in binary classification problem: logistic regression, a neural network model with a logistic output layer and a prior that favours smooth solutions, a Gaussian process with a probit likelihood function

8 EXAMPLE: MARKETING ACTION FOR PRODUCT UPDATE Marketing action Probability model: P(update X, y) Targeted marketing action with the model Given observations from a marketing action, build a conditional probability model for an update. Response variable: update / no update P(update X, y) X = explanatory variables, y = response variable For example, in binary classification problem: logistic regression, a neural network model with a logistic output layer and a prior that favours smooth solutions, a Gaussian process with a probit likelihood function Instead of modelling the probability for an update, we should model the probability of an update because of the marketing action!

9 DESIGN OF EXPERIMENT FOR INCREMENTAL IMPACT Marketing action Control group Incremental response model: P(update X, y, t) Targeted marketing action with the model Given observations from a marketing action and from a control group, build a conditional probability model for an update given the marketing action. Response variable: update / no update conditioned to the marketing action. P(update X, y, t) X = explanatory variables y = response variable t = indicator whether the marketing action was received or not

10 INCREMENTAL RESPONSE MODELLING Different approaches for incremental response modelling (or uplift modelling)

11 INCREMENTAL RESPONSE MODELLING Different approaches for incremental response modelling (or uplift modelling) Single probability model P update X, y, t Simulate the incremental impact of a marketing action using the model P update X, y, t

12 INCREMENTAL RESPONSE MODELLING Different approaches for incremental response modelling (or uplift modelling) Single probability model P update X, y, t Simulate the incremental impact of a marketing action using the model P update X, y, t Two separate probability models: one for the target group and one for the control group Subtract the predicted probabilities obtained using the control group from the predicted probabilities obtained using the target group: P t update X, y P c update X, y

13 INCREMENTAL RESPONSE MODELLING Different approaches for incremental response modelling (or uplift modelling) Single probability model P update X, y, t Simulate the incremental impact of a marketing action using the model P update X, y, t Two separate probability models: one for the target group and one for the control group Subtract the predicted probabilities obtained using the control group from the predicted probabilities obtained using the target group: P t update X, y P c update X, y

14 INCREMENTAL RESPONSE MODELLING Different approaches for incremental response modelling (or uplift modelling) Single probability model P update X, y, t Simulate the incremental impact of a marketing action using the model P update X, y, t Two separate probability models: one for the target group and one for the control group Subtract the predicted probabilities obtained using the control group from the predicted probabilities obtained using the target group: P t update X, y P c update X, y Algorithms: For example in binary classification where y {0,1} instead of a response variable y use a transformed variable z that consists of responses y from the target group unchanged and responses from the control group inverted 1 y. With certain assumptions: P t update X, y P c update X, y = 2P "update due to a marketing action" X, z 1

15 INCREMENTAL RESPONSE MODELLING Different approaches for incremental response modelling (or uplift modelling) Single probability model P update X, y, t Simulate the incremental impact of a marketing action using the model P update X, y, t Two separate probability models: one for the target group and one for the control group Subtract the predicted probabilities obtained using the control group from the predicted probabilities obtained using the target group: P t update X, y P c update X, y Algorithms: For example in binary classification where y {0,1} instead of a response variable y use a transformed variable z that consists of responses y from the target group unchanged and responses from the control group inverted 1 y. With certain assumptions: P t update X, y P c update X, y = 2P "update due to a marketing action" X, z 1 Semi-supervised style solutions where in addition to a single uplift model, two additional models for the target and control groups are built. An algorithm defines the training observations to each model and information flows via training samples between the models.

16 EVALUATING INCREMENTAL IMPACT To evaluate incremental impact (or uplift), we need to measure the number of updates both from the target and control groups but for a single observation only one of them is known

17 EVALUATING INCREMENTAL IMPACT To evaluate incremental impact (or uplift), we need to measure the number of updates both from the target and control groups but for a single observation only one of them is known One solution is to measure updates at different times but the measurement times can affect the number of updates Need to build a model for adjusting the effect of different measurement times

18 EVALUATING INCREMENTAL IMPACT To evaluate incremental impact (or uplift), we need to measure the number of updates both from the target and control groups but for a single observation only one of them is known One solution is to measure updates at different times but the measurement times can affect the number of updates Need to build a model for adjusting the effect of different measurement times Alternative solution: evaluate incremental impact for a group of observations The assumption: Similarly modelled obervations behave similarly Example: Uplift% for the highest decile = Update% for the observations ranked at the highest decile in the target group Update% for the observations ranked at the highest decile in the control group Cumulative uplift% can be computed at each decile

19 EXAMPLE: AREA UNDER THE UPLIFT CURVE (AUUC) A point at 100% gives the total uplift% in success probability if the whole target group is contacted A diagonal line connecting points corresponding to 0% and 100% describes the random selection for the marketing action One measure to summarise the model performance: the Area Under the Uplift Curve (AUUC)

20 MODEL SELECTION USING AUUC Model performance can be summarised using the Area Under the Uplift Curve (AUUC) Models can be compared, for example, by computing the differences in AUUC To assess the model performance with respect to different data partitions, use cross-validation

21 OPTIMISING OUTBOUND MARKETING CAMPAIGN

22 EXAMPLE: OUTBOUND MARKETING Outbound marketing campaigns can be optimised with incremental response modelling Relevant communication to customers Reduce unnecessary marketing costs

23 EXAMPLE: OPTIMISING MARKETING CAMPAIGN In addition to incremental response modelling, the marketing profits and costs can be included into the model Return on Investment (ROI) can be used as a measure to summarise a marketing campaign ROI can be optimised, for example, with respect to the number of customers contacted given a follow-up time period (as illustrated)

24 REFERENCES

25 REFERENCES

26 THANK YOU!