How to approach a segmentation project for ten million customers with SAS Enterprise Miner. Luis Miguel Muruzabal Endesa - Spain

Transcription

1 How to approach a segmentation project for ten million customers with SAS Enterprise Miner Luis Miguel Muruzabal Endesa - Spain Florence, 1 st June, 2001

2 Index Introduction to Endesa Starting point Short term solution: Segmentation project using SAS

3 Electricity Liberalization Process in Spain Main Agents Monetary relation POOL - OME RETAILERS (liberalized) GENERATION DISTRIBUTORS (regulated) FINAL CUSTOMERS Physical Relation REE (Transport) DISTRIBUTORS Planned and real Schedules of liberalization >15 GWh >9 GWh >5 GWh >1 GWh >15 GWh >1 GWh 2000 High Voltage 2003 Mass Market Sharp acceleration of the liberalization proccess Mass Market FREE MARKET

4 Endesa in Spain and other countries (year 2000) Supplies GWh FECSA ERZ GESA Sevillana Unelco Viesgo Endesa Energía TOTAL Spain Chile > > Argentina > > Colombia > > Brazil > > Perú > >6.900 TOTAL Latin America > > Endesa Suppliers Competitors Holland > Total Europe > TOTAL > > NATIONAL MARKET SHARE = 44,7% Countries where Endesa distributes electricity

5 Main market segments of Endesa in Spain (1999) CUSTOMERS USAGE REVENUE 1% 16% 0% 25% 29% 36% 46% 13% 17% 83% 16% 18% Residential Small businesses Commercial + Industrial Key Customers Customers GWh Mill. Ptas kwh/cust. Residential Small businesses C & I Key Customers Total % Mass Market 99,9% 34% 47% 2.800

6 Index Introduction to Endesa Starting point Starting point Data Mart Implementation Operating Plan (medium term) Short term solution: Segmentation project using SAS

7 Starting Point No real need to know well the customers and their needs, as there was a monopolistic situation All the efforts focused on cost reductions DATA BASES SYSTEMS SEGMENTATION Seven data bases and massive volume (10 Mill. customers) Few relevant variables for segmentation Data quality not equal among the different data bases Analysis information was, mainly, in papers Oriented to the customer administration, not for analysis Data requests to the S.I. Department on-need High Time response - Low flexibility No specific Marketing tools Home-made, with very basic tools (Excel, Access ) Baseda on internal variables: Tariff, usage, kw, revenue... Descriptive statistics and basic activities for C&I Difficult to define common strategies for the whole Group Need to know better the customers and to improve the marketing campaigns

8 Data-Mart Launching Approach Beat unbelief on the tool Very low initial budget (implementation, tools, etc) Acquire experience Use very few relevant data INITIAL VERSION Revision Revision Revision DATA WAREHOUSE Incorporate more data and External Data Bases Incorporate other tools (OLAP, Data-Mining) Increase processing power Increase the number of users

9 Implementation Plan Scheme Data-Mining Pilot Human Resources Tools (OLAP, Data Mining, GIS) Prove Benefits Implement Data-Mart Increase specialization Increase functionality Increased Knowledge Marketing Campaigns Technical environment Improve data quality Short term solution Medium term solution

10 Existing Tools in the market Company decission support capabilities Source : IBM Decission support Presentation Visualization tools Data Mining Discover Information Exploration OLAP, ROLAP, Statistical Analysis, Query reporting Data Warehouse / Data Mart Data Sources Internal and External Databases, files... Final User Business Analysts Technical Analysts Data Base Administrator

11 Main Analysis Tools Query Reporting Decission Support System (DSS) / EIS Data Mining SAS Enterprise Miner TM Geomarketing / GIS Data extraction (usually relational data bases) Basic segmentations Data preparation for marketing campaigns Users need high technical knowledge Visualization and Multidimensional Analysis (presentation) Pre-defined queries Solid information upon the time Users need less technical knowledge Discover relations among variables Behaviour patterns Customer profiles Requires very expert users Visualize customers on the territory Search potential customers based on standard profiles Easy to use, but with some limitations Requieres data normalization

12 Operating plan (Medium Term) MARKET SEGMENTATION FRAUD DETECTION CHURNING DETECTION - ACTION LOCATE CUSTOMERS (OWN - OTHERS) OPERATING PLAN PREDICTIONS MARKETING CAMPAIGNS DEFINITION AND MONITORING OF GOALS OTHER USES

13 Structure and Resources Needed FUNCTIONAL TECHNICAL / SYSTEMS CREATE A SPECIFIC TEAM ALIGNEMENT OF THE FUNCTIONAL AND TECHNICAL AREAS SUPPORT FROM THE ORGANIZATION LEARN WITH EXTERNAL SUPPORT EXTERNAL (CONSULTING) MANAGEMENT / ORGANIZATION

14 Index Introduction to Endesa Starting point Short term solution: Segmentation project using SAS Main goals Methodology Results

15 Specific questions to answer Is the existing pure electric data enough to make a good segmentation? What can we know about our customers from the electric usage data? What should we know of our customers to define segments? To sell electricity To sell other products and services How good is our database? (quality) What kind of problems do we have from our structure of companies? Which are the main steps to make an advanced market reseach?

16 Main goals of the Data Mining Pilot Project Make a segmentation of the residential and SMEs, using Data- Mining tools Check the feasibility of these tools for Endesa s needs Availability of the description for the main tipologies of customers Be able to make strategic analysis until the implementation of the final Data Mart and its tools Define the main segmentation variables Detect the potential improvement of the data quality Identify the main features of the future Data Mart, and the tools needed

17 SAS Methodology applied to Endesa: Data Mining SEMMA SAMPLING SELECTION OF THE INITIAL VARIABLES SAMPLING SELECTION WITHOUT SAMPLING EXPLORATION MODIFICATION MODELING VISUAL EXPLORATION CREATION OF NEW VARIABLES NEURONAL NETWORKS VARIABLES REMOVE IMPUTATION OF MISSING VALUES DECISSION TREES LOGISTIC REGRESSION DESCRIPTIVE STATISTICS USER DEFINED MODELS IMPROVE AND REFINE ASSESSMENT FINAL MODEL COMPARE MODELS AGAINST STATISTIC AND BUSINESS RULES SELECT THE BEST MODEL Tool: External help: Data source: SAS Enterprise Miner TM SAS + Apex Group Endesa s Distributors files (supplies - customers)

18 Sampling Model Residential Representative sample of supplies data (tariff, power, usage, revenue, geographic zone, climate zone ) Random stratified sample, through proportional criteria on the variables: Tariff Company Habitat Activity code Sample size: supplies No personal data, to avoid regulation problems SMEs Without sampling Only companies (not personal data) Analyze the total population, once the data has been modified Sample size: supplies

19 Exploration and Modification EXPLORATION MODIFICATION Analysis of the sample distribution Descriptive analysis of the variables to study Generation of frecuency counts for the main variables and cross tabulation Visual exploration of data Correlation analysis Identification of the main discriminant variables Creation of new variables, through direct and statistic methods (factorial analysis) Transformation and replacement of variables to the model that is going to be used Imputation of missing values Creation of typologies (target variable), through cluster analysis (k-mean algorithm) Data partition using proportional criteria for the same variables of the sample: 50% Training data set 30% Validation data set 20% Test data set

20 Modelizng Create a predictive model for the different segments: Neuronal Networks Decission trees, created authomatically and interactively Logistic Regression Creation of Specific Models ( combination of different models)

21 Validation and Scoring There is no unique best model to discriminate all segments Selection of the best discrimination model for each segment Execution of models and scoring to the whole data base Clasification of supplies according to the best probability of segment belonging

22 Pilot Project consecuences Significative improvement of the customer knowledge Organization changes: Oriented to main market segments Improvements in the campaing preparation Strong need to improve the data quality Need to obtain aditional information (no electric) Need to create a Data-Mart: Obtain customer authorization to use personal data Primary and secondary information Need to invest in specific tools

23 Data Mart Structure (Implementation scheme) SECONDARY SOURCES (i.e, surveys) STATISTIC INFORMATION (i.e, EGM, INE) EXTERNAL INFORMATION PERCEIVED QUALITY SYSTEM Public Data Bases CUSTOMERS COMERCIAL SYSTEMS DATA QUALITY PROCEDURES AND DATA PROTECTION CUSTOMER MANAGEMENT SYSTEMS CUSTOMERS CONTACTS PROSPECTS Data Mart: Residential SMEs... MARKET RESEARCH ANALYSIS TOOLS i.e, Enterprise Miner TM Extractions Reports Analysis Operating Plan...

24 Additional data from customers SURVEYS (samples) (Previous authorization from the customer) DIRECT ACTION INTERNAL ANALYSIS CUSTOMER PROFILES PUBLIC INFORMATION CAMPAIGNS (only authorized) CORRECTION (Supply Companies) DATA MART ACTUALIZATION COMERCIAL SYSTEM DATA MART + PROFILES CAMPAIGNS (profiles)

25 Results application in Endesa APPLICATIONS Descriptive Segmentation Basic Segmentation Behaviour Patterns Customer Scoring Marketing Campaigns Churning Analysis - Prevention Customer Value Fraud Detection Strategic Analysis IMPLEMENTATION Actual Actual Future Actual /Future Actual / Future Future Actual / Future Future Actual / Future

26 Summary... It is key that the functional area leads the project, with the support of the technical area It is easier to start with a low cost pilot, rather than full scale Focus the pilot on proving the benefits of these tools You don t need to have all the tools from the begining You need an operating plan and a growth plan You need external partners (i.e. SAS), who know well these techniques and have solid and proven tools