Data Mining using SPSS Modeler 2nd Session

Transcription

1 IBM Taiwan Claire Lin Data Mining using SPSS Modeler 2nd Session 2014 IBM Corporation

2 Agenda Data Mining Process Business Understanding Data Understanding Live Demo and Exercise Data Preparation and Manipulation Live Demo and Exercise IBM Corporation

3 What is Data Mining? The analysis step of the Knowledge Discovery in Databases (KDD) process, it encompasses a number of techniques to extract useful information from (large) data files, without necessarily having preconceived notions about what will be discovered. The goal of data mining is to extract information from a data set and transform it into an understandable structure for further use IBM Corporation

4 Data mining process Cross Industry Standard Process for Data Mining(CRISP) IBM Corporation

5 Data mining process Cross Industry Standard Process for Data Mining(CRISP) What SPSS Modeler can do? Input raw data Data understanding Check missing data Check anomalous and outlier data Data preparation Filter, derive, reclassify nodes Modeling Output IBM Corporation

6 Business Understanding Determining business objectives Finding what people will buy together with 粽子 during Dragon Festival Predicting who is likely to not renew and contract for mobile phone service Assessing the situation Determining data mining goals Producing a project plan IBM Corporation

7 Data Understanding Need to understand Includes What your data resources are What the characteristics of those resources are Collecting initial data Describing data Exploring data Verifying data quality Missing Data Anomalous Data IBM Corporation

8 Data Understanding - Missing Data Blank Contain no information. White space if the field is string and Null value (non-numeric) if the field is numeric Empty string A string field may be empty, which means that it contains nothing (This is common in databases) Value blanks Represent missing or invalid information IBM Corporation

9 Data Understanding - Missing Data IBM Corporation

10 Data Understanding - Anomalous Data What is Anomalous Data? Far from the center of the distribution Measured by the mean or median and using the standard deviation as a measure of spread Far from other values Whether close to the center of the distribution, or not IBM Corporation

11 Data Understanding Anomaly detection IBM Corporation

12 SPSS Modeler User Interface IBM Corporation

13 IBM Corporation

14 Data Sources Database: ODBC source Var. File: free-field text file Fixed File: fixed-field text file Statistics File/SAS File/Excel File IBM Corporation

15 Data Understanding The Data Audit node Provide report Missing values Outlier data and Extreme data Information on a field s distribution IBM Corporation

16 Data Understanding Anomaly detection models identify outliers or unusual cases by using clustering analysis Each record is assigned an anomaly index It's the ratio of the group deviation index to its average over the cluster that the case belongs to Cases with an index value greater than 2 could be good anomaly candidates IBM Corporation

17 Data Understanding Outliers Data Live Demo Live Demo SPSS Modeler UI Read data into SPSS Modeler Check missing data Check anomalous and outlier data Data Audit Node Anomaly Node IBM Corporation

18 Live Demo & Exercise I IBM Corporation

19 Data Preparation and Manipulation Objective: Construct the final dataset for modeling Record Operations Select partial data from dataset Sort the data Field Operations IBM Corporation

20 Type: Specifies field metadata and properties Type Continuous Categorical Nominal Description Used to describe numeric values, such as a range of or A continuous value can be an integer, real number, or date/time. String values Used to describe data with multiple distinct values, each treated as a member of a set. Ordinal Used to describe data with multiple distinct values that have an inherent order IBM Corporation Flag Used for data with two distinct values that indicate the presence or absence of a trait. Such as true and false, Yes and No or 0 and 1.

21 Filter: Filters, renames fields IBM Corporation

22 Derive: Modifies data values or creates new fields IBM Corporation

23 Reclassify IBM Corporation

24 Live Demo & Exercise II IBM Corporation

25 Trugarez Breton Merci French Gracias Spanish Grazie Italian Hindi Arabic Obrigado Brazilian Portuguese Traditional Chinese Korean go raibh maith agat Gaelic Dankon Esperanto Simplified Chinese Hebrew Tack så mycket Swedish Tak Danish Danke German Japanese Thank You English Tamil Dank u Dutch Thai Dekujeme Vam Czech IBM Corporation