Making Your Data a Strategic Asset. Tony Fisher

Transcription

1 Making Your Data a Strategic Asset Tony Fisher Tony.Fisher@SAS.com

2 Intelligent Architecture Data Quality is a fundamental component in the Intelligence Value Chain. Poor data yields poor decisions.

3 Data Quality Gets NO Attention Over 75% of companies have no data quality process. Why? Unattractive subject matter Data Quality implementations traditionally difficult to implement and costly Can t demonstrate a return on investment Don t recognize the problems!

4 Business Impact Failed Projects Missed Revenue Telecommunications: A European telecommunications company discovered through a data audit that it was not invoicing 4 percent of its orders. For a company with $2 billion in revenues, this meant that $80 million in orders went unpaid. CRM: FleetBoston Financial spent $38M over 3 years on a failed CRM project. The project failed because of the banks inability to link customer information across the different systems.

5 Business Impact Wasted Expenses Poor Customer Relationships Healthcare: The U.S. Attorney General s office has stated that approximately $200 billion, or 14 percent of health care dollars, are wasted in fraud or inaccurate billing. Mailing Costs: The Data Warehousing Institute estimates that poor quality customer data costs U.S. businesses a staggering $611 billion a year

6 The Bottom Line Lost sales opportunities Missed project deadlines Poor customer relationships Duplication of effort Failed business initiatives Poor data leads to poor decisions.

7 What are you going to do about it? Initiate Data Integration Processes: Provides companies with a consistent, reliable view of their business by making data a strategic asset. Data Integration Methodology: Data quality Data linking and consolidation Data enhancement

8 Ad hoc and Reactive

9 Problems with Data Warehouses

10 Data Warehouse & Data Quality

11 The Need for Integration

12 True Data Integration

13 Technology Stack

14 IDI Methodology Data Quality Linking and Consolidation Enhancement

15 Data Quality Profiling Table/Column Analysis Cleansing Field/Element Analysis

16 Data Quality Profiling Identify Data Defects Identify Non-standard Data Table Column Analysis Frequency Distribution Min/Max/Outlier Detection Datatype Analysis Unique/Null Analysis Metadata Analysis

17 Data Profiling Examples Invalid input! Account Number should be Annnnnnnn and unique Validate against industry or corporate standards! Business Analyst understands valid Loan Amounts Metadata! Might determine through analysis that there are faulty or inefficient table relationships or that table columns do not contain what the column type indicates Numerical Analysis! Frequency Distribution, Max, Min of Loan Amount Data Type Analysis! Date or numeric fields may not be accurately depicted

18 Data Profiling Account Number Non-missing, Non-unique and Pattern Matching Analysis

19 Data Profiling Outliers Loan Amount

20 Data Quality Cleansing Standardization Defect Correction Value in Range Unique/Missing Compliance Transformation Verification

21 Data Cleansing Examples Uniqueness/Valid Value! Enforce Account Number rules Range Checking! Loan Amounts out of Range Verification! Invalid City/Province Standardization! Permutations of Organization, Street Name, etc.

22 Data Cleansing Segment for Country Specific Analysis

23 Data Cleansing Non-standard Data Variations First Merit Bank First Merit 1 st Merit First Merit Corp >Fist Merit Corp. first merit 1 st Merit Bank 1 st Merit Bank Corp The 5 th 3 rd Bank Fifth Third Bank 5 th Third Bank 5 th Thrid Bank Fifth 3 rd Bank

24 Linking and Consolidation Identifying Duplicates Linking Disparate Sources Consolidation Householding Siting Duplicate Elimination

25 Data Linking and Consolidation Proc Match Customer data integration

26 Data Linking and Consolidation Proc Match Householding

27 Data Enhancement Demographics Geographic Data Spending Habits Financial Reports

28 What are you going to do about it? By 2005 more than 50% of data warehouse and CRM projects will fail, with one of the points of business failure including denial about data quality issues. Stated by a senior research analyst at a recent Gartner Group Symposium

29 Don t Be One of the Failures! In today s information economy, the data in your enterprise systems is your business Don t let poor data sabotage your supply chain, CRM, or Web-based implementations