The Evolution of Analytics

Transcription

1 The Evolution of Analytics Ed Colet Capital One Financial Corporation SAS Global Forum, Executive Track Presentation April, 2011

2 Outline Looking back at the evolution of analytics Standard views, and the Capital One difference Current approaches for managing change Strong processes: Patented Model Governance System Clean technology stack to enable analytics Consolidation and rationalization for growth Looking ahead to the future and evolving trends Forces driving change Opportunities worth pursuing 2

3 Introduction: A collaborative approach to evolving analytic capabilities Decision Science Infrastructure Tools for data access Tools for decision making Environments for Analytics Capital One IT Fix & Stabilize Focus of Add Near Term Value today s discussion Do strategic re-invention 3

4 Analytic Sophistication Evolution of Analytics: The typical evolution Predictive/ Prescriptive Analytics Data collection, Reporting, Dashboards Ad hoc query, OLAP, Data Mining Why Did it Happen? What Will / should Happen? What Happened? Time 4

5 Evolution of Analytics: The Capital One view Skill Levels Required Experimental Design and Hypothesis Testing Why Did it Happen? Predictive Analytics and Optimization What Will / should Happen? Reporting, Dashboards What Happened? Better Decisions Time 5

6 Analytic tools: The desired solution (a parallel coordinates view) High Ideal Situation Med Low $ Cost Analytic User depth population Decision Proximity 6

7 Analytic tools: A qualitative comparison (parallel coordinate view) High-end analytics Higher Your own code Open Source Fewer specialized users Still separated from decision making Med Lower Other $ Cost Analytic User depth population Decision Proximity Top-10 tools from the kdnuggets.com annual survey, June

8 Five stages of analytic competitiveness 5 Competitor 4 Analytical companies Big Results, sustainable advantage Possibility thinking Primary driver of performance and value Enterprise-wide Roadmap to future Drives performance and value 3 Analytical aspirations Integrated data & analytics Extrapolation Drives future performance 2 Localized analytics Opportunistic ROI of individual apps 1 Analytically impaired Flying blind Reactive mode Source: Davenport & Harris (2007). Competing on Analytics: The New Science of Winning. 8

9 Outline Looking back at the evolution of analytics Standard views, and the Capital One difference Current approaches for managing change Strong processes: Patented Model Governance System Clean technology stack to enable analytics Consolidation and rationalization for growth Looking ahead to the future and evolving trends Forces driving change Opportunities worth pursuing 9

10 Strong processes for statistical model building: MGS Model Governance System (patented): 10

11 An Enterprise-wide technology stack Technology and Capabilities User Access and Analytic Tools A Vision of Integration Universal stats & reporting platforms with access to customer data across all LOBs Customer Channel Data Storage Bank FS Card Other Access to data across LOBs, enabling deeper insights customer focused analysis. Metadata and data lineage standardized Insights in analytical arena can be more easily transferred for operational use Data Integration One location for all data movement for the company, easing integration Raw Data, System and External Files Bank Files Internet Files Auto Files Mortgage Files Other Card Files (Privacy/AML) Data files moved from source to one centralized location to ensure down stream analytics use consistently sourced information 11

12 SAS is part of our computing and analytics environment SAS usage is widespread Platform management is flexible Usage: Business Strategies & Model Development Data Analysis & Reporting Model Execution (Batch and Real time) ~100 of PhD-level quants, Thousands of data-centric analysts SAS Used for analytics and other activities across lines of business Several hundred predictive SAS Models in use (we also add new models, retire models, refine models) Consolidation: Rationalization of users, data, products, and processes Efficient product and infrastructure foot print Strengthen: Implement appropriate controls and governance Expand Infrastructure Capabilities Enhance: Exploit and explore new capabilities An effective strategy to accommodate present and future needs 12

13 Product Rationalization Server Consolidation New SAS MLA Consolidation: Product, server and desktop consolidation have helped manage complexity Product Rationalization Matched SAS desktop product needs by User Job roles Technology Consolidation Utilize Superdome as a target Platform for SAS Decommission legacy environments 13

14 A platform that supports a continuum of analytic work Analysis Unpredictable workload Ad hoc research, analysis, model development Collaborative, team-based efforts Fewer controls Access to wide variety of data sources Execution Predictable and tuned workload Limited or no end-user access SLA s for reliability, availability, Increased audit and controls Logging, monitoring, incident management Configuration management, version control Access to specific production data sources Garage Shared Laboratory Factory Utilities Idea incubation environment Evaluate new software tools Small subset of users Insights would be migrated to Laboratory for wider adoption Outsourced solution possible Analysis and model development with approved software tools Monitor and measure the execution results Large majority of users Solutions need to be migrated to Factory or Utility for reliable execution General purpose production environment Mimics laboratory layout for easy deployment Controlled end-user involvement Access to analytical data sources Batch and real-time scoring services built on optimized engines Lights-out automation Pre-defined inputs and outputs 14

15 SAS Start Index Superdome usage patterns trending upward SAS Dome - SAS Starts by Month 300% 250% 200% 150% EG Index CMD Index 100% 50% 0% March 2010 Dec 2010 Jan 2011 Feb

16 Outline Looking back at the evolution of analytics Standard views, and the Capital One difference Current approaches for managing change Strong processes: Patented Model Governance System Clean technology stack to enable analytics Consolidation and rationalization for growth Looking ahead to the future and evolving trends Forces driving change Opportunities worth pursuing 16

17 Factors driving the future-state of analytic infrastructure Issue #1 New data means more data Take advantage of costeffective storage Issue #2 New data may be unstructured (or structured differently) Issue #3 New data may require different analytic approaches Take advantage of data management capabilities Take advantage of emerging analytic techniques that find signals in noise Newer analytic paradigms and platforms Cheaper, Faster, Better? 17

18 Data growth is rapidly expanding Source: Washington Post, 2/10/11 18

19 Cost Ratio Cheaper: Data storage costs are decreasing with advances in technology 100% 80% 100% Data Storage Costs 60% 40% 20% 0% 34% 28% 18% 7% 4% Current Vendor 1 Vendor 2 Vendor 3 SWB SAS Dome Vendor Current - which includes license, disk, CPU, maintenance, backup, DR, Dev, Test and divide the total cost of the platform by the usable storage to come up with the cost per GB number. Vendor 1 - List Price - which includes license, disk, CPU, maintenance Vendor 2 - List Price - which includes license, disk, CPU Vendor 3 - List Price - which includes license, disk, CPU SAS Dome (T3) - which includes just storage (doesn't include the cost of the SAS license, support-maintenance, CPU) SWB this is a leasing arrangement. This includes: license, disk, CPU, maintenance, backup, and development 19

20 Performance Index Seconds Faster: New capabilities have faster processing times SAS In Database Performance Results About the test: 2.00% 100% 1. Converted existing model to 1.80% 1.60% use SAS In Database processing 1.40% 1.20% 1.00% 0.80% 0.60% 0.40% 0.630% 0.883% 1.031% 2. Ran the current model code on the Current Infrastructure during a maintenance window to get a benchmark 3. Ran the new in-database code on the vendor platforms 0.20% 0.00% Vendor 1 Vendor 2 Vendor 3 Current Infrastructure 20

21 Query Run Time Index But performance can degrade as the number of concurrent users increase 350% Total Run Time against # Concurrent Queries 300% 250% 200% 150% 100% 50% 0% Vendor 1 Vendor 2 Vendor 3 Current Vendor 4 # Concurrent Sessions The appliance systems are not built for high concurrency usage patterns like our current environment. However, they are able to handle a workload that is scaled down in terms of users and number of queries. **We have normalized the performance data by multiplying by the list price of the vendor POC systems. 21

22 Conclusion & Summary Looking back at the evolution of analytics Markets will drive the general evolution of analytics, Your particular circumstances and competitive situation can be addressed via effective collaborations between Business and IT Foundational elements for managing change Strong processes and governance Powerful, stable, scalable technologies that integrate well within a stack and also across lines of business Cost-effective management may drive consolidation and rationalization, but should not compromise growth and expansion Looking ahead to the future and evolving trends Data is foundational, and there is likely to be much more of it Data + Analytics + Technology can result in newer and better insights 22

23 Contact Info Ed Colet Vice President, Analytics & Technology 23