McCain & Teradata Quick overview of a journey through Big Data

Size: px

Start display at page:

Download "McCain & Teradata Quick overview of a journey through Big Data"

Rodger Mitchell
6 years ago
Views:

1 McCain & Teradata Quick overview of a journey through Big Data

2 McCain Foods Today Founded in 1957 by brothers Wallace, Harrison, Robert and Andrew, McCain Foods is the world's largest manufacturer of frozen French Fries and potato specialties. 1 in every 3 French Fries across the globe is a McCain fry.

2017 2017 Second agile BI project and many other TAS based projects 2012 2011 2010 Global SAP

3 Our Data Warehouse journey Implementation of TAS (Teradata Analytics for SAP), start of BI Self-Service initiative, and first agile BI project Pilot of Teradata Replicator Second agile BI project and many other TAS based projects Global SAP Implementation 2009 Consolidation of 36 Sales Data Marts into 1 Enterprise Data Warehouse 2008 Acquisition of Teradata system

4 McCain today Non-SAP Sources Enterprise view with ALL data from SAP and EDW Extraction Teradata System Integrated Data Warehouse Data Acquisition OEE Data Integration Vistex POS Data Access Reports + dashboards Forecast Core Database Disputes Analysis SOURCES TAS Solution Data Replication Extraction More than 120TB of database space Using ~48% of allocated space Serving ~613,000 queries/day or ~18M/month Daily Sales BPC Staging On Demand BPC Extract Intelligent Extraction, Loading and Transformation Safety Data Mining Reconciliation Reports BI abstraction layer

5 Cultivating Better Outcomes by Integrating IoT Data Rob Armstrong, Teradata Sr. Technologist 5

6 6 Devices are talking and they won t shut up.

7 This Changes Everything Product/Service Operations Support Business Goals Periodic plans and design Silos, manual operations Unexpected failures Product-centered Continuous design + feedback Automation + feedback Fixed before it breaks New digital business revenue 7

8 Plenty of Growth in IoT But Most Companies Are Not Keeping Up $7.1T 20% CAGR 80 % $2.7T IoT Worldwide Revenue Companies with unclear IoT goals, narrow use cases, limited analytics scope 8 Sources: IDC, Worldwide and Regional Internet of Things (IoT) Forecast: A Virtuous Circle of Proven Value and Demand Gartner, Internet of Things: The Foundation of the Digital Business, Jan. 6,2016

other data, such as service histories, inventory

9 9 This new product data is valuable by itself, yet its value increases exponentially when it is integrated with other data, such as service histories, inventory locations, commodity prices, and traffic patterns... Dr. Michael Porter

Integration Demands Agility Data Agility Model data for reuse, not application specific usage User self-service Business Led, IT supported Balance between modeled and un-modeled data Analytic Agility

10 Integration Demands Agility Data Agility Model data for reuse, not application specific usage User self-service Business Led, IT supported Balance between modeled and un-modeled data Analytic Agility Self service and Unconstrained queries Analytics across multiple analytic engines and data stores Multiple languages and tools Local and corporate visibility Ecosystem Agility Minimize data movement and duplication Best of breed environment Avoid technology lock in Increase adoption of new technology into ecosystem Companies become Agile by being data and analytic centric 10

11 11 What Does Centricity Mean?

12 Different Points of View Application / Process Centric Define known data Define access and output Model for performance Collect data Transform and Store Data / Analytic Centric Identify subject area Model data for relationships Collect and store data Access and cast for output Optimize performance for often run analytics based on business value When you change how you look at things, what you look at changes 12

13 13 Is not about Volume, Velocity and Variety anymore. It is about how you integrate the data and analytics

14 but there are no free lunches in Information Management merely more and different options Tightly Coupled Loosely Coupled Non Coupled Data migrates based on usage Data is leveraged dynamically Up until the late 1990s, almost all the data used in business originated internally from operational applications built by IT according to predefined business requirements Today, much of the information used by business no longer originates internally. 14 Dr. Barry Devlin, 9sight Consulting

EXPERIENCE Customers Campaign History Customers Products Pipeline CUSTOMER CARE AUDIO RECORDINGS EXTERNAL

15 FINANCE OPERATIONS Telemetry RFID ELECTRONIC COMMERCE SERVER LOGS SENSOR Maps Inventory Returns Manufacturing Supply Chain Customer Service 2855 Orders Revenue Expenses Case History IVR Routing SOCIAL MEDIA CUSTOMER EXPERIENCE Customers Campaign History Customers Products Pipeline CUSTOMER CARE AUDIO RECORDINGS EXTERNAL INTERACTIONS DIGITAL ADVERTISING CUSTOMER PORTAL INTERACTIONS CLICKSTREAM INTERACTIONS MARKETING RATINGS & REVIEWS SALES 15

16 16 Sensors are good but

17 FINANCE OPERATIONS SENSOR MANUFACTURING COSTS 2855 CASE HISTORY PRODUCTS CUSTOMER EXPERIENCE CUSTOMERS CAMPAIGN HISTORY MARKETING SALES 17

What is the forecast of demand for our hybrid cars in the Western region?

What can text based service forms tell us about potentially larger safety issues?

18 What is the forecast of demand for our hybrid cars in the Western region? What is the typical path to purchase for an extended warranty? What can text based service forms tell us about potentially larger safety issues? How many customers that called Customer Service expressed a frustrated tone of voice? Which customers are highly influential on social media and regularly post about our hybrid vehicles? SQL ANALYTICS PATH / TIME SERIES ANALYTICS TEXT ANALYTICS RICH MEDIA ANALYTICS GRAPH ANALYTICS 18

19 Business Projects, not Science Projects Avoid rather than survive Flying around your Business Turbulence Predict and Prevent Next Friday not Last Friday Don t take the call, make the call No thing as high risk, just low premiums 19

20 20

21 Engine Failure Prediction and Monitoring Problem Equipment availability issues on mine site - big direct $$$ impact - customer sat impact Challenge Initial customer solution took TOO long, making prediction for full fleet infeasible. Tools such as Matlab; no in-database; lots of time extracting, decompression, reorganizing the data. Need to detect, predict, define remaining useful life Data Sources: Sensor data from engine operation - Last oil change - Oil pressure - RPM shifts - Fuel issues. Engineering data Vehicle master data This is a $100 Million problem 21

22 Changing the Game products become information nodes enabling equipment management solutions that create sustainable value for customers, dealers, and manufactures'! Technology Enabled Solutions Real-time Monitoring Remote Flash & ET Large File Transfer Multi-mode sat, cell, Wi-Fi Hi Bandwidth Low Communication Costs Low Hardware Costs 2-way Communication Real-time Data Basic Product Data Health & Asset Mgmt. Predictive Analytics Track & Trace Remote Diagnostics Data Logging Product Configuration Customer Support Agreement Administration Component Life Prediction Enhanced Diagnostics Service Recommendation Maintenance Planning & Scheduling Prime Product Design Repair Before Failure 22

23 Continuously monitor performance of heavy machinery and perform predictive maintenance Predicted two failure modes around fuel pump system and lubrication system Reduced operating capital by $ 50million 23

24 24

25 Important Factor Extraction Process Factor Type Indicator Parameter No. Unit operating Condition Total unit operating time 833 Wind Speed Wind speed 821 Wind speed 1 second average 822 Gear oil temperature 1 second average 757 Engine room outside temperature 1 second average 797 In 635 measurement points, 65 attributes could be used in modelling Preliminary model filter results, humanadjusted factors, factor deviation Sample reselection, new model created excluding seasonal factors Conversion of power consumption and other data, iterative modelling Temperature Factors Engine room temperature 1 second average 794 Generator cooling temperature 1 second average 778 Tower base cabinet temperature 1 second average 763 Tower base temperature 1 second average 809 Hub temperature 1 second average 790 Blade 3 motor temperature 1 second average 732 Power Grid Factors Grid voltage L2 818 Grid voltage L3 819 Grid voltage L1 817 Other Concerning Factors Total cable torsion angle 734 Active turbine power consumption (30 minutes) 207_x 25

rotor overcurrent is most likely to occur

26 There is a strong correlation between

26 Turning data into action Total Unit Operation Time Histograms Wind Speed Histograms Power Consumption Rotor overcurrent is likely to occur between day 170 and 255. Cable Torsion HIstograms We can see that rotor overcurrent is most likely to occur with a wind speed around 2m/s. 26 There is a strong correlation between generator rotor overcurrent and yaw untwisting. Turbines consume significantly more power when a fault occurs.

Confirm, Re-think, and Learn More accurate recognition and determination of business regularities Discovery of new business regularities Optimisation of existing business rules Enhanced data quality

overcurrent commonly occurs when wind turbines have been in use for some time. Overcurrent commonly occurs at low wind speeds.

27 Confirm, Re-think, and Learn More accurate recognition and determination of business regularities Discovery of new business regularities Optimisation of existing business rules Enhanced data quality and level of data control Big Data Provides Multi-faceted Capability Enhancement Validation of Common Business Sense Overturning of Existing Business Thinking Discovery of New Problems Rotor overcurrent commonly occurs when wind turbines have been in use for some time. Overcurrent commonly occurs at low wind speeds. Temperature has a significant influence on rotor current and is worth analyzing further. The risk of rotor overcurrent is highest at a wind speed of 2m/s; this must be further analyzed. Yaw untwisting faults are strongly correlated with rotor overcurrent. Data quality problem: SCADA power consumption data is not sufficiently precise. Business problems: The changes in business logic for determining rotor overcurrent were not recorded. The lack of information affects analysis. 27

28 Things Public Clouds Invest in Both IoT and AoT Gateway Networks Analytics The Edge Data Center Measuring Thinking Doing 28

Analytical architectures have evolved: The Data Mart Era (application focused) The EDW Era (process focused) The Logical Data Warehouse Era (data and analytic focus) Just Give Me Any Old Data And

29 Analytical architectures have evolved: The Data Mart Era (application focused) The EDW Era (process focused) The Logical Data Warehouse Era (data and analytic focus) Just Give Me Any Old Data And Fast! (Never our advocated approach!) Give me integrated, high quality data that enables me to optimise end-toend business processes cost-effectively. Centralise the data that are widely re-used and shared - but integrate all of the data and the analytics. 29

30 Sources Acquisition Data Engines Analytics Access REAL TIME Users Audio and Video Sensors Text Web and Social Machine Logs INGEST Listener DATA LAKE Hadoop Teradata Database DATA WAREHOUSE VIRTUAL QUERY QueryGrid No SQL OPERATIONAL IN MEMORY COMPUTE CLUSTER MULTI GENRE Aster Analytics EMERGING R, Spark, Giraph CONVENTIONAL APP FRAMEWORK AppCenter Business Intelligence Languages Marketing Executives Operational Systems Customers Partners Knowledge Workers Business Analysts CRM SCM Teradata Database SAS, SPSS, KXEN Integrated Development Environment Data Scientists Platform Services DATA DEVELOPMENT OPERATIONS ERP Cloud Deployment PUBLIC PRIVATE HYBRID Engineers Teradata

31 Data Warehouse extended by Data Lake Usage Pattern Example Relational MPP Best at Secure data Integrated data Quality data Scrubbed data Privacy requirements Product-plan analysis Regulatory reporting Enterprise reporting Financial reporting Better at Enriched data Historical data Temporal analysis YoY trend analysis Good at Descriptive analytics Predictive analytics Product forecasting Campaign mgmt. Better at Machine learning Document mgmt. Discovery analysis Streaming data Fraud detection Contract mgmt. Net Promoter Score drivers Sensor Hadoop Distributions Best at Unstructured data Raw data Schema evolution Natural Language MRI / CT images CEP analysis Evolving protocols Google like search 31

32 The Data is Just the Beginning Track and forecast trends Dropping/increasing demand Decreasing/increasing risk Appropriate/competitive pricing Changes will not be small percentages Understand customer better than ever More visibility into customer journey Fast changing preferences Avoid major business failures Holding onto shrinking business models Find and exploit new opportunities Who is first to adjust rates for using automation features? Who wins the fleet? Who turns parking lots into gold mines? Who starts building houses without garages? Or remodels of garages for new use? Who figures out how to get riders attention since they are not driving? And lets all advocate for retraining programs At our companies and from the government For all the folks who are going to have to change careers For our children who are going to have to continuously adapt 32

33 Data without Analytics is Worth-less Analytics without Action is Use-less Action without Measurement is Sense-less 33

Enabling Self-Service Analytics Across The UDA With Teradata AppCenter

Enabling Self-Service Analytics Across The UDA With Teradata AppCenter Chaitanya Atreya Director, AppCenter Engineering, Teradata Jeremy Wilken AppCenter Architect, Product Manager, Teradata #TDPARTNERS16