IBM PureData System for Analytics Overview

Transcription

1 IBM PureData System for Analytics Overview Chris Jackson Technical Sales Specialist

2 Traditional Data Warehouses are just too complex They do NOT meet the demands of advanced analytics on big data. Too complex an infrastructure Too complicated to deploy Too much tuning required Too long to get answers Too inefficient at analytics Too many people needed to maintain Too costly to operate 2

3 3 3

4 4

5 IBM PureData System for Analytics (Powered by Netezza technology) The Simple Data Warehouse Appliance for Serious Analytics Purpose-built analytics appliance Integrated database, server and storage Standard interfaces Low total cost of ownership What makes it different? Speed x faster than traditional custom systems 1 Simplicity - minimal administration and tuning Scalability - petabyte+ scale user data capacity Smart - high performance, advanced analytics 1 Based on IBM customers' reported results. "Traditional custom systems" refers to systems that are not professionally pre-built, pre-tested and optimized. Individual results may vary. 5

6 Evolution of Netezza & PureData System for Analytics World s First appliance with no cost encryption World s Fastest and Greenest Analytical Appliance PureData System for Analytics N200x PureData System for Analytics N300x World s First Analytic Data Warehouse Appliance World s First Petabyte Data Warehouse Appliance TwinFin TwinFin with i- Class Advanced Analytics 6 World s First 100 TB Data Warehouse Appliance World s First Data Warehouse Appliance NPS 8000 Series NPS Series

7 What s New in PureData System for Analytics N3001 Security Improved security with Self Encrypting Drives Kerberos support Performance Faster performance with upgraded CPUs with more cores New appliance models New rack mountable, ultra lite appliance for midsize businesses New 8-rack, Petabyte capacity appliance New Netezza Platform Software (NPS) 7.2 Faster load rates Performance Portal enhancements and more 7

8 Logical Data Warehouse Capabilities Ready to go! PureData appliance with IBM Fluid Query and IBM BigInsights for Hadoop Included with the PureData System for Analytics N3001 Data Warehouse Appliance Hadoop Data Services BigInsights for Apache Hadoop 5 virtual Servers in aggregate per customer number Exceptional value provided IBM Fluid Query included with NPS appliance software Business Intelligence Cognos Business Intelligence, 5 Analytics User licenses, plus 1 Analytics Administrator license Data Integration & Transformation InfoSphere DataStage 280 PVUs of DataStage engine, 2 DataStage Designer User licenses, and InfoSphere Data Click Real-time Analytics InfoSphere Streams Developer Edition 2 non-production User licenses and includes additional select accelerators 8

9 IBM Fluid Query connects to Hadoop Unifying PureData System for Analytics with Hadoop Cross platform query & data movement between PureData System for Analytics and Hadoop Question SQL Query Answer Data Movement 9

10 IBM Fluid Query extends your data warehouse to RDBMS* sources Unifying PureData System for Analytics with Structured Databases Cross platform query from PureData System for Analytics to dashdb, DB2, Oracle, and PureData System for Analytics Question SQL Query Answer 10 *Relational Database Management System

11 Introducing PureData System for Analytics N Bringing speed and simplicity to midsize organizations for big outcomes Simple Same user experience as all PureData System for Analytics appliances Full function Netezza Platform Software with IBM Netezza Analytics Support tools and Netezza Performance Portal ODBC/JDBC/OLE-DB/SQL Driver integration Load and go with no tuning or administration Speed x faster than traditional custom systems 1 Smart Rich set of in database analytic functions Protection of all data from unauthorized access Includes starter kits for Big Data and Business Intelligence Agile Easily incorporated into the data center with simplified installation into an existing rack Affordable Purchase or lease Solution Highlights Rack mountable Production ready Full function appliance User data capacity 16 TB* High availability - All redundant hardware, 4 disk spares, hot swap power supply Self encrypting drives, Kerberos support, LDAP/Active directory *Assumes 4x compression 11 1 Based on IBM customers reported results. Traditional custom systems refers to systems that are not professionally pre-built, pre-tested and optimized. Individual results may vary.

12 Introducing PureData System for Analytics N rack System 1.5 PB of user data capacity 1 Hosts: 2x x3750m4 and 600 GB Self Encrypting Drives Blades: 56x HS23 with 20 core IvyBridge processors Storage: 96 EXP2524 disk enclosures with 24x 600 GB Self Encrypting Drives 12 1 Assumes 4x compression

13 PureData System for Analytics Family N2002 N3001-xxx DB2 Analytics Accelerator for z/os (now with N3001) N x faster than custom systems 1 3.3x faster I/O scan rate 2 Load and go, no tuning Designed to run complex analytics in minutes, not hours Rich set of in-database analytics...plus Entitled software capability for real-time analytics, Hadoop data services, data movement and business intelligence Advanced security Partial rack to 8-rack configurations plus Rack mountable appliance Ideal for small and medium business with up to 16 TB of user data The hybrid computing platform integrating Netezza technology with zenterprise technology Supports transaction processing and analytic workloads concurrently, efficiently & cost effectively Accelerates complex queries, up to 2000x faster Required security compliance with Data-at-Rest Encryption 13 1 Based on IBM customers' reported results. "Traditional custom systems" refers to systems that are not professionally pre-built, pre-tested and optimized. Individual results may vary. 2 Comparing N1001 scan rate of 145 TB/hour to N2002 scan rate of 478 TB/hour

14 Advanced analytics the traditional way Analytical Tools Data warehouse Analytics grid Data SQL ETL ETL Demand forecasting ETL SQL C/C++, Java, Python, Fortran, Fraud detection SQL 14 IBM PureData System for Analytics Data Warehouse Appliance

15 IBM PureData System for Analytics Simplifying serious analytics Analytical Tools Data warehouse Analytics grid Data SQL ETL ETL Demand forecasting ETL SQL SQL C/C++, Java, Python, Fortran, Fraud detection SQL 15 IBM PureData System for Analytics Data Warehouse Appliance

16 The PureData System for Analytics AMPP Architecture Field Programmable Gate Array = a blank canvas until it s configured CPU FPGA Memory Advanced Analytics CPU Memory FPGA Lite Host (IBM xseries, Red Hat Linux) BI ETL CPU FPGA Disk Enclosures Memory S-Blades Network Fabric PureData System for Analytics Appliance Loaders Applications 16

17 S-Blade Data Stream Processing FPGA Core CPU Core Stream via Zone Map From Decompress Project Restrict Visibility SQL & Advanced Analytics From Select Where Group by Select State, Age, Gender, count(*) From From MultiBillionRowCustomerTable Where BirthDate Where BirthDate < 01/01/1960 < And 01/01/1960 State in ( FL, And GA, State in SC, ( FL, NC ) Group GA, by State, SC, Age, NC ) Gender Order Group by State, by State, Age, Age, Gender Gender Order by State, Age, Gender 17

18 Inside the IBM PureData System for Analytics N N does not have Hardware Acceleration (FPGA) Optimized Hardware + Software Hardware accelerated AMPP Purpose-built for high performance analytics Requires no tuning 18 SMP Hosts SQL Compiler Query Plan Optimize Admin Disk Enclosures User data, mirror, swap partitions High speed data streaming Snippet Blades Hardware-based query acceleration with FPGAs Blistering fast results Complex analytics executed as the data streams from disk

19 Hardware Overview: Model N Disk Enclosures Total GB SAS2 Self Encrypting Drives 240 for User Data 14 for S-Blades 34 Spare RAID 1 Mirroring 2 Hosts (Active-Passive) 2 Intel 10 Core Ivy Bridge CPUs 5X600 GB SAS Self Encrypting Drives Red Hat Linux 6 64-bit 7 PureData for Analytics S-Blades 2 Intel 10 Core Ivy Bridge CPUs 2 8-Engine Xilinx Virtex-6 FPGAs 128 GB RAM + 8 GB slice buffer Linux 64-bit Kernel Scales up to 8 full Racks Terabyte to Petabyte+ Capacity Up to 10TB/hr load rate in multi-rack configurations User Data Capacity: 192 TB 1 Data Scan Speed: 478 TB/hr* Load Speed: 10 TB/hr Power Requirements: 7.5 kw Cooling Requirements: 27,000 BTU/hr 1 Assuming 4X compression 19

20 The analytic enterprise Optimization Predictive analytics BI reporting and ad hoc analysis What is the best choice? What will happen? What will the impact be? What happened? When and where? How much? 20 IBM PureData System for Analytics Data Warehouse Appliance

21 IBM Netezza Analytics v3.2 New extensions for ESRI, Spatial and Open Source R ESRI Spatial R 21

22 IBM Netezza Analytics In-database Analytics For Every Role in Your Enterprise Included Use cases Reduce hospital admissions or personalize disease treatments Achieve an order of magnitude improvement in manufacturing quality Better understand the risk of catastrophic events and many more Bring the analytics to the data not the data to the analytics Data Preparation 22 Features Built-in, in-database analytic functions - Data mining, prediction, transformations, statistics, geospatial, data preparation Full integration with tools for BI & visualization - IBM Cognos, Microstrategy, Business Objects, SAS, MS Excel, SSRS, Kognitio, Qlikview Full integration with tools for model building & scoring - IBM SPSS, SAS, Open Source R, Fuzzy Logix Full integration for custom analytics - Open Source R, Java, C, C++, Python, LUA Predictive Analytics Geospatial Analytics Advanced Statistics

23 Questions? 23

24 24