Session: 0909 Big Data Velocity- Leveraging High Speed Event Streams with ESP and SAP HANA

Transcription

1 Session: 0909 Big Data Velocity- Leveraging High Speed Event Streams with ESP and SAP HANA Prasad illapani, Jeff Wootton SAP HANA Product Management & Strategy SAP Labs LLC.,

2 Legal Disclaimer he information in this document is confidential and proprietary to SAP and may not be disclosed without the permission of SAP. his presentation is not subject to your license agreement or any other service or subscription agreement with SAP. SAP has no obligation to pursue any course of business outlined in this document or any related presentation, or to develop or release any functionality mentioned therein. his document, or any related presentation and SAP's strategy and possible future developments, products and or platforms directions and functionality are all subject to change and may be changed by SAP at any time for any reason without notice. he information on this document is not a commitment, promise or legal obligation to deliver any material, code or functionality. his document is provided without a warranty of any kind, either express or implied, including but not limited to, the implied warranties of merchantability, fitness for a particular purpose, or non-infringement. his document is for informational purposes and may not be incorporated into a contract. SAP assumes no responsibility for errors or omissions in this document, except if such damages were caused by SAP intentionally or grossly negligent. All forward-looking statements are subject to various risks and uncertainties that could cause actual results to differ materially from expectations. Readers are cautioned not to place undue reliance on these forward-looking statements, which speak only as of their dates, and they should not be relied upon in making purchasing decisions SAP AG. All rights reserved. 2

3 Agenda Big Data, Volume, Velocity - Overview Real ime Data Challenges SAP HANA Real ime Data Platform SAP Sybase Event Stream Processor with SAP HANA SAP HANA and ESP Example Use Cases Demo Key akeaways 2012 SAP AG. All rights reserved. 3

4 Big Data: Volume, VELOCIY, Variety Web Enterprise Applications witter Smart Machines Sensors his image cannot hiscurrently be image displayed. cannot current ly be display ed. h hi s i his image cannot currently be displayed. hi hi s s im im ag e ca his image cannot currently be displayed. h i h h is im ag h i h i s h h i h h his image i cannot s currently be i displayed. hi s i his image cannot currently be displayed. h i his image cannot h currently be is displayed. im h his image is cannot currently im be displayed. ag e ca nn his image cannot h current is ly be i his image cannot display m currently be ed. a displayed. his image cannot currently be displayed. Data Marts Data Warehouses racking his image cannot currently be displayed. Databases Events Data Doubles Every 18 months Many Opportunities are Short-lived Information Is a Strategic Corporate Asset 2012 SAP AG. All rights reserved. 4

5 he Challenge: Extracting Insight from the flood of raw data 2012 SAP AG. All rights reserved. 5

6 Why Real ime? Real-ime is a competitive advantage Business Value Business-relevant Event occurs Value proposition of real-time Monitoring Value lost through latency Event data stored Analysis information delivered Action taken Data Latency Analysis Latency Infrastructure Latency Reaction ime Decision Latency ime Acceleration through realtime Monitoring Source: compare Hackathorn, SAP AG. All rights reserved. 6

7 SAP HANA Real ime Data Platform 3 rd -Party BI Client SAP Business Suite SAP NetWeaver BW SAP Big Data Applications SAP Analytics SAP Mobile Custom Apps SAP NetWeaver (On-Premise/Cloud) SAP Real-ime Data Platform Open Developer APIs and Protocols Common Modeling SAP Sybase PowerDesigner MPP Scale-Out SAP Sybase Adaptive Server Enterprise (ASE) SAP Sybase Replication Server SAP Sybase SQL Anywhere SAP HANA Platform SAP Sybase ESP SAP Data Services SAP Sybase IQ SAP MDG, MDM HADOOP 3 rd -Party DB Common Landscape Management SAP Smart Data Services Platform SAP innovation without customer disruption 2012 SAP AG. All rights reserved. 7

8 SAP Sybase Event Stream Processor 2012 SAP AG. All rights reserved. 8

9 Basic Stream Capture for SAP HANA Sensors ransactions Price Feeds ESP 1 SAP HANA Click Streams Etc, etc 1 ESP receives streaming data from any number of sources Continuous queries transform the data Micro-batching optimizes load performance into SAP HANA 2012 SAP AG. All rights reserved. 9

10 High Speed Capture echniques Micro-batching: configurable to balance load efficiency with max latency Array column bindings Multiple-parallel loading channels with support for table partitions ested at over 1 million messages per second continuous loading into SAP HANA 2012 SAP AG. All rights reserved. 10

11 Smart Capture echniques Sensors ransactions Price Feeds ESP 1 SAP HANA Click Streams Etc, etc 2 1 High speed capture but after applying continuous queries to apply complex filters and rules to eliminate noise 2 ESP pulls data from HANA to provide context, such as historical norms, reference data, thresholds, etc 2012 SAP AG. All rights reserved. 11

12 Immediate Response Sensors ransactions Price Feeds ESP 1 SAP HANA Click Streams Etc, etc High speed smart captureesp publishes alerts or triggers response process Historical/reference data and/or analysis from HANA ESP publishes an alert or triggers an automated response based on changes in conditions 2012 SAP AG. All rights reserved. 12

13 SAP Precision Gaming Maximize Monetization A suite of cloud services specifically designed for gaming Personalized Offers and Messages delivered in real-time Increase the conversion rate from free to paying players Increase the purchase of specific big ticket items Reduce abandonment by first-time & long-term players Big Data Visualization & Analytics Improve game design by identifying poorly designed features Increase marketing ROI by identifying the most effective customer acquisition campaigns 40+ Easy-to-use and Easy-to-understand Standard Reports 2012 SAP AG. All rights reserved. 13

14 Architecture Diagram Customer System Existing SAP Component SAP Gaming Compnent Game Provider BOBJ Desktop Reports & Dashboards SAP Precision Gaming Solutions Precision Gaming Application Server Personalized Messaging Visualization & Analytics Performance Optimization Accounting & Finance Game Engine j Consumer Master Data Real-time Offers Real-time Event Data Sybase Event Stream Processor 5.1 SP01 Event Filter Context Analysis SAP HANA Hot Data Industry Data Model Queries / Views Predictive Models ransaction History Disk Storage Warm Data Financial System Cold Data 2012 SAP AG. All rights reserved. 14

15 Energy Production: It's too late to save resources, if they have already been spent Reports - Reports with plan and facts Real-ime Analysis - Compliance with operating mode - Liquid Leakage t V Vandt Gas Electricity Water - Environmental temperature - Metering Unit of Electricity - Metering Unit of Water W V Vandt Heat Energy - Metering Unit of Gas - Metering Unit of Flow Rate and temperature - Metering Unit of Heat 2012 SAP AG. All rights reserved. 15

16 Monitoring Energy Production Event Driven Analytics Analytical reporting on the fly DB RA #1 Billions of events per day Realtime Dashboard RS (Replication Server) RS input adapter SOI2 input adapter ESP (Event Stream Processor) HANA output adapter SAP HANA DB RA# 2 SOI 2 serv er Real-time data stream: consumption of fuel, energy consumption.. Filtering Recoding Imposition Real-ime analytical calculation Replication flow SCADA SCADA SCADA Event flow 2012 SAP AG. All rights reserved. 16

17 SAP Sybase ESP Enables Continuous Intelligence Situational Awareness Pro-active Notification Immediate Response Sometimes you just can t wait for the report 2012 SAP AG. All rights reserved. 17

18 Demo

19 Key akeaways KEY LEARNINGS 1. SAP Sybase Event Stream Processor can be used to capture streaming data in SAP HANA even at extreme message rates 2. Smart Capture techniques can be used to only capture the data that s meaningful 3. In addition to capturing data, continuous queries can be used to generate alerts or trigger a response based on observed patterns, trends or other changes in the business environment 2012 SAP AG. All rights reserved. 19

20 For More Information SAP on the Web Event Processing on SCN: ESP Developer Center: SAP Real-time Data Platform: SAP AG. All rights reserved. 20

21 2012 SAP AG. All rights reserved. No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP AG. he information contained herein may be changed without prior notice. Some software products marketed by SAP AG and its distributors contain proprietary software components of other software vendors. Microsoft, Windows, Excel, Outlook, PowerPoint, Silverlight, and Visual Studio are registered trademarks of Microsoft Corporation. IBM, DB2, DB2 Universal Database, System i, System i5, System p, System p5, System x, System z, System z10, z10, z/vm, z/os, OS/390, zenterprise, PowerVM, Power Architecture, Power Systems, POWER7, POWER6+, POWER6, POWER, PowerHA, purescale, PowerPC, BladeCenter, System Storage, Storwize, XIV, GPFS, HACMP, REAIN, DB2 Connect, RACF, Redbooks, OS/2, AIX, Intelligent Miner, WebSphere, ivoli, Informix, and Smarter Planet are trademarks or registered trademarks of IBM Corporation. Linux is the registered trademark of Linus orvalds in the United States and other countries. Adobe, the Adobe logo, Acrobat, PostScript, and Reader are trademarks or registered trademarks of Adobe Systems Incorporated in the United States and other countries. Oracle and Java are registered trademarks of Oracle and its affiliates. UNIX, X/Open, OSF/1, and Motif are registered trademarks of the Open Group. Citrix, ICA, Program Neighborhood, MetaFrame, WinFrame, VideoFrame, and MultiWin are trademarks or registered trademarks of Citrix Systems Inc. HML, XML, XHML, and W3C are trademarks or registered trademarks of W3C, World Wide Web Consortium, Massachusetts Institute of echnology. Apple, App Store, ibooks, ipad, iphone, iphoto, ipod, iunes, Multi-ouch, Objective-C, Retina, Safari, Siri, and Xcode are trademarks or registered trademarks of Apple Inc. IOS is a registered trademark of Cisco Systems Inc. RIM, BlackBerry, BBM, BlackBerry Curve, BlackBerry Bold, BlackBerry Pearl, BlackBerry orch, BlackBerry Storm, BlackBerry Storm2, BlackBerry PlayBook, and BlackBerry App World are trademarks or registered trademarks of Research in Motion Limited. Google App Engine, Google Apps, Google Checkout, Google Data API, Google Maps, Google Mobile Ads, Google Mobile Updater, Google Mobile, Google Store, Google Sync, Google Updater, Google Voice, Google Mail, Gmail, Youube, Dalvik and Android are trademarks or registered trademarks of Google Inc. INERMEC is a registered trademark of Intermec echnologies Corporation. Wi-Fi is a registered trademark of Wi-Fi Alliance. Bluetooth is a registered trademark of Bluetooth SIG Inc. Motorola is a registered trademark of Motorola rademark Holdings LLC. Computop is a registered trademark of Computop Wirtschaftsinformatik GmbH. SAP, R/3, SAP NetWeaver, Duet, PartnerEdge, ByDesign, SAP BusinessObjects Explorer, StreamWork, SAP HANA, and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP AG in Germany and other countries. Business Objects and the Business Objects logo, BusinessObjects, Crystal Reports, Crystal Decisions, Web Intelligence, Xcelsius, and other Business Objects products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of Business Objects Software Ltd. Business Objects is an SAP company. Sybase and Adaptive Server, ianywhere, Sybase 365, SQL Anywhere, and other Sybase products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of Sybase Inc. Sybase is an SAP company. Crossgate, m@gic EDDY, B2B 360, and B2B 360 Services are registered trademarks of Crossgate AG in Germany and other countries. Crossgate is an SAP company. All other product and service names mentioned are the trademarks of their respective companies. Data contained in this document serves informational purposes only. National product specifications may vary. he information in this document is proprietary to SAP. No part of this document may be reproduced, copied, or transmitted in any form or for any purpose without the express prior written permission of SAP AG SAP AG. All rights reserved. 21