In-Memory: Self-service Business Intelligence & Information Discovery

Transcription

1 In-Memory: Self-service Business Intelligence & Information Discovery BI / DWH Konferenz Kassel, 19. März 2013 Harald Erb Solution Architect BI & DWH 1

2 Safe Harbor Statements The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle s products remains at the sole discretion of Oracle. 2

3 Agenda From traditional BI to Business Analytics New Data Warehouse & BI Perspective Self-service BI by Example Endeca Information Discovery by Example Wrap-up 3

4 From traditional BI to Business Analytics 4

5 Traditional BI Elimination of Data Silos, multiple Tools with an integrated BI Platform Oracle Approach 5

6 Center of Excellence Standardizing technologies & processes to maximize the potential of every DWH/BI/EPM project Positioning of DWH/BI/EPM CoE s within the Organizational Structure Typical tasks of a DWH/BI/EPM Center of Excellence 6

7 Business Analytics New challenge: Pace of Change Has Rapidly Accelerated Today Big Data In-Memory Mobile Social Cloud 1980s Static Reporting 1990s Interactive Dashboards 2000s Ad-hoc Query And Analysis 7

8 New Data Warehouse Perspective 8

9 Big Data Extends the Breadth of Data Big Data Decisions based on all your data Social Data Video and Images Documents Machine-Generated Data Information Architectures Today Decisions based on database data Transactions DATA WAREHOUSE Historic Data 9

10 Customer expected a large quantity of data More than 500TB next 5 years Is it a Big Data solution? Is it a classic Data Warehouse solution? Or is it a hybrid solution? 10

11 What Makes it Big Data? SOCIAL BLOG SMART METER VOLUME VELOCITY VARIETY VALUE 11

12 What Makes it Big Data? Volume Petabyte scale Low structured data streams Weblogs, stock ticks, machine generated logs, social media streams Complex data types Spatial, graph, image, video Combine with existing data management systems 12

13 Data Samples Sensor data Machine logs {"custid": ,"movieid":null,"genreid":null,"time":" :00:33:18","recommended":null,"activity":9} {"custid": ,"movieid":768,"genreid":9,"time":" :00:33:39","recommended":"n","activity":6} {"custid": ,"movieid":null,"genreid":null,"time":" :00:34:01","recommended":null,"activity":8} {"custid": ,"movieid":null,"genreid":null,"time":" :00:34:18","recommended":null,"activity":8} {"custid": ,"movieid":null,"genreid":null,"time":" :00:34:28","recommended":null,"activity":8} {"custid": ,"movieid":27205,"genreid":9,"time":" :00:35:09","recommended":"y","activity":11,"price":3.99} {"custid": ,"movieid":20352,"genreid":14,"time":" :00:35:12","recommended":"n","activity":7} {"custid": ,"movieid":null,"genreid":null,"time":" :00:35:27","recommended":null,"activity":9} {"custid": ,"movieid":null,"genreid":null,"time":" :00:35:38","recommended":null,"activity":8} {"custid": ,"movieid": ,"genreid":11,"time":" :00:36:08","recommended":"n","activity":1,"rating":3} {"custid": ,"movieid":null,"genreid":null,"time":" :00:36:27","recommended":null,"activity":8} JSON files Apache Weblogs Twitter feeds 13

14 Infrastructure Requirements Big Data, Data Warehousing & Business Intelligence Acquire Organize Analyze Decide Low, predictable Latency High Transaction Count Flexible Data Structures Deep Analytics Agile Development Massive Scalability High Throughput In-Place Preparation All Data Sources/Structures Real-time High Concurrency Broad Access 14

15 Divided Solution Spectrum Data Variety Low-structured Schema-less Distributed File Systems Transaction (Key-Value) Stores MapReduce Solutions NoSQL Flexible Specialized Developer Centric Schema High-structured DBMS (OLTP) ETL DBMS (DW) Advanced Analytics SQL Trusted Secure Administered Information Density Acquire Organize Analyze Decide 15

16 Data Warehouse Foundation High Density Data Oracle Database Oracle BI Enterprise Edition Dashboard Ad-Hoc Query Acquire Organize Analyze Decide 16

17 Deep Analysis of Current Data High Density Data Oracle Database Spatial and Graph Oracle BI Enterprise Edition Dashboard Ad-Hoc Query Churn Locality Advanced Analytics Acquire Organize Analyze Decide 17

18 Architect for Volume and Variety Low Density variable Data High Density Data Hadoop Aggregate Pre-Analyze Oracle Database Spatial and Graph Advanced Analytics Oracle BI Enterprise Edition Dashboard Ad-Hoc Query Churn Locality Relationship Comments Acquire Organize Analyze Decide 18

19 Architect for Velocity Low Density Batch Data Streaming Data High Density Data Hadoop Aggregate Pre-Analyze Event Processing Oracle Database Spatial and Graph Advanced Analytics Model Oracle BI Enterprise Edition Real Time Decisions Dashboard Ad-Hoc Query Churn Locality Relationship Comments Recommend Act Act Acquire Organize Analyze Decide 19

20 Discover New Information Endeca Information Discovery Low Density Batch Data Streaming Data High Density Data Hadoop Aggregate Pre-Analyze Event Processing Oracle Database Spatial and Graph Advanced Analytics Model Oracle BI Enterprise Edition Real Time Decisions Dashboard Ad-Hoc Query Churn Locality Relationship Comments Recommend Act Act Discover Acquire Organize Analyze Decide 20

21 Oracle Integrated Software Solution Data Variety Low-structured Schema-less Schema HDFS Oracle NoSQL DB Oracle (OLTP) OLH ODI Hadoop MapReduce In-DB MR Oracle (DW) In-DB Analytics: Mining R Spatial Graph Endeca Information Disocvery OBI EE End User Queries Trusted Secure Administered High-structured Information Density Acquire Organize Analyze Decide 21

22 Oracle Engineered Systems Data Variety Low-structured Schema-less Schema High-structured Big Data Appliance Hadoop NoSQL Database Oracle Loader for Hadoop Oracle Data Integrator Oracle Exadata OLTP & DW Data Mining & Oracle R Semantics Spatial Exalytics Speed of Thought Analytics Information Density Acquire Organize Analyze Decide 22

23 Oracle Engineered Systems = Business Analytics Solution Oracle Big Data Appliance InfiniBand Oracle Exadata InfiniBand Oracle Exalytics Acquire Organize Analyze Decide Dashboard Ad-Hoc Query Churn Locality Relationship Comments Recommend Act Discover 23

24 New Business Intelligence Perspective 24

25 Oracle Business Analytics Strategy Any Data, Any Source Full Range of Analytics Integrated Analytic Applications On Premise, On Cloud, On Mobile 25

26 Oracle Business Analytics Tapping into Diverse Data Sets Any Data, Any Source Your Data: Decisions based on your data Transactions Documents Oracle SAP Big Data: Decisions based on all data relevant to you Social Data Machine-Generated Data 26

27 Oracle Business Analytics Full Range of Analytics powered by Exalytics In-Memory Machine Full Range of Analytics Apps & Tools Platform & Tech Reporting & Analysis Modeling & Planning Unstructured Analytics Predictive Analytics Super fast information discovery, interactive reporting, visualizations, and planning Can be extra powered by first ever In-Memory engineered system spanning Relational Multi-Dimensional and Schema Free Tightly integrated with Exadata and Big Data Appliance 27

28 Oracle Business Analytics Open, Integrated and Most Complete Business Analytics Suite Integrated Analytic Applications YOUR CUSTOM Analytics EPM Applications INDUSTRY Analytics Pre-Built Analytic Warehouse ERP Analytics CRM Analytics Best Practice Metrics Over 80 Analytic Apps 500+ Dashboards and Pages reports ~10000 metrics Role-Based Dashboards Certified Application Integration ~300 star schemas Common Enterprise Information Model 28

29 Oracle Business Analytics Open, Integrated and Most Complete Business Analytics Suite Integrated Analytic Applications YOUR CUSTOM Analytics EPM Applications INDUSTRY Analytics ERP Analytics CRM Analytics Common Enterprise Information Model Over 80 Analytic Apps 500+ Dashboards and Pages reports ~10000 metrics Siebel Oracle EBS JDE PSFT Fusion Apps SAP Universal Syndicated ~300 star schemas 29

30 Oracle Business Analytics Mobile Enterprise - Analytics available with no additional development Mobile Advanced Visualizations Enhanced right-click interactions Better interactivity with new native gestures Offline access to content with fine-grained security to grant or revoke privileges IT Controlled Managed, Consistent Support users throughout their work. Enterprise or Operational Sophisticated and reusable content 30

31 Oracle Business Analytics On Premise, On Mobile, On The Cloud 1 Mobile & Cloud Deployment Self-service Multi-tenant Full range of BI capabilities Mobile consumption without any content re-work 1 Cloud Deployment for Business Analytics is on the roadmap, but not yet available 31

32 Oracle Business Analytics Complete Solution Any Data, Any Source Full Range of Analytics Integrated Applications On the Cloud, On Mobile Oracle Exadata Oracle Exalytics In-Memory Machine Oracle Database Exadata Software Predictive Analytics Oracle BI Oracle TimesTen Oracle Essbase Oracle Endeca Oracle Big Data Appliance Oracle NoSQL Oracle R Apache Hadoop 32

33 Self-service Business Intelligence by Example 33

34 Self-service Business Intelligence Some Aspects Oracle BI Foundation Integrated Suite of Tools improves Decisions User Groups vs. User Requirements 34

35 Self-service Business Intelligence Example Overview Siebel CRM with Analytics Oracle BI Applications today Oracle Fusion Applications & Transactional BI WebCenter Integration Oracle BI Planned Features *) *) Oracle BI Version is on the roadmap, but not yet available 35

36 Introduced with Siebel CRM Pre-packaged BI Solutions 36

37 37

38 38

39 39

40 40

41 Unrivaled Integration with Oracle Applications Extends BI Value. Lowers TCO. 41

42 Alignment across Functions & the Enterprise 42

43 Consistent Views and Results Oracle Business Warehouse & Common Enterprise Information Model 43

44 Oracle Business Applications today Improved Usability, runs on Exalytics 44

45 Oracle Business Applications today Open, Integrated and Most Complete Business Analytics Suite Integrated Analytic Applications Financials HR Procure & Spend Projects Supply Chain Order Management Manufacturing Sales Marketing Service Contact Center Price Loyalty Financial Close Strategic Planning Planning & Forecasting Profitability Management Run 5-10X Faster Leverage Built-In Best Practices Communications Healthcare Retail Public Sector Financial Services 45

46 Demonstration Human Capital Management Goal: Drive a more effective workforce strategy to retain valuable employees Flight Delay Analysis to demonstrate new BI 11g Visualisations and analytic Features for Self-service BI Solutions 46

47 Oracle Fusion Application Leveraging Oracle s BI Foundation on Transactional Data Embedded Analytics in Transaction Pages BI Composer = Analysis Editor light 47

48 Oracle Fusion Application Collaboration 48

49 Data Access not limited to DWH and OLAP Sources New Design paradigm 49

50 Oracle BI & ADF Integration of ADF View Objects as a Data Source in the BI repository Real-life example: Oracle Fusion Applications 50

51 WebCenter Pre-built BI Integration BI Portlet in WebCenter Integrated BI Applications Dashboard 51

52 WebCenter Pre-built BI Integration Web Social Mobile User Engagement Complete Oracle WebCenter User Engagement Platform Business Process Management Content Management Business Intelligence Sites Best-in-Class Web Experience Management Portal Application Dashboards & Self- Service Portals Service Integration Data Integration Identity Management & Security Integrated Social Enterprise Social Software Open Content Enterprise Content Management Development Tools Cloud Application Foundation Enterprise Management 52

53 Endeca Information Discovery by Example 53

54 Simplicity of Search, Power of BI Oracle Endeca Information Discovery Self-Service Data Discovery Walk-up ease of use Contextual Search + Faceted Navigation + Visual Analysis In-Memory Performance Leverage Existing BI Investments Unstructured and Structured Data Content Acquisition and Text Enrichment 54

55 Oracle Endeca Information Discovery Understand the Complete Picture with Context from Any Source 55

56 Oracle Endeca Information Discovery Extend Business Analytics with Unstructured Data 56

57 Diverse Oracle Endeca Use Cases Call Centers & Knowledge Mgmt Claims Analysis Maintenance, Repair & Overhaul Project Staffing Digital Asset Mgmt Healthcare Correlations Enterprise Search & Knowledge Mgmt Quality& Warranty Analysis Sales & Customer Analysis Market Intelligence Field Service Analysis Customer Risk Analysis Pricing Analysis Inventory & Delivery Service, Support & Maintain Market, Sell & Deliver Product Information Mgmt Strategize, Plan & Discover Measure, Plan & Operate Research, Develop & Produce Operations & Planning Manufacturing & Quality Part, Commodity & Supplier Analysis Human Capital Management Program & Portfolio Mgmt R&D Knowledge Discovery Financial Analysis Inventory & Demand Visibility Spend Analysis 57

58 Demonstration Human Capital Management Goal: Drive a more effective workforce strategy to retain valuable employees Marketing Analysis Goal: Targeted offerings Project Staffing Goal: Improve utilization, client satisfaction, reduce costs 58

59 Discovery Application Lifecycle Structured Perform Content Acquisition and Enrichment Ingest information on the Endeca Server no predefined model required Build the Information Discovery application with a Drag-and-drop approach Semi-Structured Unstructured Oracle Corporation Confidential Oracle Internal

60 Discovery Application Lifecycle Structured Perform Content Acquisition and Enrichment Ingest information on the Endeca Server no predefined model required Build the Information Discovery application with a Drag-and-drop approach Semi-Structured Oracle Endeca Integrator Oracle Endeca Server Oracle Endeca Studio Unstructured Oracle Corporation Confidential Oracle Internal

61 Oracle Endeca Information Discovery Enterprise Platform for Business Self-Service and Discovery Studio Interactive Business Discovery Create and Share Apps Endeca Server Hybrid Search/Analytical Database Flexible Data Model Integration Suite Data Integration and Enrichment Structured and Unstructured Helps organizations quickly explore all relevant data. Combines diverse and changing information from disparate systems. Automatically organizes information for exploration and analysis. Enables rapidly assembly of easy-touse discovery applications. 61

62 Oracle Exalytics with Information Discovery Fastest, most powerful hardware platform for Unstructured Analytics In-Memory BI, EPM, and Unstructured Analytics Designed for Performance Endeca Software Breakthroughs Multi-Core Parallel Query Evaluation In-Memory Columnar Storage and Dynamic Cache Adaptive In-Memory Data Mapping Embedded Search Indices Free Form Search, Exploration, and Analysis Optimized Hardware Processor: 4 Intel Xeon E7-4870, 40 cores total Memory: 1 Terabyte DRAM Hard Disk: 3.6 Terabytes 62

63 Oracle Endeca Information Discovery Integration Suite Studio Interactive Business Discovery Create and Share Apps Endeca Server Hybrid Search/Analytical Database Flexible Data Model Integration Suite Data Integration and Enrichment Structured and Unstructured Integration Suite Lightweight ETL with native Endeca Server connectivity Java SDK for extensibility easily connect to additional data feeds and APIs Modules for unstructured data: Text Enrichment Sentiment Analysis Content Management System Connectors 63

64 Integration Suite 64

65 Oracle Endeca Information Discovery Endeca Server Studio Interactive Business Discovery Create and Share Apps Endeca Server Hybrid Search/Analytical Database Flexible Data Model Integration Suite Data Integration and Enrichment Structured and Unstructured Oracle Endeca Server Hybrid search/analytical database Search, navigation, and analytics on diverse, changing information Designed for Discovery Flexible data model Columnar storage In-memory analytics 65

66 Endeca Server Designed for Discovery Endeca Server Flexible natively semi-structured data model as the foundation for the system architecture Unified query capability spanning structured search and navigation, analytics, and unstructured search Modern database implementation techniques such as in-memory analytics, columnar physical storage, and data parallelism optimized for multicore architectures 66

67 Endeca Server Flexible Data Model Key-Value Store No segmentation into tables No overarching schema Simple concepts: Attributes like columns, except may be sparse, multi-valued, hierarchical Records each record is a collection of attribute/value pairs Accommodates: Idiosyncratic structure each record is self describing, has its own possibly unique schema Multi-valued fields Large fields of unstructured text TxnID = ProductID = 506 Category = Mountain Bike Amount = $ Suspension = Fox 32 F-Series FrameType = Aluminium Saddle = Bontrager SSR Mountain Accessories = Fork and shock sag meter Mountain Accessories = Water Bottle Review = A great bike for off road. Smooth ride over the bumps ReviewSentiment = Positive ReviewTerm = Great ReviewTerm = Off Road ReviewTerm = Smooth ReviewTerm = Bumps TxnID = ProductID = 507 Category = Road Bike Amount = $ Weight = 20lb. FrameType = Composite Saddle = Bontrager Race Review = Disappointing for the price. The frame feels heavier than I expected. ReviewSentiment = Negative ReviewTerm = Disappointing ReviewTerm = Price ReviewTerm = Heavy Structured data Transaction TxnID ProductID Category Amount Mountain Bike Road Bike 1399 Semi-structured data e.g. XML Unstructured data + Text enrichment 67

68 Endeca Server Optimized for Exploration and Analysis Storage architecture High performance column store, enhanced with: Integrated index trees Membership column Full text search indices Enhanced in-memory analytics Data mapped and cached in memory 68

69 Endeca Server Optimized for Exploration and Analysis Discovery Capabilities Contextual Navigation Data filtering and metadata analysis Search Full text and metadata search Dynamic Data Summarizations / Visualizations Storage architecture High performance column store, enhanced with: Integrated index trees Membership column Full text search indices Enhanced in-memory analytics Data mapped and cached in memory 69

70 Endeca Server Full Text Search Search index storage and analysis build on same column storage core as structured data store / indexes Structured Data Column value, id Acme Corp, 375 Search Index Column word, id, position Corp, 375 (89,72) Supports a breadth of structured and unstructured search capabilities: Guided Navigation Keyword search Boolean search Parametric search Wildcard search Dimension search Dimension filters Dimension precedence rules Numeric range filters Geospatial filters Date/Time filters Security filters Spell correction/suggestion, DYM Find similar 1- and 2-way synonyms Stemming and lemmatization Keyword-in-context snippeting Results clustering Relevance ranking Sorting and paging Language support 72

71 Endeca Server Endeca Query Language (EQL) SQL-based syntax for specifying dynamic aggregation queries Operates on the intersection results of security constraints and all user-applied search, navigation, and range (e.g. numeric, geospatial) filters DEFINE Customers AS SELECT CustomerAge AS CustomerAge, CustomerState AS CustomerState GROUP BY CustomerKey ; RETURN AvgAges AS SELECT AVG(CustomerAge) AS AvgAge FROM Customers GROUP BY CustomerState DEFINE Terms AS SELECT COUNT(1) AS Assignments GROUP BY Term, Category ; RETURN TermCounts AS SELECT COUNTDISTINCT(Term) as NumTerms, SUM(Assignments) AS NumAssign FROM Terms GROUP BY Category Example 1: A simple, structured query Example 2: Operating on semi-structured (multi-valued) data extracted from unstructured content 73

72 Oracle Endeca Information Discovery Studio Studio Interactive Business Discovery Create and Share Apps Endeca Server Hybrid Search/Analytical Database Flexible Data Model Integration Suite Data Integration and Enrichment Structured and Unstructured Studio Interactive, visual discovery environment Drag-and-drop configuration Full-featured component library Based on best practice UI design patterns Built on Java portal technology standards 74

73 Endeca s Unique User Experience Interactive Data Exploration and Analysis Advanced Search Search across all data Dynamic typeahead Automatic spell correction Unlocks unstructured data + + Contextual Navigation Data-Driven. Freely browse data without predefined paths or writing queries Interactive. Shows only valid next steps Easy to Use. Familiar online experience Visual Analysis Charts, crosstabs, key metrics Geospatial visualization Tag clouds 75

74 Studio Component-Based Application Composition Each component includes a user control and an editor 76

75 Studio Component Library Configurable components: Alerts Bookmarks Breadcrumbs Chart Compare Crosstab Data Explorer Guided Navigation Map Metrics Bar Range Filters Record Details Results Table Results List Search Box Tag Cloud 77

76 New: Endeca integrated in ebusiness Suite 12.1 EBS Applications Extensions Structured and Semi-Structured Data Unstructured Data EBS Databases Files Social 78

77 New: Endeca integrated in ebusiness Suite 12.1 Display Names Metrics Tag Clouds Charts 79

78 Modify Attribute Display Names 80

79 Modify Metrics 81

80 Modify Tag Clouds 82

81 Modify Charts 83

82 Self-service BI & Information Discovery Wrap up 84

83 Comprehensive Big Data & Analytics Solution 85

84 Oracle Business Analytics 86

85 87