When Big Data Meets Fast Data

Size: px

Start display at page:

Download "When Big Data Meets Fast Data"

Mitchell Leonard
5 years ago
Views:

1 15 November 2016 When Big Data Meets Fast Data - London 2016 Ted Orme VP Technology EMEA

2 When Big Data Meets Fast Data The Evolution of Hadoop Enterprise ready From batch to real-time Now add Cloud It s all About the Architecture Lambda / Kappa How Change Data Capture (CDC) Enables Big and Fast Data

3 The Evolution of Hadoop The First 10 Years Operationalising Hadoop Real-time and streaming Hadoop and the cloud

4 Operationalising Hadoop Hadoop has won the battle Moved out of Dev and Test Enterprise demands more Closer to production Business impact Sqoop Financial Services Use Case business applications Classic data lake Moved to level 3 infrastructure supporting level 1 applications

5 The Evolution of Hadoop The First 10 Years Operationalising Hadoop Real-time and streaming Hadoop and the cloud

6 Real-time Streaming with Kafka Real-time processing Real-time monitoring Real-time Hadoop Big and Fast Data Scalable to 1000 s applications Online Retail Use Case Streaming Data to Kafka Multiple data Feeds Multiple data Consumers Online Credit card transaction failure Under 20 seconds a real person calling you back and helping with the transaction

7 The Evolution of Hadoop The First 10 Years Operationalising Hadoop Real-time and streaming Hadoop and the cloud

8 Combining Hadoop and the Cloud Data Warehouse RDBMS Cloud Platform Hadoop On Premises RDBMS Hadoop Data Warehouse Sources Targets

9 Universal Data Availability Fortune 100 automotive maker 4500 applications DB2 MF SQL Oracle Results Consolidating massive data volumes for global analytics Hadoop Data Lake with Kafka Minimising labour and cost Realising faster insights and competitive advantage

10 The Evolution of Hadoop Operationalising Hadoop Financial services enterprise use cases Real-time and streaming Online retail customer success Hadoop and the cloud Global Data Lake

11 It s all About the Architecture Lambda or Kappa

12 Lambda Batch, Speed, Server Speed Layer Server Layer Query1 Query2 n 2 Data Feeds Batch Layer View1 View2 View3 n 2 1

13 Kappa Speed, Server then Batch Speed Layer Server Layer Query1 Query2 n 2 Data Feeds Batch Layer View1 View2 View3 n 2 1

14 MapR - Architecture

15 How Change Data Capture (CDC) Enables Big and Fast Data With Attunity Replicate

16 CDC Enables the Flow Kafka Streams In memory optimised metadata management and data transport Transactio n logs CDC Data Streaming MSG MSG MSG n 2 1 JSON data forma t Message broker Batch data Bulk Load Data Streaming MSG MSG MSG n 2 1 JSON data forma t Message broker

17 Intuitive User Experience Configurable, Pre-defined Automation Intuitive and easy to use web-based interface Simple to configure and manage replication tasks Single interface for any supported source to target Easy management with alerts and notifications

${"col2": "val2"},., {"coln": "valn"}] "bu_data": [{"col1": "val1"}, {"col2": "val2"},.$

18 Easily Create and Manage Kafka Endpoints Attunity Replicate "table": "table-name", "schema": "schema-name", "op": "operation-type", "ts": "change-timestamp", Command Line "data": [{"col1": "val1"}, {"col2": "val2"},., {"coln": "valn"}] "bu_data": [{"col1": "val1"}, {"col2": "val2"},., {"coln": "valn"}], Eliminate manual coding Drag and drop interface for all sources and targets Monitor and control data stream through web console Bulk load or CDC Multi-topic and multipartitioned data

19 Attunity Replicate s Differentiators 1. Easy Configurable, powered by automation (no dev or scripting) 2. Change Data Capture (CDC) Efficient, low latency and no downtime 3. Zero Footprint No agent on source database server 4. Heterogeneous Supports many data sources and targets 5. Optimised Integrated with native target system APIs

Attunity Corporate Overview Global Organisation Over 2000 Customers in 65 Countries Financial Services Manufacturing / Industrials Health Care Government USA EMEA APAC Enterprise Data Management On

20 Attunity Corporate Overview Global Organisation Over 2000 Customers in 65 Countries Financial Services Manufacturing / Industrials Health Care Government USA EMEA APAC Enterprise Data Management On Premise Cloud Across Platforms Accelerate data delivery across enterprise and cloud Empower rapid utilisation of data by the business Continually optimise with intelligent insight Technology / Telecommunications Other Industries

Attunity Trusted Technology and Partner Trusted by Microsoft with 3 OEMs, bundled inside SQL

by IBM and Oracle with respective OEMs of Attunity technology Trusted by Teradata as a

and analytics market Trusted by Accenture, Capgemini and Cognizant as SI partners Trusted by

21 Attunity Trusted Technology and Partner Trusted by Microsoft with 3 OEMs, bundled inside SQL Server Trusted by Amazon (AWS) with technology licensing for cloud migration service Trusted by IBM and Oracle with respective OEMs of Attunity technology Trusted by Teradata as a reseller for Data warehouse and Hadoop market Trusted by HP as a reseller for Data warehouse and analytics market Trusted by Accenture, Capgemini and Cognizant as SI partners Trusted by Hortonworks, Cloudera and MapR for Hadoop solutions Trusted by over 2000 customers in over 65 countries

22 CDC enables the Evolution of Hadoop Operationalising Hadoop Financial services enterprise use cases Real-time and streaming Online Retail customer success Hadoop and the cloud Global Data Lake

23 Q&A and Additional Resources Contact Us Follow Us Come see us at our stand FREE TRIA L attunity.com/replicateexpress

24 Thanks! attunity.com

THE MAGIC OF DATA INTEGRATION IN THE ENTERPRISE WITH TIPS AND TRICKS

THE MAGIC OF DATA INTEGRATION IN THE ENTERPRISE WITH TIPS AND TRICKS DATA HOLDS ALL THE POTENTIAL TO HELP BUSINESSES WIN CUSTOMERS INCREASE REVENUE GAIN COMPETITIVE ADVANTAGE STREAMLINE OPERATIONS BUT