Leveraging Oracle Big Data Discovery to Master CERN s Data. Manuel Martín Márquez Oracle Business Analytics Innovation 12 October- Stockholm, Sweden

Size: px
Start display at page:

Download "Leveraging Oracle Big Data Discovery to Master CERN s Data. Manuel Martín Márquez Oracle Business Analytics Innovation 12 October- Stockholm, Sweden"

Transcription

1

2 Leveraging Oracle Big Data Discovery to Master CERN s Data Manuel Martín Márquez Oracle Business Analytics Innovation 12 October- Stockholm, Sweden

3 Manuel Martin Marquez Intel IoT Ignition Lab Cloud and Big Data Munich, September 17th CERN - European Laboratory for Particle Physics 3

4 A World-Wide Collaboration Manuel Martin Marquez Intel IoT Ignition Lab Cloud and Big Data Munich, September 17th 4

5 Manuel Martin Marquez Intel IoT Ignition Lab Cloud and Big Data Munich, September 17th 5

6 6

7 11/30/2016 7

8 CERN Aerial View 11/30/2016 Document reference 8

9 LHC Installation 11/30/2016 Document reference 9

10 CMS Detector 11/30/2016 Document reference 10

11 11

12 Hadoop and Analytics IT-DB-SAS New scalable data services Scalable databases Hadoop ecosystem Time Series databases Big Data Analytics Activities and objectives Support of Hadoop Components Further value of Analytics solutions Define scalable platform evolution Hadoop Production Service 12

13 requests per day. Direct SQL access is not permitted. A generic Java GUI called TIMBER is also provided CERN Accelerator as a means visualize and Logging extract logged data. The Service +800 extraction clients +5 million extraction requests per day 130 custom applications Credit: BE-CO-DS ~ Signals ~ 50 data loading processes ~ 5.5 billion records per day ~ 275 GB / day 100 TB / year throughput Filters for data Reduction tool is heavily used, with more than 800 active users Data ingestion Per day GB 2014 Figure 2: Logging ervice architecture overview. The Java APIs for both logging and extracting data are procedures have been h an understanding of h performing, in terms of how it is being done, and Optimal Use of Softw ~ 1 million signals ~ 300 data loading processes ~ 4 billion records per day ~ 160 GB / day 52 TB / year stored The database mod business logic (written Java infrastructure inter engineered to use the Oracle, to maximize per the LS systems are be aforementioned instrum which features and tech performance. Data Quality Contro The MDB (introduc filtering capabilities wh for long-term storage. effort [4] to ensure 13 configurations, the MD

14 CERN Accelerator Logging Service New Landscape bring new challenges Better Performance on bigger datasets Big Data queries: Impala, Spark SQL Leverage analytics capabilities Spark Analytics: Python, ML, R More heterogeneous data access models Storage Evolution - Size in GB / day Credit: BE-CO-DS QPS 14

15 CERN Accelerator Logging Service Speed Log. Proc. Log. Proc. 100mS Kafka 1min 7 min Gobblin 1min 7 min Batch Compactor HBase HDFS Credit: BE-CO-DS CCDB Schema Partition Provider Storage 15

16 Accelerator Postmortem Analysis Postmortem Analysis Diagnostic on failures Continue operations safely Intervention Required Designed for CERN LHC Extended to injectors complex (SPS) External Post Operational Checks Injection Quality Checks 16

17 Accelerator Postmortem Analysis Challenges: Stringent Timing Constraint Better scalability data storage IO throughput Big Data Streaming Analytics 17

18 Post-LHC accelerator projects ( km)

19

20 Architecture overview CDH nodes, 24 GB ram Intel Xeon 2.27GHz 165 TB HDFS C o o r d i n a t i o n Oracle Big Data Discovery Libraries + Hive table detector Resource Management (YARN) Data Storage Data Integration Big Data Discovery v1.2.2 Dgraph & Studio 4x Xeon E v2 (15 cores each) 2 TB RAM 4.8 TB Flash + 6 x 1.2 TB 10K HDD

21 Oracle Big Data Discovery Overview Data Exploration & Discovery Interactive catalog of all data Assess attribute statistics, data quality and outliers Quick data exploration or create dashboards and applications Data Transformation with Spark in Hadoop Apply built-in transformations or write your own scripts Data Enrichment Text: Entity extraction, relevant terms, sentiment, language detection Geographical information: address, IP, reverse Preview results, undo, commit and replay transforms Collaborative environment Share and bookmarks Create and share transformed datasets

22 Data Transformation UI - ETL 26/01/ USA BIWA 16 22

23 Discovery Applications 26/01/ USA BIWA 16 23

24 Advance Analytics - Notebooks Easy to create and share documents that contain live code Step by step execution reproduce the analysis, charts, etc. Support for multiple languages/kernels Multiple notebook software available Jupyter/IPython BDD provides notebook from version (BDD Shell) Can be used with Jupyter/IPython HUE notebooks Apache Zeppelin More

25 Scalable Analytics Reliability of degrading components of valves in the cryogenic system of the LHC (University of Delft) BDD -> Data Extraction -> Refine Calculations Scalable solutions apply to all the cryogenics valves

26 Conclusions Hadoop is not the solution for all your problems but.. Unlock new ways to exploit your investment on data overcome technical limitations for several CERN use cases Allows heterogeneous data access not only SQL or custom java APIs Once the data is in Hadoop only half of the way is done Data visualization and discovery Notebooks are easy to use and powerful for advanced analytics Self-service tools improve productivity Users should be able to do what they need without IT intervention 11/30/

27

Data Analytics and CERN IT Hadoop Service. CERN openlab Technical Workshop CERN, December 2016 Luca Canali, IT-DB

Data Analytics and CERN IT Hadoop Service. CERN openlab Technical Workshop CERN, December 2016 Luca Canali, IT-DB Data Analytics and CERN IT Hadoop Service CERN openlab Technical Workshop CERN, December 2016 Luca Canali, IT-DB 1 Data Analytics at Scale The Challenge When you cannot fit your workload in a desktop Data

More information

Hadoop and Analytics at CERN IT CERN IT-DB

Hadoop and Analytics at CERN IT CERN IT-DB Hadoop and Analytics at CERN IT CERN IT-DB 1 Hadoop Use cases Parallel processing of large amounts of data Perform analytics on a large scale Dealing with complex data: structured, semi-structured, unstructured

More information

New Big Data Solutions and Opportunities for DB Workloads

New Big Data Solutions and Opportunities for DB Workloads New Big Data Solutions and Opportunities for DB Workloads Hadoop and Spark Ecosystem for Data Analytics, Experience and Outlook Luca Canali, IT-DB Hadoop and Spark Service WLCG, GDB meeting CERN, September

More information

Course Content. The main purpose of the course is to give students the ability plan and implement big data workflows on HDInsight.

Course Content. The main purpose of the course is to give students the ability plan and implement big data workflows on HDInsight. Course Content Course Description: The main purpose of the course is to give students the ability plan and implement big data workflows on HDInsight. At Course Completion: After competing this course,

More information

20775A: Performing Data Engineering on Microsoft HD Insight

20775A: Performing Data Engineering on Microsoft HD Insight 20775A: Performing Data Engineering on Microsoft HD Insight Duration: 5 days; Instructor-led Implement Spark Streaming Using the DStream API. Develop Big Data Real-Time Processing Solutions with Apache

More information

20775 Performing Data Engineering on Microsoft HD Insight

20775 Performing Data Engineering on Microsoft HD Insight Duración del curso: 5 Días Acerca de este curso The main purpose of the course is to give students the ability plan and implement big data workflows on HD. Perfil de público The primary audience for this

More information

20775: Performing Data Engineering on Microsoft HD Insight

20775: Performing Data Engineering on Microsoft HD Insight Let s Reach For Excellence! TAN DUC INFORMATION TECHNOLOGY SCHOOL JSC Address: 103 Pasteur, Dist.1, HCMC Tel: 08 38245819; 38239761 Email: traincert@tdt-tanduc.com Website: www.tdt-tanduc.com; www.tanducits.com

More information

20775A: Performing Data Engineering on Microsoft HD Insight

20775A: Performing Data Engineering on Microsoft HD Insight 20775A: Performing Data Engineering on Microsoft HD Insight Course Details Course Code: Duration: Notes: 20775A 5 days This course syllabus should be used to determine whether the course is appropriate

More information

Oracle Big Data Discovery The Visual Face of Big Data

Oracle Big Data Discovery The Visual Face of Big Data Oracle Big Data Discovery The Visual Face of Big Data Today's Big Data challenge is not how to store it, but how to make sense of it. Oracle Big Data Discovery is a fundamentally new approach to making

More information

Big data is hard. Top 3 Challenges To Adopting Big Data

Big data is hard. Top 3 Challenges To Adopting Big Data Big data is hard Top 3 Challenges To Adopting Big Data Traditionally, analytics have been over pre-defined structures Data characteristics: Sales Questions answered with BI and visualizations: Customer

More information

BIG DATA AND HADOOP DEVELOPER

BIG DATA AND HADOOP DEVELOPER BIG DATA AND HADOOP DEVELOPER Approximate Duration - 60 Hrs Classes + 30 hrs Lab work + 20 hrs Assessment = 110 Hrs + 50 hrs Project Total duration of course = 160 hrs Lesson 00 - Course Introduction 0.1

More information

Oracle Big Data Discovery Cloud Service

Oracle Big Data Discovery Cloud Service Oracle Big Data Discovery Cloud Service The Visual Face of Big Data in Oracle Cloud Oracle Big Data Discovery Cloud Service provides a set of end-to-end visual analytic capabilities that leverages the

More information

Spark and Hadoop Perfect Together

Spark and Hadoop Perfect Together Spark and Hadoop Perfect Together Arun Murthy Hortonworks Co-Founder @acmurthy Data Operating System Enable all data and applications TO BE accessible and shared BY any end-users Data Operating System

More information

Data Analytics Use Cases, Platforms, Services. ITMM, March 5 th, 2018 Luca Canali, IT-DB

Data Analytics Use Cases, Platforms, Services. ITMM, March 5 th, 2018 Luca Canali, IT-DB Data Analytics Use Cases, Platforms, Services ITMM, March 5 th, 2018 Luca Canali, IT-DB 1 Analytics and Big Data Pipelines Use Cases Many use cases at CERN for analytics Data analysis, dashboards, plots,

More information

Microsoft Azure Essentials

Microsoft Azure Essentials Microsoft Azure Essentials Azure Essentials Track Summary Data Analytics Explore the Data Analytics services in Azure to help you analyze both structured and unstructured data. Azure can help with large,

More information

EXAMPLE SOLUTIONS Hadoop in Azure HBase as a columnar NoSQL transactional database running on Azure Blobs Storm as a streaming service for near real time processing Hadoop 2.4 support for 100x query gains

More information

Big Data Hadoop Administrator.

Big Data Hadoop Administrator. Big Data Hadoop Administrator www.austech.edu.au WHAT IS BIG DATA HADOOP ADMINISTRATOR?? Hadoop is a distributed framework that makes it easier to process large data sets that reside in clusters of computers.

More information

Transforming Analytics with Cloudera Data Science WorkBench

Transforming Analytics with Cloudera Data Science WorkBench Transforming Analytics with Cloudera Data Science WorkBench Process data, develop and serve predictive models. 1 Age of Machine Learning Data volume NO Machine Learning Machine Learning 1950s 1960s 1970s

More information

Oracle Big Data Cloud Service

Oracle Big Data Cloud Service Oracle Big Data Cloud Service Delivering Hadoop, Spark and Data Science with Oracle Security and Cloud Simplicity Oracle Big Data Cloud Service is an automated service that provides a highpowered environment

More information

Intro to Big Data and Hadoop

Intro to Big Data and Hadoop Intro to Big and Hadoop Portions copyright 2001 SAS Institute Inc., Cary, NC, USA. All Rights Reserved. Reproduced with permission of SAS Institute Inc., Cary, NC, USA. SAS Institute Inc. makes no warranties

More information

KnowledgeENTERPRISE FAST TRACK YOUR ACCESS TO BIG DATA WITH ANGOSS ADVANCED ANALYTICS ON SPARK. Advanced Analytics on Spark BROCHURE

KnowledgeENTERPRISE FAST TRACK YOUR ACCESS TO BIG DATA WITH ANGOSS ADVANCED ANALYTICS ON SPARK. Advanced Analytics on Spark BROCHURE FAST TRACK YOUR ACCESS TO BIG DATA WITH ANGOSS ADVANCED ANALYTICS ON SPARK Are you drowning in Big Data? Do you lack access to your data? Are you having a hard time managing Big Data processing requirements?

More information

Cloudera Data Science and Machine Learning. Robin Harrison, Account Executive David Kemp, Systems Engineer. Cloudera, Inc. All rights reserved.

Cloudera Data Science and Machine Learning. Robin Harrison, Account Executive David Kemp, Systems Engineer. Cloudera, Inc. All rights reserved. Cloudera Data Science and Machine Learning Robin Harrison, Account Executive David Kemp, Systems Engineer 1 This is the age of machine learning. Data volume NO Machine Learning Machine Learning 1950s 1960s

More information

ABOUT THIS TRAINING: This Hadoop training will also prepare you for the Big Data Certification of Cloudera- CCP and CCA.

ABOUT THIS TRAINING: This Hadoop training will also prepare you for the Big Data Certification of Cloudera- CCP and CCA. ABOUT THIS TRAINING: The world of Hadoop and Big Data" can be intimidating - hundreds of different technologies with cryptic names form the Hadoop ecosystem. This comprehensive training has been designed

More information

Copyright 2015, Oracle and/or its affiliates. All rights reserved.

Copyright 2015, Oracle and/or its affiliates. All rights reserved. Copyright 2015, Oracle and/or its affiliates. All rights reserved. Finding new business potential with Big Data Analytics Carsten Frisch Oracle Business Analytics DOAG 2015 Business Solutions Conference

More information

5th Annual. Cloudera, Inc. All rights reserved.

5th Annual. Cloudera, Inc. All rights reserved. 5th Annual 1 The Essentials of Apache Hadoop The What, Why and How to Meet Agency Objectives Sarah Sproehnle, Vice President, Customer Success 2 Introduction 3 What is Apache Hadoop? Hadoop is a software

More information

Big Data Introduction

Big Data Introduction Big Data Introduction Who we are Experts At Your Service Over 50 specialists in IT infrastructure Certified, experienced, passionate Based In Switzerland 100% self-financed Swiss company Over CHF8 mio.

More information

ORACLE DATA INTEGRATOR ENTERPRISE EDITION

ORACLE DATA INTEGRATOR ENTERPRISE EDITION ORACLE DATA INTEGRATOR ENTERPRISE EDITION Oracle Data Integrator Enterprise Edition delivers high-performance data movement and transformation among enterprise platforms with its open and integrated E-LT

More information

Hortonworks Connected Data Platforms

Hortonworks Connected Data Platforms Hortonworks Connected Data Platforms MASTER THE VALUE OF DATA EVERY BUSINESS IS A DATA BUSINESS EMBRACE AN OPEN APPROACH 2 Hortonworks Inc. 2011 2016. All Rights Reserved Data Drives the Connected Car

More information

Your Top 5 Reasons Why You Should Choose SAP Data Hub INTERNAL

Your Top 5 Reasons Why You Should Choose SAP Data Hub INTERNAL Your Top 5 Reasons Why You Should Choose INTERNAL Top 5 reasons for choosing the solution 1 UNIVERSAL 2 INTELLIGENT 3 EFFICIENT 4 SCALABLE 5 COMPLIANT Universal view of the enterprise and Big Data: Get

More information

Introduction to Big Data(Hadoop) Eco-System The Modern Data Platform for Innovation and Business Transformation

Introduction to Big Data(Hadoop) Eco-System The Modern Data Platform for Innovation and Business Transformation Introduction to Big Data(Hadoop) Eco-System The Modern Data Platform for Innovation and Business Transformation Roger Ding Cloudera February 3rd, 2018 1 Agenda Hadoop History Introduction to Apache Hadoop

More information

BIG DATA and DATA SCIENCE

BIG DATA and DATA SCIENCE Integrated Program In BIG DATA and DATA SCIENCE CONTINUING STUDIES Table of Contents About the Course...03 Key Features of Integrated Program in Big Data and Data Science...04 Learning Path...05 Key Learning

More information

Cask Data Application Platform (CDAP) Extensions

Cask Data Application Platform (CDAP) Extensions Cask Data Application Platform (CDAP) Extensions CDAP Extensions provide additional capabilities and user interfaces to CDAP. They are use-case specific applications designed to solve common and critical

More information

How In-Memory Computing can Maximize the Performance of Modern Payments

How In-Memory Computing can Maximize the Performance of Modern Payments How In-Memory Computing can Maximize the Performance of Modern Payments 2018 The mobile payments market is expected to grow to over a trillion dollars by 2019 How can in-memory computing maximize the performance

More information

Hadoop Course Content

Hadoop Course Content Hadoop Course Content Hadoop Course Content Hadoop Overview, Architecture Considerations, Infrastructure, Platforms and Automation Use case walkthrough ETL Log Analytics Real Time Analytics Hbase for Developers

More information

Apache Hadoop in the Datacenter and Cloud

Apache Hadoop in the Datacenter and Cloud Apache Hadoop in the Datacenter and Cloud The Shift to the Connected Data Architecture Digital Transformation fueled by Big Data Analytics and IoT ACTIONABLE INTELLIGENCE Cloud and Data Center IDMS Relational

More information

MapR: Solution for Customer Production Success

MapR: Solution for Customer Production Success 2015 MapR Technologies 2015 MapR Technologies 1 MapR: Solution for Customer Production Success Big Data High Growth 700+ Customers Cloud Leaders Riding the Wave with Hadoop The Big Data Platform of Choice

More information

Modern Analytics Architecture

Modern Analytics Architecture Modern Analytics Architecture So what is a. Modern analytics architecture? Machine Learning AI Open source Big Data DevOps Cloud In-memory IoT Trends supporting Next-Generation analytics Source: Next-Generation

More information

Insights-Driven Operations with SAP HANA and Cloudera Enterprise

Insights-Driven Operations with SAP HANA and Cloudera Enterprise Insights-Driven Operations with SAP HANA and Cloudera Enterprise Unleash your business with pervasive Big Data Analytics with SAP HANA and Cloudera Enterprise The missing link to operations As big data

More information

EXECUTIVE BRIEF. Successful Data Warehouse Approaches to Meet Today s Analytics Demands. In this Paper

EXECUTIVE BRIEF. Successful Data Warehouse Approaches to Meet Today s Analytics Demands. In this Paper Sponsored by Successful Data Warehouse Approaches to Meet Today s Analytics Demands EXECUTIVE BRIEF In this Paper Organizations are adopting increasingly sophisticated analytics methods Analytics usage

More information

Bringing the Power of SAS to Hadoop Title

Bringing the Power of SAS to Hadoop Title WHITE PAPER Bringing the Power of SAS to Hadoop Title Combine SAS World-Class Analytics With Hadoop s Low-Cost, Distributed Data Storage to Uncover Hidden Opportunities ii Contents Introduction... 1 What

More information

Outline of Hadoop. Background, Core Services, and Components. David Schwab Synchronic Analytics Nov.

Outline of Hadoop. Background, Core Services, and Components. David Schwab Synchronic Analytics   Nov. Outline of Hadoop Background, Core Services, and Components David Schwab Synchronic Analytics https://synchronicanalytics.com Nov. 1, 2018 Hadoop s Purpose and Origin Hadoop s Architecture Minimum Configuration

More information

Cask Data Application Platform (CDAP)

Cask Data Application Platform (CDAP) Cask Data Application Platform (CDAP) CDAP is an open source, Apache 2.0 licensed, distributed, application framework for delivering Hadoop solutions. It integrates and abstracts the underlying Hadoop

More information

Big Data & Hadoop Advance

Big Data & Hadoop Advance Course Durations: 30 Hours About Company: Course Mode: Online/Offline EduNextgen extended arm of Product Innovation Academy is a growing entity in education and career transformation, specializing in today

More information

Enterprise Analytics Accelerating Your Path to Value with an Open Analytics Platform

Enterprise Analytics Accelerating Your Path to Value with an Open Analytics Platform Enterprise Analytics Accelerating Your Path to Value with an Open Analytics Platform Federico Pozzi @fedealbpozzi Mathias Coopmans @macoopma Characteristics of a badly managed platform No clear data

More information

PNDA.io: when big data and OSS collide

PNDA.io: when big data and OSS collide .io: when big data and OSS collide Simplified OSS / BSS Stack [Build Slide] Order Customer Bills and Reports Order Mgmt BSS Billing and Reporting Orchestration is responsible for service provisioning and

More information

Accelerating Your Big Data Analytics. Jeff Healey, Director Product Marketing, HPE Vertica

Accelerating Your Big Data Analytics. Jeff Healey, Director Product Marketing, HPE Vertica Accelerating Your Big Data Analytics Jeff Healey, Director Product Marketing, HPE Vertica Recent Waves of Disruption IT Infrastructu re for Analytics Data Warehouse Modernization Big Data/ Hadoop Cloud

More information

Apache Spark 2.0 GA. The General Engine for Modern Analytic Use Cases. Cloudera, Inc. All rights reserved.

Apache Spark 2.0 GA. The General Engine for Modern Analytic Use Cases. Cloudera, Inc. All rights reserved. Apache Spark 2.0 GA The General Engine for Modern Analytic Use Cases 1 Apache Spark Drives Business Innovation Apache Spark is driving new business value that is being harnessed by technology forward organizations.

More information

Big Data Application Engineer/ Developer. Specialization in Apache Spark, Kafka, Airflow, HBase

Big Data Application Engineer/ Developer. Specialization in Apache Spark, Kafka, Airflow, HBase BIG DATA COURSE Big Data Application Engineer/ Developer Specialization in Apache Spark, Kafka, Airflow, HBase In Exclusive Association with 21,347+ Participants 10,000+ Brands 1200+ Trainings 45+ Countries

More information

Data Analytics. Nagesh Madhwal Client Solutions Director, Consulting, Southeast Asia, Dell EMC

Data Analytics. Nagesh Madhwal Client Solutions Director, Consulting, Southeast Asia, Dell EMC Data Analytics Nagesh Madhwal Client Solutions Director, Consulting, Southeast Asia, Dell EMC Last 15 years IT-centric Traditional Analytics Traditional Applications Rigid Infrastructure Internet Next

More information

Machine Learning For Enterprise: Beyond Open Source. April Jean-François Puget

Machine Learning For Enterprise: Beyond Open Source. April Jean-François Puget Machine Learning For Enterprise: Beyond Open Source April 2018 Jean-François Puget Use Cases for Machine/Deep Learning Cyber Defense Drug Discovery Fraud Detection Aeronautics IoT Earth Monitoring Advanced

More information

HDInsight - Hadoop for the Commoner Matt Stenzel Data Platform Technical Specialist

HDInsight - Hadoop for the Commoner Matt Stenzel Data Platform Technical Specialist HDInsight - Hadoop for the Commoner 10-1-2016 Matt Stenzel Data Platform Technical Specialist SQL Saturday #557 Thank you Sponsors! Please visit the sponsors and enter their end-of-day raffles. Event After

More information

Edge Analytics for IoT Device Intelligence

Edge Analytics for IoT Device Intelligence Edge Analytics for IoT Device Intelligence 1. IoT Trends 2. IoT Analytics 3. Edge Analytics Platform: Kanga 4. Future Direction 2017. 3. 10 IoT Trends - Business/Technology (1/3) Google : IoT Solution

More information

Cloudera, Inc. All rights reserved.

Cloudera, Inc. All rights reserved. 1 Data Analytics 2018 CDSW Teamplay und Governance in der Data Science Entwicklung Thomas Friebel Partner Sales Engineer tfriebel@cloudera.com 2 We believe data can make what is impossible today, possible

More information

BIG DATA PROCESSING A DEEP DIVE IN HADOOP/SPARK & AZURE SQL DW

BIG DATA PROCESSING A DEEP DIVE IN HADOOP/SPARK & AZURE SQL DW BIG DATA PROCESSING A DEEP DIVE IN HADOOP/SPARK & AZURE SQL DW TOPICS COVERED 1 2 Fundamentals of Big Data Platforms Major Big Data Tools Scaling Up vs. Out SCALE UP (SMP) SCALE OUT (MPP) + (n) Upgrade

More information

NFLABS SIMPLIFYING BIG DATA. Real &me, interac&ve data analy&cs pla4orm for Hadoop

NFLABS SIMPLIFYING BIG DATA. Real &me, interac&ve data analy&cs pla4orm for Hadoop NFLABS SIMPLIFYING BIG DATA Real &me, interac&ve data analy&cs pla4orm for Hadoop Did you know? Founded in 2011, NFLabs is an enterprise software company working on developing solutions to simplify big

More information

Azure ML Data Camp. Ivan Kosyakov MTC Architect, Ph.D. Microsoft Technology Centers Microsoft Technology Centers. Experience the Microsoft Cloud

Azure ML Data Camp. Ivan Kosyakov MTC Architect, Ph.D. Microsoft Technology Centers Microsoft Technology Centers. Experience the Microsoft Cloud Microsoft Technology Centers Microsoft Technology Centers Experience the Microsoft Cloud Experience the Microsoft Cloud ML Data Camp Ivan Kosyakov MTC Architect, Ph.D. Top Manager IT Analyst Big Data Strategic

More information

Sr. Sergio Rodríguez de Guzmán CTO PUE

Sr. Sergio Rodríguez de Guzmán CTO PUE PRODUCT LATEST NEWS Sr. Sergio Rodríguez de Guzmán CTO PUE www.pue.es Hadoop & Why Cloudera Sergio Rodríguez Systems Engineer sergio@pue.es 3 Industry-Leading Consulting and Training PUE is the first Spanish

More information

Common Customer Use Cases in FSI

Common Customer Use Cases in FSI Common Customer Use Cases in FSI 1 Marketing Optimization 2014 2014 MapR MapR Technologies Technologies 2 Fortune 100 Financial Services Company 104M CARD MEMBERS 3 Financial Services: Recommendation Engine

More information

Managing explosion of data. Cloudera, Inc. All rights reserved.

Managing explosion of data. Cloudera, Inc. All rights reserved. Managing explosion of data 1 Customer experience expectations are converging on the brand, not channel Consistent across all channels and lines of business Contextualized to present location and circumstances

More information

Azure Data Analytics & Machine Learning Seminar. Daire Cunningham: BI Practice Area Manager

Azure Data Analytics & Machine Learning Seminar. Daire Cunningham: BI Practice Area Manager Azure Data Analytics & Machine Learning Seminar Daire Cunningham: BI Practice Area Manager AGENDA 09:00 AM 09:30 AM Registration & Refreshments 09.30AM 10:00 AM 10:00 AM 10:30 AM Welcome & Keynote, Ger

More information

D a t a J u g g l i n g a t S k y B e t t i n g a n d G a m i n g. A b r i e f l o o k i n s i d e t h e D a t a S c i e n c e t o o l b o x

D a t a J u g g l i n g a t S k y B e t t i n g a n d G a m i n g. A b r i e f l o o k i n s i d e t h e D a t a S c i e n c e t o o l b o x D a t a J u g g l i n g a t S k y B e t t i n g a n d G a m i n g A b r i e f l o o k i n s i d e t h e D a t a S c i e n c e t o o l b o x Intro to SB&G 100% online sports betting and gaming operator

More information

Cloud Based Analytics for SAP

Cloud Based Analytics for SAP Cloud Based Analytics for SAP Gary Patterson, Global Lead for Big Data About Virtustream A Dell Technologies Business 2,300+ employees 20+ data centers Major operations in 10 countries One of the fastest

More information

Business is being transformed by three trends

Business is being transformed by three trends Business is being transformed by three trends Big Cloud Intelligence Stay ahead of the curve with Cortana Intelligence Suite Business apps People Custom apps Apps Sensors and devices Cortana Intelligence

More information

Intel Public Sector 3

Intel Public Sector 3 Intel technologies features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. No computer

More information

Analytics for the NFV World with PNDA.io

Analytics for the NFV World with PNDA.io for the NFV World with.io Speaker Donald Hunter Principal Engineer in the Chief Technology and Architecture Office at Cisco. Lead the MEF OpenLSO project which uses.io as a reference implementation for

More information

GET MORE VALUE OUT OF BIG DATA

GET MORE VALUE OUT OF BIG DATA GET MORE VALUE OUT OF BIG DATA Enterprise data is increasing at an alarming rate. An International Data Corporation (IDC) study estimates that data is growing at 50 percent a year and will grow by 50 times

More information

Simplifying the Process of Uploading and Extracting Data from Apache Hadoop

Simplifying the Process of Uploading and Extracting Data from Apache Hadoop Simplifying the Process of Uploading and Extracting Data from Apache Hadoop Rohit Bakhshi, Solution Architect, Hortonworks Jim Walker, Director Product Marketing, Talend Page 1 About Us Rohit Bakhshi Solution

More information

Towards Seamless Integration of Data Analytics into Existing HPC Infrastructures

Towards Seamless Integration of Data Analytics into Existing HPC Infrastructures Towards Seamless Integration of Data Analytics into Existing HPC Infrastructures Michael Gienger High Performance Computing Center Stuttgart (HLRS), Germany Redmond May 11, 2017 :: 1 Outline Introduction

More information

SOLUTION SHEET Hortonworks DataFlow (HDF ) End-to-end data flow management and streaming analytics platform

SOLUTION SHEET Hortonworks DataFlow (HDF ) End-to-end data flow management and streaming analytics platform SOLUTION SHEET Hortonworks DataFlow (HDF ) End-to-end data flow management and streaming analytics platform CREATE STREAMING ANALYTICS APPLICATIONS IN MINUTES WITHOUT WRITING CODE The increasing growth

More information

New Approach for scheduling tasks and/or jobs in Big Data Cluster

New Approach for scheduling tasks and/or jobs in Big Data Cluster New Approach for scheduling tasks and/or jobs in Big Data Cluster IT College, Chairperson of MS Dept. Agenda Introduction What is Big Data? The 4 characteristics of Big Data V4s Different Categories of

More information

Achieving Agility and Flexibility in Big Data Analytics with the Urika -GX Agile Analytics Platform

Achieving Agility and Flexibility in Big Data Analytics with the Urika -GX Agile Analytics Platform Achieving Agility and Flexibility in Big Data Analytics with the Urika -GX Agile Analytics Platform Analytics R&D and Product Management Document Version 1 WP-Urika-GX-Big-Data-Analytics-0217 www.cray.com

More information

Cloudera Enterprise Data Hub Reference Architecture for Oracle Cloud Infrastructure Deployments O R A C L E W H I T E P A P E R J U N E

Cloudera Enterprise Data Hub Reference Architecture for Oracle Cloud Infrastructure Deployments O R A C L E W H I T E P A P E R J U N E Cloudera Enterprise Data Hub Reference Architecture for Oracle Cloud Infrastructure Deployments O R A C L E W H I T E P A P E R J U N E 2 0 1 8 Disclaimer The following is intended to outline our general

More information

Berkeley Data Analytics Stack (BDAS) Overview

Berkeley Data Analytics Stack (BDAS) Overview Berkeley Analytics Stack (BDAS) Overview Ion Stoica UC Berkeley UC BERKELEY What is Big used For? Reports, e.g., - Track business processes, transactions Diagnosis, e.g., - Why is user engagement dropping?

More information

Spark, Hadoop, and Friends

Spark, Hadoop, and Friends Spark, Hadoop, and Friends (and the Zeppelin Notebook) Douglas Eadline Jan 4, 2017 NJIT Presenter Douglas Eadline deadline@basement-supercomputing.com @thedeadline HPC/Hadoop Consultant/Writer http://www.basement-supercomputing.com

More information

Amsterdam. (technical) Updates & demonstration. Robert Voermans Governance architect

Amsterdam. (technical) Updates & demonstration. Robert Voermans Governance architect (technical) Updates & demonstration Robert Voermans Governance architect Amsterdam Please note IBM s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice

More information

Real-time IoT Big Data-in-Motion Analytics Case Study: Managing Millions of Devices at Country-Scale

Real-time IoT Big Data-in-Motion Analytics Case Study: Managing Millions of Devices at Country-Scale Real-time IoT Big Data-in-Motion Analytics Case Study: Managing Millions of Devices at Country-Scale Real-time IoT Big Data-in-Motion Analytics Case Study: Managing Millions of Devices at Country-Scale

More information

Applying Automated Methods of Managing Test and Evaluation Processes

Applying Automated Methods of Managing Test and Evaluation Processes Applying Automated Methods of Managing Test and Evaluation Processes Chad Stevens, CTEP Presented to the ITEA 35th International T&E Symposium December 2018 1 Outline Purpose Background and Athena Usage

More information

SAP Predictive Analytics Suite

SAP Predictive Analytics Suite SAP Predictive Analytics Suite Tania Pérez Asensio Where is the Evolution of Business Analytics Heading? Organizations Are Maturing Their Approaches to Solving Business Problems Reactive Wait until a problem

More information

Making Data Science Simple

Making Data Science Simple Making Data Science Simple IBM Code Tech Talk Oct 18 th 2017 https://developer.ibm.com/code/videos/tech-talk-replay-making-datascience-simple/ IBM Code Tech Talk Making Data Science Simple David Taieb

More information

Data - tools for data integration, access, preparation, discovery, and data streaming.

Data - tools for data integration, access, preparation, discovery, and data streaming. Licensed for distribution Summary The unifying concept that defines FICO and its substantial technology and solutions stack is Decision Management. This term has not yet become mainstream - but it will.

More information

Monetizing the Lake. Kirk Haslbeck, Hortonworks Dan Kernaghan, Pitney Bowes

Monetizing the Lake. Kirk Haslbeck, Hortonworks Dan Kernaghan, Pitney Bowes Monetizing the Lake Kirk Haslbeck, Hortonworks Dan Kernaghan, Pitney Bowes Hadoop is Lower Cost and more Scalable 14000 Cost Per Terabyte 12000 10000 8000 6000 4000 2000 0 HDP Oracle X Teradata Netezza

More information

Taking Advantage of Cloud Elasticity and Flexibility

Taking Advantage of Cloud Elasticity and Flexibility Taking Advantage of Cloud Elasticity and Flexibility Fred Koopmans Sr. Director of Product Management 1 Public cloud adoption is surging 2 Cloudera customers are leading the way 3 Hadoop was born for the

More information

CASE STUDY Delivering Real Time Financial Transaction Monitoring

CASE STUDY Delivering Real Time Financial Transaction Monitoring CASE STUDY Delivering Real Time Financial Transaction Monitoring Steve Wilkes Striim Co-Founder and CTO Background Customer is a US based Payment Systems Provider Large Network of ATM and Cashier Operated

More information

SOLUTION SHEET End to End Data Flow Management and Streaming Analytics Platform

SOLUTION SHEET End to End Data Flow Management and Streaming Analytics Platform SOLUTION SHEET End to End Data Flow Management and Streaming Analytics Platform CREATE STREAMING ANALYTICS APPLICATIONS IN MINUTES WITHOUT WRITING CODE The increasing growth of data, especially data-in-motion,

More information

The Applicability of HPC for Cyber Situational Awareness

The Applicability of HPC for Cyber Situational Awareness The Applicability of HPC for Cyber Situational Awareness Leslie C. Leonard, PhD August 17, 2017 Outline HPCMP Overview Cyber Situational Awareness (SA) Initiative Cyber SA Research Challenges Advanced

More information

Analytics in Action transforming the way we use and consume information

Analytics in Action transforming the way we use and consume information Analytics in Action transforming the way we use and consume information Big Data Ecosystem The Data Traditional Data BIG DATA Repositories MPP Appliances Internet Hadoop Data Streaming Big Data Ecosystem

More information

Pentaho 8.0 and Beyond. Matt Howard Pentaho Sr. Director of Product Management, Hitachi Vantara

Pentaho 8.0 and Beyond. Matt Howard Pentaho Sr. Director of Product Management, Hitachi Vantara Pentaho 8.0 and Beyond Matt Howard Pentaho Sr. Director of Product Management, Hitachi Vantara Safe Harbor Statement The forward-looking statements contained in this document represent an outline of our

More information

ADVANCED ANALYTICS & IOT ARCHITECTURES

ADVANCED ANALYTICS & IOT ARCHITECTURES ADVANCED ANALYTICS & IOT ARCHITECTURES Presented by: Orion Gebremedhin Director of Technology, Data & Analytics Marc Lobree National Architect, Advanced Analytics EDW THE RIGHT TOOL FOR THE RIGHT WORKLOAD

More information

SAS FORUM RUSSIA Welcome

SAS FORUM RUSSIA Welcome SAS FORUM RUSSIA 2016 Welcome SAS Technology Directions Anand Chitale Senior Manager, SAS Global Technology Practice C opyr i g ht 2016, SAS Ins titut e Inc. All rights res er ve d. PURPOSE & LEGAL DISCLAIMER

More information

ETL on Hadoop What is Required

ETL on Hadoop What is Required ETL on Hadoop What is Required Keith Kohl Director, Product Management October 2012 Syncsort Copyright 2012, Syncsort Incorporated Agenda Who is Syncsort Extract, Transform, Load (ETL) Overview and conventional

More information

Insights to HDInsight

Insights to HDInsight Insights to HDInsight Why Hadoop in the Cloud? No hardware costs Unlimited Scale Pay for What You Need Deployed in minutes Azure HDInsight Big Data made easy Enterprise Ready Easier and more productive

More information

Operational Hadoop and the Lambda Architecture for Streaming Data

Operational Hadoop and the Lambda Architecture for Streaming Data Operational Hadoop and the Lambda Architecture for Streaming Data 2015 MapR Technologies 2015 MapR Technologies 1 Topics From Batch to Operational Workloads on Hadoop Streaming Data Environments The Lambda

More information

Analytics in the Cloud, Cross Functional Teams, and Apache Hadoop is not a Thing Ryan Packer, Bank of New Zealand

Analytics in the Cloud, Cross Functional Teams, and Apache Hadoop is not a Thing Ryan Packer, Bank of New Zealand Paper 2698-2018 Analytics in the Cloud, Cross Functional Teams, and Apache Hadoop is not a Thing Ryan Packer, Bank of New Zealand ABSTRACT Digital analytics is no longer just about tracking the number

More information

Pentaho 8.0 Overview. Pedro Alves

Pentaho 8.0 Overview. Pedro Alves Pentaho 8.0 Overview Pedro Alves Safe Harbor Statement The forward-looking statements contained in this document represent an outline of our current intended product direction. It is provided for information

More information

TechArch Day Digital Decoupling. Oscar Renalias. Accenture

TechArch Day Digital Decoupling. Oscar Renalias. Accenture TechArch Day 2018 Digital Decoupling Oscar Renalias Accenture !"##$ oscar.renalias@acenture.com @oscarrenalias https://www.linkedin.com/in/oscarrenalias/ https://github.com/accenture THE ERA OF THE BIG

More information

MapR Pentaho Business Solutions

MapR Pentaho Business Solutions MapR Pentaho Business Solutions The Benefits of a Converged Platform to Big Data Integration Tom Scurlock Director, WW Alliances and Partners, MapR Key Takeaways 1. We focus on business values and business

More information

Two offerings which interoperate really well

Two offerings which interoperate really well Microsoft Two offerings which interoperate really well On-premises Cortana Intelligence Suite SQL Server 2016 Cloud IAAS Enterprise PAAS Cloud Storage Service 9 SQL Server 2016: Everything built-in built-in

More information

DataAdapt Active Insight

DataAdapt Active Insight Solution Highlights Accelerated time to value Enterprise-ready Apache Hadoop based platform for data processing, warehousing and analytics Advanced analytics for structured, semistructured and unstructured

More information

Power BI for Data Science Integration and exploration capabilities

Power BI for Data Science Integration and exploration capabilities Power BI for Data Science Integration and exploration capabilities J AV I E R G U I L L E N C H A R LOT T E B I G R O U P C H A R LOT T E, N C - 2018 Power BI for Data Science exploration Different mindset

More information

Adobe and Hadoop Integration

Adobe and Hadoop Integration Predictive Behavioral Analytics Adobe and Hadoop Integration JANUARY 2016 SYNTASA Copyright 1.0 Introduction For many years large enterprises have relied on the Adobe Marketing Cloud for capturing and

More information