Azure ML Data Camp. Ivan Kosyakov MTC Architect, Ph.D. Microsoft Technology Centers Microsoft Technology Centers. Experience the Microsoft Cloud

Transcription

1 Microsoft Technology Centers Microsoft Technology Centers Experience the Microsoft Cloud Experience the Microsoft Cloud ML Data Camp Ivan Kosyakov MTC Architect, Ph.D.

2 Top Manager IT Analyst Big Data Strategic Business Unit (3v) Analytics Analytics Business User End User Advanced Data Scientist Analytics Analytics

3 Microsoft Technology Centers Experience the Microsoft Cloud Modern Data Warehouse Business User Customer Top Manager Analyst IT Office 365 Data Scientist Developer Security Officer Active Directory Multi-Factor Authentication Key Vault

4 Cortana Analytics Suite Unparalleled security Hyper scalability Most comprehensive Intelligent by design Preconfigured Solutions Dashboards and Visualizations Machine Learning and Analytics Big Data Store Information Management Face APIs Speech APIs Personal Digital Assistant Cortana Perceptual Intelligence Computer Vision APIs Text Analytics Transform data into intelligent action in the cloud

5 Big Data Flow Business apps Custom apps People Sensors and devices

6 MICROSOFT BIG DATA SOLUTIONS Big Data as part of Cortana Intelligence Data Sources Information Management Big Data Stores Machine Learning and Analytics Intelligence People Data Factory Data Lake Store Machine Learning Cognitive Services Data Catalog SQL Data Warehouse Data Lake Analytics Bot Framework Web Apps Event Hubs HDInsight (Hadoop and Spark) Cortana Apps Mobile Stream Analytics Bots Dashboards & Visualizations Sensors and devices Power BI Automated Systems Data Intelligence Action

7 Reference architecture for Big Data Data Generation Real-Time Storage Processing Visualizations Example reference architecture Analyze Data Stored in Website SQL DB SQL DW SQL Server Analysis Services Machine Learning Marketplace Track realtime data from IOT Suite: collect data from IOT Suite in permanent store (ADLS) Track other data ( Website generating web logs) and store in ADLS Run Machine Learning through R Server for HDInsight to find patterns in data Devices Crawlers Bots Sensors Event Hubs Stream Analytics Spark Streaming for HDInsight Document DB Data Lake Store HDInsight Hive Data Lake Analytics Spark SQL R Server SSRS SharePoint BI Excel BI Show results in BI tools (Power BI) HBase on HDInsight Sqoop Pig Spark MLib Power BI Storm for HDInsight Oozie Datazen Data Factory

8 Microsoft Technology Centers Experience the Microsoft Cloud Big Data Decision Tree Big Data is often described as a solution to the three V s problem, and how we choose right solution depends on which one of these problems we are trying to solve first: Volume: need to store and query hundreds of terabytes of data or more, and the total volume is growing. Processing systems must be scalable to handle increasing volumes of data, typically by scaling out across multiple machines. Velocity: need to collect data at an increasing rate from many new types of devices, from a fastgrowing number of users, and from an increasing number of devices and applications per user. Processing systems must be able to return results within an acceptable timeframe, often almost in realtime. Variety: situation when data do not match any existing data schema semi-structured or unstructured data. Source:

9 ML STUDIO API

10 10

11 Community Commercial R Open SQL Server R Services Windows Red Hat R Server SUSE Hadoop Teradata

12 Cognitive Services Give your solutions a human side

13 Microsoft Technology Centers Experience the Microsoft Cloud Machine Learning Decision Tree Source: