Azure Data Factory Hybrid data integration, at global scale. Erika Harris Senior Program Manager AzureCAT

Transcription

1 Azure Data Factory Hybrid data integration, at global scale Erika Harris Senior Program Manager AzureCAT

2 Data Cloud AI

3 There are barriers to getting value from data Data silos Incongruent data types Complexity of solutions Multi cloud enviro nment Rising costs

4 Derive real value from your data Data silos Incongruent data types Per formance constraints Complexity of solutions Rising costs One hub for all data Suppor t for diverse types of data Unlimited data scale Familiar tools and ecosystem Lower TCO On-premises, hybrid, Azure

5 A Z U R E D A T A F A C T O R Y Hybrid data integration, at global scale P R O D U C T I V E H Y B R I D S C A L A B L E T R U S T E D Drag & Drop UI Codeless Data Movement Orchestrate where your data lives Lift SSIS packages to Azure Serverless scalability with no infrastructure to manage Certified compliant Data Movement

6 A Z U R E D A T A F A C T O R Y Hybrid data integration, at global scale LOB Integrate via Azure Data Factory CRM INGEST STORE PREP & TRANSFORM MODEL & SERVE Cloud Graph Image VNet Social Data orchestration and monitoring Data lake and storage Hadoop/Spark and machine learning Data warehouse & Data lake Apps and Insights IoT On-premise

7

8 A Z U R E D A T A F A C T O R Y Modernize your enterprise data warehouse at scale Orchestrate with Azure Data Factory INGEST STORE PREP & TRAIN MODEL & SERVE On-premises data Oracle, SQL, Teradata, fileshares, SAP Azure Databricks Cloud data Azure, AWS, GCP SaaS data Azure Data Factory Azure Blob Storage Polybase Azure SQL Data Warehouse Azure Analysis Services Power BI Salesforce, Dynamics Microsoft Azure also supports other Big Data services like Azure HDInsight, Azure SQL Database and Azure Data Lake to allow customers to tailor the above architecture to meet their unique needs.

9 A Z U R E D A T A F A C T O R Y Create intelligent data driven applications Orchestrate with Azure Data Factory D A T A T I E R A P P L I C A T I O N T I E R On-premises data Oracle, SQL, Teradata, fileshares, SAP INGEST STORE PREP & TRAIN MODEL & SERVE Cloud data Azure, AWS, GCP Azure Data Factory Azure Blob Storage Azure Databricks Azure Cosmos DB Application Logic SaaS data Salesforce, Dynamics Microsoft Azure also supports other Big Data services like Azure HDInsight, Azure SQL Database and Azure Data Lake to allow customers to tailor the above architecture to meet their unique needs.

10 A Z U R E D A T A F A C T O R Y Lift your SQL Server Integration Services (SSIS) packages to Azure Azure Data Factory SSIS Integration Runtime Cloud data sources SSIS Cloud ETL SQL DB Managed Instance Cloud On-premises VNET On-Premise data sources Microsoft SQL Server Integration Services SQL Server

11 Azure Data Factory PaaS Data Integration DATA DRIVEN APPLICATIONS Author, orchestrate and monitor with Azure Data Factory On-Prem SaaS Apps Public Cloud DATA SCIENCE AND MACHINE LEARNING MODELS ANALYTICAL DASHBOARDS USING POWER BI

12

13 ADF Integration Runtime (IR) Portal Application & SDK Azure Data Factory Service ADF compute environment with multiple capabilities: - Activity dispatch & monitoring - Data movement - SSIS package execution To integrate data flow and control flow across the enterprises hybrid cloud, customer can instantiate multiple IR instances for different network environments: - On premises (similar to DMG in ADF V1) - In public cloud - Inside VNet Bring a consistent provision and monitoring experience across the network environments Self-Hosted IR Data Movement & Activity Dispatch on-prem, Cloud, VNET Azure IR Data Movement & Activity Dispatch In Azure Public Network, SSIS VNET coming soon

14 UX & SDK Azure Data Factory Service On Premises Apps & Data Cloud Apps, Svcs & Data

15 UX & SDK Azure Data Factory Service On Premises Apps & Data Cloud Apps, Svcs & Data

16 HP Inc (Global) Customer 1 HP Prod Firewall Border Customer 1 firewall border 443 Public Internet Border Azure (US West) Data Factory (Orchestration micro-service) HP Hadoop Cluster IR Integration ADF Foo Fabric On-prem On-prem 443 Storage (Azure) 443 SQL Data Warehouse UAM Server

17 Public Internet Border Azure (US West) HP Inc (Global) Customer 2 Data Factory (Orchestration micro-service) HP Customer Prod Firewall 2 firewall Border border 443 Customer 2 vnet HP VNET HP Hadoop Cluster IR Integration ADF Foo Fabric in vnetin VNet Storage (Azure) SQL Data Warehouse expressroute Private circuit UAM Server

18

19 Get started with Azure Data Factory View pricing Hands-on-lab