Improve your webinar experience

Transcription

1 Improve your webinar experience Download the presentation Ask questions Turn up your device s volume Rate this presentation Gartner, Inc. and/or its affiliates. All rights reserved. Gartner is a registered trademark of Gartner, Inc. and its affiliates.

2 Ask your questions! Gartner, Inc. and/or its affiliates. All rights reserved. Gartner is a registered trademark of Gartner, Inc. and its affiliates.

3 Download attachments, including the presentation! Gartner, Inc. and/or its affiliates. All rights reserved. Gartner is a registered trademark of Gartner, Inc. and its affiliates.

4 Beyond the Data Warehouse: New Data Management for Analytics Find out more about all of Gartner s upcoming events. Connect with Gartner Rick Greenwald Research Director Connect with Rick 4 years at Gartner 34 years industry experience Rick Greenwald is a data management expert in GITL and covers a number of areas, including operational database issues, data warehousing, high availability, data integration and cloud databases. Mr. Greenwald has published research on deploying databases in the cloud and has spoken at Catalyst and at the Infrastructure, Operations and Data Center Conference in India Gartner, Inc. and/or its affiliates. All rights reserved. Gartner is a registered trademark of Gartner, Inc. and its affiliates.

5 Polling Question 1 of 1 Which of these best describes your organization s level of satisfaction with current analytic deliverables? A. All our needs are being met B. We have good analytics, but not fast enough C. We have rapid delivery of analytics, but not all our needs are met D. Business has stopped going to IT for their analytic needs E. What are analytics? How to participate in our polling If you are in full screen mode click Esc The poll question is on the Vote tab. Please click the box to make your selection. Upon voting you will see the results. Thank you! Q. Polling Question (please choose 1 answer) A. Answer B. Answer C. Answer D. Answer E. Answer Gartner, Inc. and/or its affiliates. All rights reserved.

6 Gartner, Inc. and/or its affiliates. All rights reserved.

7 Key Issues 1. How can you address the increased scope and breadth demanded by DMSA platforms today? 2. Do traditional data warehouse platforms still have a place? 3. How do you build an architecture that leverages the traditional while using innovations? Gartner, Inc. and/or its affiliates. All rights reserved.

8 Key Issues 1. How can you address the increased scope and breadth demanded by DMSA platforms today? 2. Do traditional data warehouse platforms still have a place? 3. How do you build an architecture that leverages the traditional while using innovations? Gartner, Inc. and/or its affiliates. All rights reserved.

9 Questions We Need a New Way to Think About Data Challenges Known Unknown Expanding Understanding and Investigating Foundational Core Innovation and Exploration Establishing Value Gartner, Inc. and/or its affiliates. All rights reserved. Known Source: "Solve Your Data Challenges With the Data Management Infrastructure Model," (G ) Data Unknown

10 The Data Management Infrastructure Model and the LDW Known Unknown Questions Expanding Understanding and Investigating Traditional DW Foundational Core Innovation and Exploration Establishing Value Context Independent Data Warehouse Known Data Unknown Gartner, Inc. and/or its affiliates. All rights reserved.

11 Questions The Data Management Infrastructure Model and the LDW The Logical Data Warehouse Known Unknown Expanding Understanding and Investigating Innovation and Exploration Diverse Data Types Diverse End Users Diverse Contexts Traditional DW Diverse Technologies Foundational Core Establishing Value Diverse Use Cases Context Independent Data Warehouse Gartner, Inc. and/or its affiliates. All rights reserved. Known Data Unknown

12 Questions Known Unknown We Can Use It to Align Roles and Skills Expanding Understanding & Investigating Foundational Core Data Scientists Casual Users Innovation and Exploration Establishing Value Casual (Apprentice) Business Analyst (Journeyman) Engineer (Master) Data Science Analytics Center of Excellence Reports, dashboards Level 1 support, possibly Level 2 Create new reports Needs technical assistance Level 2 and Level 3 Ops process, systems analyst, data architect, reliable tech Modeling theory, graph theory, mathematics, program languages User Distribution 1,000 Source: "Organizing Your Teams for Modern Data and Analytics Deployment," (G ) Known Gartner, Inc. and/or its affiliates. All rights reserved. Data Unknown

13 Questions Known Unknown and to Address Governance Concerns Expanding Understanding & Investigating STOP Innovation and Exploration Governance needs to be applied to data flowing in this direction Strong Governance Light Governance Foundational Core Data can flow this direction with minimal integrity risk Establishing Value Gartner, Inc. and/or its affiliates. All rights reserved. Known Data Unknown

14 Business Questions Known Unknown It Aligns to the LDW SLAs Expanding Understanding Contender & Investigating Innovation Candidate Exploration Foundational Compromise Core Establishing Contender Value Gartner, Inc. and/or its affiliates. All rights reserved. Known Data Unknown

15 Known Unknown Questions and Can Be Used to Map Changing Business Context A Expanding Understanding and Investigating 4 3 Foundational Core A Innovation and Exploration 1 2 Establishing Value A Data Exploration: The question development lab Testing Hypotheses: What are the right data sources to answer my questions? Optimization: Apply structure and optimize for the questions we are asking Exploration on Curated Data: What else can I do with this? ETL Load of Known Data for Known Purpose Known Data Unknown Gartner, Inc. and/or its affiliates. All rights reserved.

16 Key Issues 1. How can you address the increased scope and breadth demanded by DMSA platforms today? 2. Do traditional data warehouse platforms still have a place? 3. How do you build an architecture that leverages the traditional while using innovations? Gartner, Inc. and/or its affiliates. All rights reserved.

17 Does the Data Lake Replace the Data Warehouse? Data warehouses do not have to be in a relational database; data lakes do not have to be in a nonrelational database. Data warehouses are optimized for consistency across aspects of performance, repeatability and integration. The data warehouse was pushed, pulled and prodded into roles that were beyond the mission. because there were other missions! And some were misuse cases. Now, it's time to adapt, evolve and add new things. * At least not yet! Gartner, Inc. and/or its affiliates. All rights reserved.

18 Strategic Planning Assumption By 2020, 30% of data lakes will be built on standard relational technology at equal or lower cost than Hadoop. Why It Will Happen: RDBMSs are the enterprise standard Most RDBMSs support nonrelational data in multiple formats, and can support a schemaon-read approach Not all "native format" data is nonrelational Most data going into data lakes is relational from operational systems RDBMS ecosystem is very mature RDBMSs are not more expensive Why It Won't Happen Rapid ingest of data into schema-on-read platforms is easier than conforming to a relational model Increasing demand for analysis of nonrelational data that does not fit easily (or efficiently) into an RDBMS Gartner, Inc. and/or its affiliates. All rights reserved.

19 DBMS Market Dynamics USD $34.4 Billion Market Pure-Play Hadoop: 1.39% Nonrelational (Including Hadoop): 5.2% Top 5 DBMS Vendors: 87.7% Traditional DBMS Technologies Still Dominate the Market! Source: "Market Share: All Software Markets, Worldwide, 2016," (G ) Gartner, Inc. and/or its affiliates. All rights reserved.

20 Questions Questions Hype vs. Investment Known Unknown Investment Known Unknown Hype Known Data Unknown Known Unknown Data Gartner, Inc. and/or its affiliates. All rights reserved.

21 Questions Hype Exceeds Investment and Does Not Always Convert! Known Unknown Known Hype Investment Unknown Data expectations File Analysis Data as a Service In-DBMS Analytics In-Process HTAP SQL Interfaces to Cloud Object Stores Innovation Trigger Data Catalog Cross-Platform Structured Data Archiving Enterprise Taxonomy and Ontology Management ipaas for Data Integration Peak of Inflated Expectations Years to mainstream adoption: Distributed Ledgers Self-Service Data Preparation Event Stream Processing Multimodel DBMSs Blockchain Point-of-Decision HTAP Data Lakes Graph DBMSs Hadoop SQL Interfaces Wide-Column DBMSs Information Stewardship Applications Spark Operational In-Memory DBMS Application Data Management Key-Value DBMSs Trough of Disillusionment time Data Quality Tools Data Virtualization Database Platform as a Service Slope of Enlightenment less than 2 years 2 to 5 years 5 to 10 years more than 10 years Database Encryption In-Memory Data Grids Content Migration Enterprise Information Archiving Data Integration Tool Suites SaaS Archiving of Messaging Data Metadata Management Solutions Analytical In-Memory DBMS Document Store DBMSs Hadoop Distributions Master Data Management Logical Data Warehouse As of July 2017 Plateau of Productivity obsolete before plateau Gartner, Inc. and/or its affiliates. All rights reserved. Source: "Hype Cycle for Data Management, 2017, 25 July 2017 (G )

22 Most Companies Struggle to Meet an Ever-Increasing Demand for Data Analytics, Especially Demand for New Data Types and Having the Ability to Model Independently What scenario best describes how your organization's data capabilities meet current demand? 38% There is increased demand for new data types and the ability to model it independently 27% Demand for existing and new data types and the ability to model independently is coming from highly skilled users only 10% 9% A data science team designs new solutions and helps IT put them into production for everyone else 8% 7% Our business intelligence and reporting capabilities meet current demand Of late, some users want to model their own data Of late, some users are demanding more data types Base: n = 175 Gartner Research Circle Members Q. What scenario best describes how your organization's data capabilities meet current demand? Gartner, Inc. and/or its affiliates. All rights reserved.

23 Key Issues 1. How can you address the increased scope and breadth demanded by DMSA platforms today? 2. Do traditional data warehouse platforms still have a place? 3. How do you build an architecture that leverages the traditional while using innovations? Gartner, Inc. and/or its affiliates. All rights reserved.

24 Magic Quadrant for DMSA 2018 The Rise of Distributed Data Management Environments LDW Reaches ~15% of Target Market Continued Adoption of Cloud and Hybrid Cloud Deployments Contracting Use Cases for Hadoop-only DMSA Offerings The Rise of China-Based Vendors Source: "Magic Quadrant for Data Management Solutions for Analytics," 13 February 2018 (G ) Gartner, Inc. and/or its affiliates. All rights reserved.

25 Invest in Skills Gartner's 2017 survey on the adoption and deployment of logical data warehouse architectures shows a troubling gap between demand for new data and analytics and the skills available to address that demand. 92% of organizations report that their current data management needs supporting analytics and reporting remain unmet and demand new data and data types. Delivery capacity for prequalified data will almost double by early 2018, forcing the data management architecture and infrastructure design to support the rapid conversion of data science discoveries into production. Low-skilled analysts are demanding access to highly complex data use cases and threaten to overwhelm the credibility of data use in the digital business Gartner, Inc. and/or its affiliates. All rights reserved.

26 Processing Flexibility Infrastructures That Balance Optimization Needs With Self-Service Demand Optimized Self-Service Embedded API/Bots None Casual User Business Experts Citizen Scientists Data Engineers Dashboards BI Platform Analytics Platform Analytics Platform Dimensions/Cubes Analytics Platform Self-Service Data Prep. Processing Languages Semantic Consistency Data Scientists Processing Languages Unobstructed Data Gartner, Inc. and/or its affiliates. All rights reserved.

27 Questions Technologies for the Known and Unknown Are Complementary Known Unknown Expanding Understanding & Investigating Spark In-DBMS Analytics Solutions Traditional DW Operational DBMS Foundational Core Traditional Data Warehouse Real-Time Processing Enterprise Reporting SQL Interfaces to Cloud Object Stores Innovation Hadoop and Exploration Spark Data Lakes Data Virtualization Establishing Self-Service Data Prep. Self-Service Value Data Exploration Context Independent Data Warehouse and the Lines Are Blurry Gartner, Inc. and/or its affiliates. All rights reserved. Known Data Unknown

28 You Can Start "Light" From Either Direction Lake or Warehouse, Simple Tools or Platform! Accessing the Data Faster/Sooner Distributed Process Semantic Integration Bridging Information Silos Enterprise Data Warehouse If you already have an enterprise data warehouse, you can extend it. Data Science Application Development Data Lake Physical Integration If you already have a data lake or Hadoop cluster that needs reuse optimization, you can extend it. Data Sources RDBMS NoSQL Gartner, Inc. and/or its affiliates. All rights reserved.

29 Action Plan Monday Morning: Identify classes of users and use cases present currently in the organization. Catalog different types of data management for analytics present. Determine which existing use cases/users are capable of self-support. Next 90 Days: Develop timelines for when missing user classes and use cases are anticipated for support. Identify platform choices available from existing enterprise vendors and perform a gap analysis for capabilities that are missing. Map your existing systems onto the data management infrastructure model. Many companies find that they already have 70% to 80% or more of the components. Next 12 Months: Target a project to extend an existing warehouse, multiple marts or a data lake with new data and new use cases. Evaluate user experiences to create user qualifications for leveraging different infrastructure components Gartner, Inc. and/or its affiliates. All rights reserved.

30 Recommended Gartner Research Solve Your Data Challenges with the Data Management Infrastructure Model Adam M. Ronthal and Nick Heudecker (G ) Efficiently Evolving Data from the Data Lake to the Data Warehouse Rick Greenwald and Ehtisham Zaidi (G ) Modern Data Management Requires a Balance Between Collecting Data and Connecting to Data Roxane Edjlali and Ted Friedman (G ) Data Management Solutions for Analytics: Current and Future States, 2017 Rick Greenwald and Adam M. Ronthal (G ) Survey Analysis: New Data and New Analytics Are All Mythology Unless You Add Skills Mark A. Beyer and Adam M. Ronthal (G ) For more information, stop by Gartner Research Zone Gartner, Inc. and/or its affiliates. All rights reserved.

31 Ask your questions! Gartner, Inc. and/or its affiliates. All rights reserved. Gartner is a registered trademark of Gartner, Inc. and its affiliates.

32 Gartner Data & Analytics Summit March Orlando, FL Lead with Purpose to Achieve Clarity in a World of Ambiguity At Gartner Data & Analytics Summit 2019 you ll learn how to tackle complex issues and generate new ideas to. At the 2019 conference our 8 track agenda will cover: Business Outcomes & Strategy Doman Analytics How to transition to a data centric architecture Innovation: Be the next disruptor Governance and foundation of trust Persuasive analytics Advanced analytics Leadership Save $325 with code: WEBINAR Find out more about all of Gartner s upcoming events.

33 ThinkCast: The Gartner Podcast Channel Recent Episodes on Data & Analytics gartner.com/podcasts Data Security s Biggest Risk: Not Adapting to Digital Infonomics: Unleash Your Best Asset Digital Runs on Data: Maximize Your Data & Analytics Gartner, Inc. and/or its affiliates. All rights reserved. Gartner is a registered trademark of Gartner, Inc. and its affiliates.

34 Watch upcoming and on-demand Must-see webinars: Learn more insight from the experts Link Data to Business Outcomes Tuesday, 2 October EDT: 11:00 a.m. PDT: 8:00 a.m. BST: 16:00 On-demand: Steps for Creating a Diverse and Data Driven Culture How Augmented Analytics Is Transforming Data and Analytics Solve Your Data Integration Challenges Tip: Download the monthly calendar from the Attachments tab for more webinars Gartner, Inc. and/or its affiliates. All rights reserved. Gartner is a registered trademark of Gartner, Inc. and its affiliates.

35 Before you go Download the presentation Ask questions Rate this presentation Gartner, Inc. and/or its affiliates. All rights reserved. Gartner is a registered trademark of Gartner, Inc. and its affiliates.

36 Let us know what you think! Gartner, Inc. and/or its affiliates. All rights reserved. Gartner is a registered trademark of Gartner, Inc. and its affiliates.