Emerging Technologies Innovations and Evolutions in BI, Analytics, and Data Warehousing

Transcription

1 Emerging Technologies Innovations and Evolutions in BI, Analytics, and Data Warehousing By TDWI Research Directors: Philip Russom, David Stodder, and Fern Halper October 14, 2015

2 TDWI would like to thank the following companies for sponsoring the 2015 TDWI Best Practices research report: Emerging Technologies For Business Intelligence, Analytics, and Data Warehousing This presentation is based on the findings of that report. * STAY TUNED * At the end of this webinar, learn how to download a free copy of the report.

3 Today s Agenda Introduction to Emerging Technologies and Methods (ETMs) Philip Russom, David Stodder, and Fern Halper ETMs for BI David Stodder ETMs for Analytics Fern Halper ETMs for Data Mgt Philip Russom David Stodder Philip Russom Fern Halper Top Ten Priorities for Emerging Technologies and Methods Philip Russom, David Stodder, and Fern Halper Question-and-Answer Period

4 ETMs tend to fall into Three Categories Based on their functions within common technology stacks ETMs for Business Intelligence (BI) David Stodder TDWI Research Director for BI ETMs for Analytics Fern Halper TDWI Research Director for Analytics ETMs for Data Warehousing (DW) Philip Russom TDWI Research Director for Data Mgt

5 Other Attributes of ETMs Some ETMs reach into multiple layers of BI/DW/analytics tech stack. These include variations of clouds and software-as-a service (SaaS), as well as advancements in development best practices based on agile and lean methods. Many ETMs span multiple layers of a technology stack. For example, in-database analytics combines innovations in both analytics and database management. Similarly, real-time BI or stream analytics get their speed from a combination of ETMs, such as in-memory functions, solid-state drives, and columnar databases. New ETMs coexist with older tools types and practices. New tools for visualization and advanced analytics are specialized, and therefore complement but don t replace general purpose BI platforms. New specialized analytic databases, Hadoop, and Cassandra complement & extend mature relational database management systems.

6 ETMs are Important to People and Organizations In your own words, why are ETMs important (or not) to your organization? In order to stay competitive, we need to be in tune with the new technologies in the market that are going to help us move our business forward. IT director, Education, USA Human actions are being captured like never before. And machine generated data is only adding different dimensions to the same. It is important to visualize trends and derive meaningful insights for the business. IT specialist, Internet, Asia Our organization needs to embrace ETMs, because they complement or improve the value that a traditional data warehouse can provide. Senior business systems analyst, Retail, USA New business operating models are emerging, and they cannot be implemented without new technologies. The more data you can access and the faster you can process it, the better services you can offer. IT specialist, Insurance, Europe

7 BENEFITS of ETMs BARRIERS to ETMs ETMs help meet or beat the competition. ETMs extend BI, analytics, and DW. ETMs automation can improve process outcomes. ETMs can be a positive response to change. ETMs spur innovation. The leading barrier to ETM adoption is the state of IT. Budgetary issues are the second most common barrier to ETMs. Like anything in IT, ETM adoption is unlikely without a business case. Innovation culture isn t as common as it should be. 7 In priority order, based on survey responses. See Figures 6 and 7 in report.

8 Speaker: DAVID STODDER Overview of ETMs for Business Intelligence New Technology Adoption and Satisfaction User Story: Primatics actively scouts for ETMs that provide a fundamentally better way of achieving desired goals, especially to meet emerging customer needs. Cary Moore, Director of BI, Primatics Self-Service BI and Analytics User Story: We are in urgent need of effective planning for the workforce of the future; they tend to be more data-driven. Michael Greene, BI Manager, Buncombe County, NC Hadoop Access from BI and Analytics Tools Gaining value from data s variety Leading edge: Mobile BI, Embedded BI, and Monetization

9 New Technology Adoption and Satisfaction Moderate satisfaction with rate of new technology adoption; only 10% said their users are very satisfied; 46% somewhat satisfied, 41% unsatisfied Most participants organizations are not at current software releases

10 Tech Needs for Self-Service BI and Analytics - Fueling selfservice: 55% said their users are dissatisfied with slow development and deployment - Reducing dependence on IT: Increasing users selfreliance is a top priority for many organizations - Heating up: Selfservice data preparation

11 Hadoop Access from BI and Analytics Tools Hadoop: The land of data lakes is a wellspring of emerging technologies - SQL and NoSQL: multiplying ways to store, access, and manage big data - How to access: Improvements emerging; less custom code - Spark SQL, SQL-on- Hadoop: More interactive query capabilities

12 Mobile BI, Embedded BI, and Monetizing Data Looking ahead at three trends Mobile ubiquity: Yet BI apps have been slow to expand due to security, authentication, and performance concerns 45% are implementing dashboards on mobile, often for KPIs ; 28% are doing more advanced report authoring, filtering, and distribution Embedding BI and analytics: 45% said this is very important; trend toward tightening integration, especially for real-time, actionable intelligence Enriching business value: Using BI ETMs to monetize data assets:

13 Speaker: FERN HALPER Overview of ETMs for Analytics Evolving data sets and analytics Analytics hits the mainstream The Internet of Things The Cloud for Data Management and Analytics

14 Evolving data sets and analytics Now - structured data Planned - disparate sources (n=344)

15 Analytics hits the mainstream Now - Dashboards and even predictive! Planned - Advanced (n=332)

16 The Internet of Things A network of interconnected devices that can send and receive data over the internet Use cases are wide and growing Usage doubling if users stick to their plans (n=303)

17 Cloud for data management and analytics Interest is (finally) growing in public cloud for analytics Data warehouses in the cloud Cloud for analytics Cloud for data reduction And more

18 Speaker: PHILIP RUSSOM Overview of ETMs for Data Warehousing Trends in ETMs, relative to data warehousing (DW) and data management (DM) Adoption of ETMs for DW/DM ETMs in use today for DW/DM Special Case: ETMs for securing data User-centric security vs Data-centric security

19 Adoption of ETMs for DW/DM Over next three years, expect increased use of ETMs for real-time DW, Hadoop, streams, self service, data prep, clouds, and SaaS Other growth areas (not charted): in-database analytics, NoSQL, data federation and virtualization, in-memory databases, event processing Based on 344 survey respondents. See Figure 17 in report.

20 ETMs in use today for DW/DM Some ETMs have emerged so much as to become common. Service-oriented architecture (51% use it today; 21% more in three years), data warehouse appliances (45%; 23%), and in-memory databases (42%; 29%). Others have proliferated to a degree, but still have room for growth. Data virtualization (40% use it today; 32% more in three years), analytic DBMSs (40%; 31%), in-database analytics (37%; 34%), and solid-state drives (37%; 26%). Some ETMs have a minimal presence in DW environments today. This includes a third of the ETMs tested by the survey, namely SQL off Hadoop, NoSQL DBMSs, stream data processing, SQL on Hadoop, realtime DW, clouds for DW, and SaaS for DW. 20% to 25% of respondents are using these today, yet an additional 33% to 39% say they will adopt them within three years. Hence, these ETMs are a bit rare today, but will soon be more widely adopted.

21 SPECIAL CASE ETMs for Securing Data User-centric security is well established Authentication, authorization, auditing Data-centric security should be used more Encryption Data at rest or in motion Masking Tokenization

22 Top 10 Priorities for Emerging Technologies & Methods These are recommendations, requirements, or rules that can guide you. 1. Adopt ETMs for the business benefits. 2. Understand your organization s goals and map them to potential ETMs. 3. Know the hurdles so you can leap over them. 4. Consider the organizational implications of emerging technologies. 5. Keep an open mind. 6. New skills may be needed. 7. Focus on how ETMs can improve agility with data and analytics. 8. Give users the freedom to personalize their BI & analytics experiences. 9. Evaluate whether ETMs for BI and analytics could create opportunities to monetize data assets. 10. Don t expect the new ETMs to replace many older systems.

23 Download a free copy of the report that this Webinar is based on Download the report in a PDF file at: tdwi.org/bpreports Feel free to distribute the PDF file of any TDWI Best Practices Report

24 Q & David Stodder A Philip Russom TDWI Research Director for Data Management Philip Russom Fern Halper prussom@tdwi.org David Stodder TDWI Research Director for Business Intelligence dstodder@tdwi.org Fern Halper TDWI Research Director for Advanced Analytics fhalper@tdwi.org