THE IMPORTANCE OF END USER DATA PREPARATION

Similar documents
Analytics empowering clients to see farther & go faster

Competitive Analysis.

Shining the Light on Dark Data Through Data Preparation and Cognitive Analytics

BI SURVEY 14. The world s largest survey of business intelligence software users

Wisdom of Crowds Business Intelligence Market Study

Pg Alliance Data

Business Intelligence & Big Data for the Accounting Firm & Your Clients

#mstrworld. A Deep Dive Into Self-Service Data Discovery In MicroStrategy. Vijay Anand Gianthomas Tewksbury Volpe. #mstrworld

Analytics in the Cloud, Cross Functional Teams, and Apache Hadoop is not a Thing Ryan Packer, Bank of New Zealand

Sentient Enterprise for Dummies Where do we start?

One platform, many benefits. Managing data as a corporate asset. BUSINESS KEY MARKET PLATFORMS

KnowledgeENTERPRISE FAST TRACK YOUR ACCESS TO BIG DATA WITH ANGOSS ADVANCED ANALYTICS ON SPARK. Advanced Analytics on Spark BROCHURE

Analytics and Business Intelligence Vendor Report Prepared for

Analytics in Action transforming the way we use and consume information

SAP Visual Intelligence Executive Summary

Market for BI & Data Analytics

Business Intelligence

BARC Score Enterprise BI and Analytics Platforms

Visual Dashboards for Self-Service BI Cindi Howson

IBM COGNOS BI OVERVIEW

Copyright 2014, Oracle and/or its affiliates. All rights reserved. 2

Mass-Scale, Automated Machine Learning and Model Deployment Using SAS Factory Miner and SAS Decision Manager

TO MARKETING ANALYTICS

Pentaho 8.0 and Beyond. Matt Howard Pentaho Sr. Director of Product Management, Hitachi Vantara

Pentaho Technical Overview. Max Felber Solution Engineer September 22, 2016

GET MORE VALUE OUT OF BIG DATA

1. Search, for finding individual or sets of documents and files

The Importance of good data management and Power BI

Oracle Big Data Discovery The Visual Face of Big Data

Ventana Research Big Data and Information Management Research in 2017

KnowledgeSTUDIO. Advanced Modeling for Better Decisions. Data Preparation, Data Profiling and Exploration

Modern Analytics Strategy & Roadmap

IBM Cognos 10.2 BI Demo

T H E B O T T O M L I N E

Datameer for Data Preparation: Empowering Your Business Analysts

Dig Deeper. Discover Diamonds. Reveal insights to drive business decisions with Pronto iq

Microsoft Big Data. Solution Brief

Effectively Capturing Business Intelligence Data

Tableau in Higher Education

E-guide BI Analytics Tools Buyer s Guide Part 2

Trifacta Data Wrangling for Hadoop: Accelerating Business Adoption While Ensuring Security & Governance

Sample, Products & Methodology

Business Intelligence

By 2020, more than half of major new business processes and systems will incorporate some element of the IoT.

Guide to Modernize Your Enterprise Data Warehouse How to Migrate to a Hadoop-based Big Data Lake

KnowledgeSEEKER POWERFUL SEGMENTATION, STRATEGY DESIGN AND VISUALIZATION SOFTWARE

GUIDEBOOK ADAPTIVE INSIGHTS

Building a Single Source of Truth across the Enterprise An Integrated Solution

# Product Business Objects Qlik View A Product

IBM BI & Analytics Briefing for OCBIG Why Cognos in 2016

EMBED ANALYTICS EVERYWHERE Tomáš Jurczyk

Pentaho 8.0 Overview. Pedro Alves

SAP Analytics Cloud All Analytics. All Users. One Product

Make Business Intelligence Work on Big Data

Information Builders Enterprise Information Management Solution Transforming data into business value Fateh NAILI Enterprise Solutions Manager

Staffing Services Portfolio Advisory Fulfilment

Data Insights with Power BI. Phua Chiu Kiang MVP, SQL Server

Enabling Self-Service BI Success: TimeXtender s Discovery Hub Bridges the Gap Between Business and IT

The Buyer s Guide to Business Intelligence Software

Bot Insight is here. Improve your company s top-and-bottom-line with powerful, real-time RPA Analytics Go be great.

Adobe and Hadoop Integration

INTRODUCING BIRST INFOR S GO-FORWARD BUSINESS INTELLIGENCE SOLUTION

PRODUCT UPDATES APJ PARTNER SUMMIT - BALI. February Software AG. All rights reserved. For internal use only

Automated Data Source Creation using Tableau SDK & Rest API 24 th Oct :00 PM 1:00 PM MCCNO - L2-297

Making Smarter Decisions for Data Discovery Solutions: Evaluating 3 Options

CONFIGMGR DATA SOLUTIONS

SAP Business Analytics Overview and Strategy. Patric Imark, Business Architect EPM SAP (Suisse) AG 29 May 2013

DLT AnalyticsStack. Powering big data, analytics and data science strategies for government agencies

Cognitive Data Governance

PORTFOLIO AND TECHNOLOGY DIRECTION ARMISTEAD SAPP & RANDY GUARD

Building data-driven applications with SAP Data Hub and Amazon Web Services

EXECUTIVE BRIEF. Successful Data Warehouse Approaches to Meet Today s Analytics Demands. In this Paper

Accelerated Data Blending and Analytics for Microsoft Power BI with Alteryx

Adobe and Hadoop Integration

WebFOCUS: Business Intelligence and Analytics Platform

SAP BusinessObjects The Power of Business Intelligence to Transform the Way the World Works

Wisdom of Crowds Business Intelligence Market Study

DataAdapt Active Insight

Bringing the Power of SAS to Hadoop Title

SAP BusinessObjects Innovation and Standardization: The Yin and Yang of BI

The Internet of Things Wind Turbine Predictive Analytics. Fluitec Wind s Tribo-Analytics System Predicting Time-to-Failure

ANALYTICS Business Intelligence / Predictive Analytics / Data integration / Data Management

Changing the direction of Risk and Control Analytics

DATASHEET. Tarams Business Intelligence. Services Data sheet

Datametica. The Modern Data Platform Enterprise Data Hub Implementations. Why is workload moving to Cloud

Changing The Business landscape SAS and Open Source, Better Together. Dr Mark Chia, Head of Advanced Analytics, SAS

CONNECTING THE DOTS FOR BETTER INSIGHT.

SAS & HADOOP ANALYTICS ON BIG DATA

Quinnox BI OBIEE Solution. For more information, visit.

Cloud Integration and the Big Data Journey - Common Use-Case Patterns

Exceed your business with SharePoint Server 2010

BARC Score Data Discovery

Automating Your Hyperion EPM and BI Environments. Tom Tortolani, VP Products January 2010

Qlik Sense. Data Sheet. Transform Your Organization with Analytics

The Industry Leader in Data Warehousing, Big Data Analytics, and Marketing Solutions

MARKETING DARK DATA: 5 DARK DATA SOURCES THAT LEAD TO BETTER MARKETING INSIGHTS WITH IBM WATSON ANALYTICS AND COGNOS ANALYTICS

MicroStrategy 10. Adam Leno Technical Architect NDM Technologies

Select and Implement a Business Intelligence and Analytics Solution

Big Data Analytics met Hadoop

Managing explosion of data. Cloudera, Inc. All rights reserved.

Transcription:

THE IMPORTANCE OF END USER DATA PREPARATION

Welcome! Dan Potter Chief Marketing Officer Howard Dresner Chief Research Officer 2 June 28, 2016

Agenda 3 June 28, 2016

Dresner Advisory Services End User Data Preparation 2016 EndUserDataPrep.report

End User Data Preparation Defined End User Data Preparation is a "self-service" capability for end users to model, prepare, and combine data prior to analysis. This may complement traditional IT-driven Data Quality/ETL processes or may be used independently.

End User Data Preparation Overview Importance of end user data preparation Current user scenarios User requirements Vendor ratings

Technologies and Initiatives Strategic to Business Intelligence 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Dashboards End-user "self service" Data warehousing Advanced visualization Integration with operational processes Data discovery Enterprise planning/budgeting Data mining, advanced algorithms, predictive Embedded BI (contained within an application, portal, etc.) Mobile device support End-user User data "blending" Data Preparation (data mashups) Location intelligence/analytics Collaborative support for group-based analysis In-memory analysis Software-as-a-service and cloud computing Search-based interface Pre-packaged vertical/functional analytical applications Big data (e.g., Hadoop) Ability to write to transactional applications Text analytics Social media analysis (Social BI) Open source software Complex event processing (CEP) Internet of Things (IoT) Cognitive BI (e.g., artificial intelligence-based BI) Critical Very important Important Somewhat important Not important

Importance of End User Data Preparation

Importance of End-User Data Preparation Not important, 5.2% Somewhat important, 14.1% Critical, 37.2% Important, 17.6% Very important, 25.9%

100% Importance of End-User Data Preparation by Function 5 90% 4.5 80% 4 70% 3.5 60% 50% 40% 3 2.5 2 Critical Very important Important Somewhat important Not important Mean 30% 1.5 20% 1 10% 0.5 0% Research and Development (R&D) Information Technology (IT) Sales and Marketing Executive Management Finance 0

100% Importance of End-User Data Preparation by Organization Size 5 90% 4.5 80% 4 70% 3.5 60% 50% 40% 3 2.5 2 Critical Very important Important Somewhat important Not important Mean 30% 1.5 20% 1 10% 0.5 0% 1-100 101-1000 1001-5000 More than 5000 0

Frequency of End User Data Preparation

Frequency of End-User Data Preparation Never, 4.4% Rarely, 8.0% Constantly, 22.7% Occasionally, 23.0% 65% Frequently, 41.9%

100% Frequency of End-User Data Preparation by Function 5 90% 4.5 80% 4 70% 3.5 60% 50% 40% 3 2.5 2 Constantly Frequently Occasionally Rarely Never Mean 30% 1.5 20% 1 10% 0.5 0% Executive Management Finance Information Technology (IT) Research and Development (R&D) Sales and Marketing 0

100% Frequency of End-User Data Preparation by Organization Size 5 90% 4.5 80% 4 70% 3.5 60% 50% 40% 3 2.5 2 Constantly Frequently Occasionally Rarely Never Mean 30% 1.5 20% 1 10% 0.5 0% 1-100 101-1000 1001-5000 More than 5000 0

Effectiveness of Current Approaches

Effectiveness of Current Approach to End-User Data Preparation 35% Totally ineffective, 8.2% Highly effective, 12.6% Somewhat ineffective, 26.5% Somewhat effective, 52.6%

100% Effectiveness of Current Approach to End-User Data Preparation by Organization Size 5 90% 4.5 80% 4 70% 3.5 60% 50% 40% 3 2.5 2 Highly effective Somewhat effective Somewhat ineffective Totally ineffective Mean 30% 1.5 20% 1 10% 0.5 0% 1-100 101-1000 1001-5000 More than 5000 0

60.00% Effectiveness of Current Approach to End-User Data Preparation 2015 to 2016 50.00% 40.00% 30.00% 20.00% 10.00% 0.00% Highly effective Somewhat effective Somewhat ineffective Totally ineffective 2015 2016

End User Data Prep Features

End-User Data Preparation Usability Features 0% 20% 40% 60% 80% 100% Visual interface for users to view and explore inprocess data sets, interactively profile and refine data transformations prior to execution Automated detection of anomalies, outliers, & duplicates Technical expertise/programming is *NOT* required to build/execute data transformation scripts Visual highlighting of relationships between columns, attributes & datasets Automatically generate data transformation code/scripts for execution Support for entire data transformation process in a single application/user interface Automated recommendations for data relationships & keys for combining data across multiple data sets and sources Critical Very important Important Somewhat important Not important

End-User Data Preparation Integration Features 0% 20% 40% 60% 80% 100% Access to file formats (e.g.,log files, CSV, Excel) Access to traditional databases (e.g.,rdbms) Ability to combine data across multiple data sets and sources through joins and merging data Ability to infer metadata by introspecting the data elements Access to Bigdata (e.g., Hadoop) Critical Very important Important Somewhat important Not important

End-User Data Preparation Manipulation Features 0% 20% 40% 60% 80% 100% Ability to aggregate & group data Simple interface for imposing structure on raw data Ability to pivot (convert table to matrix) & reshape (convert matrix to table) data Ability to derive new data features from existing data (text extraction, math expressions, date expressions, etc.) Ability to normalize, standardize & enrich data Support for cutting, merging & replacing of values Ability to manipulate the order of data transformation steps Ability to unnest data (e.g. json/xml parsing) Critical Very important Important Somewhat important Not important

100% End-User Data Preparation Supported Outputs 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Files (e.g., Excel, CSV) Database (e.g., MySQL, MongoDB) Popular (third-party) business intelligence tool formats

End-User Data Preparation Deployment Features 0% 20% 40% 60% 80% 100% Ability to schedule the execution/replay of data transformation processing Ability to monitor ongoing data transformation processing to alert on anomalies or changes in the structure Ability to iteratively sample data to provide an interactive testing of transformation logic Push-down processing of data transformations into the native data source for script execution (SQL, Pig, etc) Critical Very important Important Somewhat important Not important

60% Location of End-User Data Preparation Capabilities 2015 to 2016 50% 40% 30% 20% 10% 0% On-Premises Cloud-based Both 2015 2016

End User Data Preparation Vendor Ratings

End User Data Preparation Vendor Ratings 2016 Jinfonet Tableau Trifacta 64 Paxata RapidMiner SAS 32 Datawatch Panorama 16 ClearStory Data TIBCO 8 Platfora Oracle Infor 4 2 1 Microsoft Datameer MicroStrategy OpenText (Actuate) Dundas Qlik Birst Domo Information Builders Informatica InetSoft Logi Analytics Dimensional Insight Pentaho (Hitachi) Sisense SAP Usability score Integration score Output score Data Manipulation score Deployment score Overall score

Conclusions Huge need for improved data prep; users wasting time valuable time using whatever tool is at hand End user data preparation empowers end users; supporting self service initiatives Key technology selection criteria: Usability (visualization, automation), manipulation and integration.

Dresner Advisory Services End User Data Preparation 2016 EndUserDataPrep.report

Agenda 31 June 28, 2016

The Datawatch Promise 1. You will spend more time analyzing not preparing data 2. You will access all of the important and trusted data you need 32 June 28, 2016

Embedded and Resold by Industry Leaders IBM selected Datawatch for: IBM Watson Analytics IBM Cognos Analytics IBM Content Manager on Demand Dell selected Datawatch for: Dell Statistica for Predictive Analytics Dell Edge Gateway for IoT 33 June 28, 2016

40,000 Organizations Use Datawatch Financial Services Healthcare Government Retail Other Industries 34 19 June 28, 2016

Why 40,000 Organizations Use Datawatch? SPEND LESS TIME PREPARING DATA Analysts waste up to 80% of their time preparing data versus analyzing $22,000 per year, per analyst wasted 35 June 28, 2016

Why 40,000 Organizations Use Datawatch? 12% Relational, Excel, Hadoop, Salesforce, etc. Only 12% of enterprise data is used to make decisions 36 June 28, 2016

Why 40,000 Organizations Use Datawatch? 12% Relational, Excel, Hadoop, Salesforce, etc. 88% Reports, Web Pages, JSON, Log Files, etc. Make Better Decisions 37 June 28, 2016

38 June 28, 2016

Datawatch Platform ACQUIRE Any data including multi-structured Remove risk of non-managed data GOVERN DISCOVER PREPARE Simply powerful for business users Proven scalability to thousands AUTOMATE 39 June 28, 2016

Utilized Across the Organization Analytics Analysts - Financial Analysts - Marketing Analysts - Sales Analysts - Operations Analysts Use Cases - Visualization & Discovery - Dashboards - Advanced Analytics Operations Broad Array of Users - Finance - Supply Chain - Risk - Sales & Marketing Use Cases - Reconciliation - Auditing - Compliance - Consolidation 40 June 28, 2016

Data Preparation Automation Service Content Repository Governance and Control Connectivity Report Mining A P I 41 June 28, 2016

Recommended Next Steps Read full Market Study Try Datawatch Attend weekly product demo 42 June 28, 2016

Thank You! Dan Potter Chief Marketing Officer Datawatch Howard Dresner Chief Research Officer Dresner Advisory Services 43 June 28, 2016