Big Data - Challenges and risks

Transcription

1 Big Data - Challenges and risks Dr. Marcel Blattner Chief Data 1 Beispielpräsentation Tamedia, Datum, Autor

2 The Tamedia Digital Analytics Team Thomas Gresch Marcel Blattner Julian Nordt Nicolas Perony Yannick Koechlin Digital CTO Chief Data Scientist Business Analyst Data Scientist Data Engineer Leading the digital transformation of Tamedia from a technical perspective Leading the development of predictive analytics Three years of business experience as a management consultant Holds a Phd from ETH Zurich, Chair of Systems Design Experienced full stack engineer with 8+ years of experience Technology background with a masters degree in bioinformatics from ETH Zurich Strong experience with agent based systems on social graphs Former CTO of Rayneer Head of the Tamedia Data Analytics Team and other potentially transversal platforms Physicist (Phd) with a strong background in quantitative analysis and machine learning Physicist with a strong Entrepreneurial background 5

3 Big Data - what else Knowing the name of something does not mean to know something - Richard P. Feynman

4 Big Data - The big promise The analysis of a vast amount of data may lead to new insights. ZH Filmfestival

5 Big Data - The big promise ZH Filmfestival

6 Big Data - why is everybody talking about it?

7 Big Data - why is everybody talking about it? Data accessibility

8 Big Data - Characteristics Big data embodies new data characteristics created by today s digital marketplace ZH Filmfestival Big data means to analyze a vast amount of data from different sources and formats in a very short time. The aim is to generate new insights leading to competitive advantages

9 Big Data - core skills Industry Vertical Domain Expertise Develop hypothesis, identify relevant business issues, ask the right questions Decision Making Executive and Management Apply information to solve business issues Analytics Skills and Tools Skills developed as a core discipline Enabled by a robust set of tools and solutions Develops action-oriented insights Information management Solid information foundation Standardised data management practices Insights accessible and available Act on the data Fact-driven leadership Analytics used as a strategic asset Strategy and operations guided by insights Visualization Expertise Interpret data sets, determine correlations and present in meaningful ways I Data Experts Data architecture, management, governance, policy Tool Developers Mask complexity and analytics to lower skills boundaries

10 Big Data - skills and job trends

11 Big Data - skill landscape for data analytics team

12 Big Data - skill shortage Among organisations worldwide today: 1 in 10 has all the skills it needs to be successful applying advanced technology for business benefit 40% report a skill shortage in the ability to manage information We face a big skill shortage. This will continue for the next years. 1/4 have major skill gaps in mobile, business analytics, and security Source: IBM Tech Report 2014

13 Big Data - Project cycle Acquisition Exploration Implementation Operationalization Iterative Iterative development Ideas Initial discussion Data discovery Formalize Business case Design service Develop & deliver service Operationalize service Initial brief Workshop Analytic canvas Business case Deliverables Gate 1: Principal checks Foundational agreement on use case Gate 2: Business case sign off Gate 3: Implementation sign off Portfolio companies and TDA generate use case ideas Ideas are noted (light weight) Use case ideas are discussed between TDA and portfolio companies Promising ideas are formulated in an initial brief and decision on use case exploration is taken Iterative and collaborative data discovery together with portfolio companies (2 3 weeks) Workshop to present findings to wider portfolio company management, present approximate business case and get principal decision on next steps Formalise business case and define goals, costs, timelines and commitments and project team Formal sign off of business case Iterative design and implementation of use case Strong collaboration with portfolio companies Build up of prototypes and testing of use case ZH Filmfestival Delivery of business case goals Operationalization of use case Training and knowledge transfer to portfolio companies

14 Big Data - Whats the difference to traditional analysis? Traditional Analytics Structured & Repeatable Structure built to store data Big Data Analytics Iterative & Exploratory Data is the structure Business Users Determine Questions IT Team Delivers Data On Flexible Platform IT Team Builds System To Answer Known Questions Business Users Explore and Ask Any Question

15 Big Data - Whats the difference to traditional analysis? Traditional Analytics Structured & Repeatable Structure built to store data Big Data Analytics Iterative & Exploratory Data is the structure Analyzed Information IT Team Delivers Data On Flexible Platform Available Information Capacity constrained down sampling of available information Business Users Explore and Ask Any Question

16 Big Data - Whats the difference to traditional analysis? Traditional Analytics Structured & Repeatable Structure built to store data Big Data Analytics Iterative & Exploratory Data is the structure Analyzed Information Available Information Capacity constrained down sampling of available information Analyze ALL Available Information Whole population analytics connects the dots

17 Big Data - Whats the difference to traditional analysis? Traditional Analytics Structured & Repeatable Structure built to store data Big Data Analytics Iterative & Exploratory Data is the structure Analyzed Information Available Information Capacity constrained down sampling of available information Analyze ALL Available Information Whole population analytics connects the dots Analyzed Information Carefully cleanse a small information before any analysis

18 Big Data - Whats the difference to traditional analysis? Traditional Analytics Structured & Repeatable Structure built to store data Big Data Analytics Iterative & Exploratory Data is the structure Analyzed Information Available Information Capacity constrained down sampling of available information Analyze ALL Available Information Whole population analytics connects the dots Analyzed Information Analyzed Information Carefully cleanse a small information before any analysis Analyze information as is & cleanse as needed & existing repeatable

19 Big Data - Whats the difference to traditional analysis? Traditional Analytics Structured & Repeatable Structure built to store data Big Data Analytics Iterative & Exploratory Data is the structure

20 Big Data - Whats the difference to traditional analysis? Traditional Analytics Structured & Repeatable Structure built to store data Big Data Analytics Iterative & Exploratory Data is the structure Hypothesis Question? Analyzed Information Answer Data Start with hypothesis Test against selected data

21 Big Data - Whats the difference to traditional analysis? Traditional Analytics Structured & Repeatable Structure built to store data Big Data Analytics Iterative & Exploratory Data is the structure Hypothesis Question Data Exploration? All Information Analyzed Information Answer Data Start with hypothesis Test against selected data Actionable Insight Correlation Data leads the way Explore all data, identify correlations

22 Big Data - Whats the difference to traditional analysis? Traditional Analytics Structured & Repeatable Structure built to store data Big Data Analytics Iterative & Exploratory Data is the structure Hypothesis Question Data Exploration? All Information Analyzed Information Answer Data Start with hypothesis Test against selected data Actionable Insight Correlation Data leads the way Explore all data, identify correlations Analyze after landing

23 Big Data - Whats the difference to traditional analysis? Traditional Analytics Structured & Repeatable Structure built to store data Big Data Analytics Iterative & Exploratory Data is the structure Hypothesis Question Data Exploration? All Information Analyzed Information Answer Data Start with hypothesis Test against selected data Actionable Insight Correlation Data leads the way Explore all data, identify correlations Analyze after landing Analyze in motion

24 Big Data - science? Big data is not science (in the traditional sense)

25 Big Data - What is missing so far 1. A comprehensive approach to using big data.

26 Big Data - What is missing so far 1. A comprehensive approach to using big data. 2. Getting the right information into the hands of decision makers.

27 Big Data - What is missing so far 1. A comprehensive approach to using big data. 2. Getting the right information into the hands of decision makers. 3. Effective ways of turning big data into big insights.

28 Big Data - What is missing so far 1. A comprehensive approach to using big data. 2. Getting the right information into the hands of decision makers. 3. Effective ways of turning big data into big insights. 4. Big data skills are in short supply.

29 Big Data - What is missing so far 1. A comprehensive approach to using big data. 2. Getting the right information into the hands of decision makers. 3. Effective ways of turning big data into big insights. 4. Big data skills are in short supply. 5. Big data privacy issues.

30 Big Data - Healthcare Personalized medicine Genome analytics in oncology Precision medicine Computer based analysis in pathology and radiology Personalized and cooperative treatment planning Personalized therapy planning