Introduction to Big Data

Transcription

1 Introduction to Big Data Assoc. Prof. Dr. Thanachart Numnonda Executive Director IMC Institute August Speaker Executive Director, IMC Institute Committee of the Council, Ubon Ratchathani University Chairman, Siameast Solutions Public Co.Ltd. Independent Director & President of Audit Committee, Thanachart Bank Public Co.Ltd. Independent Director, Vintcom Technology Public Co.Ltd. Independent Director, Humanica Ltd. 2 1

2 3 3 4 ห นยนต เภส ชกร Wearable device อ นเตอร เน ต โทรศ พท ฝ งอย ในต ว เคร องพ มพ สามม ต พ มพ อว ยวะคน รถยนต ไร คนข บ Sharing car 4 2

3 5 5 6 Every activities create data 6 3

4

5

6

7 13 In an era of the Internet of Things, companies that have the right IT architecture and infrastructure, and the right talent, capable of both handling fast-moving technologies and finding meaning in big data, will be able to leapfrog their competitors. Harvard Business Review

8 15 Big Data Big Data Analytics Data Science Machine Learning Artificial Intelligence Deep Learning 15 Big Data Source:

9 Source: IBM 17 Source: IBM 18 9

10 Source: IBM 19 Source: IBM 20 10

11 21 Big Data : Why Now? Source: William EL KAIM, Enterprise Architecture and Technology Innovation

12 Source: Big Data Analytics: The Revolution Has Just Begun, SAS Software 23 Source: Bernard Marr 24 12

13

14 27 We are forecasting the future based on the past

15 Use Cases Source Big Data Analytics with Hadoop: Phillippe Julio WALMART: Retail Industry - Largest retailer in the world - 20,000 stores in 28 countries. - Has Big Data and analytics department since The world s largest private data cloud - Process 2.5 PB every hour Problem Solving Realtime analytics: product recommendation Right place, right time, right customer Monitors public social media conversations,and attempts to predict what products people will buy Source: Big Data in Practice, Bernard Marr,

16 31 WALMART: Retail Industry Data Data Café uses database consisting of 200 billion rows of transactional data 200 other sources, including meteorological data, economic data, telecoms data, social media data, gas prices Technology 40 petabytes of data Hadoop (since 2011) Spark Cassandra R SAS Source: Big Data in Practice, Bernard Marr, WALMART: Retail Industry Results Data Café system has led to a reduction in the time it takes from a problem being spotted in the numbers to a solution being proposed from an average of two to three weeks down to around 20 minutes. Source: Big Data in Practice, Bernard Marr,

17 Netflix: Entertainment - Streaming movie and TV service - 65 million members in over 50 countries - one-third of peak-time Internet traffic in the US Problem Solving To understand customer viewing habits Improve in the number of hours customers spending They launched the Netflix Prize Source: Big Data in Practice, Bernard Marr,

18 35 Netflix: Entertainment Data Customer ID, movie ID, rating and the date the movie was watched Streaming data Technology 40 petabytes of data Amazon Web Services Hadoop, Hive and Pig Originally used Oracle databases, but they switched to NoSQL and Cassandra Source: Big Data in Practice, Bernard Marr,

19 37 Netflix: Entertainment Results They added 4.9 million new subscribers in Q1 2015, compared to four million in the same period in Q alone, Netflix members streamed 10 billion hours of content. Source: Big Data in Practice, Bernard Marr, Uber: Transportation - A smartphone app-based taxi booking service. - Now valued at $41 billion. - Firmly in Big Data, and leveraging this data in a more effective way than traditional taxi firms. Problem Solving Big Data principle of crowdsourcing. Store and monitor data on every journey to determine demand, allocate resources and set fares. Big Data-informed pricing, which call surge pricing Source: Big Data in Practice, Bernard Marr,

20 39 Uber: Transportation Data mixture of internal and external data. GPS, traffic data public transport routes Technology Hadoop data lake. Apache Spark Source: Big Data in Practice, Bernard Marr, Uber: Transportation Results This case is less about short-term results and more about long-term development of a data-driven business model. But it s fair to say that without their clever use of data the company wouldn t have grown into the phenomenon they are. Source: Big Data in Practice, Bernard Marr,

21 Amazon - one of the world s largest retailers of physical goods, virtual goods such as ebooks and streaming video and more recently Web services. Problem Solving recommendation engine technology is based on collaborative filtering. 360-degree view of you as an individual customer monitor, track and secure its 1.5 billion items in its retail store Source: Big Data in Practice, Bernard Marr,

22 43 Amazon Data Data from users as they browse the site. Location data and information about other apps use on your phone. External datasets such as census information Streaming data Technology 187 million unique monthly website visitor. Hewlett-Packard servers running Oracle on Linux 5 TB of data Source: Big Data in Practice, Bernard Marr, Amazon Results Amazon have grown to become the largest online retailer in the US based on their customer-focused approach to recommendation technology. Last year, they took in nearly $90 billion from worldwide sales. Source: Big Data in Practice, Bernard Marr,

23 45 45 Big Data is changing the world 46 23

24 Data Sources Technology Analytics 47 Data, Data Everywhere Structure data Semi-structure data Unstructure data Internal data External data Activity data Conversation data Photo data Sensor data IoT data 48 24

25 Technology Data Collection Data Storage Data Analysis/ Processing Data visualisation 49 Big Data Technology 50 25

26 51 The old way: Ask, then collect The new way: Collect, then ask Source: Data Science and Critical Thinking, A, Croll Data Warehouse BI Big Data Data Lake Data Science 52 26

27 53 54 Image: rodneyrohrmann.blogspot.com 27

28 Data Warehouse Source: dinesql.blogspot.com 55 Differences between Data Lake and Data Warehouse 56 Source: martinfowler.com/bliki/datalake.html 28

29 Data Sources Technology Analytics 57 Data, Data Everywhere Structure data Semi-structure data Unstructure data Internal data External data Activity data Conversation data Photo data Sensor data IoT data 58 29

30 Technology Data Collection Data Storage Data Analysis/ Processing Data visualisation 59 Big Data Technology 60 30

31 61 The old way: Ask, then collect The new way: Collect, then ask Source: Data Science and Critical Thinking, A, Croll Data Warehouse BI Big Data Data Lake Data Science 62 31

32 63 64 Image: rodneyrohrmann.blogspot.com 32

33 Data Warehouse Source: dinesql.blogspot.com 65 Differences between Data Lake and Data Warehouse 66 Source: martinfowler.com/bliki/datalake.html 33

34 67 Don't Think Big Data Technology, Think Business Transformation Don't Think Business Intelligence, Think Data Science 68 34

35 69 Don't Think Data Warehouse, Think Data Lake Don't Think What Happened, Think What Will Happen 70 35

36 71 How to Use Big Data? Using data to make better business decisions Using data to improve your business operations Transforming your business model: data as a business asset Big Data Business Model Maturity Index Source: Big Data MBA, Bernard Marr 72 36

37

38 75 Thank you