Big Data s Big Impact on Businesses. Webconference : Jan 29, 2013

Transcription

1 Big Data s Big Impact on Businesses Webconference : Jan 29, 2013

2 Key Takeaways Slide 3 Introduction to Big Data Slide 5 Global Landscape and Trends Slide 12 The Big Data Opportunity Slide 20 Big Data s Big Impact on Businesses

3 Key Takeaways Big Data market opportunity is expected to witness strong growth in the next 5 years Expected to touch US$25 billion globally; the BIG opportunity for India lies in the IT & IT-enabled Services space, which is likely to be ~US$ billion market globally in 2015 India is likely to garner a ~10% share of the ~US$ billion global Big Data IT Services Market by 2015 Data-related regulations like Dodd-Frank and Basel III to impact Big Data implementations Initially, North America & Europe are likely to drive the Big Data opportunity since over 85% of the world s data is today residing in these 2 regions New database architectures and innovative analytics tools & techniques to facilitate Big Data implementations By end of 2012, around 90% of Fortune 500 companies had some initiatives underway related to Big Data Key verticals driving demand for Big Data analytics: Financial services, Retail, Telecom, Healthcare and Manufacturing Key risk potential shortfall of 1.5 million Data-Savvy Managers and 140, ,000 Data Scientists in the US by 2018 Source: CRISIL GR&A analysis 3

4 Key Takeaways Slide 3 An Introduction to Big Data Slide 5 Global Landscape and Trends Slide 16 Definition of Big Data Big Data ecosystem The Big Data Opportunity Slide 23 Benefits of Big Data to enterprises Key applications for end consumers

5 Big Data is Defined by Volume, Variety and Velocity What is Big Data? Big Data relates to rapidly growing, Structured and Unstructured datasets with sizes beyond the ability of conventional database tools to store, manage, and analyze them. In addition to its size and complexity, it refers to its ability to help in Evidence-Based Decision-making, having a high impact on business operations Speed, Accuracy and Complexity of Intelligence Small Data Sets Advanced analytics Small Data Sets Traditional analytics Size of Data Big Data analytics Big Data Gigabytes Terabytes Petabytes Zetabytes Source: CRISIL GR&A analysis Big Data Traditional analytics 3Vs Velocity Volume Variety 3 1 Large quantity of data which may be enterprisespecific or general and public or private 2 Diverse set of data being created, such as social networking feeds, video and audio files, , sensor data and other raw data Speed of data inflow as well as rate at which this fast-moving data needs to be stored Source: CRISIL GR&A analysis 5

6 The Global Data likely to Grow at a CAGR of 41% Zetabytes Growth of global data, CAGR ( ) 41.0% Implication for an organization Need for large storage capacity and quick retrieval of data Enable informed decision-making effectively, leveraging large data sets Turn 12 TB of Tweets created each day into improved product sentiment analysis Convert 350 billion annual meter readings to better predict power consumption Note: ZB stands for Zetabytes; Source: IDC; CRISIL GR&A analysis 6

7 Today 80% of Data Existing in any Enterprise is Unstructured Data Volume Variety Velocity Introduction Variety of sources from where data is being generated has also undergone a shift The types of data being created has changed from structured to semi-structured to unstructured data Structured Data Resides in formal data stores RDBMS and Data Warehouse; grouped in the form of rows or columns Accounts for ~10% of the total data existing currently RDBMS (e.g., ERP and CRM Data Warehousing Microsoft Project Plan File Implication for organization Need to manage broad range of data types Process analytic queries across numerous data types Semi- Structured Data A form of structured data that does not conform with the formal structure of data models Accounts for ~10% of the total data existing currently Solutions required Need to extract meaningful analysis from this data has led to several technologies to gain traction Examples include NoSQL databases to store unstructured data as well as innovative processing methods like Hadoop and massive parallel processing (MPP) Unstructured Data Video Web logs & clickstreams Audio Comprises data formats which cannot be stored in row/ column format like audio files, video, clickstream data, Accounts for ~80% of the total data existing currently Text message Blogs Weather patterns Sensor data/ M2M Social media Location co-ordinates Geospatial data Source: Industry reporting; CRISIL GR&A analysis 7

8 Big Data will Enable Real Time Analytics Volume Variety Velocity 7,000+ photos on flickr 700,000+ Facebook updates 1,500+ blog posts US$ 300,000+ are spent on online shopping Source: Industry reporting; CRISIL GR&A analysis 600+ videos on YouTube 200 million+ s sent Data velocity per minute 400,000+ tweets on Twitter ticks per minute in securities trading 2 million+ Google search queries 400,000+ minutes of Skype calling Big Data velocity enabling real time use of data Big Data is also characterized by velocity or speed i.e. frequency of data generation or the frequency of data delivery New age communication channels such as mobile phones, s, social networking has increased the rate of information flows Examples: Telcos adopting location based marketing based on user location sensed by mobile towers Satellite images can help monitor and analyze troop movements, a flood plane, cloud patterns, or forest fires Video analysis systems could monitor a sensitive or valuable facility, watching for possible intruders and alert authorities in real time 8

9 Big Data Analytics is Application of Advanced Techniques on Big Datasets; Answers Questions Previously Considered Beyond Reach Evolution of analytics Level of Complexity Prescriptive analytics Predictive analytics Descriptive analytics Standard reports Adhoc reports Big Data analytics is where advanced analytic techniques are applied on Big Data sets The term came into play late 2011 early 2012 Query drill down Alerts Statistical analysis Big Data analytics Complex event processing Extreme SQL Forecast - ing Predictive modeling Optimization Stochastic optimization Natural Language Processing Multivariate statistical analysis Data mining Time series analysis Basic analytics What happened? When did it happen? What was the its impact? Behavioral analytics Social network analytics Semantic analytics Visualization Online analytical processing (OLAP) Constraint based BI Analytic database functions Advanced analytics Why did it happen? When will it happen again? What caused it to happen? What can be done to avoid it? Late 1990s 2000 onwards Time Analytics as a separate value chain function In-database analytics Source: CRISIL GR&A analysis 9

10 Big Data Management, Analytics, IT Services & Applications are the Key Constituents of Big Data Ecosystem Four key elements: What does the Big Data Ecosystem Constitute? 1. Big Data Management & storage: Data storage infrastructure and technologies 2. Big Data Analytics Includes the technologies and tools to analyze the data and generate insight from it 3. Big Data s Application & Use Involves enabling the Big Data insights to work in BI and end-user applications 4. IT services including System Integration Consulting Project management and customization IT services (SI,customization, consulting, system design) Developer Environments (Languages (Java), Environments (Eclipse & NetBeans), programming interfaces (MapReduce)) Data Sources Unstructured data (Text, web pages, social media content, video etc.) Structured data (stored in MPP, RDBMS and DW*) Components of Big Data Ecosystem Big Data Analytics Data Architecture Hadoop/ Big Data tech y framework (MapReduce etc.) Analytics products (Avro, Apache Thrift) Big Data Data administration tools ETL & Data integration products Input data Workflow/ scheduler products NoSQL Hadoop based System tools End users Applications (mobile, search, web) BI &visualization tools Business analysts Operational Data NoSQL MPP RDBMS DW Data analytics & its application and use Data management & storage *MPP Massively parallel processing; RDBMS - Relational Data Base Management Systems; DW Data warehouse Source: CRISIL GR&A analysis 10

11 Key takeaways Slide 3 An Introduction to Big Data Slide 5 Global Landscape and Trends Slide 16 The Big Data Opportunity Slide 23 Big Data Geographic Analysis Market Trends & Developments

12 North America & Europe Drives the Big Data Opportunity with over 85% of the World s Data As North America and Europe account for the lion s share of the world s data the initial opportunity of both Big Data implementations and analytics lies in the these geographies i.e. developed economies North America Key verticals: Healthcare, Manufacturing, Retail, Digital Marketing Demand trend: High demand of Big Data analytics Europe Key verticals: Technology, Financial services, Oil & Gas, Utilities, Manufacturing Demand trend: European MNC s are still in the early stages of the adoption cycle >2,000 Japan >3,500 South America >40 Middle East >200 Demand trend: Current demand appears to be limited, however, lack of skills may drive outsourcing of Big Data analytics India >50 China >250 >400 Key verticals: Manufacturing, Telecom, Health & Life Sciences Demand trend: Demand for BI to derive operational efficiency Key verticals: Telecom, Bioinformatics, Retail Demand trend: Industry is in nascent stage with demand catching up, particularly in retail Low awareness levels Data generated: High to low Amount of new Big Data stored (Petabytes), 2010 Source: McKinsey Global Institute; CRISIL GR&A analysis Key verticals: Telecom, Retail, Banking Demand trend: Still embryonic; most organizations have wait and watch approach 12

13 Emergence of Niche Startups and Large IT Players Enhancing their Big Data Capabilities are key enablers for the Industry Market Trends and Developments 1 Emergence of niche Big Data startups driving technological innovation 2 Large IT players leveraging M&As to add Big Data capabilities to their service portfolios 3 Financial Services, Retail and Telecom are likely to be the early adopters in the Big Data space 4 Talent shortage is one of the biggest challenges of the Big Data space Source: CRISIL GR&A analysis 13

14 Emergence of niche Big Data start-ups to boost technological innovation 1 A new class of companies, specializing in Big Data technologies have emerged, to capitalize on the opportunities in the Big Data domain Big Data start-ups Key characteristics Technology Area Players* 1 2 Specialized in niche Big Data technologies like Hadoop, NoSQL systems, in-memory analytics, multiple parallel processing, and analytical platforms Majority of start-ups generate revenue less than USD 50 million and exhibit double digit revenue growth annually Hadoop distributions Non Hadoop Big Data Platforms 3 Most start-ups raising funding by private ventures or being acquired by large IT players Analytic Platforms and Applications Cloud-based Big Data Applications *Indicative list of players Source: Industry reporting; CRISIL GR&A analysis 14

15 Large IT Players Leveraging M&As to add Big Data Capabilities to their Service Portfolios 2 Area Acquirer Target Company Date Deal value Rationale Data Management Oct. '11 USD 1.1 billion Develop a comprehensive platform to analyze Big Data Mar. '11 USD 263 million Strengthen position in data warehousing market through expertise in SQL and MapReduce-based analysis Jun. '12 N.A. Extend Smarter Commerce suite with qualitative analytics software May. '12 N.A. Leverage data navigation technologies for Big Data by automating discovery of through innovative index and search capabilities Advanced analytics May. '12 N.A. Addition of sales performance analytics May. '12 N.A. Enhance Big Data marketing analytics Apr. '12 N.A. Acquisition of spend and procurement analytics Mar. '12 N.A. Accelerate development of Big Data analytic applications Mar. '11 N.A. Enhance real time business analytics for Big Data Key highlights M&As in the Big Data space had tripled in the first half of 2012 Acquisition targets are mainly innovative Big Data start-ups M&As with bigger deal value are happening in data management N.A. is not available. Source: Industry reporting; CRISIL GR&A analysis 15

16 1. Retail: Sears is leveraging Big Data analytics internally and is also keen on offering analytics services externally 3 Sears Holding is a leading integrated retailer with ~4,000 full-line and specialty retail stores in the US and Canada. It operates through its subsidiaries including Sears, Roebuck and Co. and Kmart Corp. IT need Challenge/Business Need Manage Increasing volumes of data like customer personal information, PoS data, online purchases, etc., posing a challenge Capacity run-out on its mainframe, and adding more capacity proving to be expensive Business need The need to set prices quickly and in real time The need to drive customer loyalty Solution Leverages its global In-house center in Pune, India for Big Data Analytics Implemented a Big Data architecture using Hadoop Used MapReduce algorithms to analyze data and feed results back into the mainframe, on individual customer activity, across all 4,000 locations Benefits Across IT environment Utilization of 100% of collected data against 10% utilization earlier Ability to run price elasticity algorithms in one week, as opposed to eight weeks previously Cost-savings of USD 600,000 per year Across business More relevant and personalized customer communications and offers to an active customer base (~80 million) Increased shopping and higher spend per transaction by active members Looking at the current and potential benefits of Big Data analytics, Sears aims to expand into newer areas and sell its data management and analytics services technology to other companies, through its subsidiary MetaScale *Massively Parallel Processing Source: Industry reporting; CRISIL GR&A analysis

17 2. Financial Services: Witnessing increased adoption of Big Data analytics, to reduce risk and uncover new market opportunities 3 The need to meet growing regulatory compliances, detect fraud and create new market opportunities is driving the growth for Big Data analytics in the financial services sector Customer & transaction data from multiple channels like branch, kiosks, mobile and web; social media; s; credit cards data; insurance claims data; stock market data; statistical data, PDF & excel files, videos, government filings, etc. are key Big Data sources Big Data application across Financial Services sub-sector Banking Credit line optimization Credit reward program analysis Capital Markets/ Trading Trading surveillance Intraday analysis Trading pattern analysis Insurance Predict client longevity, along with analyzing perspective clients medical status Using weather and calamity information for managing exposures and losses Pre-trade decision support analytics Risk management/assessment Fraud detection Portfolio analytics Compliance & regulatory reporting CRM,, Entering new markets Source: Industry reporting; CRISIL GR&A analysis

18 Potential Shortfall of 1.5 million Data-Savvy Managers and ~150,000 Data Scientists in the US in Demand-supply gap for data scientists* in US, 2018 Role in Ecosystem Requisite educational qualifications Other expertise 300K 140K 190K 50%-60% gap relative to supply 440K-490K Demand-supply gap for data-savvy managers* in US, 2018 Data Scientists Data-savvy Managers Big Data analytics Business intelligence Visualization Project management across the Big Data ecosystem Consulting services Implementation Infrastructure management Analytics Advanced degree like M.S. or Ph.D., in mathematics, statistics, economics, computer science or any decision sciences Advanced business degree such as MBA, M.S. or managerial diplomas Expertise in data analytics skills to extract data, use of modeling & simulations Multi-disciplinary knowledge of business to find insights Knowledge of statistics and/or machine learning to frame key questions and analyze answers Conceptual knowledge of business to interpret and challenge the insights Ability to make decisions using Big Data insights 4.0 million 2.5 million 1.5 million 60% gap relative to supply Technical Engineers Technical support in hardware & software across the Big Data ecosystem for: Data architecture Data administration Developer environment Applications Having a degree in computer science, information technology, systems engineering. or related disciplines Possessing data management knowledge IT skills to develop, implement, and maintain hardware and software *Analysts with deep analytical training; **Managers to analyze Big Data and make decisions based on their findings; Source: McKinsey Global Institute; CRISIL GR&A analysis 18

19 Key Takeaways Slide 3 An Introduction to Big Data Slide 5 Global Landscape and Trends Slide 16 The Big Data Opportunity Slide 23 Forecasted market size Future outlook

20 Global Big Data market to reach ~USD 25 billion by 2015,with a 45% share of IT & IT-enabled services The global Big Data market is expected to grow by about a CAGR of 46% over IT & ITES, including analytics, is expected to grow the fastest, at a rate of more than 60% Its share in the total Big Data market is expected to increase to ~45% in 2015 from ~31% in 2011 The USD 25 billion opportunity represents the initial wave of the opportunity. This opportunity is set to expand even more rapidly after 2015 given the pace at which data is being generated. Global Big Data Market Size, E US$ billion Global Big Data Market Size, 2015F ~US$25 billion Big Data analytics & IT & IT-enabled services US$ billion Opportunity for India lies in capturing the slice of IT services that includes Big Data analytics and IT & ITenabled services Software Hardware US$ billion US$ billion Lion s share of the Big Data hardware and software market is expected to be occupied by IT giants like IBM, HP, Microsoft, SAP, SAS, Oracle, etc. Source: Industry reporting; CRISIL GR&A analysis 20

21 Conclusion Big Data market opportunity is expected to witness strong growth in the next 5 years Expected to touch US$25 billion globally; the BIG opportunity for India lies in the IT & IT-enabled Services space, which is likely to be ~US$ billion market globally in 2015 Data-related regulations like Dodd-Frank and Basel III to impact Big Data implementations New database architectures and innovative analytics tools & techniques to facilitate Big Data implementations Key verticals driving demand for Big Data analytics: Financial services, Retail, Telecom, Healthcare and Manufacturing Key risk potential shortfall of 1.5 million Data-Savvy Managers and 140, ,000 Data Scientists in the US by 2018 Source: CRISIL GR&A analysis 21

22