Business Intelligence and Analytics. Elio Velazquez

Transcription

1 Business Intelligence and Analytics Elio Velazquez

2 Business Intelligence and Analytics What s been happening... (Gartner reports) By 2014: Business Intelligence and Analytics will Remain CIO s Top Technology Priority. By 2015, the majority of BI vendors will make data discovery their prime BI platform offering, shifting BI emphasis from reporting-centric to analysis-centric. By 2017, more than 50 percent of analytic implementations will make use of event data streams generated from instrumented machines, applications and/or individuals. By 2018, most business users and analysts in organizations will have access to self-service tools to prepare data for analysis as part of the shift to deploying modern BI platforms. Between 2016 and 2019, spending on real-time analytics will grow three times faster than spending on non-real-time analytics.

3 Business Intelligence and Analytics MODELS OF DELIVERING ENTERPRISE-WIDE VISIBILITY REPORT CENTRIC IT ORIENTED APPROACH ANALYSIS CENTRIC SELF-SERVE DATA EXPLORATION

4 Business Intelligence and Analytics Source: TDWI 2014 Survey

5 Business Intelligence vs. Operational Intelligence Business Intelligence - Mid to long term strategic planning - Look at the past to find trends - Data snapshots: hours, days, weeks, etc. - Longer term, offline decisions (apply to many processes, depts., etc.) - Data-centric (Interactive data discovery) Operational Intelligence - Use to address immediate concerns ( front line day to day production/support activities) - Leads to changes in execution - Event-centric (continuous monitoring) - Data virtualization (data from different sources, structured/unstructured) Source: Gartner (December 2016)

6 Advanced Analytics What is Advanced Analytics? The use of analytic techniques beyond traditional BI reporting and dashboards. Yes, but what does it mean? - Predictive modeling - Big data visualization - Analysis of large volume of structured and non-structured data for which traditional DW approaches are not sufficient. Why do companies do it? - Support decision making - Understand customer behavior and improving business performance. - New revenue opportunities

7 Advanced Analytics What happens when we do it? Companies that understand the value of advanced analytics and want to be predictive and proactive in their decision-making process are on average 6% more profitable that those who don t. (McAfee and Brynjolfsson, Big Data: The Management Revolution, Harvard Business Review 2012) Revenue: 136B - 35% of what consumers purchase on Amazon comes from recommendations based on predictive models. - 75% of the content watched on Netflix are based on recommendation engines.

8 Business Intelligence Model Source: TDWI

9 Some Examples Business Operations Marketing

10 Recommendations: In October 2006 Netflix held a competition for the best algorithm to predict user ratings of movies. The winner must improve Netflix own algorithm (Cinematch) by at least 10%. Award was given in September Based on Collaborative Filtering. Difficult movies to predict: Napoleon Dynamite, Lost in Translation, Fahrenheit 9/11, Kill Bill: Volume 1

11 Sports Analytics Building your data product: Beyond Moneyball (The defensive shift) Operationalizing your Analytic Results

12 Cholera outbreak in London 1854 Physician John Snow links the outbreak to a contaminated well by plotting number of cases on a map. Started the science of Epidemiology Source: Wikipedia

13 What has changed? Big Data Data Volume (from GB -> TB -> PB, etc.) Velocity (Real-time or near real-time access to information) Variety (sensors, social media, IoT) Computing Power! VS.

14 But, not everything is perfect! IBM Watson: The Jeopardy Challenge Category: U.S. Cities ITS LARGEST AIRPORT IS NAMED FOR A WORLD WAR II HERO: ITS SECOND LARGEST, FOR A WORLD WAR II BATTLE.

15 Another Example: Children in homes with more books do better at school. In 2004, Governor Rob Blagojevich proposed a plan to mail one book a month to every child in Illinois until they reached kindergarten. Estimated cost: $26 million/year. Further analysis showed that children from homes with several books performed better in school, even if they have never read the books! Well-educated parents More Books Better grades

16 Another Example: Lesson for Data Scientists: - Question your assumptions (be especially skeptical when predicting a rare event with limited history using human behavior. - Examine data quality - in this election polls were not reaching all likely voters. - Beware of your own biases: many pollsters were likely Clinton supporters and did not want to question the results that favored their candidate.

17 Advanced Analytics: Where to start? The Data Science Process HOW Exploratory Data Analysis WHAT Data Extraction WHY Data Cleaning Machine Learning, Statistical Models Build Data Product Communicate and Report Findings

18

19 Advanced Analytics: Where to start? Business Intelligence Data Science

20 Advanced Analytics Key Takeaways: 1. Accept it! There is not silver bullet Creating an analytic ecosystem takes time, risk and experimentation. 2. Consider new infrastructure technology Combine new technologies with existing (well-established) approaches (e.g., data warehouses). Select the right tool for the job (massive parallel systems, in-memory processing, columnar databases, cloud, etc.). 3. Start with a proof of concept To get attention and prove value, start with a familiar metric and demonstrate that it can be predicted. 4. Transition into the use and adoption of more advanced analytics Starting with predictive analytics and manual interventions and gradually move to a more operationalized approach. 5. Use disparate data Go beyond the traditional DW by using non-structured data, text, etc. 6. Democratization of analytics Training and developing the skills needed for data management and creating the predictive models. 7. Quality control Ensure proper model validation before they can be deployed into production. 8. Monitor your analysis Revisit the models periodically to make sure that the data is still relevant and evaluation criteria are still met. 9. Leverage internal resources Break silos by sharing data and knowledge.