Is Machine Learning the future of the Business Intelligence Fernando IAFRATE : Sr Manager of the BI domain Fernando.iafrate@disney.com Tel : 33 (0)1 64 74 59 81 Mobile : 33 (0)6 81 97 14 26
What is Business Intelligence *Wikipedia, The Free Encyclopedia 2
What is Business Intelligence Why the Big Data is shaking this model To create Smart Data (data that every one can understand) we need people with: Strong analytical skill Strong business awareness Strong communication skill Those talents are hard to acquire, hard to scale, with a high cost. The volume of data to analyze is growing faster than the capacity of the enterprise to analyze it via the historical Business Intelligence organization. 3
What is Machine Learning Machine Learning (Mitchell 1997) Learn from past experiences Improve the performances of intelligent programs Definitions (Mitchell 1997) A computer program is said to Computer learn from Sciences experience E with respect to some class of tasks T and performance measure P, if its performance at the tasks improves with the experiences 4
What is Machine Learning Machine learning is a subfield of computer science (CS) and artificial intelligence (AI) that deals with the construction and study of systems that can learn from data, rather than follow only explicitly programmed instructions.* Artificial Intelligence Computer Sciences Machine Learning *Wikipedia, The Free Encyclopedia 5
Evolution of Analytics Landscape As with other technology in an evolving analytics landscape, machine learning capabilities are maturing to address emerging questions and will serve as a competitive advantage. And beyond BI 2.0 Prescriptive Analytics Real-Time Guest Centric Decisioning Value BI 1.0 Descriptive Analytics Predictive Analytics Analytical & Operational Decisioning Issue with Big Data! Integrated Big Data Assets Machine Learning, will help bridging between BI 2.0 and Big Data management Ad Hoc Recommendations Siloed Data Most of the enterprises are here, managing issues with Big Data Complexity & Sophistication 6
Big Data Landscape Big Data = More Data Variety/Complexity Data Complexity Additionally, with big data assets available, additional complexity is introduced. Data assets alone do not translate to value, but machine learning identifies new business insights despite complexity. Cross-Platform Guest Interaction Click Stream Images Google s Alon Halevy believes that the real breakthroughs in big data analysis are likely to come from integration specifically, integrating across very different data sets. Text OLAP DB Relational DB Hundreds Thousands Millions # Variables BI Landscape Big Data Landscape (Machine Learning ) http://www.wired.com/2013/10/computers-big-data/all/ 7
Historical BI compare to Machine Learning Machine learning compared with Historical Regression Models Prerequisites Model goal / objective Model training Historical BI Regression models (more static) A data set with samples of what you re trying to predict Data scientist selects the right answer from the data Data scientist trains the model using the data set Machine Learning models (more dynamic) The ML system must interact with the environment Select a goal. Examples: clicks, bookings, revenue, etc Initially the decisions will be random, the ML system will learn as decisions are made Model updates Data scientist gets a new data set, repeats training Learning is continuous in a closed loop between action and reaction Definitions Goal: What you want the system to do: increase bookings, increase clicks, etc Options: what the ML system chooses between. Reward: a measurement of the goal CONFIDENTIAL 8
How Machine Learning works
The Formal Neuronal The formal neuron is a mathematical representation of the biologic one Dendrites Axon Nucleus Dendrites = receivers (input) Axon = main conductor Axon terminals = transmitters (output) Nucleus = where the processing is made (processing)
The Formal Neuronal The formal neuron using back-propagation (an auto learning system) X1 and X2 are the input Source of the information The question is : how do I choose between X1 & X2 Checking of the error and back-propagation to adjust the Weight (W) W1 and W2 are the Weight of each one of the input (how much you trust them) The nucleus does the processing (X1 x W1) + (X2 x W2) If the result is closed to X1 the decision will be X1 else X2
Types of Learning Several key classes of machine learning approaches include supervised learning, unsupervised learning and reinforcement learning. Machine learning Machine learning Deep learning SUPERVISED REINFORCEMENT UNSUPERVISED Predict an outcome given prior knowledge from training data Predict an outcome/reward by incorporating feedback from interactions Discover patterns and structures without prior knowledge (unlabeled data) Training Data Classifier Training Data Classifier Data Neuronal networks 12
Supervised Learning: using training data We have an idea of the expected output We train the system with known labeled data for the models & decision algorithms We than use raw data to be processed by the ML (to predict class, value label)
Unsupervised Learning: Neuronal Network 14
Unsupervised Learning: Neuronal Network Step by step deep analysis from one network layer to the next one 15
Unsupervised Learning: Neuronal Network 16
Machine Learning Is everywhere
Example: Amazon.com DATA User profile (even anonymous) Purchase history Item rating Shopping cart content METHOD Collaborative Filtering Is a technique commonly used to build personalized recommendations on the Web ACHIEVEMENT 35% purchase comes from recommendation based on collaborative filtering KEY TAKEAWAY Collaborative filtering can meet the challenge in scalability, relevancy (declarative approach), and speed (response time). http://en.wikipedia.org/wiki/collaborative_filtering http://www.mckinsey.com/insights/consumer_and_retail/how_retailers_can_keep_up_with_consumers 244M active users 232M products 426 items per second during pick period 18
Identity Resolution for anonymous How can I augment my customer knowledge Using cookies Device ID IP Address IP Address #1 Cookies Email #1 Email #N The Key Ring concept We link together step by step, session by session the information to help resolving the identity IP Address #1 (can be augmented by external sources) IP Address #N > 98% of Web browsing activity is done by anonymous (person not using a formal login name ) Device ID #N Device ID #1 Identity ID use to index The Knowledge Database 19
Collaborative Filtering (CF) What is Collaborative Filtering Collaborative filtering (CF) is a technique commonly used to build personalized recommendations on the Web. In collaborative filtering, algorithms are used to make automatic predictions about a user's interests by compiling preferences from several users. Different types of collaborative filtering are as follows: Memory Based: This method makes use of user rating information (declarative mode) to calculate the likeness between the users or items. This calculated likeness is then used to make recommendations. Model Based: Models are created by using data mining (analytical mode), and the system learns algorithms to look for habits according to training data. These models are then used to come up with predictions for actual data. Hybrid: Various programs combine the model-based and memory-based CF algorithms. 20
Example: Netflix DATA Video streaming activities User rating Searches User profile METHOD (Hybrid mode) Ensemble Learning (is usually used to average the predictions of different models to get a better prediction), Collaborative Filtering ACHIEVEMENT 75% ~ 80% viewing comes from recommendation KEY TAKEAWAY Everything is a Recommendation Business optimization successes driven by data science innovations 30 million plays 4M subscriber ratings 3M searches DAILY http://www.nytimes.com/2013/02/25/business/media/for-house-of-cards-using-big-data-to-guarantee-its-popularity.htmlpagewanted=all&_r=0 http://gigaom.com/2012/06/14/netflix-analyzes-a-lot-of-data-about-your-viewing-habits/ 21
Example: Google(deep mind) Image Recognition DATA 16,000 processors 10 million YouTube video images (without label) 3-day training period ACHIEVEMENT 82% accuracy identifying human faces,... 75% accuracy identifying cats KEY TAKEAWAY Unsupervised learning algorithm able to self-learn and classify objects, without being told what to look for (unlabeled data). Le et al., Building high-level features using large scale unsupervised learning, International Conference in Machine Learning 2012 http://www.wired.com/2012/06/google-x-neural-network/ 22
Unsupervised Learning: Neuronal Network Augmented reality using deep learning 23
In Conclusion Enablers The GPU technical architecture will enable the implementation of deep neuronal networks, Machine Learning will help to unlock the power of Big Data. Added value Machine learning will enable an auto learning process (learning based on the experience). Assumption Machine learning will still need to be managed, monitor by human (the ones who creates the algorithms), this is where the BI organization will be necessary. Outcome Machine Learning is sacking the current BI processes and organization where the temporality for decision & action is now in milliseconds (or less). 24
Thanks for listening Questions, comments