Leveraging Big Data Analytics in Business Processes

Transcription

1 Leveraging Big Data Analytics in Business Processes Jeroen van Grondelle Be Informed With Big Data technologies delivering predictive, real time analytics on production data of unparalleled volumes, offering these insights in an actionable way across the operation becomes critical. Constraint based business processes allow organizations to leverage big data analytics in their daily operations. Processes can be optimized based on the emerging trends and unexpected side effects detected across a broad spectrum of datasets. At runtime, autonomous actors are empowered by real time, case specific analytics to make expert decisions how to best proceed within the governing constraints. The Big Data Transformation The amount of data available to organizations is increasing drastically. A number of recent technologies and approaches, collectively dubbed Big Data, aim to capture, manage and process data at these new scales to provide insight and support decision making based on all these data. An often cited definition by Gartner 1 introduces volume, variety and velocity as the three characteristics that define Big Data. The Big Data approach breaks with conventional data management strategies in a number of important ways. In a series of research notes, Be Informed s research director Jeroen van Grondelle shares his perspective on emerging challenges in the BPM field, and how they can be addressed by a paradigm shift towards declarative business processes. We welcome your ideas and suggestions on this subject. Please submit your contributions to the author at j.vangrondelle@beinformed.com. Don t Prepare for Expected Questions Classically, many data management initiatives have been guided by the questions that are known to be asked later on. Decisions on the scope of the data to store, or on what data model to store the data in, were all heavily influenced by the expected queries to be answered. The Big Data approach acknowledges that unexpected questions will be asked, and that being guided by the ones that are anticipated, hampers the ability to answer the unexpected ones. Instead, new querying execution models, such as massively distributed architectures based on for instance Hadoop and new database paradigms like streaming and in-memory databases, compensate at query time for the effects of not optimising storage for query processing at design time. 1 Douglas Laney. The importance of big data : A definition. Gartner, 2012 New Analytics Models The availability of wide ranges of data for unified processing has lead to new models of analytics.

2 Jeroen van Grondelle First of all, many data sources nowadays become available in the form of event streams, rather than as (interpreted) entities with attributes and complex constructs to account for time aspects. These events represent the context in which the data were captured and by whom far better, thereby implicitly addressing the issue of contextual semantics which hampered the creation of large, shared canonical data models in the past. The event based nature of data enables new kinds of analytics, based on machine learning or statistical techniques. Rather than reporting on the past, predictive models are derived from (patterns in) historical data that in real time can predict likely future events. Also, with many data sources being brought together on the basis of availability rather than up front understood relations, discovering statistically relevant phenomena across seemingly unrelated data sources can uncover unknown trends and emerging change, that would not have been available in conventional data management strategies. Analytics based on Production Data Traditionally, there has been a strict decoupling of the data as maintained and managed in production and the data used for business intelligence. The underlying technologies were so different and inherently optimized for their specific tasks that data had to be copied to the different infrastructures for both types of services to be offered. In Big Data, this distinction is fading and hybrid usage of the same data collection becomes possible. This of course has important consequences for the range of (real time) analytics that can be offered, and the (production) situations these can be used in. In the whitepaper Towards Webscale Business Processes, the concept of constraint based processes was explained by the analogy of a maze. In a maze, the hedges form constraints, that need to be respected while searching a way through the maze. 2 The Corresponding Process Transformation In BPM, a similar transition is occurring. Most organizations implementing BPM strategies move away from being prepared for all possible questions they might face, and as a consequence, from designing explicit flows up front for all these potential situations. This is the only way to allow business processes to offer customers and citizens an individualised experience, while providing its actors with a maximum of autonomy and, finally, have everybody stick to compliance rules, all at the same time. In the BPM stacks that emerged to this end, offering what is often called intelligent processes, the strict decoupling of designing a specific process at specification time, and just performing it at execution time is disappearing. When applying a constraint based process management approach for instance, a baseline of constraints is modelled up front that ap-

3 leveraging big data analytics in business processes plies at all times, but often a lot of autonomy is inherently left to the individual actors at execution time 2. Depending on constraint levels, they get to make operational choices on how to deal with individual cases based on their expertise, as long as they stay within constraints. They typically are supported by smart process engines that compute for each specific case, at execution time, the options they have to proceed within constraints. 2 Jeroen Van Grondelle. Declarative Business Processes: The Unification of BPM Approaches. Be Informed, 2013 The New Role of Analytics in BPM Current process oriented business intelligence typically focuses on collective performance within existing business processes. It is mostly used as feedback to the design process, for instance in business process improvement initiatives, or to accommodate in the process design those exceptional situations that lead to overrides in execution. Big Data analytics, however, can help guide business processes at both specification and implementation time by providing actionable data to all stakeholders throughout the policy lifecycle. Policy Making "Impact Analysis at Policy Change" Big Data "Discovering the Unexpected" Policy Evaluation Figure 1: The Use of Big Data Analytics throughout the Policy Lifecycle "Guiding Individual Decisions" Policy Execution By mining traces of historical process instances, frequently taken routes can be detected, and the different tasks can be offered to the users accordingly. Guiding Individual Decisions Rather than restricting the use of analytics to design time, the real time and predictive nature of analytics enables its use in operational situations, providing answers to very individual/contextual questions. In processes where its actors have a high degree of autonomy to make choices in how to perform a process, analytics can guide these decisions by providing insight into choices made by colleagues in similar situations, or choices that turned out successful in similar cases. Alternatively, suggested next steps can be guided by operational as- 3

4 Jeroen van Grondelle pects such as workloads (and consequent congestion) across the organization, or even cognitive load of the individual actor. One way of supporting knowledge workers is of course presenting them with the right insights, for them to decide and act on. However, we are also researching how, in a rule oriented environment, analytics can influence business processes without requiring explicit action from its actors. This way, instead of presenting an actor with other peoples choices, or presenting evidence of what strategies have proven to be effective, the computed processes (gradually) adapt based on contextual analytics. Discovering the Unexpected When interpreting the correlations discovered between datasets, one of the important challenges will still be distinguishing correlation from causality. One of the major risks obviously is that what surfaces as a correlation between tow factors in reality is causal relations to a third, unknown phenomenon that is not in the available datasets. An important aspect of the Big Data idea, is that it inherently takes into account the unexpected. This offers important opportunities when evaluating policy. Conventionally, policy effectiveness has often been evaluated against preset performance indicators, based only on the data that was anticipated or intended to be affected by the policy change. Big Data analytics potentially helps to discover unexpected, and maybe even undesirable, side effects of that same policy. Also, by looking for correlation on a broad range of datasets, other contributing factors may be discovered that also contributed to the change or even were the actual reason. In highly contextualised processes, that know many highly individual outcomes, signal is sometimes hard to distinguish from noise. As a result, it is hard to determine when exceptions are business as usual, and when they are early warnings of significantly changing circumstances, that need to be addressed. Impact Analysis at Policy Change With large datasets available, internally in organizations or publicly as part of Open Data initiatives, many effects of policy change can be evaluated through simulation in the phase of policy making. Next years policy, applied to last years case load, or sometimes better, applied to the case load as expected next year, creates valuable data when deciding on that same policy. The same analytics that are available on production data, can now be applied on the simulated data and give insight into the impact of the proposed policy change. Apart from estimates on the effectiveness of the policy in reaching the desired goals, they also provide important feedback on the effects on for instance the workload incurred by the new policy. This allows policy makers to make informed trade-offs between operational capacity and ultimate objectives. 4

5 leveraging big data analytics in business processes Conclusions Many organizations acknowledge that information management strategies based on anticipated use alone do not meet the requirements of the dynamic world of today. By adopting Big Data strategies, they are preparing for unexpected data needs and analytics across their organizational boundaries. Business process management strategies are affected by the same trend. By not designing exhaustive process flows for all anticipated situations up front, but by providing its expert employees maximal autonomy within the inherent constraints at runtime, they are able to deal with changing circumstances and exceptions. By combining the two transitions, big data analytics can be used to its full potential in business processes that allow stakeholders to adjust their constraints based on emerging trends discovered, and enable customer facing actors to autonomously act, based on real time analytics. References [1] Douglas Laney. The importance of big data : A definition. Gartner, [2] Jeroen Van Grondelle. Declarative Business Processes: The Unification of BPM Approaches. Be Informed, About Be Informed Be Informed is an internationally operating, independent software vendor. The Be Informed business process platform transforms administrative processes. Thanks to Be Informed s unique semantic technology and solutions, business applications become completely modeldriven, allowing organizations to instantly execute on new strategies and regulations. Organizations using Be Informed often report cost savings of tens of percents. Further benefits include a much higher straight-through processing rate leading to vastly improved productivity, and a reduction in time-to-change from months to days. Copyright 2013 Be Informed BV All rights reserved. Nothing in this publication may be duplicated, published or transmitted, using any means or in any form whatsoever, without the permission of Be Informed. Technical Report BIRN Version October 3,