Craig Stedman Executive Editor SearchBusinessAnalytics. EDITOR S NOTE Nothing Like the Present? HOME EDITOR S NOTE

Size: px

Start display at page:

Download "Craig Stedman Executive Editor SearchBusinessAnalytics. EDITOR S NOTE Nothing Like the Present? HOME EDITOR S NOTE"

Felix Willis
6 years ago
Views:

1 Operational Intelligence Adds to Business Smarts Stream processing and real-time analytics tools can give organizations an immediate eye on their business operations and help drive rapid changes in how things are done. Some internal changes might be needed first, though. SHEDS NEW LIGHT ON MAINTEGRATED ANALYTICS OPERATIONAL BI HELPS BOOST BUSINESS AGILITY

2 EDITOR S NOTE Nothing Like the Present? Business intelligence applications often are backward-looking. In many organizations, the primary job of BI systems is still to generate reports that tell corporate executives and other business users what happened on revenues, profits, the success of marketing programs and so on. There s nothing wrong with doing that the information it uncovers can pay big business dividends. Taking a more operational approach to BI and analytics might lead to even bigger benefits, though. That s because operational intelligence initiatives can, in the words of consultancy Ventana Research, enable business units to detect and respond to events as they are happening. With up-to-date information on ongoing business processes in hand, Ventana says, frontline managers and workers can make faster decisions and revise plans and procedures on the fly. Or in highly automated applications, like stock trading, business rules can be developed to trigger systems to take prescribed actions based on incoming data. It isn t easy, though. Real-time data is required, typically from a variety of data sources both internal and external. Stream processing, real-time integration and big data technologies might all be needed, adding a new layer (or more) of complexity to IT architectures. This three-part guide offers insight and advice to help organizations get started. First, we detail a stream processing deployment designed to open a more real-time window into the status of medical insurance claims. Next, we examine operational intelligence benefits and what companies need to do to get them. We close with a look at operational BI s role in running a business. n Craig Stedman Executive Editor SearchBusinessAnalytics 2

3 CASE STUDY Data Streaming Sheds New Light on Medical Claims RelayHealth, a unit of McKesson Corp. that runs claims processing applications for healthcare providers, has been a Hadoop user since Its Hadoop cluster holds 150 TB of claims-related data from hospitals and health systems, which is used to meet their regulatory compliance requirements, track the progress of claims and run analytics applications aimed at improving the claims and billing process. But now the company wants to do more with the Hadoop data for its customers and more quickly. Its new goal is to provide more real-time analytics to clients so they can get immediate insight into their business operations, enabling them to make fast adjustments in an effort to improve efficiency and resolve problems before they take a big financial hit. And to make that happen, RelayHealth is expanding its Clouderabased cluster and turning to the stream processing capabilities of Apache Spark. The expanded analytics initiative is being partly driven by demand from customers looking to upgrade from after-the-fact reports to real-time alerts about operational issues, according to Raheem Daya, director of product development and manager of the Hadoop platform at RelayHealth. The expectation is that if there s information they need, they need to have it immediately available, Daya said. They want actionable intelligence quickly, and that s the model we re moving toward. For example, he pointed to accounts receivables. Ideally, medical claims are processed promptly by insurers so providers can send final bills to patients and get payments back in a timely manner. But Daya said claims frequently get held up at the approval stage because of eligibility questions and other issues. Predictive models can look for claims that are likely to be flagged; armed with that information, workers in a providers s billing 3

4 CASE STUDY department can take action to try to expedite the approval process. A PARTIAL FIRST STEP RelayHealth deployed Spark, an open source data processing engine that can work with Hadoop or on a standalone basis, in late But Daya s team initially used the software s primary batch-processing functionality to pull in transaction data from an HBase database tied to the Hadoop cluster for analysis through machine learning algorithms. That helped, he said but it didn t provide the real-time information the company and its customers were looking for. In batch mode, it can take two to three hours to use incoming data to score a predictive model for accuracy, Daya noted. And then the scoring process typically needs to be repeated, often multiple times, as the model is refined. To try to accelerate things, the company is implementing the Spark Streaming module, with an expected go-live date in February In tests, the stream processing technology has been able to score models in seconds, Daya said. He added that if Spark Streaming is set to pull in data every five to 10 minutes, you re potentially going from waiting an entire day to get a result to waiting minutes. One reason for waiting to put the streaming software in use was the need to expand the Hadoop cluster, partly to handle the increased RelayHealth turned to stream processing to help its clients get immediate insight into their business operations. data volumes that Spark Streaming will generate. RelayHealth pulls an average of 28 million transactions per hour into the cluster, and Daya said more processing power had become necessary to get data both into and out of the system. An increase from 10 to 45 compute nodes was due to be completed shortly before Spark Streaming was turned on. Another prerequisite step was upgrading to a new version of Cloudera s Hadoop distribution that became available in December 2014 with support for Spark Release Korin Reid, a 4

5 CASE STUDY data scientist at RelayHealth, said the machine learning library built into earlier releases of Spark was very primitive, making it hard to build good algorithms. Reid added that she had started using the library in Spark to expand the set of algorithms the company plans to employ for analyzing the claims data. BE PREPARED FOR WHAT S COMING Daya said creating an overall IT architecture that can take advantage of stream processing technology is a must for a successful deployment. In addition to expanding its cluster, RelayHealth is adding the Apache Kafka message queuing technology to take data from HBase and feed it into Spark. Upstream business systems also need to be able to handle the real-time analytics information coming their way from Spark, including automated updates and actions, he said. And business processes at the operational end likely will have to be modified to some degree, which can create people issues for project teams to contend with. Really, it s a universal shift from doing things the way we ve always done them to being more data-driven, Daya said. It requires buy-in from senior management that this is a direction you want to go in. William McKnight, president of McKnight Consulting Group, agreed that stream processing and real-time analytics applications call for new ways of thinking in many organizations. There are a lot of mind things that go along with it, he said, adding that business managers and workers may need to be convinced of the wisdom of changing internal processes to get the full benefit of the new capabilities. Craig Stedman 5

6 BENEFITS Integrated Analytics Gives Users Operational Edge According to TechTarget s definition of the term, operational intelligence is an emerging discipline that allows businesses to intelligently transform vast amounts of operational data into actionable information that is accessible anywhere, anytime and across many devices. More concretely, it integrates business intelligence and analytics tools into operational processes to enable organizations to monitor business performance and empower executives and other end users to make better-informed decision. If all goes well, those decisions will lead to more business coming in, and in turn, higher revenue and profits. One of the key benefits of operational intelligence is the ability to proactively alert people on the front lines of an organization to business opportunities and influence the actions they take. Let s look at a few of the many business processes that can be improved by using operational intelligence technologies. Customer service. A call center application can be enhanced by pairing up customer profile data with analytical models evaluating the responses of callers in real time. As a conversation progresses, the probabilities of a customer responding positively to possible promotional offers are calculated and the call center agent s script can be modified on the fly. Hopefully, the end result is increased customer retention. Logistics. Deliveries can be made more quickly while improving driver safety through the use of an operational intelligence application that blends spatial data and information on traffic, weather and emergency alerts to calculate the fastest route with the lowest number of issues and hazards. If need be, new routing instructions can be transmitted to a driver s mobile device. Energy. At a utility company, smart grid data 6

7 BENEFITS and information on network status, equipment failure patterns and the weather can be combined with sensor-based automated monitoring to preempt electrical grid breakdowns and reroute transmissions as needed. That can help reduce the impact of power outages. Manufacturing. Productivity can be increased and failures resulting in downtime reduced through real-time monitoring of what s happening on the production floor, courtesy of continuous reporting on the status of assembly-line components. Operations managers can be notified when the metrics indicate an imminent equipment failure and take pre-emptive steps to replace parts. QUALIFYING CRITERIA So, what are the characteristics of a business process that can benefit from operational intelligence tools and techniques? To qualify, it should have well-defined performance metrics and specific business objectives associated with each metric. There should also be one or more decision points involving one or more of the users involved in the process, along with an internal recognition that business decisions can be impaired by an absence of information and that business performance can be impaired by ill-informed decisions. Other criteria include the ability to generate relevant insights by analyzing operational data and to take actions that can improve the business process based on those insights. It s important to recognize that none of these criteria requires any of the participating users to be technically savvy or have analytics knowhow. In fact, embedding the results of analytics applications in the framework of an operational process is what makes operational intelligence so powerful. To be most effective, though, an operational intelligence initiative should encompass these features: Real-time integration of data from a variety of sources, potentially including both human-generated and machine data plus the means to integrate analytics functionality into business processes and their supporting applications in order to get results and drive 7

8 BENEFITS operational actions in a prescribed window of time. Easy-to-grasp data presentation methods suited to specific business roles so that, for example, a manufacturing plant floor manager gets a different view of data than a production line operator does. Seamless integration with the various devices and applications used by the employees involved in the business process. Stream-oriented and event-driven notification, so that the system can send alerts and other messages to the right people in real time. HURDLES TO GET OVER There are challenges to making it all work. Performance is one: A mixed workload of data analysis tasks might need to be run in parallel for different participants in the process. Dealing with increased data volume and velocity is another: An operational intelligence architecture might have to combine stream processing and big data technologies to ensure that the required information can be captured and processed in a timely way. Despite the challenges, the growing adoption of big data systems coupled with the expanding use of sensors and computer-equipped machinery connected to corporate networks via the Internet of Things provides new opportunities for enhancing operational intelligence efforts. More complex analyses can be performed against a broader set of data sources and larger amounts of information. And a greater focus on integrating operational intelligence outputs with mobile apps will broaden information delivery capabilities. Adopting these approaches will help organizations meet the goal of making actionable information accessible anywhere, anytime and across many devices and in time to enable their business operations to take full advantage of that information. David Loshin 8

9 Q&A Operational BI Helps Boost Business Agility As organizations increasingly look to use data analytics to help drive their business operations, accelerate decision making and improve their ability to respond to changing conditions, operational business intelligence tools and techniques have become much more mainstream, according to Rick Sherman, founder of consultancy Athena IT Solutions. Operational BI is also becoming a more necessary component of running a business, no matter what size a company is, Sherman said in an interview with SearchBusinessAnalytics. At its most basic level, operational BI enables corporate executives and other end users to look at how their business processes are running, Sherman said. More advanced deployments can feed BI data directly to operational workers and automate both the analytics process and the business decisions that are based on what it finds. He noted that operational BI doesn t have to be a real-time analytics initiative but it usually is. In the interview, Sherman also talked about the business benefits and potential problems that can result from operational intelligence efforts, as well as emerging trends and best practices for successful deployments. What benefits can companies get from embedding operational intelligence capabilities into their business processes? As the operational BI functionality gets more advanced and sophisticated, they can better manage their business. The more sophisticated it gets, the better they re able to prospect for customers, manage and care for their customers, build products and make their operations more profitable, because they can look at various processes and see how well they work. It gives them more details on their business and more examination of what s working. Instead of intuition-based [management], it s 9

10 Q&A information-based. What kinds of challenges do operational BI initiatives pose that prospective users should know about? No matter what kind of operational intelligence you have, you always need to get into more detail. One of the issues you face is how extensive the pre-built operational BI capabilities you have with your enterprise applications [are]. The more extensive, the less extensively you have to fill in the gaps. The challenges you face beyond that [are] how do you handle it, what kind of data are you trying to go after, how is it accessible, what data do you have to integrate it with and how consistent are different data sets with each other. If you have multiple operational BI applications, which companies tend to do, then the issue is how do your business users manage moving between BI applications and how do you get consistent usage from them? Can you list some best-practices steps that project managers should take to help make operational BI deployments successful? They need to figure out what their business community needs. The best technique for that is storyboarding. Rather than going to a business user and asking them to list the operational reports they want, what you really want to ask them is, What information do you need in order to make a business decision? What decisions do you need to make? Look at the sequence of business decisions users need to make, then develop your operational reports based on those needs. The second thing is, based on what they need, what information is there, what can [be used] as-is, what needs to be supplemented or added. The third aspect is what kind of analytics people need in relation to BI. Do they need to see some kind of visualization? That impacts the design. And the last thing is, if you re pulling data from multiple systems, what s the consistency of data across the systems and how do you design for that? What are some noteworthy trends that are emerging in operational intelligence now? I think the first trend is the movement from operational reports to operational BI. We re moving from static reports to more 10

11 Q&A sophisticated visualization of the data and more data being able to be presented at one time to [help people] make decisions. Another thing is we re getting more self-service-enabled BI the ability to present not just pre-built tabular reports, but a selection of data in front of them that [business users] can pick and choose from. The third one is just that we re getting more and more data not just [a wider] variety but a much more comprehensive set of data that a business user can analyze about the processes of their business. Corlyn Voorhees 11

12 ABOUT THE AUTHORS CRAIG STEDMAN is executive editor of TechTarget s SearchBusinessAnalytics and SearchDataManagement websites. He has been an IT journalist for more than 25 years and was an editor and reporter at Computerworld before joining TechTarget in him at cstedman@techtarget.com and follow him on DAVID LOSHIN is president of Knowledge Integrity Inc., a consulting and development services company that works with clients on big data, business intelligence and data management projects. He also is the author or co-author of various books. him at and follow him on CORLYN VOORHEES is working as an editorial assistant for SearchBusinessAnalytics and SearchDataManagement through Northeastern University s co-op program. She is an undergraduate student at Northeastern, where she is double-majoring in journalism and communication studies. her at cvoorhees@techtarget.com. STAY CONNECTED! today Operational Intelligence Adds to Business Smarts is a SearchBusinessAnalytics.com e-publication. Scot Petersen Editorial Director Jason Sparapani Managing Editor, E-Publications Joe Hebert Associate Managing Editor, E-Publications Craig Stedman Executive Editor Linda Koury Director of Online Design Neva Maniscalco Graphic Designer Doug Olender Publisher dolender@techtarget.com Annie Matthews Director of Sales amatthews@techtarget.com TechTarget 275 Grove Street, Newton, MA TechTarget Inc. No part of this publication may be transmitted or reproduced in any form or by any means without written permission from the publisher. TechTarget reprints are available through The YGS Group. About TechTarget: TechTarget publishes media for information technology professionals. More than 100 focused websites enable quick access to a deep store of news, advice and analysis about the technologies, products and processes crucial to your job. Our live and virtual events give you direct access to independent expert commentary and advice. At IT Knowledge Exchange, our social community, you can get advice and share solutions with peers and experts. COVER ART: FOTOLIA 12

IoT Analytics Explores New Business Frontiers

IoT Analytics Explores New Business Frontiers As companies battle to gain a competitive edge, IoT analytics can provide critical insight into manufacturing processes, product development and customer purchase