Predictive Analytics within Tivoli

Transcription

1 Predictive Analytics within Tivoli Simon Coote Technical Professional IBM Denmark

2 Statement Of Direction And Intent All statements of direction or intent are provided for planning purposes only and are subject to change or withdrawal without notice This presentation represents our current goals and objectives Our development teams are now studying the product source code We are listening to customers views on the future directions of our solutions 2

3 Agenda General Principles Tivoli s strategy & approach Examples and Tivoli Capabilities ITM overview, features and roadmap Q & A 3

4 What is Predictive Analytics Management Predictive Analytics is all about Outage Avoidance! Moving IT from reactive to proactive/predictive CIOs want to share this outage avoidance data with management Helps to show what IT is saving and delivering versus just being a cost center 4

5 Why is it important? What if you could guarantee a customer they would never run out of capacity? What if you took the chance out of decision making? What if machines could act alone to prevent or contain problems? What if downtime became a thing of the past? 5

6 What we have heard so far.. I d like 30 minutes warning to know when my user experience is going to deteriorate. What are realistic baselines for my environment? I d like to adjust for repeating traffic situations so I don t receive false alerts. This device is out of service and critical to one of my core business services. I need to know the instant it comes online. I d like my operators to be able to more quickly diagnose certain events and patterns to implement fixes. 6

7 Key Elements of Predictive Analytics Predictive Business Service Health: Highlight potential service impacts before they occur by analyzing resource, service, and business data, prioritizing IT operations. Learning/Adaptive Management: Self-adjusting management of IT measurements based on business priorities, behavioral patterns, and troubleshooting Baseline Dynamic Threshold Predictive Alerts Baselining: Short and long term collection of metrics from performance and event data for normal range of values as applied to discrete time intervals across a calendar. Dynamic Thresholding: Self learns and suggests / sets thresholds based on baseline analysis. This allows Operations Management to set reliable thresholds. Predictive Alerting: Proactive notification of abnormal/normal behavior with sufficient notice prior to a failure, applicable to any data set. Knowledge Libraries Perf. Data Store Knowledge Libraries: Predefined knowledge to focus effectiveness of adaptive management, best practices for correlation, and maximize product value. Runbook Automations: Extensible collection of automated actions and policies orchestrated around common operational tasks. Rules Files: Maintained knowledge base of problem/cause/resolution analytics built from thousands of hours of experience Situation Workflows: Maintained best practices collection of resource management policies Performance Data-Store: Centralized repository for performance and event data at domain, event, and business service layer 7

8 The Opportunity Stand-alone, Availability & Performance Management has generally been a reactive practice Recent developments at Tivoli will enable predictive capabilities allowing outage avoidance Now you can fix things before they break! 8

9 Agenda General Principles Tivoli s strategy & approach Examples and Tivoli Capabilities ITM overview, features and roadmap Q & A 9

10 Tivoli s strategy for Predictive Analytics Management Built in predictive analytics & value at all management layers - not an overlay product! Internal organic growth' of PAM from our existing product sets No additional add-on component(s) to our existing BSM/COM solutions Inline adaptive technologies included everywhere Additional analytics for specific technology domains 10

11 A modular approach to Predictive Analytics Management BSM Events Monitoring Dynamic Thresholding of service KPIs. Context-driven root cause analysis to identify the impact to services and users. Evaluate SLA and financial impact of services associated with Predicted Alerts Tags and displays Predictive Alerts Supports 3 rd party feeds in real-time Associates Predicted Alerts with impacted IT resources Dynamic Thresholding includes 3 rd party and Mainframe! Improved Trend Baselining fewer false/positive alerts, proactive forecasting 11

12 Predictive Analytics: A New Approach to Deliver Outage Avoidance Resource Power App/Sys Database Network Host Security Storage Service 12

13 Predictive Analytics: A New Approach to Deliver Outage Avoidance Adaptive Baselines Peaks Baseline Transact. Memory Bandwidth CPU Intrusion Detection Space KPI Resource Power App/Sys Database Network Host Security Storage Service PMDB Experience Libraries 13

14 Predictive Analytics: A New Approach to Deliver Outage Avoidance Dynamic Threshold Adaptive Baselines Dynamic Threshold Peaks Baseline 10% Errors % Change Time of day Transact. Memory Bandwidth CPU History Local External All Intrusion Detection Space KPI Resource Power App/Sys Database Network Host Security Storage Service PMDB Experience Libraries 14

15 Predictive Analytics: A New Approach to Deliver Outage Avoidance Ability to aggregate or focus by domain/resource, local need, schedule, history, or external. Predictive Alerting Dynamic Threshold Adaptive Baselines Predictive Alerts Dynamic Threshold Peaks Baseline 10% Errors % Change Time of day Transact. Memory Bandwidth CPU History Local External All Intrusion Detection Space KPI Resource Power App/Sys Database Network Host Security Storage Service PMDB Experience Libraries 15

16 Predictive Analytics: A New Approach to Deliver Outage Avoidance Evolve Your Approach to Management Ability to aggregate or focus by domain/resource, local need, schedule, history, or external. Predictive Alerting Dynamic Threshold Adaptive Baselines Predictive Alerts Dynamic Threshold Peaks Baseline 10% Errors % Change Time of day Transact. Memory Bandwidth CPU History Local External All Intrusion Detection Space KPI Resource Power App/Sys Database Network Host Security Storage Service 16 PMDB Experience Libraries

17 A Differentiated Approach to Predictive Analytics Competitive Products Add a layer for predictive: generate intelligence from one perspective only. Limited collection/integration: require mapping into 3rd-party sources, build baselines from the scratch Lack domain experience: thrashing of realtime data only. Not efficient or scalable: collecting any/all data that may not be pertinent. Visibility gaps: infrequent data collection (30 min) impacts reliability. Uneven data capture and trending: gaps in historical perspective Tivoli Solution PA across all layers: Built-in PA span all levels of technology stack! Broad collection/integration: Largest available experience library of collectors, integrations, and run-books! Robust domain experience: We re investing more intelligence up-front! Efficient & scalable: We collect the right data, not just lots of data! Robust visibility: Get the metrics that matter most, more frequently! Maximum intelligence: Nimble approach to collecting & storing data for maxim Intel Only Tivoli collects the right quantity & quality data, at the right intervals, to ensure more meaningful, accurate analysis across the broadest mix of technology & layers! 17

18 Extending the Tivoli Suite with Predictive Analytics Built in predictive analytics & value at all management layers -- not an overlay product! Predictive Analytics Predictive service impact, RCA, SLA tracking & event/perf mgmt. Multi-layer Approach Assures predictive value at any phase of customer maturity. Broad integrations across IBM & 3 rd party collection 18 Broad predictive collection & experience across: mainframe, power, virt., SOA Common warehouse & Visual, Nav, SSO, Process automation Extended analytics for metric forecasting Additional domains including security and storage Roadmap to Predictive Leadership Robust CMDB integration Run Book Automation Extended 3rd party agent library Maximizes scalability to manage any size environment Maximizes OOTB Intel, while minimizing collection of irrelevant data Leverages and improves ROI on existing investments Performance Management Database

19 Bringing Predictive Analytics to Tivoli Future Q Adaptive Monitoring delivers a self tuning management environment TBSM KPI integration expand the offering Future offering broadens support to Security, Storage, and all key other data sets. IBM Philosophy on PA Dynamic Thresholding & Predictive capability built-in to products you already know Adaptive Monitoring Predict Service Outages Predictive Alerting and Event Tagging Dynamic Thresholding 19 No additional charges Easier data source integration. Broad domain coverage & works with 3 rd party data

20 A step-wise approach to Predictive Analytics Learn your normal system behaviour (including seasonal changes, e.g. weekends)..and automatically determine thresholds based upon what s learnt Ability to trend forwards and predict threshold violations Multi-variable analysis learn dependencies between metrics Trend for all of the above combined with a Service Dependency Model RCA Predictions about Service Predictive Analytics specific TBSM dashboards Event Pattern learning and recognition Problem Fingerprinting 20

21 Predictive Analytics Technologies: Tomorrow (2009+) Complementary to Inline Calculations and Optimization adds: Out-of-the-Box Integrations 3 rd Party Systems Workflow GUI for Runbooks Abnormal behavior pattern detection/alerts Adaptive service models Cross domain correlation on discovered relationships Inline adaptive technologies included everywhere Additional analytics for specific technology domains Inline calculations without PMDB Storage Systems Experience Libraries BSM Integrations and Mappings Experience Libraries Experience Libraries Events Integrations and Mappings Monitoring Agent or Agentless Network Dynamic Threshold Predictive Alerts Baseline Dynamic Threshold Predictive Alerts Baseline Dynamic Threshold Predictive Alerts Baseline Optimization Predictive Alerts Baselining Dynamic Thresholding Applications PMDB PMDB PMDB Dynamic Threshold Predictive Alerts Baseline adds: Additional metric forecasting methods Optional inline for monitoring baselines without the PMDB 21

22 Agenda General Principles Tivoli s strategy & approach Examples and Tivoli Capabilities ITM overview, features and roadmap Q & A 22

23 Leveraging Base lining and Dynamic Thresholds Thresholds defined like this today 80 Process abc CPU% profile.... Thresholds defined with base lining and dynamic thresholding. Varying by time 0 6am 7am 8am 9am 10am 11am 12am 1pm 2pm 3pm 4pm 5pm 6pm 7pm 8pm 9pm 10pm Automated definitions with + or - variation Proactive warning when abnormal behavior occurs during nonpeak periods Automated updates when changes take place 23

24 Auto-generated for each key metric, per collection agent, based on learned behavior Can adapt to important time windows based on business calendar overrides Dynamic Thresholding settings can be automatic or change controlled Automatic Thresholds set but provides override and centralized coordination for remote collector learning behavior 24

25 Predictive Alerts generated based on adaptive baselines and intelligence on monitoring, event, and BSM data sets Forecast predictions are based on many learned behavior factors 25

26 Outage Avoidance: Biz Prioritization from Predicted & Actual State of Services IT Operations now managing from the Predictive and Actual states of Services. Proactive Management 26 BSM helping further drive IT Operations by aligning Correlated Actual and Predictive States into Business Reprioritized actions

27 Advanced Monitoring Analytics - ITPA What ITPA Does Provides hands off capacity monitoring Automates performance analysis and reporting Enables prediction of application bottlenecks and creation of alerts for potential service threats. Supported Scenarios What will my resources look like tomorrow, next week. next month or next year? What IT Resources should I worry about next? Will I have enough capacity to get me through Monday? A seamless extension to ITM 6 Uses existing ITM agents Derive new metrics using arithmetic expressions Predictive trending and forecast reports Out of the box reports for distributed systems Fully Extensible Supports the Universal Agent and Agent Builder 27

28 Example of Predictive Analytics within BSM Event and Performance Bring IT, Change, Business, and Predictive events together in advanced visualization techniques to drive further correlation of any Service Affecting Event with the real-time Health of Services. 28

29 Example of Predictive Analytics within BSM Event and Performance Bring IT, Change, Business, and Predictive events together in advanced visualization techniques to drive further correlation of any Service Affecting Event with the real-time Health of Services. Reflects impact of Events on Services in Real-time 29

30 Example of Predictive Analytics within BSM Event and Performance Bring IT, Change, Business, and Predictive events together in advanced visualization techniques to drive further correlation of any Service Affecting Event with the real-time Health of Services. Reflects impact of Events on Services in Real-time Delivers Outage Avoidance 30

31 Example of Predictive Analytics within BSM Event and Performance Bring IT, Change, Business, and Predictive events together in advanced visualization techniques to drive further correlation of any Service Affecting Event with the real-time Health of Services. Reflects impact of Events on Services in Real-time Delivers Outage Avoidance Move from reactive to proactive and measure 31

32 Example of Predictive Analytics within BSM Event and Performance Bring IT, Change, Business, and Predictive events together in advanced visualization techniques to drive further correlation of any Service Affecting Event with the real-time Health of Services. Reflects impact of Events on Services in Real-time Delivers Outage Avoidance Move from reactive to proactive and measure 32

33 Predictive Analytics to extend Manager of Manager Adds Hybrid Fault/Perf. capture, dynamic thresholds, alerting at collection level Manager of Manager Predictive Alerts Situation and Predictive Events EIF Probe Omnibus Webtop Console Alerts.status Baseline PMDB *Performance Analyzer TDW Gateway for Processed Events Health PM Data Baseline Predictive Alerts Automations TEMS Dynamic Threshold Dual-headed Probe/Agent Dynamic Threshold Probes Same API integration, dual feeds outbound 3 rd Party Agents Fault and Monitoring event feeds Performance polling and feeds 33

34 Example of Adaptive Monitoring in OMNIbus Silent Node A Link down Buffer full BER HIGH Buffer full No event from Manager A for 24 hours Last Event from Node X Time Custom Automation Custom Custom Automation Automation Regular Event flow X 34

35 Example of Adaptive Monitoring in OMNIbus Signal Success 12:42: Signal Failure 12:38: Signal Success 12:23:19 12:05:45 12:42: Signal Failure 12:00:05 12:20:51 12:38:26 Time elapsed: 1 hour ObjectServer Admin log Past hour: signal failures 35 Probe Engineer paged: no. of failures

36 Agenda General Principles Tivoli s strategy & approach Examples and Tivoli Capabilities ITM overview, features and roadmap Q & A 36

37 Service Availability and Performance Optimizing Architecture, Applications and Service Performance Visibility Visualize service performance and health across all network, server, middleware and application components. A Lifecycle Based Availability Solution for Success Plan for growth & service enhancements Fix the problem Determine the root cause Isolate the bottleneck Remediate Diagnose Isolate Optimize IT Availability Life Cycle Service Management Discover Correlate Sense Application Availability Business Service Management Discovery of Application landscape and supporting infrastructure Monitor end user experience, applications and supporting infrastructure Correlate & Analyze Events Consolidated Operations Management Control Increase effectiveness and productivity, reduce errors and improve availability through consolidated tooling. Automation Keep costs under control as all aspects of infrastructure grows with integrated policy-based automation. 37

38 ITM and ITCAM Portfolio ITCAM for Transactions Comprehensive Set of Response Time Capabilities ITCAM for Applications Broad Application and Application Infrastructure Monitoring Capabilities ITCAM for SOA Platform Integrated Service, Middleware and SOA Enterprise Management Offering ITCAM for WebSphere/J2EE Deep dive problem determination and resolution ITM for Microsoft Applications Health Monitoring of Microsoft Operating Systems, Application Infrastructure and Applications ITM Health Monitoring of your Operating System ITM for Energy Management Reduce Power Consumption ME for AIX System p Optimization 38

39 Tivoli Monitoring Architecture Highly Scalable from SMB to Largest Enterprise. (1 to over monitored endpoints) Agent-based and Agent-less both natively supported Agent-based technology resides directly on a managed server and collects data based on policy set locally or by the management server Agentless technology resides primarily on a management server and gets its data via a remote application programming interface (API) Extensible architecture at the agent and server level Universal Agent Agent Builder Fully Integrated Warehousing Adaptive Event Management to allow customized thresholds for individual resources/servers 39

40 ITM v6.2.1 Overview A release that makes larger scale deployments more effective, dynamic and accurate and allows an extended team to directly contribute expertise! Customers can leverage ITM s capability for self-adjusting management based on business priorities, behavioral patterns, and troubleshooting Dynamic Thresholding Self learns baseline data and sets thresholds using real-time and historic data to set reliable thresholds. Works with all solutions (agent and agentless, custom and out of the box) Extend the reach of the monitoring team by enabling self-service monitoring, that is SMEs to set and manage thresholds for their resources. SMEs who know their resource s behaviours and business requirements may now directly manage thresholds for their resources.. 40

41 ITM v6.2.1 Overview A release that makes larger scale deployments more effective, dynamic and accurate and allows a broader operator set to contribute expertise! Customers can now choose the out of the box monitoring technology for each resource and situation agent or agentless. Agentless technology for easy and fast deployments that is less intrusive Agent technology for secure, scalable, continuous mission critical operations Capability to deploy mixed solutions (agent/agentless) deploy what makes business sense! Extend out of the box monitoring solutions with IBM Tivoli s Agent Builder with agent-based or agentless custom monitoring solutions Use a graphically driven, easy to use Agent Builder that can deliver extended monitoring or performance data in hours or days Automatically integrate the solution into the Tivoli Data Warehouse, build reports or forward alerts to an event manager like OMNIbus. 41

42 ITM v6.2.1 Overview A release that makes larger scale deployments more effective, dynamic and accurate and allows a broader operator set to contribute expertise! Customers benefit from improvements in infrastructure performance, management and reliability Significant performance improvements in the scalability of remote deploy parallel distribution can dramatically reduce deployment and upgrade times. Additional health monitors improve availability and management TDW and S&P monitors, Agent Watchdog. Broad ITM solution can be extended across the portfolio Comprehensive solution included in the box -- TEP/TPM/TCR Touch points to other solutions ITM, ITCAM, Green Energy, ME for AIX etc. Dynamic event content mappings for OMNIbus/TEC (EIF Formated events) TPM/TCM integration with SPB/SPDs 42

43 Dynamic Thresholds the drivers What s the right alert for DB2 cache buffer pools? I can t manage thousands of individual situations! How can I easily create different thresholds for different times of day? 43

44 ITM Dynamic Thresholds Fixed Thresholds Adaptive Thresholds Same thresholds all targeted agents Customized to individual agent / resources Self-Learning based on history or Data warehouse analysis Effective based on schedule Derived based on local needs and observations 44

45 ITM Dynamic Thresholds The Policy Editor allows for easy creation of policies, which can be created and managed at the TEMS or edited on the monitored servers and synchronized. Use Central Warehouse to establish Baselines Distribute Thresholds for real-time evaluation of thresholds to spot exceptions Enables Distributed Evaluation in real-time with Central Control Centralized Editing Policy Editor Adaptive Monitoring Policy Repository Local Threshold Policy ITM Agent HUB TEMS ITM Enterprise Management Remote TEMS Local Threshold Policy ITM Agent 45

46 Dynamic Threshold - example CPU_Warning Situation CPU >= 75% Agent B CPU_Warning CPU >= 85% Agent C CPU_Warning CPU >= 80% from 8AM 5PM Agent A CPU_Warning CPU >= 75% on day 30 or 31 of month TEMS Agent E CPU_Warning CPU >= 90% Only after 10 Minutes Agent D CPU_Warning CPU >= 90% during weekend 46

47 TEP Situation Override Editor Situation formula being overridden (reference formula) Selected distribution for override (MS or MSL) Calendar condition Empty condition represents <default> override Highlighted items represent overridden values <threshold> Attribute condition <key> 47

48 Dynamic Thresholds Apply to Anything Full Resource Monitoring Coverage zseries to Distributed Integrated Virtualization Broad Resource Coverage Any Metrics Monitored using ITM or TEP Baseline any Metric in the Warehouse Agent or Agent-less Monitoring Out of the Box or Custom Agents End to End Resource Monitoring - Operating Systems - Vmware, pseries, zseries, Citrix - DB2, MS SQL, Informix - Exchange, Domino, SAP - Storage and SANs - IMS, CICS, USS - Custom & Agentless Solutions Security Operations Monitoring 48 Storage Systems - SANs and Storage Devices - DR and Data Recovery Network Monitoring - Multi-Protocol Topology and Root Cause - Telecom Class Data Collection End User Response and Application Monitoring - User Transactions - Web Resources - Tracking and Diagnostics

49 Other notable features Agentless / Remote monitoring Out of the box option for remote monitoring Out of the box situations Agent Builder Script and Log usability enhancements User friendly browse for script and log files CIM data provider Agent Management Services Monitor the health and availability of ITM agents GUI for remote deployment of agents Visibility into current status Asynchronous deployment in parallel Enhanced ITM / TADDM integration 49

50 ITM - Roadmap Product 1H09 1H10 2H10 Current version ega: Nov 2008 v6.2.1 V6.2.2 v7.1 Dynamic Thresholding Agentless OS monitoring packages Agent Builder Support 100+ connections for remote monitoring TADDM exploitation of ITM agent for discovery ITM Infrastructure Event Slot Customization Improved EIF Event Integration Reduced Agent Disk Footprint FIPS 140 Compliance for TEPS CLI Approx 32 commands added, incl. CLI for historical data configuration collection Remotely invoke pdcollect tool Expand tacmd createsit (display item, consecutive samples, state) zseries Support TDW on z 64-bit zlinux OTEA Support TEC Events from z HUB 50 Visualization Enhancements for Dynamic Thresholding Time-to-value enhancements for ITM and ITCAM Family of Products Agent Configuration for Autonomous Operations Reduced agent disk footprint Local agent config files (optional) SNMP Eventing from agent (optional) ITPA Bundled into base ITM Product (2-in-a-box) Agent Builder Java, JDBC, SSH, HTTP(S) Providers Data Warehouse Granular Collection Infrastructure Agent Management Services Platform Coverage CLI tepslogin aka v6.3 Granular RBAC Security Model Options customized by user/action/object Very Large Scale Mgmt Policy-based admin/agent mgmt Federation of multiple installations at UI Agent Autonomy http server update to config Scheduling Service 64b exploitation Warehousing Granular S&P Autonomous agent warehousing ITM TEP-TIP Integration Serviceablity Maintenance application Audit Self-Monitoring Service Status Display Platform currency

51 Questions? 51