Managing Oracle E-Business Applications with Concord s ehealth Suite

Transcription

1 Focus on Value 1 Managing Oracle E-Business Applications with Concord s ehealth Suite Concord's ehealth Suite provides an end-to-end, real-time view of Oracle application, system, and network performance across the enterprise. Oracle E-Business Suite applications drive an enterprise's business in real time. They are at the center of resources allocated and dollars spent to develop, improve, and maintain critical relationships with customers, suppliers, partners, and employees. Any downtime or performance lag is measured not only in seconds, but also in dollars lost and business opportunities missed. Concord's ehealth Suite assures the performance and availability of the critical business processes delivered by Oracle applications and the Oracle database. Concord s Business Service Management (BSM) products deliver the end-to-end visibility and real time management used by IT and business unit managers to proactively manage the technology on which the Oracle services rely. Companies that depend on Oracle applications to provide superior business performance also rely on Concord's ehealth Suite to provide superior application, system, and network performance for Oracle, , Internet, and other user services. The ehealth Suite Application Response, Service Availability, and Live Health empower systems and application managers to keep performance levels optimal and troubleshoot problems before they affect business performance. Deliver What Matters Most: Fast Response Seconds matter to an enterprise that depends on Oracle applications. The success of Oracle s real-time business processes depends on rapid, consistent response times from the underlying applications, systems, and networks. To keep performance high and highly available you must monitor and manage the entire application pathway continually. The best approach is a single, unified performance management system that alerts you to problems, helps you troubleshoot in real-time, and gives you the tools to manage and control the costs of application performance. Use the ehealth Suite to: Detect and resolve response time problems before they affect users. Achieve optimum performance by regularly tuning Oracle applications and infrastructure. Drill down from onscreen alarms for rapid troubleshooting. Implement 24x7 robotic testing of application availability and response. Maintain optimal performance following system or software upgrades. Forecast capital purchases based on accurate historical trend reports. With Concord's ehealth Suite of reports, alarms, and historical data, you can continually monitor Oracle applications and proactively resolve performance, availability, and capacity issues.

2 2 Troubleshooting Real-time Alerts T oday's Finance and Accounting staff plays a critical and fast-paced role in a fiscally efficient global enterprise. As they juggle a wide variety of tasks rapidly to meet company needs, governance requirements, and shareholder requests, Oracle's Financials gives them a needed edge. In their key position, application response time is crucial. If poor response from Oracle Financials applications slows them down, work falls behind and complaints rise as do costs. To prevent application slowdowns from hindering Finance group effectiveness, an application support team has a rapid response plan in place for dealing with Oracle Financials system problems. Application Response and Live Health are key components of their plan. When Application Response detects the early stages of declining response time, Live Health promptly alerts the on-call IT staff. Armed with seamless access to ehealth reports, team members use ehealth s robust historical and real-time analysis to quickly troubleshoot the situation. Using Threshold Alarms. The basis for the team s rapid response plan is the Service Level Agreement (SLA) developed jointly by IT and Finance group management. For each Oracle Financials transaction, the SLA states a precise, maintainable response time threshold with escalating levels of severity. For example, the SLA defines these thresholds: Warning -- response time over 3.0 seconds Minor alarm -- response time over 4.0 seconds Major alarm -- response time over 4.5 seconds Critical alarm -- response time over 5.0 seconds The team uses Application Response to measure response time against each threshold, and sets Live Health to trigger an alert if the time reaches a threshold. Live Health sends alerts via or pager message to on-call IT personnel. The message states the alarm s severity level, along with the client system, server, and transaction involved. When Application Response detects a threshold problem, the alert also quickly appears on a Live Status diagram (Figure 1). Live Status provides a real-time, high-level map of the company s business processes presented in a graphic form that resembles a fishbone. The icons in the diagram represent elements of the IT infrastructure and business applications and are arranged in logical groupings. For example, Figure 1 shows the Live Status diagram for the Oracle applications. Elements below the fishbone s center spine depict the status of application response times; those above the spine depict underlying IT components such as servers, switches, and LAN/WAN connectivity. Minor alarm for Oracle Financials transactions at one location (Dallas). Figure 1. Live Status Display for Oracle Applications (close-up). The icons are color-coded to indicate each element s real-time status. Green means the element is operating within an acceptable threshold; if a problem erupts and response time rises above a threshold, the color changes from green to yellow to orange to red as severity increases. Multiple, escalating alarms give the IT staff time to recognize, troubleshoot, and repair the problem before response times degrade significantly. At any time, a glance at the Live Status display gives the IT team and Finance group managers a complete real-time overview of the Oracle system status. On it they can readily spot alarms, know the severity, and can quickly assess the problem s scope and impact. Determining the Problem s Scope. When an alarm occurs, a look at the Live Status diagram (Figure 1) highlights the affected elements: there is a minor level alarm (yellow icons) involving the Oracle applications. When an IT team member hovers the mouse cursor over the yellow icons, the problem s location displays: Dallas. With the problem and location identified, the IT team is ready to find a solution. The Dallas location uses many Oracle applications, and the IT team needs to understand whether the problem is affecting all Dallas Oracle users or just the Finance group. To answer this question, they run the ehealth Ser-

3 3 vice Performance report. The Service Performance report shows an application s average response time for a specific location or functional group over time, and compares it to other locations or functional groups. In this case, the Service Performance report is filtered to show Oracle users in Dallas during the previous seven days. The resulting report (Figure 2) shows a separate trend line for each department in Dallas. A quick look at the report reveals that only the DallasFin group (yellow line) is experiencing the response time problem. It s critical to restore fast response to that group before Oracle Financials users become frustrated by slow service. Identifying the Problem s Source. To identify which part of the infrastructure is contributing most to response time problems (the client, the network, or the server), the IT team next runs a CIO Summary report tailored to show Oracle application use during the last 24 hours for DallasFin users. The resulting Response Observed chart (Figure 3) shows the average response time for each Finance transaction in Dallas, with a breakdown into client (red), network (green), and server (orange) time. From the Response Observed chart the team learns that all DallasFin transactions are over their 4.0 second minor thresholds and server time (orange bars) accounts for most of the response time. ehealth CIO Report CIO Report Fin_Login At the Dallas Financial offices, all Oracle transactions are slow (taking nearly five seconds) mostly due to server time (orange bars). AccntRec AccntUpd GL_Entry Fin_GL_Upd Figure 2. Service Performance Report for Oracle Users in Dallas. By learning that the problem affects all Oracle Financials transactions, the IT team understands that it is not caused by a specific transaction within the Financials application suite. By learning that slow response time is due to the server, they don t waste time checking the network for problems, nor do they need to check for client-side issues. Instead, they immediately focus their efforts on troubleshooting the servers. To check the servers, the team runs a Daily Health report (Figure 4). The Daily Health report focuses on a group of Oracle Financials servers used by DallasFin. The Element Detail section of the report includes a Response/Limit chart which summarizes application response times by server. According to the Response/ Limit chart (top of Figure 4), Response Endpoint Group Dallas_RS Figure 3. CIO Summary Report for Oracle Financial Transactions in Dallas. Fin_GL_Inq Service Performance Report Application/Group Performance GL_Glbl_Con 5K 4K 3K 2K 1K 0 Thu-03/04 Fin_iRecInq DallasAdmin DallasEng DallasMkg DallasSales DallasFin GL_HistInq Fri-03/05 Fin_Treas Location/Group Performance Sat-03/06 Sun-03/07 The Oracle slowdown is only affecting the Dallas Financial group (yellow line). Mon-03/08 Oracle Server2 is having significantly more trouble than the other Oracle Financials servers the red bar indicates it s reaching its threshold 90% of the time. For more detail on this situation, the IT team clicks on the red bar to drill down to an At-a-Glance report for Oracle Server2 (bottom of Figure 4). This report includes both system-level information and response information. The report s CPU Utilization chart shows that CPU use has jumped from an average of 40% to nearly 90%. Tue-03/09 Wed-03/10

4 4 A Successful Conclusion. To find the cause of the high CPU usage rate, the IT team checks the server for high-use CPU processes and finds the problem s source: a nightly backup job is running during the day. The backup log indicates that the job failed during the night and automatically restarted itself this morning. This backup job is interfering with Oracle response times for DallasFin users. The IT manager stops the job, and response times for the Oracle Financials users in Dallas return to normal levels immediately. Daily Health Report Subject: Oracle_Servers_RD OracleDb1 OracleDb2 OracleDb3 Oracle Server2 is not performing well. Clicking the bar drills down to an At-a-Glance report for the server. OracleDb4 OracletSvr1 OracleSvr2 OracleSvr3 This At-a-Glance report shows that CPU utilization on Oracle Server2 increased suddenly. OracleSvr4 OracleSvr5 WebSvr1 Response Endpoint WebSvr2 At-a-Glance Report Generic Response Destination Element OracleSvr2 Figure 4. Daily Health Report for Oracle Dallas Servers; At-a-Glance for Server2. Tuning For Faster Response E very second matters in a contact center. No one likes to wait on hold, wait for an answer, or wait for a callback. Seconds spent waiting add up quickly when thousands of calls per day are involved. The result is higher costs, lower productivity, and service complaints. That s why contact center managers who use Oracle TeleService keep an eye on a key measure of contact center performance: Average Handle Time (AHT) the amount of time a teleagent is occupied with an incoming call. Knowing the AHT is not enough; a dedicated manager wants to understand the why behind the number and looks for ways to lower AHT. System response time is a key contributor to AHT. Since a teleagent usually performs many individual transactions during a call, reducing response time can reduce AHT. Concord s Application Response tracks the response time for contact center transactions, and breaks response into server, network, and client time for ease of investigation. Watching Response Times. One savvy contact center manager who recognizes the impact that system response time has on AHT uses Application Response to help continually tune a contact center s systems. This contact center manager uses Application Response to track response times for Oracle TeleService transactions. The manager then compares current response times with target thresholds, looking for slow transactions and information bottlenecks to investigate and troubleshoot. One tool the manager uses daily for this purpose is a custom performance report (Figure 5). The report, readily made by importing ehealth historical data into Microsoft Excel, makes it easy to compare the average response times experienced by teleagents in two locations: Boston and Seattle. The response time for each type of monitored transaction (Account, Service Profile, Invoice, and Service Order) is depicted by a green bar, while their performance thresholds appear as red lines. Comparing Response Times. The report makes it clear that in both centers, the Account and Service Profile transactions are under their thresholds, Invoice is at its limit, and Service Order exceeds the maximum response time at both locations. The report s Transaction Count indicates that Invoice is performed less frequently than other transactions in both locations. Therefore, the manager gives it a lower priority and

5 5 instead investigates the Service Order transaction first, due to its high frequency of use and slow response time. The report also shows that, while the Service Order transaction is slow in both locations, the Seattle contact center experiences a noticeably slower response on all transactions than the Boston contact center does an item also worth examining. Investigating the Results. To investigate response times for the Service Order transaction, the manager runs ehealth At-a-Glance reports focusing on the Service Order transaction: one each for Seattle and Boston. Figure 6 shows the At-a-Glance reports for Seattle. The Seattle At-a-Glance report indicates that average response time for the Service Order transaction consists of a small amount of client time and significant amounts of network and server time. However, the Boston At-a-Glance report (not shown) indicates that, The Service Order transaction uses mostly network (yellow) and server time (blue), and a small amount of client time (green). It also requires many round trips to complete. Account and ServProfile are within limits. Application TeleService-ServOrder Invoice is at its threshold. Service Order significantly exceeds limits. Figure 5. Daily Contact Center Performance Spreadsheet while the Boston group experiences a similar level of server response time, their network is faster. The manager can also see that, at both locations, the Service Order transaction requires several network round trips per transaction. Based on this information, the manager has the application developers redesign the Service Order transaction to reduce the number of network round trips by gathering several pieces of required data at one time, rather than in separate network/server queries. The IT team also investigates and corrects slow network connections at the Seattle office. Together, these actions result in Service Order response times dropping to within thresholds at all contact centers and improving AHT overall. Figure 6. At-a-Glance Report for the Service Order Transaction in Seattle. Because Concord understands the complexities of managing the Oracle application infrastructure, the ehealth Suite equips you with the reporting and diagnostic tools you need to consistently meet your users requirement for application reliability and fast response time.

6 6 Testing Availability Continuously K eeping Oracle s Sales application available and responsive 24x7 for a global sales force is a daily challenge for the IT team at a multi-national company. The company s farflung host of sales reps operate on every continent, and supporting them requires that complex pricing and product configurations, inventory status, and quote approval come together promptly when the customer is ready to commit, or months of high-end sales negotiation will be delayed. To help meet their demanding service level agreements, the IT group uses Concord s Service Availability to continuously and actively test system, application, and network availability and response. Because Service Availability s robotic testing does not rely on user activity, it provides a steady stream of response data 24x7. The data stream allows them to use Application Response and Live Health displays, alerts, and reports to assure uninterrupted service worldwide around the clock for all functions of the Sales application. Plus, the IT group uses the information to monitor response trends and compare performance among distant sites. Configuring the Active Tests. To accurately monitor the system end-to-end, the IT team co-locates Service Availability test systems in the field with users. In this case, field sales regions in Berlin, Buenos Aires, and Johannesburg access an Oracle server cluster in New York. To continually test the Oracle servers, the IT team deploys a set of tests to run against the server: a ping test, DNS test, TCP Connect test, an HTTP test, and a custom test of several key Customer Order Management functions. The IT team runs Top N reports each week to compare the availability and response statistics among the three major sales regions centered around Berlin, Johannesburg, and Buenos Aires. The Top N report in Figure 7 shows the average response times for each test, as well as the deviation in response time from the goal of an average five second response. The report shows that, while the application was available 100% of the time at each location, there is a difference in response times. The Buenos Aires and Johannesburg regions have similar average response times (3.3 seconds). However, the Berlin service is much slower (nearly 4.5 seconds). While Berlin is still operating within acceptable limits, the team will Berlin response time is far higher than other regions. It is near the five second maximum threshold. Figure 7. Top N Report Compares Test Results Among Sales Regions. investigate the reason for the longer response time before the problem results in complaints and business interruptions. Getting to the Details. To obtain details about the response performance, IT drills down to an At-a- Glance report for the response path for the previous 24 hours for each of the cities. When the team compares the reports, they find the Berlin report (Figure 8) shows that time spent shuttling data over the network is a major factor in Berlin s total response time. To improve performance, the IT team provides a faster network connection from Berlin to the New York Oracle server cluster. When that is done, active tests show that Sales users in Berlin once again experience response times closer to those of the other regions. Network time accounts for most of Berlin s average transaction response time (yellow). Figure 8. At-a-Glance Chart Showing Average Response for Berlin.

7 7 Monitoring an Upgrade s Impact A pplication and systems upgrades are both a blessing and a curse. While they bring helpful new features and functions, they also introduce changes that can disrupt a stable infrastructure and inflate well-tuned response times. So when Oracle released an upgrade to the iprocurement application, the IT staff took a cautious approach to putting it online. They will use Application Response to spot any changes in response time as they upgrade each server in the cluster and to immediately troubleshoot problems if they occur. Plus, since Application Response has logged many prior months of historical data, they can also compare pre- and post-upgrade performance in any time frame hour, day, week, etc. With Application Response, they can assure users that the rolling upgrade will be closely monitored to avoid response degradation and the IT team can quantify any performance improvements the upgrade might bring. Applying the Upgrade. As they upgrade each server, the IT team runs an At-a-Glance report for the iprocurement application. The report, based on aggregated response time data for all servers in the iprocurement cluster, shows that the average response time for the application is rising a little with each successive upgrade it goes from two seconds to nearly four seconds. The Average Response Observed chart (Figure 9) from the At-a-Glance report shows the change in response time and a breakdown of response time into client, network, and server time. Server response time (blue) increases during the upgrade. Figure 9. At-a-Glance Chart of Average Response During Upgrade. The report shows that the largest contributor to response time is the server. When the IT staff looks more closely at the upgraded servers, they conclude that the servers need adjustments made to their performance parameters to accommodate the software upgrade s new functions. After tuning, a new At-a-Glance report shows that server response times returned to normal ranges with no increase in overall response time from the previous version of the iprocurement application. Planning Accurately for the Future T he company has been growing steadily, new employees are arriving each week, and sales forecasts are up. The IT manager and the business unit manager know they have to do some planning. They need to determine how long the existing infrastructure can support increasing levels of Oracle application activity. They need to know where to make capital investments to assure adequate capacity and response times as the business grows. And they want form their plan using facts, not guesswork. To meet these needs, they turn to the analytical features in the ehealth Suite. For months, Application Response has been measuring actual, observed response time from the end user's point-of-view and storing aggregate historical data. The data has been used many times by the IT team to troubleshoot problems and to maintain system performance. Now the managers can mine this historical data to spot long term trends and weak infrastructure areas that impact response time. Plus, ehealth will provide the statistical analysis and reports to validate and document plans. Getting Insight. The first place the managers turn for insight is the Situations to Watch report. The Situations to Watch report is an analytical report that identifies the top 10 conditions within the IT infrastructure that require attention or resolution. The managers examine the Situations to Watch table (top of Figure 10) to learn which devices or resources are approaching their defined thresholds. Ranked number one on the list is the Login transaction

8 8 on Oracle Server4. For that Login transaction, the Days to Threshold column indicates that, based on the current trend, response time will reach its limit in 95 days (about three months). The remaining situations-towatch indicate similar trends for other Oracle application-related elements. Clearly, the managers must take some action, soon. But they have a question to answer first: in what equipment or resource must they invest to head off this impending response time problem? To answer the question, the managers take a closer look at the elements listed in the Situations to Watch table. Starting with the element at the top of the list, they drill down to an At-a-Glance report (bottom of Figure 10). Because this first element is a server, the Ata-Glance report graphs the average response time for all transactions performed by the server, with a breakdown into client, network, and server time. It also shows the number of transactions per minute. The report shows that, as the rate of transactions increases, it is server time not network latency that is causing the response time increase. Making a Sound Plan. Armed with this hard evidence, the managers quickly make plans to purchase and deploy two additional servers as soon as possible. By examining trends and making use of ehealth s forecasting features, the managers are able to plan for and justify this capital investment appropriately to avoid serious degradation in service. The increase will cause average response time to exceed its threshold in about three months. At-a-Glance Report Generic Response Destination Element OraclSvr4-RD Over the past two months, server time for Oracle Server4 transactions has increased while client and network time have remained constant. The increase in response time corresponds with a parallel increase in the number of transactions per minute (from 120 to 180 transactions per minute). Figure 10. The Situations to Watch Report (top) Lists the Top 10 Conditions to Monitor. The At-a-Glance Report Shows Details.