Capacity Management - Telling the story

Size: px

Start display at page:

Download "Capacity Management - Telling the story"

Beverly Mathews
6 years ago
Views:

1 Capacity Management - Telling the story

What is a Story? It is either: a. an account of incidents or events b. a statement regarding the facts pertinent to a situation in question Data is nothing more than 1 s and 0 s.

2 What is a Story? It is either: a. an account of incidents or events b. a statement regarding the facts pertinent to a situation in question Data is nothing more than 1 s and 0 s. It s what we do with it that makes it powerful. We can collect or capture as much data as we want to from our systems, from our business level, from our service delivery level and store it but it s what we do with this data that is the important thing.data used in a meaningful way is very powerful. What is the Story? What is happening in the current environment this is typically called the baseline. When analyzing data in our systems we want to identify a normal behavior period which shows the demands an application is making on resources in usual circumstances. Concise information We need to state the facts and not over complicate matters, presenting clear and concise information. Display forecasts How do we present this story to our audience? In terms of capacity management we could be using forecasting methods such as: Trends Models We need to describe to our audience so that they can understand easily the point/ message that we are trying to get across to them. We must also ensure that we are getting the right information to the right people in the right way. Gather further information Do we need to actually gather any additional information or do we have enough information? You may need to supplement resource data with some business data, or perhaps speak to the Service Delivery Managers to get the SLA information. When we are forecasting it is important to have as much information as possible from the business because we need a full understanding of what we are forecasting to. 2 Capacity Management - Telling the story

What are we attempting to display? Display current state of the environment this is our baseline, a period or periods which reflect our normal pattern of behavior.

3 What are we attempting to display? Display current state of the environment this is our baseline, a period or periods which reflect our normal pattern of behavior. You need an understanding of the business to class a period as normal behavior. Display possible anomalies This involves looking at periods and determining whether the peaks in a set period of time are an anomaly or whether they are just normal user busy periods of the day, explaining peaks in the data. Has there been some kind of change made that you are aware of? You may need to perform some root cause analysis to get to the bottom of peaks where you don t already have an understanding of what is causing them. Display forecasts Trends are very useful. You can trend to a limit, you can trend to a threshold etc but if you wish to get a better degree of accuracy to your forecasting you may wish to model. Models analytical modeling can provide you with a better degree of accuracy when forecasting, especially when it comes to things like utilizations where you have to take in to consideration the utilization curve. Different stories for different audiences When we say different audiences what we are talking about is the different layers of your Organization, the people who are involved in the process. C level executives At the very top we have the C level executives. When we are telling the story to our C level executives these are the key points that we need to make sure we follow when presenting our information: Keep the information concise we have to make sure that they understand very quickly what we are trying to say to them. Elevator talk keep your information as brief and to the point as possible, they don t have the time or inclination to wade through pages of reports. Summary and findings first make sure at the top there is always something called the management or executive summary, where it clearly states the relevant points Capacity Management - Telling the story 3

4 of our findings. They just need the facts. Findings Do we need more resources? When are we going to hit our limit? What hardware do we require? How much money do we need and when do we need the money? Make sure that the requests are clear and concise. Leave detail reports in pocket The detail which provides the basis of our findings is important as it backs up our management or executive summary. They need to be included but are unlikely to be read at this level. Business owners At the Business level, you will need to focus in on a specific area and the business owner will want to know: What are the trends for their area? How does it affect their area? What do they need to budget for? The information provided to the business owner will be only that which is pertinent to their department. Technical At the technical level you will need to provide all the information that went in to the pocket of the management or executive summary. Show me the details they will want to see the full reports, graphs, data and drill downs. At this level the figures mean something. Show me the trends they will want to see the trend reports and the what if scenarios. Be prepared when imparting good or bad news at any level anticipate your audience reaction and be ready for it. Different presentation mediums There are a variety of different presentation methods that we can use: Dashboards / Web portals these typically show red, amber and green status as they are very quick to assimilate. More suited for technical staff and business users Different look for each area 4 Capacity Management - Telling the story

5 Word documents More suited for C level executives Up front summary is critical Historical document PowerPoint documents Used for different types of presentations Good for discussion groups Types of Data required Technical data in ITIL terms our data should be coming from business, service and resource levels. This data is then fed in to our CMIS and allows us to see what is happening in two ways: Current how everything is performing now History the more historical data we have on our systems and applications the more accurate our trending and modeling will be. Business metrics what is happening in the business will dictate the resources needed to support it. Current what is happening now in the business Forecast what is planned for the future, in terms of growth, new services, increased user numbers etc Key Performance Indicators we need some idea of how we can measure our performance going forward. Threshold levels we need to know what thresholds we are going to be planning towards to enable us to put them on reports and see when/if they are going to be breached. Capacity Management - Telling the story 5

business data and KPI s to your CMIS. This allows you to build the most accurate picture of your environment as you can.

6 Capacity Management The diagram below shows 360 Capacity Management. A combination of our capacity management software athene and SharePath allows you to bring resource, application, application transaction response times, service, business data and KPI s to your CMIS. This allows you to build the most accurate picture of your environment as you can. Display of different presentation types Below is a selection of the types of presentations that you can use. Humans like visual representation so using these charts in the right way and gauging which are right to represent the information to your audience is crucial. 6 Capacity Management - Telling the story

Dashboard More aligned to presenting real time information. The key thing to remember is that any dashboard you use should auto update. Analysis Present the drill down of a problem.

7 Dashboard More aligned to presenting real time information. The key thing to remember is that any dashboard you use should auto update. Analysis Present the drill down of a problem. The root cause analysis, where we know there is a problem and we want to drill down and show what is causing the issue. Where s the bottleneck? Was there a change? Capacity Management - Telling the story 7

Virtualization Report on virtualization data, understand what it

8 Advice We can provide some automatic advice, automatic interpretation of the data that you are reporting on. Virtualization Report on virtualization data, understand what it is telling you about your virtual environment. 8 Capacity Management - Telling the story

We can look for correlations by measuring component data against business metrics.

9 Business We have discussed about bringing in business data to the CMIS. Why do we want to do that? We can look for correlations by measuring component data against business metrics. Trending As I mentioned previously trending is useful. We can show what-if trending which can give you a time to live value. Capacity Management - Telling the story 9

any future bottlenecks are likely to occur.

10 Modeling More accurate reports to show are modeling reports. You can show things like future system response times or identify where any future bottlenecks are likely to occur. Breakdown Shows further analysis on the data in an easy to understand way. 10 Capacity Management - Telling the story

Management Overview Applications Let s move on to what we will see at the management level, particularly for applications. This is called an application summary.

11 Management Overview Applications Let s move on to what we will see at the management level, particularly for applications. This is called an application summary. The example summary below shows grouping by category, in this instance by location. We have a call center, a warehouse, datamart and sales and it shows you that horizontal scale progress through use of color. For each arrival workflow that we have coming in it shows you the progress. Application Summary As you can see there are some reds being displayed in this report, the comment section clearly describes what these issues are. This allows you to clearly display to management the issues, what is causing them and enables you to discuss with them what you are doing about it. In this case, the warehouse needs some new architecture. Capacity Management - Telling the story 11

12 Resources and Costs The report below is a dashboard view of resources and costs. In the top left hand corner we have a cost by application. This is a breakdown of our applications and how much each of these is actually costing us. In this instance the biggest cost to us is development. In the top right hand corner is an analysis of how much memory each of our applications is using and again it is development who are consuming the most memory resources. In the bottom left hand corner we can view the sum of the CPU, this is the usage of the CPU by the application and in this case an in-house application is consuming the most CPU. In the bottom right hand corner we have the numbers of CPU s that are being used by applications and again in this example development are using the largest amount. This is a very clear and concise way of displaying the information to your audience. Now we ll look in more detail at dashboards. 12 Capacity Management - Telling the story

13 Dashboard Dashboard Overview Scorecard In the following example of a dashboard we can immediately see that we have a green, 2 reds and some greys. Based on the green, amber and red status we can immediately see that we have an issue with a couple of these categories, memory and I/O. Is this enough information? Who is viewing this information and does it tell them enough? If management were looking at this information they would be worried as they can see red in the status. It does scare senior management when they see a red status, mainly due to the fact that they do not have the time or inclination to see what is behind the issue. They would immediately be on the phone to their capacity management team asking why there are issues and it then causes more pressure further down the tree. It may be that this particular issue is not an immediate problem. Maybe one of the thresholds was breached during a certain time period and needs investigation. Dashboard Overview Scorecard Detail We can drill down and find out some further information on the issue in this case. In the report below there is still some red showing so it is going to have to be investigated fully and we would need to drill down even further to find out what applications are involved here. Capacity Management - Telling the story 13

14 In the further drill down report below we can see that we have some paging activity on Unix that has breached the threshold. These red, amber and green scorecards have to be based on thresholds. Where the grey is shown this simply means that there is no threshold data attached to that. We need to get in to the details to understand what the root cause of the issue is and understand whether the issue is serious or not. 14 Capacity Management - Telling the story

15 Unix Reports So let s look at a couple of examples which have been created for a Linux box. Below is a simple utilization report for a Linux box running an application, it is for a day and it shows us that there are a couple of spikes where it has breached our CPU threshold. Linux Server - CPU Utilization Looking at this report we can see that these peaks take place during the middle of the day. Is that normal behavior? Do you know enough about the system or application that you are reporting on to make that judgment? Do we need to perform a root cause analysis? If we believe the peaks to be normal then maybe we need to adjust the threshold settings or if we are unsure then we need to carry out further investigation. Has some extra load been added to the system? Has there been a change? Are there any other anomalies that you need to find out about? Remember when reporting don t make your reports over complicated. Don t put too much data on to one chart, it will look messy and be hard to understand. Capacity Management - Telling the story 15

16 Linux Server Disk Utilization Below is an example of a report on disk utilization of a Linux server. The reason I chose to share this report is that it is an instance based report, displaying the top 5 disks and their utilization on this system. You have the ability to pick out our top 5 or our bottom 5 to display to your audience because we don t want too much noise on our chart. We want to keep things clear and concise, don t flood reports with meaningless data and keep it relevant to our audience. Business Metric Correlation As mentioned previously it is important to get business information in to the CMIS to enable us to perform some correlations. As in the example below we have taken business data and taken component data and we can now report on this together to see if there is some kind of correlation. 16 Capacity Management - Telling the story

Business Transactions vs. CPU Utilization In this example we can see that the number of customer transactions (shown in dark blue) reasonably correlates with the amount of CPU utilization.

17 Business Transactions vs. CPU Utilization In this example we can see that the number of customer transactions (shown in dark blue) reasonably correlates with the amount of CPU utilization. Can we make some kind of judgment based on just what we see here? Do we need to perform some further statistical analysis on this data? What is the correlation co-efficiency for our application data against the CPU utilization? Closer to the value of 1 indicates that there is a very close correlation between the application data and the underlying component data. What can we do with this information back to the business? An example would be: This graph indicates that there is a very close correlation between the number of customer transactions and the CPU utilization. Therefore, if we plan on increasing the number of customer transactions in the future we are likely to need to do a CPU upgrade to cope with that demand. Modeling Scenario I have talked about bringing your KPI s, resource and business data in to a CMIS and about using that data to produce reports in a clear, concise and understandable way. Let s now take a look at some analytical modeling examples, based on forecasts which were given to us by the business. Below is an example of an Oracle box, we have been told by the business that we are going to grow at a steady rate of 10% per month for the next 12 months. We can model to see what the impact of that business growth will be on our Oracle system. Capacity Management - Telling the story 17

18 In the top left hand corner is our projected CPU utilization and on the far left of that graph is our baseline. You can see that over a few months we begin to go through our alarms and our thresholds pretty quickly. Model oracleq000 10% growth server change In the bottom left hand corner we can see where bottlenecks will be reached indicated by the large red bars which indicate CPU queuing. On the top right graph we can see our projected device utilization for our busiest disk and we can see that within 4 to 5 months it is also breaching our alarms and thresholds. Collectively these models are telling us that we are going to run in to problems with CPU and I/O. In the bottom right hand graph is our projected relative service level for this application. In this example we started the baseline off at 1 second, this is key. By normalizing the baseline at 1 second it is very easy for your audience to see the effect that these changes are likely to have. In this case, once we ve added the extra workload we can see that we go from 1 second to 1.5 seconds (a 50% increase) and then jumped from 1 second to almost 5 seconds. From 1 to 5 seconds is a huge increase and one that your audience can immediately grasp and understand the impact of. We would next want to show the model for change in our hardware. 18 Capacity Management - Telling the story

19 Model Linux server change and disk change In the top left hand corner we are showing that once we reach the pain point and then make a hardware upgrade the CPU utilization drops back to within acceptable boundaries for the period going forward. In the bottom left hand corner you can see from the primary results analysis that the upgrade would mean that the distribution of work is more evenly spread now. The model in the top right hand corner has bought up an issue on device utilization with another disk so we would have to factor in an I/O change and see what the results of that would be and so on. In the bottom right hand corner we can see that the service level has been fine for a couple of periods and then it is in trouble again, caused by the I/O issue. Whilst this hardware upgrade would satisfy our CPU bottleneck it would not rectify the issue with I/O, so we would also need to upgrade our disks. When forecasting modeling helps you to make recommendations on changes that will be required and when they will need to be implemented. Capacity Management - Telling the story 19

20 VMware Reports Let s take a look at some examples of VMware reports. The first report below looks at the CPU usage of clusters in MHz. It is a simple chart and this makes it very easy to understand. VMware CPU Usage all Clusters You can immediately see who the biggest user of the CPU is, Core site 01 The next example is a trend report on VMware resource pool memory usage. The light blue indicates the amount of memory reserved and the dark blue line indicates the amount of memory used within that reservation. This information is then trended going forward, allowing you to see at which point in time the required memory is going to exceed the memory reservation. 20 Capacity Management - Telling the story

21 VMware Resource Pool Memory Usage Trend A trend report like this is useful as an early warning system, you know when problems are likely to ensue and can do something to resolve this before it becomes an issue. We need to keep ahead of the game and setting up simple but effective reports, provided automatically, will help you to do this and to report back to the business regarding requirements well in advance. I ll show you one final report on VMware, which looks at headroom available in the Virtual Center. On the example below we re showing CPU usage. The average CPU usage is illustrated by the green bars, the light blue represents the amount of CPU available across this particular host and the dark blue line is the total CPU power available. Capacity Management - Telling the story 21

22 VMware Virtual Center Headroom We have aggregated all the hosts up within the cluster to see this information. We can see from the green area at the bottom how much headroom we have to the blue line at the top, although actually in this case we will be comparing it to the turquoise area as this is the amount of CPU available for the VM s. This is due to the headroom taken by VMkernel which has to be taken in to consideration and explains the difference between the dark blue line and the turquoise area. 22 Capacity Management - Telling the story

23 Summary When reporting: Stick to the facts Elevator talk Show as much information as needs to be shown Display the information appropriate for the audience Talk the language for the audience.tell the Story Capacity Management - Telling the story 23

24 Metron Metron, Metron-Athene and the Metron logo as well as athene and other names of products referred to herein are trade marks or registered trade marks of Metron Technology Limited. Other products and company names mentioned herein may be trade marks of the respective owners. Any rights not expressly granted herein are reserved.

ITIL Capacity Management for the Newbie For those new to system z. Charles Johnson Principal Consultant

ITIL Capacity Management for the Newbie For those new to system z Charles Johnson Principal Consultant Agenda ITIL Definition of Capacity Management Capacity Management and ITIL activities Difference between