UNITE Managing Service Delivery, Workloads, and Capacity Burn on Metered and Non-Metered Systems

Transcription

1 UNITE 2009 Managing Service Delivery, Workloads, and Capacity Burn on Metered and Non-Metered Systems UNITE Conference Minneapolis, MN 11 November 2009 Session GE4028 Guy Bonney Bob Morrow

2 MGS, Inc. Software Engineering, Product Development & Professional Services firm founded in 1986 We solve business problems with: Products: SightLine, CheckOut, MGSWEB Web Services, Deliver, C.A.T.T., SecureCATT, and others Professional Services IT Management Planning Capacity Planning and Management Consulting and Technical Services including Performance Management and Hardware- Software-Network Integration Application Development Services including Java/J2EE development and platform rehosting Training Services Software Engineering Services on ClearPath MCP, Windows, and UNIX platforms. 2

3 Agenda Concepts Defining Workloads Identify Characterize Establish Service Levels Managing Service Delivery Managing Capacity Burn 3

4 Service Delivery Concepts Whether using ITIL, COBIT, ISO/IEC 20000, Quality Management, or other integrated process framework they all include areas similar to the following: Service Level Management Financial Management of IT Services Capacity and Performance Management IT Service Continuity Management Availability Management This session discusses service level management and capacity management 4

5 What is Service Delivery? Delivery of defined IT services to requestors (typically users) at: an expected performance level, with expected reliability and availability, and within budget constraints. 5

6 What is Performance? Meeting Expectations. Complete Processing a Unit of Work in the Expected Time. On-line transactions - Response time Batch runs - Elapsed time/deadlines On-Line Batch - Elapsed (Response) time Processing work requires Capacity. 6

7 Capacity Defined Ability to process work Maximum work that can be done while meeting Performance objectives (service levels) Performance cannot be achieved without sufficient Capacity to process the Work Performance and Capacity are inextricably linked 7

8 Managing Service Delivery Know the Workload(s) Know the Service Requirements (performance) Know the Demand Pattern KNOWLEDGE ENABLES: Planning for Capacity needs Managing Services to Budget 8

9 Define and Characterize Workloads Workload Identification Identify programs associated with a business function: Payroll Accounting (A/R, A/P, G/L) Order Entry Outpatient Admissions Determine whether there are on-line and batch elements and when they are active on the system For on-line workloads: Identify COMS programs, TIP transactions, or IIS entities For batch workloads: Identify major batch processes Is it a single-threaded process or parallel processing? 9

10 Define and Characterize Workloads Equate business volume indicators to identified workloads Changes in the business environment affect services Business volumes are natural business units (NBU s) Orders processed Occupied hospital beds Outpatient admissions Student registrations System transaction volumes are natural forecasting units (NFU s). Inquiry, Update, or Order Entry transactions Admissions, patient accounts, and discharge inquiry and update transactions 10

11 Business Volume History % Increase Annual Increase of 2% 25.1% Increase Annual Increase of 7.8% DDA SAV COD LAS Total Linear (DDA) Linear (Total) Linear (COD) Linear (SAV) Linear (LAS) /25/2001 8/25/ /25/ /25/2001 2/25/2002 4/25/2002 6/25/2002 8/25/ /25/ /25/2002 2/25/2003 4/25/2003 6/25/2003 8/25/ /25/ /25/2003 2/25/2004 4/25/2004 6/25/

12 Define and Characterize Workloads Identify and document service level indicators for each principle workload Number job requests processed Required end-user response time Report deadlines Service errors Number of service requests 12

13 Perform a Workload Service Level Analysis Workload Resource Analysis Determine the heaviest resource per workload Determine resource contention for each workload Workload Analysis Workload Resource Times Interval * CPU% 10 seconds Interval * ReadyQ% 13 seconds I/O time * IO% 5 seconds Unknown (DMS, etc) 2 seconds Sample Interval 30 seconds Other contention indicators OtherPbit % CPU Stretch 13

14 On-Line Workload Profile Apr :00-20: :00 08:00 09:00 10:00 11:00 12:00 13:00 14:00 15:00 16:00 17:00 18:00 19:00 SISFIN Online Transaction Count SETBN Transaction Count Credit Card Transaction Count Internet Banking Transaction Count 14

15 On-Line Response Time A19: COMS Response Avg 22 Aug :00-15:00 14:00 14:05 14:10 14:15 14:20 14:25 14:30 14:35 14:40 14:45 14:50 14:55 15:00 PGU0020 PGU0034B PGU0034 PGU0015 PGU0043B PGU0043 PGU0060 PGU0060B 15

16 On-Line Workload Projection Average Weekly Transaction Counts vs. Total Accounts Weekly COMS Transaction Count Total Accounts 16

17 Perform a Workload Service Level Analysis Transaction Resource Analysis Determine the heaviest resource per program Determine resource contention for each program Reconcile where response time is spent Transaction Analysis Breakdown Average Response time 900 ms CPU time 150 ms Readyq time 300 ms I/O time 200 ms DMS/COMS overhead 250 ms COMS Program Q-Depth.5 transactions DMS BTR Delay 150 ms 17

18 Workload Service Level Analysis Response Time Jul :10:00-10:10:00 09:10:00 09:20:00 09:30:00 09:40:00 09:50:00 10:00:00 10:10:00 COMS CPU/Tran for #EBB COMS IO/Tran for #EBB COMS RdyQ/Tran for #EBB COMS Queuing Delay for #EBB 18

19 Workload Service Level Analysis Response Time :10:00 09:20:00 09:30:00 09:40:00 09:50:00 10:00:00 10:10:00 COMS CPU/Tran for #EBB COMS IO/Tran for #EBB COMS RdyQ/Tran for #EBB COMS Queuing Delay for #EBB Jul :10:00-10:10: Jul :10:00-10:10:00 09:10:00 09:20:00 09:30:00 09:40:00 09:50:00 10:00:00 10:10:00 CPU Total Pct 19

20 Workload Service Level Analysis Response Time Jul :10:00-10:10:00 09:10:00 09:20:00 09:30:00 09:40:00 09:50:00 10:00:00 10:10:00 COMS CPU/Tran for #PRM COMS IO/Tran for #PRM COMS RdyQ/Tran for #PRM COMS Queuing Delay for #PRM 20

21 Workload Service Level Analysis Response Time Jul :10:00-10:10:00 09:10:00 09:20:00 09:30:00 09:40:00 09:50:00 10:00:00 10:10:00 COMS CPU/Tran for #PRM COMS IO/Tran for #PRM COMS RdyQ/Tran for #PRM COMS Queuing Delay for #PRM UAS: COMS Q-Depth Avg 1 Jul :10:00-10:10:00 09:10:00 09:16:00 09:22:00 09:28:00 09:34:00 09:40:00 09:46:00 09:52:00 09:58:00 10:04:00 10:10:00 #PRM #ADS #ATM #DFT #EBB #FMS #PDS #TTM 21

22 Perform a Workload Service Level Analysis Batch Workload Resource Analysis Must utilize SWA vs. SPA performance data Identify the programs with the highest resource times Determine resource contention per program Reconcile where elapse time is spent Batch elapsed analysis breakdown Average elapsed time 8100 seconds (2:30) CPU time * Program Count 500 seconds Readyq time * Program Count 2500 seconds I/O time * Program Count 1000 seconds Unknown overhead 4100 seconds DMS BTR Delay * BTR WaitCnt Program RSVP Other Delay 500 seconds 3000 seconds 600 seconds 22

23 Workload Distribution Deadlines Identify processing constraints Identify input availability constraints Identify output delivery requirements Map Requirements vs. Resources Foundation/constant demands Variable demands Plot on a timeline with deadlines Determine Demand Peaks 23

24 Workload CPU Demand by Type 21 Aug :00-22 Aug 20: Aug21 20:00 Aug22 00:00 Aug22 04:00 Aug22 08:00 Aug22 12:00 Aug22 16:00 Aug22 20:00 MCP Online Processing Critical Path Nightly Batch Adhoc Batch Processing 24

25 Application CPU Distribution Quarter Hour Averages - CPU Busy 29 Nov :00-30 Nov 00: :00 07:00 09:00 11:00 13:00 15:00 17:00 19:00 21:00 23:00 CPU MCP Pct Work CPU Pct for 1-CISDBOL Work CPU Pct for 1-CISDBBT Work CPU Pct for 1-BUDGETDBBT Work CPU Pct for 1-TXDBBT Work CPU Pct for 1-ABSDBOL Work CPU Pct for 1-SYSTEM Work CPU Pct for 1-OTHER 25

26 Management Methodology Establish baseline Performance requirements (objectives) Capacity usage Workload volume Maintain performance/capacity history Perform trend analysis Capacity usage Workload volume Projections Future capacity usage Potential hardware solutions Alerting and periodic reporting Capacity consumption vs plan Performance vs service levels Capacity usage alerts 26

27 Management Methodology Monitor Capacity Usage MIPS 30 Nov :00-7 Dec 00:00 0 Nov30 Dec1 Dec2 Dec3 Dec4 Dec5 Dec6 Average MIPS Average MCP MIPS 27

28 Management Methodology Monitor CPU Busy Percent CPU Pct 6 Apr :00-13 Apr 00:00 0 Apr6 Apr7 Apr8 Apr9 Apr10 Apr11 Apr12 Apr13 CPU MCP Pct CPU User Pct 28

29 Management Methodology Demand Pattern - Capacity by Day MIPS 30 Nov Dec Nov30 Dec1 Dec2 Dec3 Dec4 Dec5 Dec6 Average MIPS 29

30 Management Methodology Trend Analysis 30

31 Management Methodology Trending Data 70 1 Jan Mar Jan 2003 Sep 2003 May 2004 Jan 2005 Sep 2005 May 2006 Jan 2007 Sep 2007 CPU Total Pct AVG 31

32 Management Methodology Trend Projection 80 1 Jan Oct Jan 2003 Jan 2004 Jan 2005 Jan 2006 Jan 2007 Jan 2008 Jan 2009 Jan 2010 CPU Total Pct AVG CPU Total Pct AVG exponential 32

33 Managing Capacity Capacity Usage Projection MIPS 450 MIP Months Dec-08 Jan-09 Feb-09 Mar-09 Apr-09 May-09 Jun-09 Jul-09 Aug-09 Sep-09 Oct-09 Nov-09 Dec-09 Jan-10 Feb-10 Mar-10 Apr-10 May-10 Jun-10 Jul-10 Aug-10 Sep-10 Oct-10 Nov-10 Dec-10 Jan-11 Feb-11 Mar-11 Apr-11 May-11 Jun-11 Jul-11 Aug-11 Sep-11 Oct-11 Nov-11 Dec-11 Jan-12 Feb-12 Mar-12 Apr-12 May-12 Jun-12 Jul-12 Aug-12 Sep-12 Oct-12 Nov-12 Dec-12 Jan-13 Feb-13 Mar-13 Apr-13 May-13 Jun-13 Jul-13 Aug-13 Sep-13 Oct-13 Nov-13 Dec-13 0 Projected Average Monthly MIPS ranges from 199 to 397 Total MIPS Months of Avg Monthly Run Rate of 277 MIPS 33

34 Managing Capacity Metering Capacity Usage Plan 34

35 Managing Capacity Metering Capacity Used Actual Cumulative Capacity Usage MIPS Months Dec-08 Jan-09 Feb-09 Mar-09 Apr-09 May-09 Jun-09 Jul-09 Aug-09 Sep-09 Oct-09 Nov-09 Dec-09 Jan-10 Feb-10 Mar-10 Apr-10 May-10 Jun-10 Jul-10 Aug-10 Sep-10 Oct-10 Nov-10 Dec-10 Jan-11 Feb-11 Mar-11 Apr-11 May-11 Jun-11 Jul-11 Aug-11 Sep-11 Oct-11 Nov-11 Dec-11 Jan-12 Feb-12 Mar-12 Apr-12 May-12 Jun-12 Jul-12 Aug-12 Sep-12 Oct-12 Nov-12 Dec-12 Jan-13 Feb-13 Mar-13 Apr-13 May-13 Jun-13 Jul-13 Aug-13 Sep-13 Oct-13 Nov-13 Dec-13 Actual Cumulative Capacity Used Annual Capacity Usage Plan 35

36 Managing Capacity Excessive Workload Capacity Usage Work MIPS 1-ACCT 12 Mar :50-19: :50 18:00 18:10 18:20 18:30 18:40 18:50 19:00 19:10 19:20 36

37 Managing Capacity Excessive Workload Usage Alarm 37

38 Managing Capacity Problem Alerting 38

39 Managing Capacity Performance vs. Service Levels Seconds Response Time vs. Serice Level 26 Nov :00-14: :00 13:06 13:12 13:18 13:24 13:30 13:36 13:42 13:48 13:54 14:00 #GUI #PMC #PDB #PDC #GUIB 39

40 To Meter or Not to Meter Metered System Option MIPS MIP Months Dec-05 Jan-06 Feb-06 Mar-06 Apr-06 May-06 Jun-06 Jul-06 Aug-06 Sep-06 Oct-06 Nov-06 Dec-06 Jan-07 Feb-07 Mar-07 Apr-07 May-07 Jun-07 Jul-07 Aug-07 Sep-07 Oct-07 Nov-07 Dec-07 Jan-08 Feb-08 Mar-08 Apr-08 May-08 Jun-08 Jul-08 Aug-08 Sep-08 Oct-08 Nov-08 Dec-08 Jan-09 Feb-09 Mar-09 Apr-09 May-09 Jun-09 Jul-09 Aug-09 Sep-09 Oct-09 Nov-09 Dec-09 Jan-10 Feb-10 Mar-10 Apr-10 May-10 Jun-10 Jul-10 Aug-10 Sep-10 Oct-10 Nov-10 Dec-10 0 Projected Average Monthly MIPS ranges from 20 to 26 Total MIPS Months of 1376 Avg Monthly Run Rate of 23 MIPS 40

41 To Meter or Not to Meter Pay for Peak Option CPU REQUIREMENT PROJECTION Pay for Peak Replacement Systems' CPU Utilization vs Effective Capacity 120% 110% CPU USAGE 100% 90% 80% 70% 60% 50% Dec 2008 Dec 2009 Dec 2010 Dec 2011 Dec 2012 Dec 2013 CPU Effective Capacity CS CS CS

42 To Meter or Not to Meter Considerations Consistent Workload? Small to Medium Load? Gradual Growth? User Based Licensing? Few software licenses needed Non-metered solution may be less expensive do detailed financial analysis note that Libra 4000 changes the HW/SW expense ratio. 42

43 Questions? Thank you for your attention Are there any questions? Note that this presentation will be available for download today at: 43