DATAPREV Rio de Janeiro 16 Dezembro de capacidade. Departamento de Ciência da Computação Universidade Federal de Minas Gerais - UFMG

Transcription

1 Treinamento em Análise Quantitativa & Planejamento de Capacidade Virgilio A. F. Almeida DATAPREV Rio de Janeiro 16 Dezembro de 2009 Módulo: introdução ao planejamento de capacidade Departamento de Ciência da Computação Universidade Federal de Minas Gerais - UFMG

2 Some Questions Why is capacity planning strategically important to business? Why is capacity planning essential to Web Services? What are future challenges to capacity planning in the context of the Web? Virgilio Almeida, All Rights Reserved,

3 Citi Critical Issues in Web Services Performance = money Service downtime = money Security = money (Right?) Virgilio Almeida, All Rights Reserved,

4 Real Examples The impact of Web performance on the user behavior: the rate of abandons. Availability is a key metric for Web services: cost of downtime. The impact of security for Web services: performance costs. Virgilio Almeida, All Rights Reserved,

5 Dowtime Cost * Institution Assumptions Estimated t Downtime Cost 1. EECS Dept. 7 hours downtime/mo $300,000/year U.C. Berkeley affects 50% employees 2. Amazon outage affects $550,000/hour 90% of revenue & 90% of employees 3. SUN outage affects 10% $825,000/hour of revenue and 90% of employees (*)Patterson, D. ``A simple way to estimate the cost of downtime 16th Systems Administration Conference (USENIX/LISA), 2002 Virgilio Almeida, All Rights Reserved,

6 When Security is a Problem Security is essential to maintain trust and confidence in Web services. Costs of security attacks are tremendous: DOS attacks may cost a fraction of a day Costs to company operations Decreased productivity Loss of confidence in Web services Web services are being used for critical applications: hospitals, utilities, banks, etc. The number of attacks in the Internet is growing g fast! Security implies performance costs. Virgilio Almeida, All Rights Reserved,

7 CERT (*) Reported Attacks in the Internet t (*) CERT: The Computer Emergency Response Team Center Virgilio Almeida, All Rights Reserved,

8 Performance Cost of Security The difference is around two orders of magnitude for XML signatures From: ``Performance Comparison of Security Mechanisms for Grid Services, S. Shirasuna et. Al., Proc. Of the 5 th IEEE/ACM 2004 Workshop on Grid Comp. Virgilio Almeida, All Rights Reserved,

9 The Strategic Importance of Capacity Planning Capacity Planning methodologies and techniques can help companies in the process of conducting IT oversight: Assure security and reliability; Avoid surprises; Watch out for legal problems; Keep an eye for fresh threats and opportunities. IT governance policies should take into account the operational and strategic t needs of an organization in order to avoid unnecessary risks and improve its competitive position. Virgilio Almeida, All Rights Reserved,

10 The Strategic Approach to IT *R. Nolan, HBR October 2005 Defensive Offensive Factory Mode Strategic t Mode *If systems fall for a minute or more, there s an immediate loss of business. *Increase in response time has serious consequences for users. -Most core business activties are online. *If systems fall for a minute or more, there s an immediate loss of business. *Increase in response time has serious consequences for users. -New systems promise major process and service transformations. Support Mode -Internal systems are almost invisible to suppliers and customers. -Even with repeated service interruptions, ti there are no serious consequences. Turnaround Mode -New systems promise major process and service transformations -New systems promise major cost reductions. -IT makes up more than 15% of total corporate expenses. Low to High Need for New IT Virgilio Almeida, All Rights Reserved,

11 Capacity Planning The IT strategic impact grid * Defensive Offensive Factory Mode Strategic t Mode *If systems fail for a minute or more, there s an immediate loss of business. *Increase in response time has serious consequences for users. -Most core business activties are online. *If systems fail for a minute or more, there s an immediate loss of business. *Increase in response time has serious consequences for users. -New systems promise major process and service transformations. Support Mode -Internal systems are almost invisible to suppliers and customers. -Even with repeated service interruptions, ti there are no serious consequences. Turnaround Mode -New systems promise major process and service transformations -New systems promise major cost reductions. -IT makes up more than 15% of total corporate expenses. *R. Nolan, HBR October 2005 Low to High Need for New IT Virgilio Almeida, All Rights Reserved,

12 Capacity Planning From the previous slides it is evident that we need sound quantitative methods and techniques to plan the infrastructure for Web Services. Capacity planning can help improving Business and IT alignment. Capacity planning is a useful technique for Web engineering. Virgilio Almeida, All Rights Reserved,

13 Outline Capacity planning and Web engineering What is engineering? What is capacity planning? A capacity planning methodology Performance modeling E-business scalability analysis Virtualization Future challenges Virgilio Almeida, All Rights Reserved,

14 What is engineering? i The disciplined i d application i of scientific ifi knowledge to resolve conflicting constraints and requirements for problems of immediate and practical significance. Characteristics of engineering activities: Create cost-effective ti solutions Apply to practical problems Make use of scientific knowledge Traditional engineering: civil, chemical, etc New engineering disciplines that are in process of consolidation: software, Web, Virgilio Almeida, All Rights Reserved,

15 Examples of the New Engineering Techniques Capacity Planning is a good example of an engineering technique: a basis in theory rooted in scientific theories: Queuing Theory, Operational Analysis, Probability Theory, Simulation, Petri Nets,... a basis in practice rooted in empirical observation: measurements a pragmatically determined collection of highlevel functions that adequately describe the workloads of Web services Virgilio Almeida, All Rights Reserved,

16 What is Capacity Planning? Capacity planning is the process of determining when the future load levels will saturate the infrastructure, violating the service level e agreements e (SLA). Capacity Planning & Management Capacity Planning: long-term resource provisioning i i process for IT systems. Capacity Management: short-term resource provisioning i i for IT infrastructure, t i.e., real-time resource and service management policies. Service level agreements (SLA) are related to: Performance, Availability,Security Cost, Revenue Virgilio Almeida, All Rights Reserved,

17 Capacity Planning: Metrics, Measurements and Models SLA measures the user's s experience interacting with a Web service, Availability, Transaction Time, System security level, Cost. Why measure performance? To take fast corrective action in case of violation (SLA) Vendor evaluation (return of investment in services and technologies) To explain customer behavior Problem detection/troubleshooting To obtain input data for performance models. Virgilio Almeida, All Rights Reserved,

18 Capacity Planning types of models Intuition High naive ideal Practical: trends clueless complex Low Low High Accuracy Suggested by Faloutsos, Dimacs Workshop 2002 Virgilio Almeida, All Rights Reserved,

19 Performance Models for Capacity Planning Back of the envelope models: Simple queuing results (eg: Little s Law) Simple models for performance bounds Elaborated models: Queuing network models that calculate response times, utilization, and queue lenght. Simulation models that calculate pdfs, CDFs, etc. Combination of different models: queuing models and optimization models Benchmarks (e.g., performance benchmarks, security benchmarks, etc) Products for modeling performance (e.g. Teamquest Products) Virgilio Almeida, All Rights Reserved,

20 Examples of capacity planning questions How can one plan, design, develop, deploy and operate Web services that meet increasing demands for better service levels, in terms of performance, availability, security and cost? What are the consequences, if the service level requirements are not satisfied? Is the infrastructure of a given Web service scalable? Can it cope with new security policies? Could the Web services work better? What are the most cost-effective scenarios for the infrastructure that support Web services? What are the bounds for the service? Virgilio Almeida, All Rights Reserved,

21 Outline Capacity planning and Web engineering What is engineering? What is capacity planning? A capacity planning methodology Performance modeling E-business scalability analysis Virtualization Future challenges Virgilio Almeida, All Rights Reserved,

22 Capacity Planning Business requirements Workload demands Cost Capacity Planning Process IT Future Scenarios SLAs Virgilio Almeida, All Rights Reserved,

23 Capacity Planning Process Business Requirements & Measurable Goals Compare Different IT Architectures & SLAs Understand Service Architecture Cost Model Predict Service Performance-Availability Performance Availability Model Characterize the Workload Model Validation And Calibration Workload Model Obtain Model Parameters Develop a Performance Model Forecast Workload Evolution Virgilio Almeida, All Rights Reserved,

24 Capacity Planning Process Business Requirements & Measurable Goals Teamquest Products Compare Different IT Architectures & SLAs Understand Service Architecture Cost Model Predict Service Performance-Availability Performance Availability Model Characterize the Workload Model Validation And Calibration Workload Model Obtain Model Parameters Develop a Performance Model Forecast Workload Evolution Virgilio Almeida, All Rights Reserved,

25 Performance Modeling Tools Each vendor site provide many interesting documents about modeling, capacity planning, performance management, etc. BMC Capacity Management (former BEST/1, Patrol Perform &Predict) TeamQuest How to Do Capacity Planning Metron Athene, "provides ITIL-compliant capacity management, automatic performance analysis and reporting for UNIX, Linux, Windows and Mainframe Servers." Virgilio Almeida, All Rights Reserved,

26 Performance Modeling Tools HyPerformix Predictive IT Management Suite includes HyPerformix Optimizer (uses discrete event simulation modeling technology), HyPerformix Capacity Manager and HyPerformix Data Manager. Integrated with LoadRunner. LoadRunner is one of data sources for Hyperformix. For some time Mercury promoted HyPerformix as Mercury Capacity Planning. Wilson Mar's notes OPNET IT Guru Systems Planner PDQ - open source from Dr. Neil Gunther Ptolemy II is a set of Java packages supporting heterogeneous, concurrent modeling and design from UC at Berkeley, free for academic and commercial use. Microsoft System Center Capacity Planner A Collection of Modelling and Simulation Resources on the Internet by Andrea Emilio Rizzoli Virgilio Almeida, All Rights Reserved,

27 Outline Capacity planning and Web engineering What is engineering? What is capacity planning? A capacity planning methodology Performance modeling E-business scalability analysis Virtualization Future challenges Virgilio Almeida, All Rights Reserved,

28 E-Business Service Scalability Analysis Problem: management is interested in the value of the arrival rate that will make the average response time for specific transactions to exceed the SLA limit. Online auction service Functions; entry, home, search, login, create_auction, auction view_bid, place_bids, etc The capacity planning approach Characterize the workload at multiple levels Anticipate future user behavior and load level Predict future performance Anticipate the increased request levels Virgilio Almeida, All Rights Reserved,

29 Auction Site s Architecture Virgilio Almeida, All Rights Reserved,

30 Input Parameters Capacity planning goal: SLA maximum transaction response time Workload Characterization Model Customer Behavior Model Graph Performance Model Service demands for each resource and each type of request total average time spent by a typical request r obtaining service from resource i, (D i,r ) Workload Intensity Arrival rate: e-business functions/sec Virgilio Almeida, All Rights Reserved,

31 A New approach: Customer, workload and resource model Frequency of business functions Service demand per business function Response time, throughput and business metrics Customer Workload Resource Model Model Model What-if questions regarding impacts of customer behavior Workload changes and forecast What-if regarding architecture and configuration changes Virgilio Almeida, All Rights Reserved,

32 Customer Behavior Model Graph (CBMG) View Portfolio transition probability The goal of the CBMG P s,b is to represent the behavior of a customer during a session, defined as a sequence of consecutive Purchase 0.2 requests from the Stock same customer during a single visit Virgilio Almeida, All Rights Reserved,

33 The Customer Behavior Model Graph (CBMG) Virgilio Almeida, All Rights Reserved,

34 Metrics Derived from the CBMG Average Number of Visits Per State E.g., average number of searches per visit to the site, Average Session Length Per Visit: n 1 k =2 Performance model input parameters: arrival rate of requests Business-oriented metrics: V place-bid V k Virgilio Almeida, All Rights Reserved,

35 Performance Model D cpu,h D disk,h Multiclass Open Queuing Network Model Virgilio Almeida, All Rights Reserved,

36 Time (sec c) Average Request Response Response Times per Class place bids SLA for create auction and place bids create auctions Session Starts/sec Home Search Login Create Bid View Virgilio Almeida, All Rights Reserved,

37 New customer demands Future Challenges Multiple metric requirements Per request response time requirements Cost-effective services IT characteristics ti High workload IT and Business Highly complex fluctuations Alignment infrastructure static management is not cost-effective more complex business models & virtualized environments Virgilio Almeida, All Rights Reserved,

38 Future Capacity Planning Architectures* Seconds to minutes Days to Weeks Months (*) IEEE-ICAC2006, Dublin, Ireland Almeida, Ardagana and Francalani Virgilio Almeida, All Rights Reserved,

39 Summary Capacity Planning methodologies and techniques can help companies to align IT with business objetives. Performance data collecting tools and performance prediction models are key to implement cost-effective capacity planning methodologies. Future capacity planning techniques should cover new technologies, such as virtualization, utility computing, and self-adaptive systems. Per-request metrics will be important for SLAs. Capacity planning will need models to predict service availability. l Future capacity planning techniques should aim at new SLA metrics such as availability and security. In the future, system planning may be guided by business-oriented metrics, such as: revenue/sec, potential business loss/sec, etc. Virgilio Almeida, All Rights Reserved,

40 References Virgilio Almeida, All Rights Reserved,

41 Reference Books Performance by Design : Computer Capacity Planning By Example, Menascé, Almeida, and Dowdy, Prentice Hall, Scaling for E-Business: technologies, models, performance, and capacity planning, Menascé and Almeida, Prentice Hall, Capacity Planning for Web Services; models, methods, and metrics, Menascé and Almeida, Prentice Hall, Virgilio Almeida, All Rights Reserved,

42 Bridging g the gap between business and IT Business View Business Model External Metrics Characteristics of the Business Functional Model Customer Model Technological View Resource Model Internal Metrics Virgilio Almeida, All Rights Reserved,

43 Workload Characterization ti Approach Business Level User Level Application Level Session Layer Function Layer Protocol Level q y Request Layer Resource Level Virgilio Almeida, All Rights Reserved,

44 Performance Cost of SOAP Security Activities iti From: ``Performance Cost of Web Security Services, Hongbin Liu, et al., 13 th Annual Mardi Gras Conference, Feb Virgilio Almeida, All Rights Reserved,

45 Performance, availability, and security problems for Web services tend to increase Proliferation of mobile devices that might increase the volume of Web transactions. Increasing load placed by agents and robots. Impacts of authentication and security protocols (e.g., SSL, TSL, WS security) on performance and costs of Web services Increase in the complexity of middleware and distributed applications. Flash crowds, that overload Web services. Point out to the need of quantitative techniques! Virgilio Almeida, All Rights Reserved,