Enterprise Software Performance Engineering Presented By: Walter Kuketz Scaling your career..
Software Performance matters - everywhere DTG Facebook IPO 2
Who is responsible for end-to-end system performance? When a website slows down? When the marketing team runs a big event? When the Storage subsystem slows down, which slows down the database, which slows down the application servers, and slows down the ipad? Where do the performance goals come from? 3
Agenda Building systems that scream Review the current challenges of being a performance engineer Overview of the five knowledge areas of Software performance engineering body of knowledge Roles and career path overview Case study 10X event (Communicate to Sr. Mgt.) 4
No clear standards in the market for PE Test automation technician Capacity planner? Performance Engineer Software Engineer?? Troubleshooting Performance tester Performance Architect? Performance analyst? Benchmarks Performance test? Performance Data engineer? 5
Opportunity to take SPE to the next level.. Define profession Define the profession of performance engineering Establish Body of Knowledge Establish a body of knowledge for performance engineering Establish common language Establish a common language for roles, responsibilities, and practice areas using the form of Knowledge Areas Define competencies Define the competencies shared across the knowledge areas Capture common techniques Capture common techniques for sharing across the industry, or your company 6
Knowledge areas The SPE Body of Knowledge For the Enterprise Planning, coordination, information sharing, and control SDLC & architecture (SA) Performance validation and testing (PVT) Capacity planning (CP) Application performance management (APM) Problem detection and resolution (PDR) Underlying competencies and roles for delivering Techniques supporting the knowledge areas 7
The Enterprise PE team: There must be a practice leader Operations & PT Architecture and production Communication Business direction Development teams 8
The Enterprise PE team: There must be a practice leader A shared service and clearing house for all performance, scalability and capacity planning information. Transforms performance information to business knowledge and advantage. Communicates across the teams to share information Operations & PT For instance, the production workload and the performance test workload are they in sync? Architecture and production Make sure the architecture team knows how the new release, or newest build behave in production. Business direction The business risks are clearly identified and communicated. Application risk profile. Development teams Can provide performance guidelines to the developers, must influence the unit testing process. 9
PE: Knowledge areas SA PVT CP APM PDR Software development lifecycle and architecture Performance testing and validation Capacity planning Application performance management Problem detection and resolution 10
PE: Software development lifecycle and architecture SA PVT CP APM PDR Software development lifecycle and architecture Performance testing and validation Capacity planning Application performance management Problem detection and resolution 11
PE: SA (software development lifecycle) Waterfall methodologies Define Design Develop Functional test Performance test Deploy Need to plan for performance testing Build code to performance requirements Define Deploy Agile methods Design Test Build Can a system be designed to support 10 TPS and 1 second response time work @ 100 TPS and 1 second? 12
Performance budgets Do your developers know how much time they have for their piece of the transaction? Code runs everywhere. SERVER WEB APP SVR PROXY MQ/ESB LDAP CORBA DCOM Respond to the user in 2 seconds. Datacenter Web Services 13
PE: Performance testing and validation SA PVT CP APM PDR Software development lifecycle and architecture Performance testing and validation Capacity planning Application performance management Problem detection and resolution 14
PE: PTV (performance testing and validation) What is changing? Where does it need to be validated? What (all) is being measured? The project: Why are we testing and what are the success criteria? Prerequisites: Business goals and non-functional requirements are defined Workload and scenarios defined Environment defined (where are we testing) Application in a stable state Configuration management and release management ready Core performance team identified Supporting team committed 15
Organization: Performance team value You still need to explain to the rest of the organization what you do Business and application area knowledge Early involvement with the applications team Test execution Environment Deep understanding of the application being tested Deep understanding of the technical architecture of the application Understanding of the business area supported by the applications Performance engineers involved in design sessions Understand the nature of the data required for testing Able to understand technical gaps to meet requirements not supported by the tool Well-understood workload model and characterization All components in the test are monitored and reported on A defined triage process for troubleshooting A process to manage the inevitable unplanned tests and test archive exists Testing environment is a close approximation of production, including database size Differences between test and production are clearly known Release and configuration management in place and procedures enforced Performance team value 16
PE: Capacity planning SA PVT CP APM PDR Software development lifecycle and architecture Performance testing and validation Capacity planning Application performance management Problem detection and resolution 17
PE: CP (capacity planning) The right system at the right price! Plan for future Forecast workloads Plan for usage Track utilization of existing systems Track workload Analyze current capacity Determine capacity requirements Define system to support workloads Test Define service levels Workloads drive system transactions, which drive resource utilization 18
Capacity: Use of resources The overall system has a capacity; each component has a capacity. SERVER WEB APP SVR PROXY MQ/ESB LDAP CORBA DCOM Datacenter Web Services 19
PE: Application performance management SA PVT CP APM PDR Software development lifecycle and architecture Performance testing and validation Capacity planning Application performance management Problem detection and resolution 20
Five elements of APM (Gartner) Monitoring the performance of complex distributed applications 1 End user experience measurement browser/mobile device 2 Create a model of the run-time environment (discovery) 3 4 5 Profile the performance and behavior of user-defined transactions Performance metrics from each of the applications/systems technical components (Webserver, App server, Database, etc.) Application performance management database 21
Rendering Track key user experience metrics Mobile Total end-user response time Browser rendering time Network latency Mobile Real User Monitoring 22
Rendering And see every hop of the transaction Mobile Dynamic Transaction Path Detection Proxy server Web server App server Data warehouse gateway Mainframe Database 23
PE: Problem detection and resolution SA PVT CP APM PDR Software development lifecycle and architecture Performance testing and validation Capacity planning Application performance management Problem detection and resolution 24
When problems occur Flash web site events (all at once) New code released causes problems Infrastructure upgrade Consolidation of applications and servers New workload model (batch and online) 25
Cause and Effect 26
Career path SPE Competencies 27
PE roles: The team many possible roles Managing and developing the core team members Performance Architect Sr. Performance Engineer Performance Engineer Sr. Test automation technician Workload and capacity Modeler Technical Project manager Test automation technician Performance data engineer Sr. Capacity planner Capacity planner 28
Underlying competencies Project management Analytical thinking problem solver Communication skills Statistics Business Knowledge Monitoring and tuning Underlying competencies Behavioral characteristics Workload model development Application and tool knowledge Test data management Test execution process Forecasting 29
Technical competencies Database monitoring tools Code profiling tools (Static analysis, diagnostics) Deep dive tools (HP Diagnostics, CA Wily, etc, Jprobe, etc) Resource utilization system level End to end performance tools Comprehensive results collection and reporting Statistics and Distributions Technical competencies Test design and execution tools Queuing and modeling tools Test data creation DB tuning SQL tuning 30
PE Techniques Brainstorming Data modeling Creating a workload model Performance testing Triage approach Static code analysis System metrics collection Logging and instrumentation 31
Career path: Performance engineer foundation Performance Engineer Grow your leadership and communication Lead technical role Communication Workload models Core technical competency Technical architecture Additional tech competency Production triage Performance testing Grow your breadth of capabilities 32
Upcoming Marketing event 33
How do these Knowledge areas work together (10X) Responsibilities CP CP PVT SA Forecast the workload for the event APM PDR Architecture and Tune application based on results Measure the current workload and resource utilization Plan and execute PT for forecast workload with support for troubleshooting 34
Questions? Thank you! wkuketz@collaborative.com 35
Development methodologies Agile Iterative RUP Relational Data models Object oriented Implementation concepts Swing Hibernate JMS Web Services Windows RESTful services SOAP Integrated Development Environments Eclipse, Windows, SQL Developer 36
Operating systems Windows Platform Unix and all its varient ios Z/OS Database platforms Oracle Microsoft DB/2 Application servers Microsoft Jboss Oracle AS Weblogic Websphere Application frameworks.net J2EE Spring Struts Programming languages Java C# XML C/C++ COBOL Ruby/ Reporting technologies Integrations technologies Software managements technologies 37
CP: Workload model CP The CP team uses this information to forecast the new workload 10X. Step Transaction by product Percentage 1 Login 100 2 Search product 100 3 View product detail 100 4 Add to shopping cart 50 5 Confirm purchase method 50 6 Shipping information 50 7 Send confirmation email 50 38
CP: Analyze current capacity CP Review current Production workload, and the under lying system utilization. Average workload Current peak workload Response time Service level System utilization 100% 39
PVT: planning, executing, and analysis PVT Use the workload model from CP Enable APM during testing Work with the Application Architect Methodology is iterative and tactical Review infrastructure & architecture Identify risk areas Review configuration settings, topology, & sizing Define points of measurement Define business activity profiles & service levels Types & numbers of users Business activities & frequencies Design & build tests Test data generation Create test scripts User & transaction profiles Infrastructure configuration Iterate testing & tuning 40