Enterprise Software Performance Engineering

Transcription

1 Enterprise Software Performance Engineering Presented By: Walter Kuketz Scaling your career..

2 Software Performance matters - everywhere DTG Facebook IPO 2

3 Who is responsible for end-to-end system performance? When a website slows down? When the marketing team runs a big event? When the Storage subsystem slows down, which slows down the database, which slows down the application servers, and slows down the ipad? Where do the performance goals come from? 3

4 Agenda Building systems that scream Review the current challenges of being a performance engineer Overview of the five knowledge areas of Software performance engineering body of knowledge Roles and career path overview Case study 10X event (Communicate to Sr. Mgt.) 4

5 No clear standards in the market for PE Test automation technician Capacity planner? Performance Engineer Software Engineer?? Troubleshooting Performance tester Performance Architect? Performance analyst? Benchmarks Performance test? Performance Data engineer? 5

6 Opportunity to take SPE to the next level.. Define profession Define the profession of performance engineering Establish Body of Knowledge Establish a body of knowledge for performance engineering Establish common language Establish a common language for roles, responsibilities, and practice areas using the form of Knowledge Areas Define competencies Define the competencies shared across the knowledge areas Capture common techniques Capture common techniques for sharing across the industry, or your company 6

7 Knowledge areas The SPE Body of Knowledge For the Enterprise Planning, coordination, information sharing, and control SDLC & architecture (SA) Performance validation and testing (PVT) Capacity planning (CP) Application performance management (APM) Problem detection and resolution (PDR) Underlying competencies and roles for delivering Techniques supporting the knowledge areas 7

8 The Enterprise PE team: There must be a practice leader Operations & PT Architecture and production Communication Business direction Development teams 8

9 The Enterprise PE team: There must be a practice leader A shared service and clearing house for all performance, scalability and capacity planning information. Transforms performance information to business knowledge and advantage. Communicates across the teams to share information Operations & PT For instance, the production workload and the performance test workload are they in sync? Architecture and production Make sure the architecture team knows how the new release, or newest build behave in production. Business direction The business risks are clearly identified and communicated. Application risk profile. Development teams Can provide performance guidelines to the developers, must influence the unit testing process. 9

10 PE: Knowledge areas SA PVT CP APM PDR Software development lifecycle and architecture Performance testing and validation Capacity planning Application performance management Problem detection and resolution 10

11 PE: Software development lifecycle and architecture SA PVT CP APM PDR Software development lifecycle and architecture Performance testing and validation Capacity planning Application performance management Problem detection and resolution 11

12 PE: SA (software development lifecycle) Waterfall methodologies Define Design Develop Functional test Performance test Deploy Need to plan for performance testing Build code to performance requirements Define Deploy Agile methods Design Test Build Can a system be designed to support 10 TPS and 1 second response time 100 TPS and 1 second? 12

13 Performance budgets Do your developers know how much time they have for their piece of the transaction? Code runs everywhere. SERVER WEB APP SVR PROXY MQ/ESB LDAP CORBA DCOM Respond to the user in 2 seconds. Datacenter Web Services 13

14 PE: Performance testing and validation SA PVT CP APM PDR Software development lifecycle and architecture Performance testing and validation Capacity planning Application performance management Problem detection and resolution 14

15 PE: PTV (performance testing and validation) What is changing? Where does it need to be validated? What (all) is being measured? The project: Why are we testing and what are the success criteria? Prerequisites: Business goals and non-functional requirements are defined Workload and scenarios defined Environment defined (where are we testing) Application in a stable state Configuration management and release management ready Core performance team identified Supporting team committed 15

16 Organization: Performance team value You still need to explain to the rest of the organization what you do Business and application area knowledge Early involvement with the applications team Test execution Environment Deep understanding of the application being tested Deep understanding of the technical architecture of the application Understanding of the business area supported by the applications Performance engineers involved in design sessions Understand the nature of the data required for testing Able to understand technical gaps to meet requirements not supported by the tool Well-understood workload model and characterization All components in the test are monitored and reported on A defined triage process for troubleshooting A process to manage the inevitable unplanned tests and test archive exists Testing environment is a close approximation of production, including database size Differences between test and production are clearly known Release and configuration management in place and procedures enforced Performance team value 16

17 PE: Capacity planning SA PVT CP APM PDR Software development lifecycle and architecture Performance testing and validation Capacity planning Application performance management Problem detection and resolution 17

18 PE: CP (capacity planning) The right system at the right price! Plan for future Forecast workloads Plan for usage Track utilization of existing systems Track workload Analyze current capacity Determine capacity requirements Define system to support workloads Test Define service levels Workloads drive system transactions, which drive resource utilization 18

19 Capacity: Use of resources The overall system has a capacity; each component has a capacity. SERVER WEB APP SVR PROXY MQ/ESB LDAP CORBA DCOM Datacenter Web Services 19

20 PE: Application performance management SA PVT CP APM PDR Software development lifecycle and architecture Performance testing and validation Capacity planning Application performance management Problem detection and resolution 20

21 Five elements of APM (Gartner) Monitoring the performance of complex distributed applications 1 End user experience measurement browser/mobile device 2 Create a model of the run-time environment (discovery) Profile the performance and behavior of user-defined transactions Performance metrics from each of the applications/systems technical components (Webserver, App server, Database, etc.) Application performance management database 21

22 Rendering Track key user experience metrics Mobile Total end-user response time Browser rendering time Network latency Mobile Real User Monitoring 22

23 Rendering And see every hop of the transaction Mobile Dynamic Transaction Path Detection Proxy server Web server App server Data warehouse gateway Mainframe Database 23

24 PE: Problem detection and resolution SA PVT CP APM PDR Software development lifecycle and architecture Performance testing and validation Capacity planning Application performance management Problem detection and resolution 24

25 When problems occur Flash web site events (all at once) New code released causes problems Infrastructure upgrade Consolidation of applications and servers New workload model (batch and online) 25

26 Cause and Effect 26

27 Career path SPE Competencies 27

28 PE roles: The team many possible roles Managing and developing the core team members Performance Architect Sr. Performance Engineer Performance Engineer Sr. Test automation technician Workload and capacity Modeler Technical Project manager Test automation technician Performance data engineer Sr. Capacity planner Capacity planner 28

29 Underlying competencies Project management Analytical thinking problem solver Communication skills Statistics Business Knowledge Monitoring and tuning Underlying competencies Behavioral characteristics Workload model development Application and tool knowledge Test data management Test execution process Forecasting 29

30 Technical competencies Database monitoring tools Code profiling tools (Static analysis, diagnostics) Deep dive tools (HP Diagnostics, CA Wily, etc, Jprobe, etc) Resource utilization system level End to end performance tools Comprehensive results collection and reporting Statistics and Distributions Technical competencies Test design and execution tools Queuing and modeling tools Test data creation DB tuning SQL tuning 30

31 PE Techniques Brainstorming Data modeling Creating a workload model Performance testing Triage approach Static code analysis System metrics collection Logging and instrumentation 31

32 Career path: Performance engineer foundation Performance Engineer Grow your leadership and communication Lead technical role Communication Workload models Core technical competency Technical architecture Additional tech competency Production triage Performance testing Grow your breadth of capabilities 32

33 Upcoming Marketing event 33

34 How do these Knowledge areas work together (10X) Responsibilities CP CP PVT SA Forecast the workload for the event APM PDR Architecture and Tune application based on results Measure the current workload and resource utilization Plan and execute PT for forecast workload with support for troubleshooting 34

35 Questions? Thank you! 35

36 Development methodologies Agile Iterative RUP Relational Data models Object oriented Implementation concepts Swing Hibernate JMS Web Services Windows RESTful services SOAP Integrated Development Environments Eclipse, Windows, SQL Developer 36

37 Operating systems Windows Platform Unix and all its varient ios Z/OS Database platforms Oracle Microsoft DB/2 Application servers Microsoft Jboss Oracle AS Weblogic Websphere Application frameworks.net J2EE Spring Struts Programming languages Java C# XML C/C++ COBOL Ruby/ Reporting technologies Integrations technologies Software managements technologies 37

38 CP: Workload model CP The CP team uses this information to forecast the new workload 10X. Step Transaction by product Percentage 1 Login Search product View product detail Add to shopping cart 50 5 Confirm purchase method 50 6 Shipping information 50 7 Send confirmation

39 CP: Analyze current capacity CP Review current Production workload, and the under lying system utilization. Average workload Current peak workload Response time Service level System utilization 100% 39

40 PVT: planning, executing, and analysis PVT Use the workload model from CP Enable APM during testing Work with the Application Architect Methodology is iterative and tactical Review infrastructure & architecture Identify risk areas Review configuration settings, topology, & sizing Define points of measurement Define business activity profiles & service levels Types & numbers of users Business activities & frequencies Design & build tests Test data generation Create test scripts User & transaction profiles Infrastructure configuration Iterate testing & tuning 40