ABCs of Performance Testing Series I August 20, 2014

Transcription

1 ABCs of Performance Testing Series I August 20, 2014

2 Brought to you by

3 Hosted by Richard Bishop Vivit United Kingdom Chapter Leader Vivit Director of Education

4 Today s Speakers Mark Tomlinson Practitioner of Performance Testing West Evergreen Consulting, LLC Silvia Siqueira PMP, ITIL Expert, Scrum Master WW Senior Product Marketing Manager HP Software

5 Housekeeping This LIVE session is being recorded Recordings are available to all Vivit members Session Q&A: Please type questions in the Questions Pane

6 Webinar Control Panel Toggle View Window between Full screen/window mode. Questions

7 NEWS August 20, 2014

8 News: Free Meetup in NY Free Meetup in NY Learn what is new for Application Performance lifecycle and DevOps presented by Prod mgmt. and RnD September 9 (Monday) 4 pm to 7 pm. Venue: New York Hilton Midtown NY Register at: Attendees will be invited to attend the Velocity Ignite Reception Event -7 pm to 8:30 pm

9 News: Velocity Conference NY Conference Velocity NY Sept HP Platinum Sponsor 4 sessions by HP Prod Mgr and RnD Venue: New York Hilton Midtown NY Take advantage of 25% discount: Code HP25 Register at:

10 News: Free Meetup in Anaheim-CA Free Meetup in Anaheim - CA Learn what is new for Mobile and Application Lifecycle October 13 (Monday) 4 pm Venue: Disneyland Hotel Register at:

11 NEWS: StarWest Conference Conference StarWest CA Oct Breaking Software HP Platinum Sponsor: 2 sessions by HP Prod Mgr and RnD Venue: Disneyland Hotel Exclusive vendor discount code S14VW to save up to $200 off their STARWEST registration! Register at:

12 Never-ending PerfBytes Podcasting As Summer comes to a close James and Mark will get back into full-time publishing in the Fall we have lots of new ideas. PerfRants, sponsor topics, interviews and guest hosts - and as always Satan will keep bringing us the News of The Damned with the hottest in website and system failures across the globe. Listen now at

13 NEWS: Discover Barcelona Dec 2014 HP Discover Barcelona Registration and Call for papers is now open:

14 NEWS: Additional events October WOPR #23 "the twenty-third gathering of the Workshop on Performance and Reliability, in Waltham, Massachusetts. And for those who don't know WOPR is a peer conference and the subject is "Performance Escalation"...and I will be serving as the content owner. For more information visit November 3rd-6th STPCON Fall - being held in the mile-high city of Denver, Colorado - a really great conference for performance testers and engineers, we have a great line up of workshops and sessions - learn more at

15 ABCs of Performance Testing Series I August 20, 2014

16 Polling Question #1 What is your knowledge level for Performance testing? a. None b. Basic c. Intermediate d. Advanced

17 Important facts about performance 44% of mobile app defects are found by end users 63 percent of testers stated they were not 63% not able to cover the number of test cases or test across all the different devices and OS versions Interface problems at 58 % Performance at 52% Functionality at 50% Device compatibility at 45% 17 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

18

19

20 What is Performance? Performance Testing research Performance Validation gatekeeping Performance Engineering design/dev Performance Management maintenance Performance Optimization re-design Performance Capacity trending, forecasting Performance Review managing people

21 Performance Requirements* Performance is an attribute of Quality: Usability is the site usable if it s too slow? Scalability can it handle more users? Reliability will it stay up? Availability how long will it stay up? Manageability can we monitor it? Maintainability can we keep it up? (Scott Barber calls them desirements more accurately)

22 Performance testing lifecycle Step I Planning the test Step II Step III Test preparation: creating the scripts and scenarios Test execution: running and monitoring scenarios Step IV Test analysis and reporting: understanding the results Step V Monitoring in production

23 Planning the testing Business goals and SLAs Test requirements Criticality Application under test (AUT) User behavior, user locations, and application timeframe usability Special factors regarding the application under test Inter-disciplinary Team

24 Performance Organizations Development (very unreal world) Performance Testing Practice Production Performance (a.k.a. the Real World )

25 Performance Test Design Proper performance test design starts by stating a hypothesis about the system and then determining the parameters for the system conditions to simulate against the system and the specific measurements to be taken during the simulation to support the calculated analysis required to determine the results of the test. ( or just throw some load at it!)

26 Test Types: Load Test 1x Load for 1-2 hours Objective: validate the system s ability to handle a sustained load at a constant specific level (e.g. Run for 2 hours at 1000 trans/sec)

27 Test Types: Stress Test Objective: apply gradually, continuously increasing load to the system until an error state and/or failure occurs (e.g. increase # of virtual users until the system crashes)

28 Test Types: Scalability Test 3x 2x 1x Objective: measure system performance at consistent intervals of increased load (1x, 2x and 3x), useful for calculating a trend for performance projections

29 Business Value of Performance Money Risk

30 Business Value of Performance Every business leader wants to: ensure a positive customer experience ensure a system can grow and scale ensure the system can sustain heavy traffic ensure the application works in diverse conditions estimate future spending to support the system optimize the system to meet new goals

31 Business Value of Performance Find ways to convey the meaning of performance results: show how performance impacts customers always find a way to make your test relevant show performance impacts to the bottom line find a way to win over your stakeholders target multiple audiences with specific results expand the sophistication of your test

32 Business Value of Performance The graphs from your testing tool can help to show interesting technical anomalies or exceptions like a logout exception shown above.

33 Business Value of Performance Graph shows a scalability test that fails to reach 2x or beyond

34 Business Value of Performance $10K/sec $1K/sec 2X Add Financial Information to the Chart

35 Join the Revolution

36 The need to be proactive: App Performance Testing The cost of doing performance testing Applications Faster user response time High Productivity Customer Satisfaction Trusted Brand Revenue Business Growth 36 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

37 Polling Question #2 Are you currently using any performance testing solutions? a. Yes b. No

38 HP Performance testing solution If you ever dream to have it all together, you have it now! StormRunner Load Simple, Smart, Scalable Cloud Testing Free public beta LoadRunner Easy, Fast, Powerful Performance and load testing Free 50 Virtual users For unlimited time Performance Center Global Collaboration and Sharing Full LoadRunner Capabilities plus Center of Excellence Free 50 Virtual users For unlimited time Network Virtualization + Service Virtualization 38 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

39 Selecting the HP Performance testing solution Customer needs for performance testing What app need to be tested? Web and mobile (test in the cloud) Any application Where the test will be executed Cloud based, on demand On Premise Type of performance test Simple, Scale, Agile AUT in the cloud Delivered by HP Easy, Fast, Powerful Site. Project based test Global and Complete End-to-End performance testing StormRunner Load LoadRunner Delivered on SaaS Performance Center On premise 39 Customer can always use a mix and match approach and share scripts between LR/PC and SRL e.g. if customers wants an on-prem deployment but wants to add capacity for a new mobile team they could use LR/PC + SRL together Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

40 Why HP Trusted by millions of users Solutions for ANY performance testing need Most comprehensive features compared to others 50 FREE Virtual users for an unlimited time 40 Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

41 Download HP LoadRunner now 50 Vusers free for unlimited time Join Facebook group: HP LoadRunner (

42 SUMMARY Get started with Performance Objectives ( requirements ) Decide which kinds of tests will best fit the goals Find a stakeholder in IT and Business for each test type Get organized and follow the right process for your work Gather your results data into a meaningful conclusion: use proper visualizations that convey the message your business leaders aren t performance engineers your technical leaders aren t thinking about money or risk

43 Polling Question #3 What other topics are you interest for webinars? a. Best Practices and Requirements b. Cloud performance testing c. Transaction simulation, Test execution, metrics analysis d. Continuous testing, Continuous integration e. Advanced topics: Websockets, HTML5, Speedy, Center of Excellence

44 Contact me Mark Tomlinson Independent Performance Consultant mtomlins.blogspot.com

45 Questions and Answers?

46 Thanks

47 HP Performance testing solutions Performance Testing solutions StormRunner Load LoadRunner Performance Center On premise Delivered on SaaS Monitor with SiteScope Profiler solution to pinpoint root cause Diagnostics Detect Network behavior Network Virtualization Virtualize test environment Service Virtualization 47 Customer can always use a mix and match approach and share scripts between LR/PC and SRL e.g. if customers wants an on-prem deployment but wants to add capacity for a new mobile team they could use LR/PC + SRL together Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.