Test Process Improvement the art of getting it right

Transcription

1 Test Process Improvement the art of getting it right BCS SIGiST 19 th September 2012 Presented by Geoff Thompson Listen Challenge Understand Interpret Create Experimentus Ltd 17a Dorset Square London NW1 6QB T: +44 (0)

2 Experimentus Was one of the first companies to be accredited by the TMMi Foundation Has undertaken over 50 formal accredited TMMi assessments Has 9 accredited TMMI assessors Has formally TMMi certified five companies: MTP Level 2, 3 and 4 UK Home Office Level 2 and 3 Aricent Level 2,3 and 4 Malaysian Software Testing Board Level 2 and 3 Steria SFR programme Level 2 and 3 2

3 Why change? To find fault is easy; to do better may be difficult Plutarch Companies invest significant amounts in maintaining processes that don t work, particularly in the areas of Software Quality, but will invest little in getting it right Martin Adcock Those who don t learn from history are doomed to repeat it George Santayana 3

4 We all want processes like this? 4

5 However reality is. 5

6 Today's Challenge PREPARATION REQUIREMENTS SPECIFICATION ANALYSIS DESIGN CODING / BUILD Q u a l i t y TESTING TESTING DEPLOYMENT PREPARATION REQUIREMENTS SPECIFICATON ANALYSIS DESIGN CODING / BUILD TESTING DEPLOYMENT Managing the RISK so that we build QUALITY into the process Testing becomes even more of an INSPECTION process and therefore introduces more RISK Testing becomes a validation exercise and can help reduce your Project timescales and costs 6

7 Failure Point 1 - When and What to improve When:- Does timing even matter? What:- Once you have a business case Only what you have a business case for A bit at a time Use a model to help understand what's needed 7

8 To model or not to model that is the question? Model based process improvement incorporates process improvement with international standards and frameworks such as the SEI s CMMI frameworks, ISO frameworks, ITIL, TMMi, PCMM and various others. Using such models allows organisations to compare their practices with internationally recognized industry good practices. Non-model based approach involves process improvements by adapting what already exists and augmenting with process components picked up at conferences/training. These processes typically tend to be home grown untried and untested 8

9 Do you feel lucky... 9

10 TMMi the only independent model 10

11 Growth in the use of TMMi model Independent model non commercial Freely available download from No defined test process it defines process components allowing the implementation to be flexible for adapted for each customer Accredited assessment methods are as applicable for Structured as well as Agile methods Accredited Assessments are repeatable by any accredited company 51 countries involved a truly international model 5 levels of maturity Aligns with CMMi Easy to recognise levels of maturity 11

12 Failure Point 2 - How! Initiate Obtain agreement to progress Measure Assess Process/monitor against plan Evolution of programme approach & targets Evolve Organisation Prioritise & Plan Action Planning Target Setting Process Technology Monitoring of results against targets Validate Define / Re-define New process definition/refinement Operate Training Pilot Full deployment 12

13 Enthusiasm Failure Point 3 - But this success is not just process. Maximum Hype Practical Benefit Actual Use Disillusionment Measure/Prioritise and RIP Plan/ Define and Redefine Operate/Validate Evolve 13

14 Failure Point 4 - Communicate 14

15 Benefits of Test Process Improvement with TMMi 40% savings made after changes made to ensure certification to Level 2, for a UK Government dept A UK bank saved 8% of their entire IT budget and improved development output by 12% An insurer saved 440,000 on a 2m project A test services company has increased their efficiency by 8% Retailer reversed their decision to outsource, when recommendations including an ROI showed them it will be a lot cheaper if they improved the internal processes 15 15

16 Test Maturity Survey - Background The need to understand the industry Data collected from 2009 Survey data updated during last quarter 2011 Over 150 respondents Many industry sectors represented Respondents based in over 12 different countries

17 Overall results TMMi level % TMMi level % 17

18 Top Industry Challenges Repeatable processes Product Risk Assessment Defect Detection Vs Defect Prevention Stakeholder Management Configuration Management Test Environment Management Monitoring & Control 18

19 Test Maturity Survey Get a copy yourself by going to And completing this years survey

20 Remember. Never start an improvement program without measuring your start position Always ensure you understand and get agreement to the benefits of the changes Communicate, communicate, and then communicate some more (Bob Nelson Ph.D) 20

21 Test Process Improvement That s one small step for a tester, one giant leap for all testers 21 Neil Armstrong 1969

22 Thank you (0)