Acceptance Testing under Control

Transcription

1 Acceptance Testing under Control Matej Sova

2 Agenda Quality in Unicorn Systems What is the challenge Solution Outsourcing as a possible way Client view Copyright Unicorn Systems 2

3 Effective acceptance testing Clearly defined and quantitatively assessable (measurable) Cost and time optimized Flexible and tailored to company s needs Manageable Copyright Unicorn Systems 3

4 Quality by Unicorn Systems

5 Unicorn Information Systems Quality of Unicorn Information Systems means ability of ICT solution to be used Unicorn Information Systems are used in almost all of Europe, as well as extending beyond its borders Copyright Unicorn Systems 5

6 Unicorn Approach For our products and services to be attractive to clients while also being beneficial in the long-term, they have to meet the following three basic characteristics: Quality Image Effectiveness Copyright Unicorn Systems 6

7 What does mean good system? Copyright Unicorn Systems 7

8 What can be wrong? 1. Wrong idea 2. Wrong design 3. Wrong programming 4. Wrong implementation 5. Wrong operation 6. Improper integration 7. Infrastructure does not work and the last cause is a situation where a user does not know how to use the ICT functionality correctly. Copyright Unicorn Systems 8

9 What is the challenge

10 The challenging constraints Copyright Unicorn Systems 10

11 Make testing in QSTB Copyright Unicorn Systems 11

12 Time / budget vs. acceptable risk Copyright Unicorn Systems 12

13 Time / buget vs. quality, quantity Copyright Unicorn Systems 13

14 Place for management Copyright Unicorn Systems 14

15 Quantity of acceptance testing Product / services Processes Information Systems Testing types Testing cycles Copyright Unicorn Systems 15

16 Example of quantity (telco project) 300 business services / products 3 types of customers 8 activation channels 10 related systems 2 test cycles test points for functional acceptance testing How can we manage this? Copyright Unicorn Systems 16

17 Solution

18 Static view Copyright Unicorn Systems 18

19 Dynamic view Copyright Unicorn Systems 19

20 Organization view Copyright Unicorn Systems 20

21 The rule 31 Copyright Unicorn Systems 21

22 Outsourcing as a possible way

23 Testing as a Service Mission If we want to achieve high-quality software, we must address two basic issues: effective product evaluation whether the product is or is not "of sufficient quality" and when we can release it, effective communication if the product is not "of sufficient quality", how do we communicate this to the other members of the development team. Task of testing vendor is to perform an objective and independent evaluation of a certain situation/status/condition and provides valuable feedback to the business team, development team and managers. Copyright Unicorn Systems 23

24 Testing as a Service Copyright Unicorn Systems 24

25 Testing as a Service Added Value Known cost of testing Flexible working model Simplicity Independent verification of quality Key employees focused on daily business Increased quality of delivered solutions Testing services provided under SLA Unique testing expertise and experience combined with business processes knowledge Copyright Unicorn Systems 25

26 Client View Peter Varga

27 QA in Telco industry Telco industry is in context of continual competitive pressure and revenue decline, heavily focusing on efficiency and costs optimization One of key levers is outsourcing/out-tasking of non core activities Telco services are being commoditized, thus quality is perceived by customers as a MUST, as basic feature of services provided Telcos are trying to find new revenue streams, by diversification of products and attacking new markets (ICT, TV, Content,.) New lines of business are demanding new skills everywhere, incl. testing Challenge for testing is how to adapt to limited internal capacities, while ensuring high Quality standards and proper management Copyright Unicorn Systems 27

28 Testing model implemented Internal Testing team is quite limited (3FTE), providing mostly methodology and test approach for projects Team is located in IT, but standalone/separated During testing execution phase, testing team is in role of QA of test vendor Testing execution is outsourced, due to More eyes principle (IT engineers are deep in topic, might miss important facts) Internal capacities (internal people usually focusing on more core-business activities) Skills (good testing is a real skill, not only question of available capacities) Testing is not delivered through body-shopping, but through Fix- Scope-Fix-Price model Copyright Unicorn Systems 28

29 FSFP model Test vendor is responsible for full test approach Agreed scope, agreed final price, included test management Internal resources are responsible only for final QA Project brief, project plan Request for testing Opportunity study Business requirement register Business requirement Test requirement Test approach chapter (part of design document for every considered alternative) Tender Extended test approach for approved alternative Opportunity study Test strategy universal document about SW testing in OSK Project brief, project plan Business requirement Business requirement Business requirement register Test plan Quotation Test book Test reports (Progress report / Exit report) Testing Test case Test case Test case Test analyses Test plan Copyright Unicorn Systems 29

30 Benefits Transparent scope and processes, clear responsibilities Sharing of pain by vendors not respecting quality targets More value for less money Fix scope clear change request rules Fix time - strong motivation of test vendors to deliver on time Pressure on strong cooperation with engineering teams Massive testing introduced as part of Quality programs Helped us to reduce IT customer complains by 50% But, appropriate level of testing requires sensitive searching of balance Copyright Unicorn Systems 30

31