4/26/11. CTG - Company overview. True stories about testing based on experiences. CTG - Company overview. CTG - Company overview

Transcription

1 CTG - Company overview True stories about testing based on experiences Corporate Headquarters Buffalo, NY Founded in 1966 University of Antwerp Patrice Willemot Pre sales test consultant Petra Haldermans - Test Consultant / Test Manager 27/04/ CTG - Company overview CTG - Company overview European Headquarters Diegem, Belgium Founded in 1976 Over employees worldwide 350 employees in Belgium Quality certification ISO 9001, Tick IT Beste Werkgever (2003, 2005, 2007, 2009 and 2011) Investor in People Bertrange, Luxembourg Reading, United Kingdom 3 4 1

2 CTG Belgium CTG Market Leader in Belgium Tought Leadership Strong & Active Partnerships Early Adopter in Software since Test Professionals Hosts Belgium s largest Software Test Seminar 3 times winner of Eurostar Best Paper Award International speakers HP Platinum Partner 3 times HP Partner of the Year in 5 years Microsoft Gold Partner Intellectual Property We Commit Focus on Innovation & Training 10+ years of experience consolidated in IP Tools STBoX FASTBoX PTBoX Prove savings Search for s Report on the value Invest in market best practices Share risks & rewards Always finish what we start CTGLabs STAcademy Centers of Excellence CTG Knowledge Center Plan Design Transition Execute Drive IT CTG as Strategic Partner Test Maturity Assessment Optimizing the software and service lifecycle Test Practices within reach STBoX Essentials Workflow Templates STBoX Prime Workflow Templates STBoX Agile Workflow Templates FASTBoX Best Utilities Practices PTBoX Best Templates Practices Instructions Examples Instructions Examples Instructions Examples Functional Libraries Instructions Instructions Examples Performance Test Governance Test Automation Implementation Excel within the Lifecycle Training & Coaching Do IT CTG as Operation Partner Technologies ST Academy Knowledge Base Training Center Wiki Centers of Excellence Knowledge sharing CTG Labs White Papers Annual Convention 7 8 2

3

4

5

6 21 22 True stories about testing based on experiences 3 Dilemmas V-model Test Plan and risks Test Design and Execution 1 How to Implementation of a test process Customer Case Company A: introduction Assignment: UAT + Test Management Test Governance Difficulties: Different parties: definition: Company A Development & (except UAT): Company B UAT : Company A & CTG Meetings via Skype (companies in Belgium and the Netherlands)

7 Customer Case Company B: introduction Dilemma 1: V-model Assignment Test Management, Acceptance No business users available for Acceptance Test Automation A company not familiar with the V-model Difficulties Distributed test teams (Trier, Paris, Gent, Bordeaux, Vienna) Different Test approaches Different development approaches (Scrum, V-model, Waterfall model) No distinction between for example and Acceptance Different level of test maturity (very low, low, medium, high) Dilemma 1: The theory Dilemma 1: The theory User Acceptance User Acceptance Global Design Global Design Detailed Design Detailed Design Code/Build Code/Build

8 Dilemma 1: The solution Dilemma 1: The solution Goal: Increase business knowledge of the tester User Acceptance Change the mindset of the end user How: Combination of end users and testers Organize guided user acceptance test sessions Split test tasks between testers and end users Test Design on business level by IT Testers Review test cases by business Test Execution by business Global Design Promote early moment of involvement of testing Detailed Design Conclusion: Think about possible risks and workarounds during test planning! Code/Build Customer Case: Company A Customer Case: Company A done in NL: Unit Unit User CTG Acceptance Company A done in BE: User Acceptance, by both CTG and Company A (business users) Test Design by CTG Review by Company A (business users) Execution by CTG and Company A Global Design Detailed Design Code/Build Company B (Netherlands)

9 Customer Case: Company B Customer Case: Company B Phase 1: Phase 3: CTG responsible for and Acceptance Test Design and Test Execution CTG responsible for testing during a Scrum Sprint Test Design and Test Execution Company B responsible for Acceptance Phase 2: Test Execution of test cases created by CTG CTG responsible for Test Design and Test Execution Company B responsible for Acceptance Phase 4: CTG responsible for testing during a Scrum Sprint Test Design and Test Execution Test Execution of test cases created by CTG CTG responsible for testing the integration of different Scrum Sprints (Scrum of Scrum) Company B responsible for Acceptance Test Execution of test cases created by CTG Customer Case: Company B Phase 1 Customer Case: Company B Phase 2 Phase 3 Acceptance

10 Customer Case: Company B Dilemma 2: Test Plans and Risks Phase 4 The client doesn t want to have a complete Test Plan A Test Plan: what do I need to add? Acceptance How to deal with risks? We never thought about them We have a Test Plan, but nobody reads it Dilemma 2: The theory Dilemma 2: The theory Important things to consider (Risks & Based ): product risks & requirements Features To Test project risks test levels & quality attributes test types priorities constraints: available resources people & expertise infrastructure tools & techniques time & budget

11 Dilemma 2: The theory IEEE Template content 16 points test plan information introduction test Items features to be tested features not to be tested approach item pass / fail criteria suspension and resumption criteria test deliverables testing tasks environmental needs responsibilities staffing and training needs schedule risks and contingencies approvals Dilemma 2: The solution Goal: Create awareness & pay attention to the importance of a Test Plan How: Organize a workshop about the content of the Test Plan (needs) Pay attention to the consequences in case no Test Plan is agreed upon Check consistency with Project Plan (e.g. escalations procedures) Rely on the Test Policy Think about the type of project (Agile / classic development) Conclusion: Convince the client that a Test Plan has a lot of advantages, even when a company is used to work in the same way for each project Reach consensus with a list of known risks Customer Case: Company A Customer Case: Company B A full version Test Plan with extra attention on: Suspension & Resumption criteria Test Project Risks Objectives per Test Level (and thus per company) Entry & Exit criteria between UAT and the other Test Levels Description of the UAT process, with pre-defined milestones / quality gates A full version of a test plan has been created based on the Company B template Test Items Environmental needs Scope Staffing and training needs Test Approach Risks and contingencies Organization Agreements Test Deliverables Glossary Test plan has never been reviewed or read by stakeholders Test plan has been ignored during the test project New template of the test plan has been created Strengths and must haves: Penalties for the Risks Process approach Roles and Responsibilities Environmental needs Formal agreement of the document

12 Dilemma 3: Test Design & Execution Dilemma 3: The theory We only want to do Exploratory Input Process My team has a lot of knowledge about Test Design Techniques I am used to work with business users, but my new test team is not able to execute the existing test cases what should be my level of detail? - High-level test case describes the input data and the predicted result on an abstract level. It can contains one or more low level test cases. - Example: A man age > 35 with >2 children - Low-level test case, the abstract values that were assigned to the input an predicted output of the high level test case, are replaced by concrete values. - Example: Dirk Honda, 70 years, 4 children Expected Result Dilemma 3: The theory Dilemma 3: The theory Test Design Techniques: Line coverage Equivalence Partitioning Statement coverage Boundary Value Analysis Decision coverage (= Branch coverage) Algorithm Test Condition coverage Decision Table Test Decision / Condition coverage Syntactic Test Multiple Condition coverage Semantic Test Condition Determination coverage Elementary Comparison Test (ECT) Real-life Test Data Cycle Test / Entity Life Cycle Test Error Guessing Process Cycle Test Random Test Program Interface Test Idiot Proofing Exploratory testing is simultaneous learning, test design, and test execution An interactive test process Using the information gained while testing to design new and better tests A formal process defined tasks, objectives and deliverables Test Sessions and charts Testers have the skills to listen, read, think and report rigorously and effectively

13 Dilemma 3: The solution Customer Case: Company A Goal: Unit : no written test cases Define the right fit for the company Unit : no written test cases How: Learn from experiences; what went OK, what went wrong in the past? Define advantages & disadvantages of high / low level test cases Think about which and whether you want to oblige Test Design Techniques Find out the knowledge of your test team : written test cases in Excel; titles imported in Microsoft Team Foundation Server 2010 User Acceptance : high level written test cases in TFS Written (Use Case ) by CTG and reviewed by the business users of Company A A list of needed test data created and finally defined by Company A Extra: error guessing done (not structured planned) Conclusion: Define the Test Strategy based on the needs and people within your test team Pitfall: all testing based on the same Use Case Specification Document, with defined objectives per test level Customer Case: Company B True stories about testing based on experiences At the start -integration test cases based on requirements on the back of a napkin High level test cases with high level information (no click on button, ) Acceptance test cases based on requirements on the back of a napkin High level test cases with high level information (no click on button, ) 3 Dilemmas V-model Test Plan and risks Test Design and Execution Evolved to 1 How to Implementation of a test process Acceptance test cases based on requirements on the back of a napkin High level test cases with high level information used by the end users Scrum test cases based user stories Only a title, no steps Uncontrolled exploratory testing Test Automation based on user stories

14 Implementation of a test process: A best practices approach 2 main phases Test Maturity Assessment Test Practices Test Maturity Assessment Test Practices Test Maturity Assessment Interview Metrics Documentation Determine SWOT Identification Assessment AS IS TO BE People Process Technology Workshop Based on STBoX TM Increase S and O Eleminate W and T Test Maturity Assessment Test Practices s Business cases Project cards Test Maturity Assessment Test Practices Roadmap s roadmap From improvement goals to roadmap Test Maturity Assessment Identification TO BE goal 1 goal 2 goal 3 s action 1 action 2 action 3 action 4 action 5 action 6 action 7 action 8 s project 1 project 2 Quick Win SMALL MEDIUM LARGE +/- 6 ac;vi;es Informal 1 formal deliverable Quickscan Size of the assessment depends on # departments # interviews # languages # test levels +/- 12 ac;vi;es Formal +/- 10 formal deliverables Regular # test types Size of organiza;on +/- 22 ac;vi;es Formal +/- 14 formal deliverables Profound Scope People Process Technology

15 Test Practices Outcome of the assessment Quality Roadmap Business cases Project Cards Reworked roadmap Re prioritization Life long learning Continuous improvement Implementation Prioritised roadmap People Process Technology Retrospective Test Maturity (Re-)Assessment Self Assessment QUESTIONS AND ANSWERS