Introduction to White box and Black Box Software Testing

Transcription

1 Introduction to White box and Black Box Software Testing Henry Muccini Computer Science Department University of L'Aquila - Italy muccini@di.univaq.it [

2 Copyright Notice» The material in these slides may be freely reproduced and distributed, partially or totally, as far as an explicit reference or acknowledge to the material author is preserved. Henry Muccini 2

3 Agenda» Recap on Software Testing» White box Testing - JUnit» Black box Testing - Model-based Testing 3

4 Recap on Software Testing 4

5 An all-inclusive definition Software testing consists of: the dynamic verification of the behavior of a program IMP on a finite set of test cases suitably selected from the (in practice infinite) input domain against the specified expected behavior 5 [A. Bertolino]

6 Main Activities & Terminology Main Activities: Test case Selection Test case Execution Regression Testing Terminology: Test Coverage Oracle Adequacy criteria Test Objective» Main principles: - Repeatable - Measurable 6

7 Main Artefacts 7» Any software artifact can be used for testing purposes

8 Main Type of Testing» Unit, Integration, and System Testing» White Box and Black box Testing» Stress Testing» Performance Testing» 8

9 White Box Testing 9

10 Black Box vs. White Box Testing SELECTED INPUTS RESULTANT OUTPUTS DESIRED OUTPUT BLACK BOX TESTING SELECTED INPUTS RESULTANT OUTPUTS INTERNAL BEHAVIOR DESIRED OUTPUT SOFTWARE DESIGN 10 WHITE BOX TESTING

11 Main principles» The code is available» Objective: Cover the software structure 11 - How?

12 12

13 Example: CFG» Build a graph representation of the source code» Cover the graph 13

14 Coverage Criteria» Based on Control-flow and Data-flow information» Among the most famous: - Statement coverage - Branch coverage - Path coverage 14

15 Category Partition» In the case of a branch, how to select the inputs? 15

16 JUnit» Unit Testing di Java classes» White box Testing» Intuizione: - Data una classe, l idea consiste nel: > Creare una instanza di tale classe, con valori specificati (classe di test) > Controllare che tale classe di test si comporti come specificato 16

17 JUnit 17

18 Black Box Testing 18

19 Black Box vs. White Box Testing SELECTED INPUTS RESULTANT OUTPUTS DESIRED OUTPUT BLACK BOX TESTING SELECTED INPUTS RESULTANT OUTPUTS INTERNAL BEHAVIOR DESIRED OUTPUT SOFTWARE DESIGN 19 WHITE BOX TESTING

20 Main Principles» The code is NOT available - How to select inputs for test cases?» Inputs can be selected based on specifications» Model-based Testing 20

21 Modeling: there is not just programming» A model is a simplification of the reality - The reality is abstracted, since it is too complex - Everything is a model?!?» Examples 21

22 Examples 22

23 Examples 23

24 Examples 24

25 Why to make models? Document and Analyze 25

26 What is MBT?» It consists in: - Extracting Test Cases from the Model - Run the test case over the system implementation» Purpose: - To validate the implementation conformance to the model > The model itself is the oracle, i.e., it represents the expected behavior 26

27 Test Selection» A test case, in MBT, is typically a possible scenario - In scenario-based testing: > A scenario itself is an abstract test case - In state-based testing > Execution flows are extraced from the state machine and represent a test case > MB coverage criteria are applied on this model 27

28 In practice 28

29 Test Execution» It consists in running the test cases on the system implementation - So to compare the real execution with the expected behavior 29

30 MBT practical concerns» Traceability, i.e. relating the abstract values of the specification to the concrete values of the implementation.» Execution, i.e. forcing the Implementation Under Test (IUT) to execute the specific sequence of events that has been selected. 30

31 AsmL TESTOR Model-based Testing Approaches UMLAUT UMLTest AGEDIS TOTEM SOOTF COW-SUITE SCENTOR SeDiTeC UCSC-System 31

32 MBT Tools» There are some automated tools for MBT: - Telelogic Tau and Tau Tester - Leirios Test Designer - Conformiq Qtronic - Rhapsody Test Conductor and Automatic Test Generation - AGEDIS 32

33 MBT Tools 33

34 MBT Tools 34

35 Remarks» Learn English» GSEEM - EURECA Project

36 References» H. Muccini. Software Testing: Testing new Software Paradigms and new Artefacts. In the Wiley Encyclopedia of Computer Science & Engineering 2008» A. Bertolino. Software Testing Research: Achievements, Challenges, Dreams, ICSE 2007 Future of Software Engineering Track» J. Gao, H. Muccini, and X. Bai, Software Test Automation from Components to Systems, John Wiley & Sons, to appear in