Introduction to

White box and Black Box

Software Testing

Henry MucciniComputer Science Department

University of L'Aquila - Italymuccini@di.univaq.it –

[www.HenryMuccini.com]

Software Testing

Copyright Notice

» The material in these slides may be freely reproduced

and distributed, partially or totally, as far as an explicit

reference or acknowledge to the material author is

2

SEA Group

© 2009 by H. Muccini

SEA GroupSEA Group

reference or acknowledge to the material author is

preserved.

Henry Muccini

Agenda

» Recap on Software Testing

» White box Testing

- JUnit

»

3

SEA Group

© 2009 by H. Muccini

SEA GroupSEA Group

» Black box Testing

- Model-based Testing

Recap on Software Testing

4

SEA Group

© 2009 by H. Muccini

SEA GroupSEA Group

Recap on Software Testing

Software testing consists of:

the dynamic verification of the behavior of a program

An all-inclusive definition

5

SEA Group

© 2009 by H. Muccini

SEA GroupSEA Group

on a finite set of test cases

suitably selected from the (in practice infinite) input

domain

against the specified expected behavior

[A. Bertolino]

IMP

Main Activities & Terminology

Main Activities:

– Test case Selection

– Test case Execution

– Regression Testing

Terminology:

– Test Coverage

– Oracle

– Adequacy criteria

– Test Objective

6

SEA Group

© 2009 by H. Muccini

SEA GroupSEA Group

– Test Objective

– …

» Main principles:

- Repeatable

- Measurable

Main Artefacts

7

SEA Group

© 2009 by H. Muccini

SEA GroupSEA Group

» Any software artifact can be used for testing

purposes

Main Type of Testing

» Unit, Integration, and System Testing

» White Box and Black box Testing

» Stress Testing

8

SEA Group

© 2009 by H. Muccini

SEA GroupSEA Group

» Performance Testing

» …

White Box Testing

9

SEA Group

© 2009 by H. Muccini

SEA GroupSEA Group

White Box Testing

Black Box vs. White Box Testing

SELECTED INPUTS

RESULTANT OUTPUTS

DESIRED OUTPUT

10

SEA Group

© 2009 by H. Muccini

SEA GroupSEA Group

SELECTED

INPUTS

RESULTANT OUTPUTS

INTERNAL BEHAVIOR

DESIRED OUTPUT

SOFTWARE DESIGN

“BLACK BOX” TESTING

“WHITE BOX” TESTING

Main principles

» The code is available

11

SEA Group

© 2009 by H. Muccini

SEA GroupSEA Group

» Objective: Cover the software structure

- How?

12

SEA Group

© 2009 by H. Muccini

SEA GroupSEA Group

Example: CFG

» Build a graph representation of the source code

» Cover the graph

13

SEA Group

© 2009 by H. Muccini

SEA GroupSEA Group

Coverage Criteria

» Based on Control-flow and Data-flow information

» Among the most famous:

- Statement coverage

- Branch coverage

14

SEA Group

© 2009 by H. Muccini

SEA GroupSEA Group

- Branch coverage

- Path coverage

Category Partition

» In the case of a branch, how to select the inputs?

15

SEA Group

© 2009 by H. Muccini

SEA GroupSEA Group

JUnit

» Unit Testing di Java classes

» White box Testing

16

SEA Group

© 2009 by H. Muccini

SEA GroupSEA Group

» Intuizione:

- Data una classe, l’idea consiste nel:

> Creare una instanza di tale classe, con valori specificati (classe di

test)

> Controllare che tale classe di test si comporti come specificato

JUnit

17

SEA Group

© 2009 by H. Muccini

SEA GroupSEA Group

Black Box Testing

18

SEA Group

© 2009 by H. Muccini

SEA GroupSEA Group

Black Box Testing

Black Box vs. White Box Testing

SELECTED INPUTS

RESULTANT OUTPUTS

DESIRED OUTPUT

19

SEA Group

© 2009 by H. Muccini

SEA GroupSEA Group

SELECTED

INPUTS

RESULTANT OUTPUTS

INTERNAL BEHAVIOR

DESIRED OUTPUT

SOFTWARE DESIGN

“BLACK BOX” TESTING

“WHITE BOX” TESTING

Main Principles

» The code is NOT available

- How to select inputs for test cases?

20

SEA Group

© 2009 by H. Muccini

SEA GroupSEA Group

» Inputs can be selected based on specifications

» Model-based Testing

Modeling: there is not just programming

» A model is a simplification of the reality

- The reality is abstracted, since it is too complex

- Everything is a model ?!?

» Examples

21

SEA Group

© 2009 by H. Muccini

SEA GroupSEA Group

» Examples

Examples

22

SEA Group

© 2009 by H. Muccini

SEA GroupSEA Group

Examples

23

SEA Group

© 2009 by H. Muccini

SEA GroupSEA Group

Examples

24

SEA Group

© 2009 by H. Muccini

SEA GroupSEA Group

Why to make models? Document and Analyze

25

SEA Group

© 2009 by H. Muccini

SEA GroupSEA Group

What is MBT?

» It consists in:

- Extracting Test Cases from the Model

- Run the test case over the system implementation

26

SEA Group

© 2009 by H. Muccini

SEA GroupSEA Group

» Purpose:

- To validate the implementation conformance to the model

> The model itself is the oracle, i.e., it represents the expected

behavior

Test Selection

» A test case, in MBT, is typically a possible “scenario”

- In scenario-based testing:

> A scenario itself is an abstract test case

27

SEA Group

© 2009 by H. Muccini

SEA GroupSEA Group

- In state-based testing

> Execution flows are extraced from the state machine and represent a

test case

> MB coverage criteria are applied on this model

In practice

28

SEA Group

© 2009 by H. Muccini

SEA GroupSEA Group

Test Execution

» It consists in running the test cases on the system

implementation

- So to compare the real execution with the expected

behavior

29

SEA Group

© 2009 by H. Muccini

SEA GroupSEA Group

MBT practical concerns

» Traceability, i.e. relating the abstract values of the specification to the concrete values of the implementation.

30

SEA Group

© 2009 by H. Muccini

SEA GroupSEA Group

» Execution, i.e. forcing the Implementation Under Test (IUT) to execute the specific sequence of events that has been selected.

Model-based Testing Approaches

TESTO

UMLAU

TOTEM

SOOT

COW-SU

SCENTO

SeDiTe

AGEDI

UMLTe

UCSC-Sy

AsmL

31

SEA Group

© 2009 by H. Muccini

SEA GroupSEA Group

OR

UT

MTF

UITE

OR

eC IS

est

ystem

L

MBT Tools

» There are some automated tools for MBT:

- Telelogic Tau and Tau Tester

- Leirios Test Designer

- Conformiq Qtronic

32

SEA Group

© 2009 by H. Muccini

SEA GroupSEA Group

- Conformiq Qtronic

- Rhapsody Test Conductor and Automatic Test Generation

- AGEDIS

MBT Tools

33

SEA Group

© 2009 by H. Muccini

SEA GroupSEA Group

MBT Tools

34

SEA Group

© 2009 by H. Muccini

SEA GroupSEA Group

Remarks

» Learn English

» GSEEM

35

SEA Group

© 2009 by H. Muccini

SEA GroupSEA Group

» GSEEM

- http://informatica.di.univaq.it/gseem

» EURECA Project

- http://www.mrtc.mdh.se/eureca

References

» H. Muccini. “Software Testing: Testing new Software Paradigms and new Artefacts”. In the Wiley Encyclopedia of Computer Science & Engineering –2008

» A. Bertolino. “Software Testing Research: Achievements, Challenges, Dreams”, ICSE 2007

36

SEA Group

© 2009 by H. Muccini

SEA GroupSEA Group

Achievements, Challenges, Dreams”, ICSE 2007 Future of Software Engineering Track

» J. Gao, H. Muccini, and X. Bai, “Software Test Automation from Components to Systems”, John Wiley & Sons, to appear in 2009