SOFTWARE TESTING (PART 2)

White Box Testing

White box testing (a.k.a. clear box testing, glass box testing, transparent box testing, or structural testing) uses an internal perspective of the system to design test cases based on internal structure.

It requires programming skills to identify all paths through the software.

The tester chooses test case inputs to exercise paths through the code and determines the appropriate outputs

White Box Testing

Since the tests are based on the actual implementation, if the implementation changes, the tests will need to change.

While white box testing is applicable at the unit, integration and system levels of the software testing process, it is typically applied to the unit.

Normally tests paths are within a unit, but test paths can be between units during integration, and between subsystems during a system level test.

White Box Testing

Typical white box test design techniques include: Control flow testing Data flow testing Branch testing Path testing

Black-Box Testing

Black box testing takes an external perspective of the test object to derive test cases.

These tests can be functional or non-functional, though usually functional.

The test designer selects valid and invalid inputs and determines the correct output. There is no knowledge of the test object's internal structure.

Black-Box Testing

This method of test design is applicable to all levels of software testing: unit, integration, functional testing, system and acceptance.

The higher the level, and hence the bigger and more complex the box, the more one is forced to use black box testing to simplify.

While this method can uncover unimplemented parts of the specification, one cannot be sure that all existent paths are tested.

Data Flow Testing

Data flow testing is testing based on the flow of data through a program.

Data-flow testing looks at the lifecycle of a particular piece of data (i.e. a variable) in an application.

By looking for patterns of data usage, risky areas of code can be found and more test cases can be applied.

Data Flow Testing

Data flows from where it is defined to where it is used.

A definition of data or def is when a value is assigned to a variable.

Data Flow Testing

Computation use or c-use is when the value of a variable is referenced.

Predicate use or p-use is when a variable appears in the condition of a decision statement.

Data Flow Testing

A p-use is assigned to both branches out of a decision statement.

A c-use occurs on the right hand side of assignment statements, when printing variables, when passing a parameter into a function or procedure by value, etc.

Data Flow Testing

D

F

E

G

HI

JK

P-use a,b,c

P-use a,b,c

P-use a,b,c

P-use a,b,c

P-use a,b,c

P-use a,b,c

P-use a,b,c

P-use a,b,c

def a,b,c

def type

def type

def type

def type

c-use type

A

B

C

def typeNode Source Line

A Read a, b, c

B Type=”scalene”

C if (a==b||b==c||a==c)

D Type=”isosceles”

E if (a===b && b==c)

F Type=”equilateral”

G If (a>=b+c ||b>=a+c||c>=a+b)

H Type=”not a triangle”

I if (a<=0||b<=0||c<=0)

J Type=”bad inputs”

K Print type

Data Flow Testing

A definition free path or a def-free path is a path from the definition of a variable to a use of that variable (either c-use or p-use) that does not contain another definition of that variable.

Data Flow Testing

D

F

E

G

HI

JK

P-use a,b,c

P-use a,b,c

P-use a,b,c

P-use a,b,c

P-use a,b,c

P-use a,b,c

P-use a,b,c

P-use a,b,c

def a,b,c

def type

def type

def type

def type

c-use type

A

B

C

def type

Find the def-free paths for each variable in this graph.

Data Flow Testing

Some data flow testing criteria:

dcu – a def-free path exists from every definition to a c-use.

dpu – a def-free path exists from every definition to a p-use.

du – a def-free path exists from every definition to a possible use.

all du paths – a def-free path exists from every definition to every possible use.

Data Flow Testing

Which criteria does this graph specify?

D

F

E

G

HI

JK

P-use a,b,c

P-use a,b,c

P-use a,b,c

P-use a,b,c

P-use a,b,c

P-use a,b,c

P-use a,b,c

P-use a,b,c

def a,b,c

def type

def type

def type

def type

c-use type

A

B

C

def type

Random Testing

Random testing is accomplished by randomly selecting the test cases.

Advantages: Fast Eliminates the bias of the testers

Easier to make statistical inferences

Random Testing

For the triangle problem: Use a pseudo-random number generator to generate three numbers within a given range.

What would be some of the problems with testing this way?

Operational Profiles An operational profile

is a specification of types and the probability those types will be encountered in normal operation.

# Description Probability

1 Equilateral 15%

2 Isosceles – right 10%

3 Isosceles – acute 20%

4 Isosceles – obtuse 5%

5 Scalene – right 10%

6 Scalene – acute 25%

7 Scalene – obtuse 15%

Random numbers could then be generated to match the probability of occurrence for each type.

Statistical Inference From Testing If random testing has been done by

randomly selecting test cases according to an operational profile, then the behavior of the software during testing should be the same as its behavior in the operational environment.

Statistical Inference From Testing

For example: If in 1000 test cases randomly selected there were three errors, then the error rate of the software could be inferred as .3%

In fact, since the error rate is small, the number of errors can be inferred to have a Poisson distribution with mean 3 and variance 3.

Boundary Testing

Often errors happen at boundaries between domains.

Consider the statement if (X < 1) {blah blah}

An on test for X is is value of X that makes the statement true

An off test is a value of X that makes the statement false.

Boundary Testing

The trick is to have test values in each region determined by the boundaries

Boundary Testing

The boundary for X<1 is X=1 and this boundary lies in the false domain.

The boundary for X<=1 is still X=1, but this boundary lies in the true domain.

Boundary testing chooses values that lie on or near the boundary.

Boundaries can lie in either the false domain or the true domain

Boundary Testing

For the triangle example, the primitive conditions a>b+c and b>a+c and c>a+b determine a boundary of the intersection of three planes in 3-space.

Show the boundaries for these inequalities graphically and choose appropriate values for boundary testing.

http://www.livephysics.com/ptools/online-3d-function-grapher.php