White Box Testing and Symbolic Execution Written by Michael Beder.

White Box Testing and Symbolic Execution

Written by Michael Beder


2

Agenda

• What is White Box Testing?

• Flow Graph and Coverage Types

• Symbolic Execution:– Formal Definition– Examples– Questions


3

What is White Box Testing?

• Software testing approach that uses inner structural and logical properties of the program for verification and deriving test data

• Also called: Clear Box Testing, Glass Box Testing and Structural Testing

• Manual: Inspection, Walkthrough

• Automatic: Syntax Parser, Symbolic Execution


4

Pros and ConsPros:

• Usage of more information on the tested object

(than BlackBox)

• Inference of real Equivalence Partitioning

• Structural Coverage Assurance

Cons:

• Expensive

• Limited Semantic Coverage


5

Example I

Sin(x) {

if (|x| < eps) return x;

if (|x| < 5*eps) return x – (x^3)/6;

…

}

• Black Box Testing: Test 0, Pi/2, -Pi/2

• White Box Testing: Test 0.5*eps, eps, 3*eps, 5*eps, 7*eps, …

Usage of logical properties makes better coverage


6

Example II: Unit Testing

f(x) g(x) { h(x) {

{ … …

if (x > 0) return g(x); } }

return h(x);

}

• Black Box Testing: Test f(x), g(x), h(x) for every x

• White Box Testing: Test g(x) for x > 0, h(x) for x <= 0 and verify f(x)

Usage of structural properties makes fewer, qualitative tests


7

Flow Graph

• Abstraction of the program

• Defines the data and control flow in the program

• Uniform representation of the program, language independent

• Simple basic elements: assignment and condition

• Further analysis is performed using Graph Algorithms


8

Flow Graph – cont.

• G = (V, E) whenV is a set of basic blocks. start, end in VE is a set of control branches

Example:

1 a = Read(b)2 c = 03 while (a > 1) {4 If (a^2 > c)5 c = c + a6 a = a - 2}

1, 2

3

4

5

end

start

T

T

6F

F

Input: b = 2

Output:a = 0, c = 2


9

White Box Coverage Types

• Statement Coverage: Every statement is executed

• Branch Coverage: Every branch option is chosen

• Path Coverage: Every path is executed

• Basic Path Coverage: Every basic path is executed

• All definition-use path Coverage: All paths between the definition and a usage of a variable are executed

Loops?


10

Basic Path Coverage• Let p1, p2 be paths from start to end. Then, p1 < p2 if exists vertex v

such that p2 contains v and p1 doesn’t contain v• Find maximal set of paths p1, p2, …, pk such that pi < pj for i < j• Such set of paths is of size E – N + 2 (Linear Complexity)• Each path is called “basic path”

• Example:

p1 = start – 1,2 – 3 – endp2 = start – 1,2 – 3 – 4 – 6 – 3 – endp3 = start – 1,2 – 3 – 4 – 5 – 6 – 3 – end

E – N + 2 = 8 – 7 + 2 = 3

1, 2

3

4

5

end

start

T

T

6F

F


11

Path Function

• A function , when D is the working domain

• Represents the current values of the variables as function of their initial values

• Each variable X is represented by a projection function

• Function composition:

• For example:

: n nf D D

: nXf D D

1( )( ) ( ( ),..., ( ))

nX Xg f v g f v f v

( , , ) ( , , )

( , , ) ( , , ) ( , , )X Y Z

f X Y Z X Y X Y XZ

f X Y Z X Y f X Y Z X Y f X Y Z XZ

( , , ) ( , , )

( )( , , ) ( ( , , ), ( , , ), ( , , ))

( , , ) (( )( ),( ) , )X Y Z

g X Y Z XY X Z Z

g f X Y Z g f X Y Z f X Y Z f X Y Z

g X Y X Y XZ X Y X Y X Y XZ XZ


12

Path Condition

• A condition that should be fulfilled for going along the path• A constraint on the initial values of the variables

For Example: p = start – 1,2 – 3 – end.1 a = Read(b)2 c = 03 while (a > 1) {4 If (a^2 > c)5 c = c + a6 a = a - 2}

The path condition is B <= 1, when B is b’s value at start

1, 2

3

4

5

end

start

T

T

6F

F


13

Symbolic Execution• A method for deriving test cases which satisfy a given path• Performed as a simulation of the computation on the path• Initial path function = Identity function, Initial path condition =

true• Each vertex on the path concludes a symbolic composition

on the path function and a logical constraint on the path condition:If an assignment was made:

If a conditional decision was made:path condition path condition branch condition

Output: path function and path condition at the final vertex of the path

f g f ( )X g X

[ ( )]X f X


14

Example IIIx = x + y

y = y + x

end

The final path function represents the values of X, Y, Z after both assignments as a function of their initial values

1 0( , , ) ( , , ) ( , , ) ( , , )g X Y Z X Y Y Z f X Y Z X Y Z

1 1 0 1( , , ) ( )( , , ) ( , , ) ( , , )f X Y Z g f X Y Z g X Y Z X Y Y Z

2 1( , , ) ( , , ) ( , , ) ( , , )g X Y Z X Y X Z f X Y Z X Y Y Z

2 2 1 2( , , ) ( )( , , ) ( , , )

( , ( ), ) ( , 2 , )

f X Y Z g f X Y Z g X Y Y Z

X Y Y X Y Z X Y Y X Z

2 ( , , ) ( ,2 , )f X Y Z X Y Y X Z


15

Concatenation and Associativity

If is the path function of path and is the path function of

path then is the path function of path

The composition is associative:

1pf 1 1( ,..., )np v v

2pf

2 ( ,..., )n kp v v1 2 2 1p p p pf f f

1 2 1( ,..., ,..., )n kp p v v v

1 2 3 1 2 3 2 3 1 2 1

1 2 3 1 2 3 3 1 2 3 2 1

3 2 1 3 2 1

( ) 3

( )

( ) )

( )

( ) ( )

p p p p p p p p p p p p

p p p p p p p p p p p p

p p p p p p

f f f f f f f

f f f f f f f

f f f f f f

Symbolic Execution is a special case when

1 1 2 1( ,..., ), ( , )n n np v v p v v

V1 Vn Vk

1pf

2pf

1 2 2 1p p p pf f f


16

Example IV1 a = Read(b)

2 c = 0

3 while (a > 1) {

4 If (a^2 > c)

5 c = c + a

6 a = a - 2

}

Find test case for path:

p = start – 1,2 – 3 – 4 – 5 – 6 – 3 – 4 – 5 – 6 – 3 – end

1, 2

3

4

5

end

start

T

T

6F

F

input = b

output = c


17

Example IV

1, 2

3

4

5

end

start

T

T

6F

F

1 a = Read(b)

2 c = 0

3 while (a > 1) {

4 If (a^2 > c)

5 c = c + a

6 a = a - 2

}

p = start – 1,2 – 3 – 4 – 5 – 6 – 3 – 4 – 5 – 6 – 3 – end

vertex path function path condition

start: (A, B, C) true

1,2 (A, B, C) true

3 (B, B, 0) true

4 (B, B, 0) (true Λ B>1) ↔ B>1

5 (B, B, 0) (B>1 Λ B^2>0) ↔ B>1

input = b

output = c


18

Example IV

1, 2

3

4

5

end

start

T

T

6F

F

1 a = Read(b)2 c = 03 while (a > 1) {4 If (a^2 > c)5 c = c + a6 a = a – 2}

p = start – 1,2 – 3 – 4 – 5 – 6 – 3 – 4 – 5 – 6 – 3 – endvertex path function path condition6 (B, B, B) B>13 (B-2, B, B) B>1

4 (B-2, B, B) (B>1 Λ B-2>1) ↔ B>3

5 (B-2, B, B) (B>3 Λ (B-2)^2>B) ↔ B>46 (B-2, B, 2B-2) B>43 (B-4, B, 2B-2) B>4

end (B-4, B, 2B-2) (B>4 Λ B-4<=1) ↔ B=5

input = b

output = c


19

Example IV

1, 2

3

4

5

end

start

T

T

6F

F

1 a = Read(b)

2 c = 0

3 while (a > 1) {

4 if (a^2 > c)

5 c = c + a

6 a = a – 2

}

p = start – 1,2 – 3 – 4 – 5 – 6 – 3 – 4 – 5 – 6 – 3 – end

end (B-4, B, 2B-2) B=5

Hence the test case is B = 5 and the expected result is

2B-2 = 8.

Is there a test case forp = start – 1,2 – 3 – 4 – 5 – 6 – 3 – 4 – 5 – 6 – 3 – 4 – 5 – 6 – 3 – end ?

input = b

output = c


20

Question (from exam)

1 d = b + c;

2 if (d > 20)

3 a = 3 * a + d;

4 if (b < a) {

5 a = 1;

6 if (d < 2 * b)

7 b = 2;

8 }

1. Draw program’s Flow Graph

2. Find minimal number of test cases for the following coverage types:

a) Statement Coverage

b) Path Coverage

c) Branch Coverage

d) Basic Path Coverage


21

White Box Testing vs. Black Box Testing

Given a function f(X1, X2, …, X10) with the following preconditions:

1. Every parameter is odd

2. Every parameter is less or equal to M

3. Some parameter is equal to M

The function should report about every precondition that is not fulfilled

f examines each parameter in turn using if statements (without else)

and handles differently the following cases:

a. Exactly one parameter is higher than M

b. Two or more parameters are higher than M

Check f’s correctness using White/Black Box Testing methods

White Box Testing and Symbolic Execution Written by Michael Beder.

Documents

Transcript of White Box Testing and Symbolic Execution Written by Michael Beder.