CS 217 Software Verification and Validation
description
Transcript of CS 217 Software Verification and Validation
CS 217 Software Verification and
Validation
Week 8, Summer 2014
Instructor: Dong Sihttp://www.cs.odu.edu/~dsi
REVIEW OF BB-testing
Black-box Testing
1. Input Space Partitioning
2. Boundary Value Analysis
3
Example 1: compare two numbers – p50 of week3
Function ‘Compare (x, y)’
Inputs: Two numbers – x and y Outputs: A larger number between x and y
4
Compare (x, y) = z
(x, y) z
– p51 of week3
5
• Equivalence Classes:
{ (x, y) | x < y }
{ (x, y) | x > y }
{ (x, y) | x = y }
{ input other than a pair of numbers,
“as&%dfget^$(&w” }
Valid inputs
Invalid inputs
6
Valid (x, y) Input Space
x = y
x < y
x > y
Three test cases:
(1, 2) --- 2
(8, 8) --- 8
(100, 30) --- 100
Plus one test cases:
(^&%*) --- ERROR
– p52 of week3
Example 2: Loan application- p53 of week3
7
Customer Name
Account number
Loan amount requested
Term of loan
Monthly repayment
Term:
Repayment:
Interest rate:
Total paid back:
6 digits, 1stnon-zero
$500 to $9000
1 to 30 years
Minimum $10
2-64 chars.
Choosing (or defining) partitions seems easy, but is easy to get wrong…
8
Customer name
Number of characters:
2 64 65invalid valid invalid
1
Conditions ValidPartitions
InvalidPartitions
ValidBoundaries
InvalidBoundaries
Customername
2 to 64 charsvalid chars
< 2 chars> 64 charsinvalid chars
2 chars64 chars
1 chars65 chars0 chars
Valid characters:Anyother
A-Za-z-’
space
9
Loan amount
500 9000 9001
invalid valid invalid
499
Conditions ValidPartitions
InvalidPartitions
ValidBoundaries
InvalidBoundaries
Loanamount
500 - 9000 < 500>90000non-numericnull
5009000
4999001
Black-box Testing
1. Input Space Partitioning
2. Boundary Value Analysis
10
Example 3 - Bonus
Consider a program IsAnOddNumber(x) which output “Yes” or “No” to the user to tell if the input integer x is a odd number or not.
If you are planning to classify all possible user inputs into two classes - valid and invalid inputs
1) How do you partition the valid input space (the integer number space)?
2) Can you give an example of invalid test data?
11
Example 3 - Bonus
If your software manager asks the program MUST ONLY take input real number x as: -100<x<100
1) How do you partition all possible user inputs into two classes - valid and invalid inputs now?
2) If the software manager further asks you to do some Boundary Value Analysis, then what do you plan to do for the BVA analysis?
12
Example 4 - Bonus
Consider a program Absolute(x) which calculate the absolute value of a input real number x.
If you are planning to classify all possible user inputs into two classes - valid and invalid inputs
1) How do you partition the valid input space (the real number space)?
2) Can you give an example of invalid test data?
13
Example 4 - Bonus
If your software manager asks the program MUST ONLY take input real number x within the range of:
-321<x<-123, 123<x<321
1) How do you partition all possible user inputs into two classes - valid and invalid inputs now?
2) If the software manager further asks you to do some Boundary Value Analysis, then what do you plan to do for the BVA analysis?
14
Today’s topic
WHITE-BOX TESTING
Styles of Testing Testing traditionally can be conducted in two
styles
Black-Box testingTry to choose "smart" tests based on the requirements,without looking at the code.
White-Box testingTry to choose "smart" tests based on the structure of thecode, with minimal reference to the requirements.
16
Definition of White-Box Testing
Testing based on analysis of internal logic (design, code, etc.). (But expected results still come from requirements.)
Use different techniques to check every visible path of the source code to minimize errors.
It is the ability to know which line of the code is being executed and being able to identify what the correct output should be.
17
Characters of WB testing Exploits the structure, control or data flow of
programs in order to design test cases.
Typically performed during coding.
Allows to test parts of the program, since you know the structure. (with black‐box is not possible)
Allows to cover systematically every part of the program.
18
WB Testing Techniques
Logic coverage: (learned in previous classes) Statement coverage Branch coverage Condition coverage …
Dataflow based testing / Path coverage: all program paths have been traversed at least once
19
Pseudo code & Control/Dataflow Graphs
20
“nodes”
“edges”Input
output
Absolute (x){ IF (x>=0) THEN y = x;
ELSE IF (x<0) THEN y = -x;
Output y;}
x
IF (x>=0) ELSE IF (x<0)
y = x; y = -x;
y
Can you draw a graph?
21
Absolute (x){ IF (x>0) THEN y = x; ELSE IF (x==0) THEN y = x;
ELSE IF (x<0) THEN y = -x;
Output y;}
Example
22
Fun (x, y){ z = 0;
IF ((x>z) or (y>z)) THEN z = 2;
ELSE z = 3;
Output z; }
Test cases:1. Fun (0, 0) 2. Fun (2, 0) 3. Fun (0, 2) 4. Fun (2, 2) 5. Fun (8, 9)
Control/Dataflow of the example
23
Fun (x, y){ z = 1;
IF ((x>z) or (y>z)) THEN z = 2;
ELSE z = 3;
Output z; }
Input x, y
z = 1;
IF ((x>z) or (y>z))
z = 2;
ELSE
z = 3;
Output z;
Test cases:1. Fun (0, 0) 2. Fun (2, 0) 3. Fun (0, 2) 4. Fun (2, 2) 5. Fun (8, 9)
Statement coverage
Has each statement in the program been executed?
24
√ √
√
Branch coverage
Has each branch of each control structure been executed?
For example, given an if statement, have both the T and F branches been executed?
Another way of saying this is, has every edge in the Control/Dataflow graph been executed?
25
Condition coverage
Has each Boolean sub-expression evaluated both to true (T) and false (F) ?
26
Test cases:1. Fun (0, 0) 2. Fun (2, 0) 3. Fun (0, 2) 4. Fun (2, 2) 5. Fun (8, 9)
X>1 Y>1
T , F ? T , F ?
Fun (x, y){ z = 1;
IF ((x>z) or (y>z)) THEN z = 2;
ELSE z = 3;
Output z; }
Exercise Consider a program ‘Compare’ which compare
the value between two numbers. The inputs are two real numbers x and y, the program outputs the larger number between x and y.
27
Compare (x, y){ IF (x>y) THEN z = x;
ELSE IF (x==y) THEN z = x;
ELSE IF (x<y) THEN z = y;
Output z;}
Exercise - Control/Data Flow Graphs
I have provided you the Pseudo code,
Can you draw a Control/Dataflow Graphs?
Try it! This is a bonus question with +5 points towards to FINAL!!
28
Exercise - Statement Coverage
Statement Coverage requires that each statement will have been executed at least once.
What is the minimum number of test cases required to achieve statement coverage for the program Compare (x, y) ?
29
Exercise - Branch Coverage
Branch Coverage requires that each branch will have been traversed at least once.
What is the minimum number of test cases required to achieve branch coverage for the program Compare (x, y) ?
30
Condition Coverage
Condition Coverage requires that each condition (sub-expression) will have been True at least once and False at least once.
What is the relationship between Branch and Condition Coverage?
Condition / branch coverage
IF ( and )
THEN …
ELSE …
32
X>0 Y>0
T F
T , F ? T , F ?
Condition Coverage example – Combination !
IF (A or B) THEN s1;
ELSE s2;
A B Branch
test 1 T F true
test 2 F F false
test 3 T T true
test 4 F T true
Combinations of condition values: TT, TF, FT, FF
Dataflow based testing -> Path coverage
All program paths have been traversed at least once.
3434
Two paths !
Input x, y
z = 1;
IF ((x>z) OR (y>z))
z = 2;
ELSE
z = 3;
Output z;
Path coverage - Example 1
35
if a then s1else if b then s2 else if c then s3 else s4 end_if_then_else end_if_then_elseend_if_then_else
# Paths: ___
# Branches: ___
WB Testing Techniques
Logic coverage: (learned in previous class) Statement coverage Branch coverage Condition coverage …
Dataflow based testing / Path coverage: all program paths have been traversed at least once
36
Coming Up Next week…
Lab 2: White-box Testing
Preparation: Go over the slides of WB testing Understand “Code coverage” & “Data flow /
Path”