CS 325: Software Engineering

March 31, 2015

Software Testing• Black-Box Testing• White-Box Testing• Regression Testing

CS 325March 31, 2015Page 2

Black-Box TestingRecall that black-box testing is used to determine whether a software system meets its functional specifications.Black-box testing comes in multiple flavors:

Equivalence PartitioningDivide the input and output domains into disjoint subsets and test one selection from each subsetBoundary Value Analysis

Set up the Equivalence Partitioning subsets; test selections at and near their boundaries

Cause-Effect AnalysisMap the different combinations of system input values to their corresponding resultant output values

CS 325March 31, 2015Page 3

Black-Box Testing:Equivalence Partitioning

For the function below, equivalence partitioning would involve three input pairs: one producing underflow, one producing uinderflow, and one producing neither.

int safe_add( int a, int b ){ int c = a + b; if ( (a >= 0) && (b >= 0) && (c < 0) ) cout << "ERROR: Overflow!\n" ); else if ( (a < 0) && (b < 0) && (c >= 0) ) cout << "ERROR: Underflow!\n" ); return c;}

CS 325March 31, 2015Page 4

Black-Box Testing:Boundary-Value Analysis

For the function below, boundary values would be where a+b=INT_MAX and where a+b=INT_MIN, while values just beyond the boundaries would be where a+b=IKNT_MAX+1 an d where a+b=INT_MIN-1.int safe_add( int a, int b ){ int c = a + b; if ( (a >= 0) && (b >= 0) && (c < 0) ) cout << "ERROR: Overflow!\n" ); else if ( (a < 0) && (b < 0) && (c >= 0) ) cout << "ERROR: Underflow!\n" ); return c;}

CS 325March 31, 2015Page 5

Black-Box Testing:Cause-Effect AnalysisConsider a loan application software system, where users can enter the amount of the monthly repayment or the number of years they want to take to pay back the loan (i.e., the term of the loan). If a user enters both, the system will make a compromise between the two if they conflict.The two conditions are the loan amount and the term, which are put in a table, along with their different combinations of possible values.

Conditions Rule 1

Rule 2

Rule 3

Rule 4

Repayment amount has been entered

T T F F

Term of loan has been entered

T F T F

Conditions Rule 1

Rule 2

Rule 3

Rule 4

Repayment amount has been entered

T T F F

Term of loan has been entered

T F T F

Outcome: Process loan amount

Y Y

Outcome: Process term Y Y

Outcome Error message YTest cases would then be developed to test all four of the cause-effect rules.

CS 325March 31, 2015Page 6

Black-Box Testing:Cause-Effect Analysis (continued)Consider a credit card system that offers discounts.

Conditions Rule 1

Rule 2

Rule 3

Rule 4

Rule 5

Rule 6

Rule 7

Rule 8

New customer (15%)

T T T T F F F F

Loyalty card (10%)

T T F F T T F F

Coupon (20%) T F T F T F T F

Note that Rules 1 and 2 represent impossible cases (new customers with loyalty cards). Also, Rule 3 assumes that customers always select the larger discount (when they have to select).

New customers who want to open a credit card account will get a 15% discount on the day they open their accounts.Existing customers who hold a loyalty card, always get a 10% discount.Finally, customers who have a coupon, can get 20% off today (but this coupon discount can’t be used with the ‘new customer’ discount). Discount amounts are added, if applicable.

Conditions Rule 1

Rule 2

Rule 3

Rule 4

Rule 5

Rule 6

Rule 7

Rule 8

New customer (15%)

T T T T F F F F

Loyalty card (10%)

T T F F T T F F

Coupon (20%) T F T F T F T F

Effect: Discount NA NA 20% 15% 30% 10% 20% 0%

CS 325March 31, 2015Page 7

White-Box TestingWhite-box testing derives test cases via knowledge of the source code itself.

Essentially, the minimum number of test cases needed for white-box testing is the number of distinct paths from the start to the finish of the code (with loops being traversed no more than once).

This number is called the cyclomatic complexity of the program.

CS 325March 31, 2015Page 8

White-Box Testing:Cyclomatic Complexity

Consider the various paths to implement the following rule in the game Monopoly.If a player lands on a property owned by an opponent, the player needs to pay the rent. If the player doesn’t have enough money, the player is out of the game. If the property is not owned, and the player has enough money, the player may buy the property for the price associated with the property.

CS 325March 31, 2015Page 9

White-Box Testing:Cyclomatic Complexity (continued)There are six independent paths through this code:1. 1-2-3-4-5-10 (property owned by others, no

money for rent) 2. 1-2-3-4-6-10 (property owned by others, pay rent) 3. 1-2-3-10 (property owned by the player) 4. 1-2-7-10 (property available, don’t have enough money) 5. 1-2-7-8-10 (property available, have money, don’t want to buy it) 6. 1-2-7-8-9-10 (property available, have money, and buy it)

So, the minimum number of test cases needed to test this would be six.

CS 325March 31, 2015Page 10

Regression TestingWhen features are added or altered, the changes might cause previously working code to fail

Common methods of regression testing include re-running previously run tests and checking whether previously fixed bugs have reemerged