Unit Testing Methods White-box Testing Black-box Testing Extensions.
Black Box Unit Testing
description
Transcript of Black Box Unit Testing
Black Box Unit Testing
1209
What is black-box testing?
• Unit (code, module) seen as a black box
• No access to the internal or logical structure
• Determine if given input produces expected output.
Input Output
Black Box Testing
• Test set is derived from requirements • Goal is to cover the input space• Lots of approaches to describing input space:
– Equivalence Classes– Boundary Value Analysis– Decision Tables– State Transitions– Use Cases– . . .
Advantage and Disadvantage
• Advantage:
• Disadvantage:
• Need to define an efficient strategy to limit number of test cases
General Black Box Process
• Analyze requirements
• Select valid and invalid inputs
• Determine expected outputs
• Construct tests
• Run tests
• Compare actual outputs to expected outputs
Equivalence Relations:
• Reflexive
• Symmetric
• Transitive
Equivalence classes: Basic Strategy• Partition the input into equivalence classes
• Test a subset from each class
Basic strategy:
Example: fact(n): if n<0, or n>=200 errorif 0<=n<=20, exact valueif 20<n<200, approximate value within .1%
Simple Example:
• Suppose you are building an airline reservation system. A traveler can be a child, an adult, or a senior.
• The price depends on the type of traveler. • The seat reservation does not depend on the type
of traveler.• How many test cases are required for the
reservation component and the billing component?
Finding equivalence classes:
• Identify restrictions for inputs and outputs in the specification
• If there is a continuous numerical domain, create one valid and two or three invalid classes (above, below, and NaN)
• If a number of values is required, create one valid and two invalid classes
Boundary Values:
• Programs that fail at interior elements of a class usually fail at the boundaries too.
• Test the boundaries.
Hints
• If a domain is a restricted set, check the boundaries. e.g., D=[1,10], test 0, 1, 10, 11– It may be possible to test the boundaries of outputs,
also.
• For ordered sets, check the first and last elements• For complex data structures, the empty list, full
lists, the zero array, and the null pointer should be tested
Boundary Value: in class
• Determine the boundary values for US Postal Service ZIP codes
• Determine the boundary values for a 15- character last name entry.
Decision Tables
• Construct a table (to help organize the testing)
• Identify each rule or condition in the system that depends on some input
• For each input to one of these rules, list the combinations of inputs and the expected results
Decision Table Example
Test Case C1
Student
C2
Senior
Result
Discount?
1
2
3
4
Theater ticket prices are discounted for senior citizens and students.
Pairwise Testing
Problem 1From Lee Copeland, A Practitioner’s Guide to Software Test Design, Artech
House Publishers, 2004.
• A web site must operate correctly with different browsers: IE 5, IE 6, and IE 7; Mozilla 1.1; Opera 7; and FireFox 2, 3, and 4.
• It must work using RealPlayer, MediaPlayer, or no plugin.• It needs to run under Windows ME, NT, 2000, XP, and
Vista, and 7.0• It needs to accept pages from IIS, Apache and WebLogic
running on Windows NT, 2000, and Linux servers
Problem 2
• A bank is ready to test a data processing system• Customer types
– Gold (i.e., normal)– Platinum (i.e., important)– Business– Non profits
• Account types– Checking– Savings– Mortgages – Consumer loans– Commercial loans
• States (with different rules): CA, NV, UT, ID, AZ, NM
From Lee Copeland, A Practitioner’s Guide to Software Test Design, Artech House Publishers, 2004.
When given a large number of combinations: (options improve …)
• Give up and don’t test• Test all combinations … miss targets, delay
product launch, and go out of business• Choose one or two cases
Orthogonal Arrays
• A 2-D array with the property– All pairwise combinations occur in every pair of
columns
• Example: consider 3 variables (columns) with {a,b}, {1,2}, and (α,β)
1 2 3
1 1 A α
2 1 B β
3 2 A β
4 2 B α
Orthogonal Array Example
1 2 3
1 1 A α
2 1 B β
3 2 A β
4 2 B α
Look at each pair of columns (1 and 2),( 1 and 3), and (2 and 3)
Does each of the 4 pairings appear in each?
(Yes, of course!)
Pairwise Testing Algorithm
1. Identify variables
2. Determine choices for each variable
3. Locate an orthogonal array
4. Map test cases to the orthogonal array
5. Construct tests
Example using Bank
1. Identify variables
2. Determine choices for each variable
3. Locate an orthogonal array 4. Map test cases to the orthogonal array5. Construct tests
Locate an orthogonal array (look up on web)
1 1 1
1 2 2
1 3 3
1 4 4
2 1 2
2 2 1
2 3 4
2 4 3
3 1 3
3 2 4
3 3 1
3 4 2
4 1 4
4 2 3
4 3 2
4 4 1
5 5 1
5 1 2
5 2 3
5 3 4
6 5 2
6 1 1
6 2 4
6 3 3
1 5 3
2 5 4
3 5 1
4 5 2
5 4 1
6 4 2
Map Test Cases (30)CA Check Gold
CA Save Plat
CA Mort Busi
CA Cons NonP
NV Check Plat
NV Save Gold
NV Mort NonP
NV Cons Busi
UT Check Busi
UT Save NonP
UT Mort Gold
UT Cons Plat
ID Check NonP
ID Save Busi
ID Mort Plat
ID Cons Gold
AZ Comm Gold
AZ Check Plat
AZ Save Busi
AZ Mort NonP
NM Comm Plat
NM Check Gold
NM Save NonP
NM Mort Busi
CA Comm Busi
NV Comm NonP
UT Comm Gold
ID Comm Plat
AZ Cons Gold
NM Cons Plat
Pairwise Summary• When the combination is large, test all pairs,
not all combinations
• Studies indicate that most defects are single or double-mode effects – Can be found in pairing– Few defects require more than two modes
• Orthogonal arrays have the pairwise property and can be found or generated
State Transition Testing
State Transition Testing
• Build STD of system or component
• Cover the STD
Example
Res MadePaid
CancelledNon Pay
Ticketed
CancelledCust
Used
Cust Order/start timer
Payment madeTicket Printed
Time Expires
Ticket DeliveredCancel/refund
Cancel/refund
Cancel
Use Case Testing