Ch.8
-
Upload
softwarecentral -
Category
Documents
-
view
456 -
download
0
description
Transcript of Ch.8
8. UNIT TESTING
Plan project
Integrate & test system
Analyze requirements
Design
Maintain
Test unitsImplement
Software Engineering Roadmap: Chapter 8 Focus
Identify corporate practices
Test units (parts) separately - use implementations - apply discipline - gain coverage
Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.
Learning Goals of This Chapter
• Understand meaning of unit testing
• Distinguish black box vs. white box testing
• Attain proper test coverage
• Learn a testing standard
• Inspect a unit test plan
Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.
1. Introduction to unit testing
Golden Rules of Testing
Goal of testing: maximize the number and severity of
defects found per dollar spent … thus: test early
Limits of testing: Testing can only determine the
presence of defects, never their absence
– use proofs of correctness to establish “absence”
Who should test: Someone other than the developer.
– Why?
Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.
Testing: the Big Picture
Methods
Combinations of methods in class
Packages of classes
OO:
Include use-cases
Function
Module
Module combination
2. Integration tests
3.
System tests
1.Unittests
Elaboration
Unified Process
Inception Construction Transition
Requirements
Analysis
Design
Implemen-tation
Test
Jacobson et al: USDP
Prelim.iterations
Iter.#1
Iter.#n
Iter.#n+1
Iter.#m
Iter.#m+1
Iter.#k
….. …..
Unit Tests Integration tests ... System tests
RoadMap for Unit Testing
1. Plan for unit testing
Requirements
Unit test plan
2. Design test cases and acquire test I/O pairs
Generate I/O pairs (often
products of prior testing)
3. Execute unit test
Test set
Test
results
Code
under test
Detailed design
Identify largest
trouble spots
IEEE, 1986
2. Test types
Black-, Gray-, & White-box Testing
Black box… requirements
Actual outputcompared
with required output
White box
Gray box… requirements &key design elements
Input determinedby...
Result
…designelements
Confirmationof expected
behavior
As for black- and white box
testing
Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.
Black-box testing
Ie
Input test data
OeOutput test results
System
Inputs causinganomalousbehaviour
Outputs which revealthe presence ofdefects
Equivalence Partitioning
• Input data and output results often fall into different classes where all members of a class are related
• Each of these classes is an equivalence partition where the program behaves in an equivalent way for each class member
• Test cases should be chosen from each partition
Equivalence Partitioning
System
Outputs
Invalid inputs Valid inputs
Test Input Possibilities
interest rate0%
25%
principal
$100 $100M
inflation
estimate
1%
20%Infinitely many legal values:
choose a finite sample.
Infinitely many illegal values:
choose a finite sample.
Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.
Test Input Partitioning and Boundaries
interest rate0%
25%
principal
$100 $100M
inflation
estimate
Boundaries
1%
20%
Equivalence partitions
An illegal region
Range of valid inputs
Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.
Testing Ranges: Elementary Cases
1. within range
2. at the boundaries of the range
3. outside the range (“illegal”)
range
Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.
• Requirement: The system must accept a 5-digit integer between 10000 and 99999 and perform various functions on the values based on the following equivalence partitions:
<10000, 10000 - 99999 and >= 100000
• Which test cases should be chosen?
Consider the boundaries of these sets …
00000, 09999, 10000, 99999, 100000, <max bin>
Equivalence Partitioning – Example
Equivalence Partitions
Between 10000 and 99999Less than 10000 More than 99999
999910000 50000
10000099999
Input values
White Box Testing
• Every statement of code should be covered by at least one test
• However, this is not sufficient since the correct opeation of a unit of code depends upon sequences of statements
Covering Every Statement is Not Sufficient (Myers)
u>1 andv==0
x = x/u
u==2 orx>0
++x
No
Yes
No
Yes
Required program
Covering Every Statement is Not Sufficient (Myers)
u>1 andv==0
x = x/u
u==2 orx>0
++x
No
Yes
Code attempt to implement flowchart
if( (u>1) && (v==0) ) (1)x = x/u; (2)
if( (u==2) || (x>3) ) (3)++x; (4)
u=2, v=0 and x=3 • executes every line (1) - (4) • gives the correct output x= 2.5 However, line (3) is wrongSO THIS IS NOT A THOROUGH TEST
No
Yes
Required program
Paths to be Checked
Parameter & settings make
sense?
Parameter name too
long?
N
YN
Set _name to “defaultName"
Y
Truncate name
Set _name to parameter
Paths to be Checked
Parameter & settings make
sense?
Parameter name too
long?
N
YN
Decision Coverage
Set _name to “defaultName"
Y
Truncate name
Set _name to parameter
Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.
Decision Coverage via Path testing
• The objective of path testing is to ensure that the set of test cases is such that each path through the program is executed at least once
• The starting point for path testing is a program flow graph that shows nodes representing program decisions and arcs representing the flow of control
• Statements with conditions are therefore nodes in the flow graph
• Describes the program control flow. Each branch is shown as a separate path and loops are shown by arrows looping back to the loop condition node
• Used as a basis for computing the cyclomatic complexity
• Cyclomatic complexity = Number of edges - Number of nodes +2
Program flow graphs
• The number of tests to test all control statements equals the cyclomatic complexity
• Cyclomatic complexity equals number of conditions in a program
• Useful if used with care. Does not imply adequacy of testing.
• Although all paths are executed, all combinations of paths are not executed
Cyclomatic complexity
Binary search (Java)
class BinSearch {
// This is an encapsulation of a binary search function that takes an array of// ordered objects and a key and returns an object with 2 attributes namely// index - the value of the array index// found - a boolean indicating whether or not the key is in the array// An object is returned because it is not possible in Java to pass basic types by// reference to a function and so return two values// the key is -1 if the element is not found
public static void search ( int key, int [] elemArray, Result r ){
int bottom = 0 ;int top = elemArray.length - 1 ;int mid ;r.found = false ; r.index = -1 ;while ( bottom <= top ){
mid = (top + bottom) / 2 ;if (elemArray [mid] == key){
r.index = mid ;r.found = true ;return ;
} // if partelse{
if (elemArray [mid] < key)bottom = mid + 1 ;
elsetop = mid - 1 ;
}} //while loop
} // search} //BinSearch
Binary search flow graph
1
2
3
4
65
7
while bottom <= top
if (elemArray [mid] == key
(if (elemArray [mid]< key8
9
bottom > top
• 1, 2, 3, 8, 9
• 1, 2, 3, 4, 6, 7, 2
• 1, 2, 3, 4, 5, 7, 2
• 1, 2, 3, 4, 6, 7, 2, 8, 9
• Test cases should be derived so that all of these paths are executed
• A dynamic program analyser may be used to check that paths have been executed
Independent paths
Grey Box Testing
• Combination of Black and White box testing
Gray box… requirements &key design elements
Correct output and
expected behaviour
• Pre-conditions satisfied, key element in array• Pre-conditions satisfied, key element not in
array• Pre-conditions unsatisfied, key element in array• Pre-conditions unsatisfied, key element not in
array• Input array has a single value• Input array has an even number of values• Input array has an odd number of values
Grey Box Testing ExampleBinary search - equiv. partitions
Binary search equiv. partitions
Mid-point
Elements < Mid Elements > Mid
Equivalence class boundaries
Binary search - test cases
3. Planning unit tests
Plan for Unit Testing1. Decide on the philosophy for unit testing
– individual engineer responsible (common)? – reviewed by others?– designed & performed by others?
2. Decide what / where / how to document– individual’s personal document set (common)?– how / when to incorporate into other types of testing? – incorporate in formal documents?– use tools / test utilities?
3. Determine extent of unit testing (i.e., in advance).– do not just “test until time expires”– prioritize, so that important tests definitely performed
4. Decide how and where to get the test input5. Estimate the resources required
– use historical data if available6. Arrange to track time, defect count, type & source
Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.
4. Checklists and examples for Method testing
Perform Method Testing (Humphrey) 1/2
1. Verify operation at normal parameter values (a black box test based on the unit’s requirements)
2. Verify operation at limit parameter values(black box)
3. Verify operation outside parameter values(black box)
4. Ensure that all instructions execute (statement coverage)
5. Check all paths, including both sides of all branches (decision coverage)
6. Check the use of all called objects7. Verify the handling of all data structures8. Verify the handling of all files
Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.
Perform Method Testing (Humphrey) 2/2
9. Check normal termination of all loops
(part of a correctness proof)
10. Check abnormal termination of all loops
11. Check normal termination of all recursions
12. Check abnormal termination of all recursions
13. Verify the handling of all error conditions
14. Check timing and synchronization
15. Verify all hardware dependencies
Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.
Method Test Case Examples
• See pages 409-414 and Case Study at end of chapter
• Each Example layouts Input, Execution and Expected Output
• Test code is developed to exercise the methods and output the specifics of the tests (as above) and the actual results to a file for analysis
5. Checklists and examples for class testing
1. Exercise methods in combination– 2-5, usually– choose most common sequences first– include sequences likely to cause defects– requires hand-computing the resulting attribute values
2. Focus unit tests on each attribute– initialize, then execute method sequences that affect it
3. Verify that each class invariant is unchanged– verify that the invariant is true with initial values– execute a sequence (e.g., the same as in 1.)– verify that the invariant still true
4. Verify that objects transition among expected states– plan the state / transition event sequence– set up the object in the initial state by setting variables– provide first event & check that transition occurred . etc.
Perform ClassUnit Tests
Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.
Encounter State-Transition Test Sequence 1 of 2
Waiting
Preparingtest step 1
Verify that the game is initially in Preparing state (by checking on the class membership of gameStateI)
Player dismisses qualities
menu
Encounter State-Transition Test Sequence 1 of 2
Waiting
Preparingtest step 1
test step 2
test step 3
Verify that the game is initially in Preparing state (by checking on the class membership of gameStateI)
Dismiss the quality menu, and verify that the game is in Waiting state.
Move the player character to an adjacent area, and verify that the game is still in Waiting state.
Player dismisses qualities
menu
Move to adjacent
area
Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.
Player dismisses qualities
menu
Character enters area
inhabited by an opponent
Move to adjacent
area
Complete Encounter State-Transition Test
Waiting
Preparing
Engaging
12
3
4
5
ReportingPlayer
dismisses encounter
report menu
Encounter completed
6
Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.
• Defect testing and debugging are distinct processes
• Inspection and testing is concerned with establishing the existence of defects in a program
• Debugging is concerned with locating and repairing these errors
• Debugging involves formulating a hypothesis about program behaviour then testing these hypotheses to find the error
Testing and debugging
The debugging process
Locateerror
Designerror repair
Repairerror
Re-testprogram
Testresults Specification Test
cases
6. Summary
Unit Testing: Summary
• Unit testing is about “pieces”
• Other testing is about “assemblies”
• Black box: input / output only
• White box: verifies processing
– Several ways , Ensure completeness
• Grey box: a bit of Black and White
• Test planning - earlier the better
– helps clarify requirementsAdapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.
Case StudyEncounterCharacter.java
/** To test this class.* @param argsP destination of method test log, class test log respectively*/public static void main( String[] argsP ) {
// Default files on which to write test output & run tests String methodOutputFileNameM = "methodOutput.txt";String classOutputFileNameM= "classOutput.txt";
if( argsP != null && argsP.length == 2 ) // use defaults if input improper { methodOutputFileNameM = argsP[0];
classOutputFileNameM = argsP[1];}
// 1. EXECUTE TESTS WHICH DO NOT REQUIRE HUMAN INTERVENTION
// Test methods individually, then test class try { testEncounterCharacterMethods( methodOutputFileNameM );
testEncounterCharacterClass( classOutputFileNameM );} catch( IOException eP ) { System.out.println( eP ); }
Listing page 1
Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.
// 2. EXECUTE TESTS WHICH DO REQUIRE HUMAN INTERVENTION
Frame[] imageTests = { // Display test cases. new testCharacterImage( // Missing image. new EncounterCharacter( "GuyWithNoImage", null ) ), new testCharacterImage( // Image is present. new EncounterCharacter( "Elena", "elena.gif" ) )};
for( int i = 0; i < imageTests.length; i++) { // Display each test window. imageTests[i].setSize(400, 250); // Adequate size for character. imageTests[i].setVisible(true); imageTests[i].show();}
try { // Let user examine windows. Thread.currentThread().sleep( 30*1000 ); } catch( Exception exc ) { }
for( int i = 0; i < imageTests.length; i++ ) // Shut the windows. imageTests[i].dispose();
System.exit( 0 );}
Listing page 2Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.
/** Tests this class by executing its methods in combination. * @param destinationP Location to write results. * @exception IOException If there’s a problem opening or accessing destinationP*/public static void testEncounterCharacterClass( String destinationP ) throws IOException { /* Prepare for the test */
PrintWriter outM = new PrintWriter( new FileOutputStream( destinationP ) );System.out.println(
"\nEncounterCharacter class test results on " + destinationP + "\n" );
/* * The following methods will be tested in sequences: * * a. adjustQuality( String qualityP, float qualityValueP ) * d. deleteFromEncounterCharacters( EncounterCharacter encounterCharacterP ) * ge. EncounterCharacter getEncounterCharacter( String nameP ) * gq. float getQualityValue( String qualityP ) * gt. float getTolerance() * io. int indexOf( String qualityP ) * ii. insertIntoEncounterCharacters( EncounterCharacter encounterCharacterP ) * m. int maxNumCharsInName() * sq. setQuality( String qualityP, float qualityValueP ) * so. float sumOfQualities()
Listing page 3
Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.
* * The following sequences occur commonly: * ge-aq-so * ge-sq-a-gq * . . . . . * The following sequences have a high potential for defects: * ge-aq-aq-gq-so * . . . . . */
/* Test C1: ge-aq-so */EncounterCharacter eC1M = new EncounterCharacter( "CharForTestC1" ); // method “ge”eC1M.adjustQuality(QUAL_STRENGTH, 40.0f ); // aqTestExecution.printReportToFile( outM,
"Class test ge-aq-so", eC1M.sumOfQualities(), 100.0f ); // so
/* Test C2: ge-aq-aq-gq-so */EncounterCharacter eC2M = new EncounterCharacter( "CharForTestC2" ); // geeC2M.adjustQuality(QUAL_STRENGTH, 40.0f ); // aqeC2M.adjustQuality(QUAL_STAMINA, 20.9876f ); // aq
Listing page 4
Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.
TestExecution.printReportToFile( outM, "Class test ge-aq-aq-gq-so: part 1",eC2M.getQualityValue( QUAL_STAMINA ), 20.9876f ); // gq
TestExecution.printReportToFile( outM, "Class test ge-aq-aq-gq-so: part 2", eC2M.sumOfQualities(), 100.0f ); // so
/* INVARIANT-ORIENTED TESTS* Check for the invariant “qualValueI[i] >=0”* -- after executing the sequences of methods executed above*/boolean truthM = true;for( int i = 0; i < qualityTypeS.length; ++i ) { /* Set truthM false if any entry in eC1M.qualValueI not >= 0 */
truthM = truthM && ( eC1M.qualValueI[i] >= 0.0f );}TestExecution.printReportToFile( outM,
"Class test for the invariant 'qualValueI[i] >=0'", truthM, true );
/* Conclude */outM.close();System.out.println( "\nClass tests of EncounterChar class concluded." );
} // end of testEncounterCharacterClass
Listing page 5
Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.
/** Tests all the methods of this class one at a time* @param destinationP Location to write results. * @exception IOException If there’s a problem opening or accessing destinationP*/public static void testEncounterCharacterMethods( String destinationP ) throws IOException { /* Prepare for the test */
FileWriter outM = new FileWriter( new File( destinationP ) );System.out.println( "EncounterCharacter method test results on " + destinationP + "\n" );
/* Tests for getEncounterCharacter() */
EncounterCharacter eCNorM = new EncounterCharacter( "qwerty" ); // normal TestExecution.reportToFile( outM,
"GetCharacter Test 1: nominal value", eCNorM.getName(), "qwerty" );
EncounterCharacter eCNullM = new EncounterCharacter( null ); // null TestExecution.reportToFile( outM, "GetCharacter Test 2: null parameter",
eCNullM.getName(), GameCharacter. DEFAULT_NAME);
Listing page 6
Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.
String tooLongM = "1234567890123456789012345678901234567890"; EncounterCharacter eCTooLongM = new EncounterCharacter(tooLongM); // too long TestExecution.reportToFile( outM, "GetCharacter Test 3: Limit parameter values, "
+ "max name len = " + eCTooLongM .maxNumCharsInName(), eCTooLongM.getName(),tooLongM.substring(0, eCTooLongM.maxNumCharsInName()) );
EncounterCharacter eCZeroM = new EncounterCharacter( "" ); // zero-len TestExecution.reportToFile( outM, "GetCharacter Test 4: zero-length",
eCZeroM .getName(), GameCharacter. DEFAULT_NAME);
EncounterCharacter eCPuncM = new EncounterCharacter( "a+b" ); // bad chars TestExecution.reportToFile( outM, "GetCharacter Test 5: bad char '+' ",
eCPuncM .getName(), GameCharacter. DEFAULT_NAME);
Listing page 7
Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.
/* Tests for indexOf() for every valid quality name. */for( int i = 0; i < qualityTypeS.length; ++i )
try { TestExecution.reportToFile( outM, "indexOf() Test 1." + i + ": valid name: " + qualityTypeS[i], indexOf(qualityTypeS[i]), i );
} catch( Exception eP ) { TestExecution.reportToFile( outM, "indexOf() Test 1: valid name: compare ",
"indexOf('" + qualityTypeS[i] + "')", "with expected " + i ); }
/* Tests for indexOf() for an invalid quality name. */try { TestExecution.reportToFile( outM,
"indexOf() Test 2: invalid name: zorch", indexOf("zorch"), -1 );} catch( Exception eP ) { TestExecution.reportToFile( outM,
"indexOf() Test 2: valid name: compare ", "indexOf(\"zorch\")", "with expected -1" ); }
Listing page 8Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.
/* Tests for setQuality() */
// Set up for test EncounterCharacter hank = new EncounterCharacter( "Hank" );
// Nominal valuehank.setQuality(QUAL_STRENGTH , 10.3f );TestExecution.reportToFile( outM, "setQuality() Test 1: nominal value",
hank.getQualityValue( QUAL_STRENGTH ), 10.3f );
// Out of range valuehank.setQuality( QUAL_PATIENCE, -6.2f ); TestExecution.reportToFile( outM, "setQuality() Test 2: nominal value",
hank.getQualityValue(QUAL_PATIENCE ), 0.0f );
// Value below close-to-zero threshold.hank.setQuality( QUAL_STAMINA, getTolerance() * 0.9f ); TestExecution.reportToFile( outM, "setQuality() Test 3: value close to zero",
hank.getQualityValue(QUAL_STAMINA), 0.0f );
Listing page 9
Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.
// Tests for adjustQuality().
// Set up for test and verify: Values should be 20 each.EncounterCharacter harvey = new EncounterCharacter( "Harvey" ); TestExecution.reportToFile( outM, "adjustQuality() test 0: verify that values add to 100", harvey.sumOfQualities(), 100.0f );
// Nominal adjustmentharvey.adjustQuality(QUAL_STRENGTH , 30.0f ); // strength 30 rest 70/4
eachTestExecution.reportToFile( outM, "adjustQuality() test 1: values sum to 100 after adjusting", harvey.sumOfQualities(), 100.0f );TestExecution.reportToFile ( outM, "adjustQuality() test 2: values adjusted as commanded", harvey.getQualityValue(QUAL_STRENGTH ), 30.0f );
// Adjustment resulting in a zero valueharvey.adjustQuality( QUAL_STAMINA, 99.0f );TestExecution.reportToFile( outM, "adjustQuality() test 3: verify low value reverts to zero", harvey.getQualityValue( QUAL_STRENGTH ), 0.0f );
// Conclude outM.close();System.out.println( "\nMethod tests of EncounterCharacter class concluded." );
}
Listingpage 10
Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.
/** Class to test repainting of characters. Creates a window, which will contain* several copies of the character image.*/private static class testCharacterImage extends Frame{
/** Instance attribute that remembers which character image to display. */private EncounterCharacter characterI; /** Basic constructor -- create a window for testing some character's image.* @param characterP Character whose image is to be tested.*/testCharacterImage(EncounterCharacter characterP){
super(characterP.getName()); // Do all normal Frame initialization. characterI = characterP; // Remember which character we're testing. }
Listing page 11
Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 20001), with permission.Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 20001), with permission.Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 20001), with permission.Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 20001), with permission.Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 20001), with permission.Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.
/** Repaint the display areaof the frame.* @param drawP Graphics context for drawing the character.*/public void paint(Graphics drawP){ Dimension frameSizeM = getSize(); // Size of the window area.
int widthUnitM = frameSizeM.width / 5; // Convenient divisions of window. int heightUnitM = frameSizeM.height / 5; characterI.showCharacter(this, drawP, // Drawn small, facing right.
new Point(widthUnitM, heightUnitM), heightUnitM, false); characterI.showCharacter(this, drawP, // Drawn large, facing left.
new Point(widthUnitM*4, heightUnitM*3), heightUnitM*2, true); characterI.showCharacter(this, drawP, // Drawn large, facing right.
new Point(widthUnitM*2, heightUnitM*2), heightUnitM*2, false); characterI.showCharacter(this, drawP, // Drawn small, facing left.
new Point(widthUnitM*3, heightUnitM*4), heightUnitM, true); }
} // End of testCharacterImage inner class
Listing page 12
Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.