Defect testing

Software EngineeringCOMP 201Lecturer: Sebastian CoopeAshton Building, Room G.18E-mail: coopes@liverpool.ac.uk

COMP 201 web-page:http://www.csc.liv.ac.uk/~coopes/comp201

Software Testing

1COMP201 - Software Engineering1Cyclomatic ComplexityThe number of tests to test all control statements equals the cyclomatic complexityCyclomatic complexity equals number of conditions in a program plus one (or equivalently, in the program flow graph it is the Number of edges - Number of nodes +2)Conditions are any type of branching operation such as each if statement or any types of loop (for, while etc.)Useful if used with care. Does not imply adequacy of testing. Although all paths are executed, all combinations of paths are not executed2COMP201 - Software EngineeringBinary search (Java)

3COMP201 - Software EngineeringBinary search flow graph

4COMP201 - Software EngineeringQuestion: What is the Cyclomatic Complexity for this program?Independent Paths1, 2, 3, 8, 91, 2, 3, 4, 6, 7, 21, 2, 3, 4, 5, 7, 21, 2, 3, 4, 6, 7, 2, 8, 9Test cases should be derived so that all of these paths are executedA dynamic program analyser may be used to check that paths have been executed5COMP201 - Software EngineeringIntegration TestingIntegration testing - tests complete systems or subsystems composed of integrated componentsIntegration testing should be black-box testing with tests derived from the specificationMain difficulty is localising errorsIncremental integration testing reduces this problem6COMP201 - Software EngineeringIncremental Integration Testing

7COMP201 - Software EngineeringIncremental Integration Testing8COMP201 - Software Engineering Note that incremental integration as on the previous slide uses the idea of regression testing, i.e., future tests also test previous test cases again. As a new module is added, we not only run a new test, we also make sure the addition of the new module does not break the previous test cases. This can sometimes by done automatically by using a test harness (a program written to automatically generate test data and record their results). [See Junit for Java if you are interested in this.]Approaches to Integration TestingTop-down testingStart with high-level system and integrate from the top-down replacing individual components by stubs where appropriateBottom-up testingIntegrate individual components in levels until the complete system is createdIn practice, most integration involves a combination of these strategies9COMP201 - Software EngineeringTop-down Testing

10COMP201 - Software EngineeringBottom-up Testing

11COMP201 - Software EngineeringFor which types of system is bottom-up testing appropriate, and why?Object-oriented systems because these have a neat decomposition into classes and methods makes testing easyreal-time systems because we can identify slow bits of code more quicklysystems with strict performance requirements because we can measure the performance of individual methods early in the testing process12COMP201 - Software EngineeringTesting ApproachesArchitectural validationTop-down integration testing is better at discovering errors in the system architectureSystem demonstrationTop-down integration testing allows a limited demonstration at an early stage in the developmentTest implementationOften easier with bottom-up integration testingTest observationProblems with both approaches. Extra code may be required to observe tests13COMP201 - Software EngineeringInterface TestingTakes place when modules or sub-systems are integrated to create larger systemsObjectives are to detect faults due to interface errors or invalid assumptions about interfacesParticularly important for object-oriented development as objects are defined by their interfaces14COMP201 - Software EngineeringInterface Testing

15COMP201 - Software EngineeringInterfaces TypesParameter interfacesData passed from one procedure to anotherShared memory interfacesBlock of memory is shared between proceduresProcedural interfacesSub-system encapsulates a set of procedures to be called by other sub-systemsMessage passing interfacesSub-systems request services from other sub-systems16COMP201 - Software EngineeringInterface ErrorsInterface misuseA calling component calls another component and makes an error in its use of its interface e.g. parameters in the wrong orderInterface misunderstandingA calling component embeds assumptions about the behaviour of the called component which are incorrectTiming errorsThe called and the calling component operate at different speeds and out-of-date information is accessed17COMP201 - Software EngineeringInterface Testing GuidelinesDesign tests so that parameters to a called procedure are at the extreme ends of their rangesAlways test pointer parameters with null pointersDesign tests which cause the component to failUse stress testing in message passing systemsIn shared memory systems, vary the order in which components are activated18COMP201 - Software EngineeringStress TestingExercises the system beyond its maximum design load. Stressing the system often causes defects to come to lightStressing the system test failure behaviour.. Systems should not fail catastrophically. Stress testing checks for unacceptable loss of service or dataParticularly relevant to distributed systems which can exhibit severe degradation as a network becomes overloaded19COMP201 - Software EngineeringObject-Oriented TestingThe components to be tested are object classes that are instantiated as objectsLarger grain than individual functions so approaches to white-box testing have to be extendedNo obvious top to the system for top-down integration and testing20COMP201 - Software EngineeringTesting LevelsTesting operations associated with objectsTesting object classesTesting clusters of cooperating objectsTesting the complete OO system21COMP201 - Software EngineeringObject Class TestingComplete test coverage of a class involvesTesting all operations associated with an objectSetting and interrogating all object attributesExercising the object in all possible statesInheritance makes it more difficult to design object class tests as the information to be tested is not localised22COMP201 - Software EngineeringWeather Station Object InterfaceTest cases are needed for all operationsUse a state model to identify state transitions for testingExamples of testing sequencesShutdown Waiting ShutdownWaiting Calibrating Testing Transmitting WaitingWaiting Collecting Waiting Summarising Transmitting Waiting

23COMP201 - Software EngineeringObject IntegrationLevels of integration are less distinct in object-oriented systemsCluster testing is concerned with integrating and testing clusters of cooperating objectsIdentify clusters using knowledge of the operation of objects and the system features that are implemented by these clusters

24COMP201 - Software EngineeringApproaches to Cluster TestingUse-case or scenario testingTesting is based on a user interactions with the systemHas the advantage that it tests system features as experienced by usersThread testingTests the systems response to events as processing threads through the systemObject interaction testingTests sequences of object interactions that stop when an object operation does not call on services from another object25COMP201 - Software EngineeringScenario-Based TestingIdentify scenarios from use-cases and supplement these with interaction diagrams that show the objects involved in the scenarioConsider the scenario in the weather station system where a report is generated26COMP201 - Software EngineeringCollect Weather Data

27COMP201 - Software EngineeringWeather Station TestingThread of methods executedCommsController:request WeatherStation:report WeatherData:summariseInputs and outputsInput of report request with associated acknowledge and a final output of a reportCan be tested by creating raw data and ensuring that it is summarised properlyUse the same raw data to test the WeatherData object

28COMP201 - Software EngineeringLecture Key PointsTest parts of a system which are commonly used rather than those which are rarely executedEquivalence partitions are sets of test cases where the program should behave in an equivalent wayBlack-box testing is based on the system specificationStructural testing identifies test cases which cause all paths through the program to be executed29COMP201 - Software EngineeringLecture Key PointsTest coverage measures ensure that all statements have been executed at least once. Interface defects arise because of specification misreading, misunderstanding, errors or invalid timing assumptionsTo test object classes, test all operations, attributes and statesIntegrate object-oriented systems around clusters of objects30COMP201 - Software Engineeringclass BinSearch {

// This is an encapsulation of a binary search function that takes an array of// ordered objects and a key and returns an object with 2 attributes namely // index - the value of the array index // found - a boolean indicating whether or not the key is in the array

// An object is returned because it is not possible in Java to pass basic types by // reference to a function and so return two values

// the key is -1 if the element is not found

public static void search ( int key, int [] elemArray, Result r )

{

int bottom = 0 ;

int top = elemArray.length - 1 ;

int mid ;

r.found = false ; r.index = -1 ;

while ( bottom