Integration Testing - Swanseacsmarkus/CS339/dissertations/PauloRA.pdf · 2007-01-12 · As...

25
Integration Testing Rog´ erio Paulo January 12, 2007

Transcript of Integration Testing - Swanseacsmarkus/CS339/dissertations/PauloRA.pdf · 2007-01-12 · As...

Integration Testing

Rogerio Paulo

January 12, 2007

Abstract

This document is written as a dissertation to complete the module “CS339 Advanced Topics in Computer Science:Testing” in University of Wales Swansea. The referencing materials are mainly based on the book“Software TestingA Craftman’s Approach”by Paul C. Jorgensen, also referred as [Jor02].

Nowadays testing plays a major role in the software development cycle. Generally, software development firmsinvest a considerable amount of resources in testing to makesure the software developed meets the specification andstandard for the user or customer. Since problems that ariseafter the software release can cause the company to loserevenue and reputation, or worse, it can cause environmental damage, injuries and deaths.

In this document, we will discuss one of the testing phase, integration testing, and its different strategies. Namelydecomposition-based integration, call-graph based integration and path-based integration.

We will see that for each strategy there are different approaches to achieve the same main concept but with differentrequirements and consequences. For each approach there will be a case study where a closer to real-life example willbe shown and discussed.

Finally, at the end of this document there will be a conclusion based on my research and personal experiences.

Table of Contents

Table of Contents 1

Introduction 2

1 Decomposition-based Integration 71.1 Big bang Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . 71.2 Top-down Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . 91.3 Bottom-up Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . 111.4 Sandwich Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . 12

2 Call Graph-based Integration 152.1 Pair-wise Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . 152.2 Neighbourhood Integration . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . 17

3 Path-based Integration 193.1 MM-Path based Integration . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . 19

Conclusion 22

Bibliography 23

1

Introduction

In the classic software development model, V-Model (Figure1), integration testing is a software testing phase locatedbefore system testing and after unit testing. Integration testing has the goal of proving that the features developed worktogether well enough for the project to be submitted for system testing.

In this model, unit testing is performed mainly by programmers and system testing is performed mainly by testers;integration testing offers the opportunity for the programmers (people with high knowledge on the code) to work withthe testers (people with high technical expertise in testing), bridging unit testing and system testing thus allowing thetransition to be smoother. Generally, the larger and more complex the project is, the more important is the integrationtesting.

As integration testing is performed after unit testing, we will assume that all units have been tested separately. Themethod of grouping the modules of the system is determined bythe strategy and the approach. It is also important tonote that these strategies and approaches are targetted to traditional procedural programming languages.

Different strategies are needed because there are different requirements and resource allowance. For instance, invery small systems grouping all the modules and testing themin a single phase may be acceptable, however, for largersystems this is impractical. The reasons and solutions for these problems will be discussed in the later chapters.

We will first introduce the example that will be using during this document, theNextDateprogram. Then we willdiscuss the different approaches of integration testing and how it is applied to our example:

• Chapter 1, we will talk about decomposition-based integration. We will see the four main approaches in thisstrategy: big bang, top-down, bottom-up and sandwich; the advantages and disadvantages of them and examplesof their application will be given.

• Chapter 2, we will examine call graph-based integration and its general improvement made towards decomposition-based integration. Namely the approaches known as pair-wise and neighbourhood. Again, their strong and weakpoints will be discussed with concrete examples shown.

• Chapter 3, we will discuss about path-based integration. We will talkmainly about an approach known asMM-path based integration, its advantages and disadvantages. A concrete example will be used to explain howit is applied.

U nit D esign & Im p lem enta tion

U n it Testing

M odu le Testing

M odu le D esign

D esign Testing

A rch itectu re D esign

R equ irem ents Testing

R equ irem ents A na lys is

Integration Testing

Module Integration

S ystem Testing

S ystem In tegra tion

A cceptance Testing

S ystem D eve lopm ent

Figure 1: V-Model, a classic software development model.

2

The NextDate Program

This example is taken directly from [Jor02], the way the pseudo-code is shown has been modified. However, the mainfunctional and structural integrity of the program is kept.This program uses three variables: month, date and year.With the input, it returns the next date of the inputted date.It has the following characteristics:

• Checks for valid input:

– Must be positive integers

– A month must be: 1≤ month≤ 12

– A day must be: 1≤ day≤ 31

– A year must be: 1812≤ 2012

• Takes leap years into consideration:

– A year is leap year if is divisible by 4

– Unless it is a century year, then it is only leap if is multipleof 400

Here is the pseudo-code of the NextDate program implementation:

input : A dateoutput : Next date of the day inputted

Type Date: Integer month,Integer day,Integer yearDate : today,tomorrow,aDate

Main integrationNextDate1

getDate(today); /* message 1 */2

printDate(today); /* message 2 */3

tomorrow= incrementDate(today); /* message 3 */4

printDate(tomorrow); /* message 4 */5

End Main6

Function(Main) integrationNextDate(date)

We want to keep the main part simple, thus we just call functions from other parts. Here we get the date from theuser input withgetDate, we print it withprintDatethen we calculate the next date withincrementDate, finally, printingthe results withprintDateagain.

Also note thatDate is a new datatype that contains three other integer variables: month, day and year. We alsodeclaredtoday, tomorrow and aDateas variables of the typeDate.

input : A yearoutput: True if is a leap year, otherwise false

Function Boolean isLeap(year)7

if year divisible by 4then8

if year is NOT divisible by 100then isLeap= True9

else ifyear is divisible by 400then isLeap= True10

else isLeap= False11

else isLeap= False12

End (Function isLeap)13

Function isLeap(year)

3

This functionisLeapchecks if the year from the input is a leap year or not. We first check if the year is divisibleby 4, if it is, then there is possibility that it is a leap year,otherwise it is not. In case it is divisible by 4, we continue tocheck if the year is a century year or not. If it is not, then theyear is a leap year. Else, we proceed to check the nextcondition, if the year is a multiple of 400. If it is then it is aleap year, else, it is not.

input : A month and a yearoutput: The last day of the month in integer

Function Integer lastDayOfMonth(month,year)14

switch monthdo15

case1,3,5,7,8,10,12:lastDayOfMonth= 3116

case4,6,9,11: lastDayOfMonth= 3017

case2:18

if isLeap(year)then lastDayOfMonth= 29; /* message 5 */19

else lastDayOfMonth= 2820

endsw21

endsw22

End (Function lastDayOfMonth)23

Function lastDayOfMonth(month,year)

In lastDayOfMonthwe calculate the upperbound of the number of days that exist in a certain month. This is neededbecause there are months with 30 and 31 days; with February a special case where it has 28 days in a standard yearand 29 days in a leap year.

This function is handled mainly by switch-case statements.For the first case, we are saying that the months:January, March, May, July, August, October and Decemberhave 31 days. For the second case, the months:April,June, September and Novemberhave 30 days. Finally, forFebruarywe are saying that it has 29 days if it is a leapyear, otherwise it has only 28 days.

input : A dateoutput: True if the date is valid, otherwise false

Function Boolean validDate(aDate)24

if (aDate.Month> 0)∧ (aDate.Month<= 12) then monthOK= True25

else monthOK= False26

if monthOKthen27

if (aDate.Day> 0)∧ (aDate.Day<= lastDayOfMonth(aDate.Month,aDate.Year))then dayOK=28

True; /* message 6 */

else dayOK= False29

endif30

if (aDate.Year> 1811)∧ (aDate.Year<= 2012)then yearOK= True31

else yearOK= False32

if monthOK∧ dayOK∧ yearOKthen validDate= True33

else validDate= False34

End (Function validDate)35

Function validDate(aDate)

Since there are certain conditions that must be met for a dateto be valid, for our program we should have a part tomake sure these conditions are met. Especially if the date isfrom user input.

In validDatewe will be checking these conditions separately:

Month - We return true if the value is between 1 and 12 (inclusive), else, we return false.

4

Day - We return true if the value is between 1 and the maximum day ofthe month (inclusive); the maximum day ofthe month is retrieved by usinglastDayOfMonth. If this condition is not met, then we return false.

Year - Here we will be placing an artificial boundry to our program.We are just allowing years between the year of1812 and 2012 (inclusive), even if there are other values that are also considered valid. Again, we return true ifthis condition is met, else, false is returned.

After checking all three variables (month,day and year), wewill just consider that the date is valid if all threeconditions are met (all of them returned true), otherwise wewill be rejecting the date.

input : A set of integers from user inputoutput: A date

Function Date getDate(aDate)36

repeat37

Output(“Enter a month:”)38

Input(aDate.Month)39

Output(“Enter a day:”)40

Input(aDate.Day)41

Output(“Enter a year:”)42

Input(aDate.Year)43

getDate.Month= aDate.Month44

getDate.Day= aDate.Day45

getDate.Year= aDate.Year46

until validDate(aDate); /* message 7 */47

End (Function getDate)48

Function getDate(aDate)

By usinggetDatewe will be able to retrieve and store the date from the user input. Here we request the inputs byprinting to the screen the desired variable usingOutput. After each request, we retrieve the value inputted by the userto the corresponding variable usingInput. Finally, we stored the retrieved values to the corresponding variable. Thesestatements are inside the repeat block, thus if the user inputs an invalid date, the program will keep requesting a newdate until a valid one is inputted.

input : A dateoutput: The date incremented

Function Date incrementDate(aDate)49

if aDate.Day< lastDayOfMonth(aDate.Month)then aDate.Day= aDate.Day+ 1; /* message 8 */50

else51

aDate.Day= 152

if aDate.Month= 12 then53

aDate.Month= 154

aDate.Year= aDate.Year+ 155

else aDate.Month= aDate.Month+ 156

endif57

End (Function incrementDate)58

Function incrementDate(aDate)

With incrementDatewe implement the main feature of this program. With a date as an input, we output a date thatcorresponds to the following date of the input. To achieve that we first check the day of the date inputted, if it is the

5

last day of the month usinglastDayOfMonth. If it is not, then we can simply increase the day by one. If it is then wewill need to change more variables.

First, we set the day to 1; this is because the successor of thelast day of a month is the first day of the next month.Then we must check if the month is the last month of the year, which is 12 (December). If it is, then we must set themonth as 1 (January) and increment the year, which corresponds to the first day of a new year. Else, we can simplyincrement the value of month by 1.

input : A dateoutput: The date in string

Procedure String printDate(aDate)59

Output(“Day is ”,aDate.Month,“/”,aDate.Day,“/”,aDate.Year)60

End (Procedure printDate)61

ProcedureprintDate(aDate)

Here withprintDate, we simply convert the date with the datetype of date to a string. This is done so that it can beprinted on the screen and allow us to put additional strings or information.

6

Chapter 1

Decomposition-based Integration

In this strategy, we do the decomposition based on the functional characteristics of the system. A functional charac-teristic is defined by what the module does, that is, actions or activities performed by the module. In this strategy ourmain goal is to test the interfaces among separately tested units.

There are four approaches for this strategy:

• Big bang integration

• Top-down integration

• Bottom-up integration

• Sandwich integration

Briefly, big-bang groups the whole system and test it in a single test phase. Top-down starts at the root of the treeand slowly work to lower level of the tree. Bottom-up mirrorstop-down, it starts at the lower level implementation ofthe system and work towards the main. Sandwich is an approachthat combines both top-down and bottom-up.

Note that many different sources refers decomposition-based integration, ifnot the only one, the main strategy usedfor integration testing, however, this is not true. In laterchapters we will see strategies that are not decomposition-based but they still allow us to achieve a systematic test forthe integration phase.

1.1 Big bang Integration

This is one of the easiest approaches to apply in integrationtesting. Here we treat the whole system as a subsystemand test it in a single test phase. Figure 1.1 shows the modules used in a single test session.

Normally this means simply integrating all the modules, compile them all at once and then test the resulting system.This approach requires little resources to execute as we do not need to identify critical components (like interactionspaths between the modules) nor require extra coding for the “dummy modules”.

This approach may be used for very small systems, however it is still not recommended because it is not systematic.In larger systems, the low resources requirement in executing this testing is easily offset by the resources required tolocate the problem when it occurs.

While it is easy to have a SUT1 for the test cases to be applied on, it is very difficult in creating the test cases. Thisis because in a large system, the input has undergone a considerable amount of interactions and conditional checks toreach the final output. It is difficult to create test cases that has a good coverage in such complex scenario.

For big bang integration the number of integration testing sessions is only 1.

1System Under Test – a system that is being tested for correct operation.

7

Figure 1.1: Big bang integration, coverage of a test session.

1.1.1 Case Study

Here we compile all the modules in the functional decomposition tree: Main, isLeap, lastDayOfMonth, GetDate,ValidDate, IncrementDate, printDateand test the whole system in a single session (Figure 1.2).

By doing this, we will increase the difficulty of test cases creation. This is because we must create asingle set of testcases that covers conditions passed by different modules. Simultaneously, this will make the analysisof the test casesharder; when there is a test case that produces incorrect value, it will be more difficult to deduce the reasons becausethe data has been heavily processed. Consequently, this will make problem isolation a big issue in this approach.

In these situations the programmer’s experience may help locating the problem, or we can use debugging methodsto locate them. While this is viable in small systems like theone used as an example, it is not cost effective for largersystems. In fact, there are other methods that we will discuss that help resolving these problems in a more systematicway.

M ain

isLeap lastD ayO fM onth getD ateva lidD ate increm entD ate prin tD ate

Figure 1.2: Big bang integration of thenextDateprogram.

1.1.2 Summary

In summary, big bang integration has the following characteristics:

• Considers the whole system as a subsystem

• Tests all the modules in a single test session

• Only one integration testing session

And the following advantages and disadvantages:

Advantages:

• Low resources requirement

• Does not require extra coding

Disadvantages:

• Not systematic

• Hard to locate problems

• Hard to create test cases

8

1.2 Top-down Integration

In top-down integration, we start at the target node at root of the functional decomposition tree and work toward theleaves. Figure 1.3 shows different sessions of integration testing. Stubs2 are used to replace the children nodes attachedto the target node. A test phase consists in replacing one of the stub modules with the real code and test the resultingsubsystem. If no problem is encountered then we do the next test phase. If all the children were replaced by real codeat least once and meet the requirements then we move down to the next level. Now we can replace the higher leveltested modules with real code and continue the integration testing.

This testing approach finish when the whole system is coveredby using the method above.Top-down integration allows us to have a prototype of the SUTat early stages. This is a very big advantage in many

customer based software development. In many cases, companies suffer large software development cost because ofa bad design that is just identified at the later stage of the development cycle, forcing the developing team to restartat the designing phase. By having an early SUT prototype manydesign issues can be identified early thus rectified.Consequently, by using top-down integration, we will be able to interleave design and implementation.

Top-down integration has the drawback of requiring stubs. While stubs are simpler than the real code, it is notstraightforward to write them; the emulation must be complete and realistic, that is, the test cases results ran on thestub should match with the results on the real code. Being a throw-away code, it does not reach the final product norit will increase functionality of the software thus it is extra programming effort without a direct reward.

Another issue of starting at higher level is that we are forced to abstract both the application layer and the hardwarelayer. For systems where it requires strict hardware compatibility, for example, embedded systems, this will createserious issues at later stages.

Finally, top-down integration starts at the higher level ofthe software thus the expected outputs are more complex,making the creation of test cases also harder.

For top-down integration the number of integration testingsessions isnnodes− nleaves+ nedges.

Figure 1.3: Top-down integration, coverage of different sessions at different levels.

2Stub – simple throw-away codes that emulate the replaced module.

9

1.2.1 Case Study

With our example programnextDate, this integration approach is not very useful. This is mainly because this is a verysimple system (thus the functional decomposition tree doesnot have many levels) and secondly the main programdoes not have many features or complex user interface.

In top-down integration, we would be starting withMain as a target node and replace the children nodes one byone with stubs (only one stub in each test session). We must build the stub such that it returns correct values to the realmodule and compatible to the test cases. A possible stub forincrementDatecould be:

input : A dateoutput: The following day of the inputted date

Date : d31121999,d28022000,d28021999,next

Function Date incrementDate(aDate)1

d12311999= Date(12,31,1999)2

d02282000= Date(02,28,2000)3

d02281999= Date(02,28,1999)4

if aDate== d12311999then next= Date(01,01,2000)5

else ifaDate== d02282000then next= Date(02,29,2000)6

else ifaDate== d02281999then next= Date(03,01,1999)7

End (Function incrementDate)8

Function lastDayOfMonth(month,year)

In a way, the test cases will be limited by how and what we code in the stub. This also means that we are limitingthe area we must look at when problems arise. For instance, ifboth incrementDateandgetDateis replaced by stubsbut there is a mismatch of results in the output, then most likely the problem is atprintDate.

Another thing to note about this approach in this example is that there are empty test sessions. The functionsisLeap, lastDayOfMonthandvalidDateare never called byMain directly so we will not be able to isolate them withstubs with a top-down approach thus no test cases are createdin these sessions (Figure 1.4).

M ain

isLeap lastD ayO fM onth getD ateva lidD ate increm entD ate prin tD ate

N odes that w ill be rep laced by stubsN ode w ith rea l code

Figure 1.4: Top-down integration of thenextDateprogram.

1.2.2 Summary

In summary, top-down integration has the following characteristics:

• Integration starts at the main program

• Moves from the higher level modules to the lower level modules

• Hasnnodes− nleaves+ nedgesnumber of integration testing sessions

And the following advantages and disadvantages:

10

Advantages:

• Early SUT prototype

• Interleaves design and implementation

Disadvantages:

• Throw-away code programming

• Late interaction tests between the main program,the application layer and the hardware

• Difficult to create test cases

1.3 Bottom-up Integration

Bottom-up integration starts at the opposite end of the functional decomposition tree, instead of starting at the mainprogram we start at the lower-level implementation of the software. As we can see in Figure 1.5, bottom-up integrationmirrors top-down integration in the sense of testing direction in the tree.

By moving in an opposite direction, the parent nodes are replaced by throw-away codes instead of the children.These throw-away codes are also known as drivers3. Most functional decomposition trees have a reasonably highscatter near the leaves. Consequently, we will not need to replace as many modules with throw-away codes as we needin a top-down approach. While we do not need that many drivers, this is offset by higher difficulty in programmingdrivers.

This approach allow us to start working with simpler and lower level of the implementation, allowing us to createtesting environments more easily because of the simpler outputs of those modules. This also allows us to handle theexceptions more easily.

Conversely, we do not have an early prototype thus the main program is the last to be tested. If there is a designerror then it will be just identified at a later stage, which implies high error correction cost.

Bottom-up integration is commonly used for object-oriented systems, real-time systems and systems with strictperformance requirements.

For bottom-up integration the number of integration testing sessions isnnodes− nleaves+ nedges.

Figure 1.5: Bottom-up integration, coverage of different sessions at different levels.

1.3.1 Case Study

For bottom-up integration, we can apply it to our example by starting with isLeap, thenlastDayOfMonth, validDateandgetDate. Some shorter but still useful test sessions includesMain andgetDate, incrementDate, printDate.

As we can see, bottom-up seems to be a more natural choice to use than top-down integration for this example.Here we do not need to substitute as many modules with temporary throw-away modules (Figure 1.6). Also, since thelower level modules do not have a high degree of coupling (they are able to function with only their own code), thetest cases are easier to be maintained.

The main purpose of the driver is passing test cases to its children nodes. A possible driver for thelastDayOfMonthto work with isLeapcould be:

3Driver – routine that replace a real module and calls a subsystem, passing test cases to it.

11

input : None (Hardcoded test cases)output: True if it match the expected result, otherwise false

Function Integer lastDayOfMonth driver()9

Output(“Pass 1900: ”,(isLeap(1900)== False))10

Output(“Pass 1999: ”,(isLeap(1999)== False))11

Output(“Pass 2000: ”,(isLeap(2000)== True))12

End (Function lastDayOfMonth driver)13

Function lastDayOfMonth driver

In case of there are results that does not match with the expected ones, we will know thatisLeapmost likely isthe module that is causing the problem. For example, if during the test session the returning value of the years1900and 2000does not correspond to the expected result then we would check the code that handles century years in themoduleisLeap.

N ode rep laced by driver M ain

isLeap lastD ayO fM onth getD ateva lidD ate increm entD ate prin tD ate

N odes w ith rea l code

Figure 1.6: Bottom-up integration of thenextDateprogram.

1.3.2 Summary

In summary, bottom-up integration has the following characteristics:

• Start at the leaves of the functional decomposition tree

• Work toward to the higher level, to the root of the tree

• Hasnnodes− nleaves+ nedgesnumber of integration testing sessions

And the following advantages and disadvantages:

Advantages:

• Less throw-away code programming

• Easy to create testing environments

• Easy exceptions handling

Disadvantages:

• No prototype

• Main program tested last

• Late to identify design errors

• High error correction cost

1.4 Sandwich Integration

Sandwich integration combines top-down integration and bottom-up integration. The main concept is to maximize theadvantages of top-down and bottom-up and minimizing their weaknesses. In top-down by starting at the root of thefunctional decomposition tree, we will be able to test the main program at early stage. In bottom-up, we will havecoverage that is easy to create test cases.

12

Sandwich integration uses a mixed-up approach where we use stubs at the higher level of the tree and drivers atthe lower level (Figure 1.7). The testing direction starts from both side of tree and converges to the centre, thus theterm sandwich. This will allow us to test both the top and bottom layers in parallel and decrease the number of stubsand drivers required in integration testing.

While this approach decreases the number of throw-away coderequired, it is still a requirement. Due of the natureof combining both approach and starting at both side of the tree, we are in fact doing a smaller version of big banginside our functional decomposition tree. Consequently, it is also more difficult of isolate problems.

For sandwich integration, the number of integration testing sessions varies but the maximum number of sessionsis the number of subtrees that exists in the functional decomposition tree.

Figure 1.7: Sandwich integration, combining top-down and bottom-up approach.

1.4.1 Case Study

Sandwich integration is also a good choice for integration testing in this system. This is because by doing eithertop-down or bottom-up approach a good part of the functionaldecomposition tree is covered. Sandwich integrationapproach is known of its flexibility in the integration, thusthere are no strict guidelines in modules grouping. Conse-quently, there are different types of sandwich integration testing, which normally are categorized by the depth of thecoverage.

In this example, we can use a top-down approach for the modules in the higher level of the tree; for exam-ple, Main, incrementDateand printDate. While at the lower level, we use bottom-up approach. For instance,isLeap,lastDayOfMonthand validDate. Resulting a combination of top-down and bottom-up approach where thetested subsystems join in the centre of the functional decomposition tree. (Figure 1.8)

At the end we will be doing a smaller version of big bang approach, problems isolation (while smaller) is still anissue.

isLeap lastD ayO fM onth va lidD ate increm entD ate prin tD ate

B ottom -up in tegration Top-dow n in tegration

getD ate

M ain

Figure 1.8: Sandwich integration of thenextDateprogram.

1.4.2 Summary

In summary, sandwich integration has the following characteristics:

• Combines top-down approach and bottom-up approach

13

• Generally, higher level modules use a top-down approach (stub)

• Normally, lower level modules use a bottom-up approach (driver)

• Testing converges to the middle

• Number of integration sessions can vary

• The maximum number of sessions is the number of subtrees of the system

And following advantages and disadvantages:

Advantages:

• Top and bottom layers can be done in parallel

• Less stubs and drivers needed

• Easy to contruct test cases

• Better coverage control

• Integration is done as soon a component is imple-mented

Disadvantages:

• Still requires throw-away code programming

• Partial big bang integration

• Hard to isolate problems

14

Chapter 2

Call Graph-based Integration

Call graph-based integration is an improvement over the approaches based on the functional decomposition. This isdone by moving to the direction of structural testing. Here we use a directed graph instead of a functional decomposi-tion tree to represent the program. The system is presented as a directed graph where the nodes are the modules andthe edges represent function invocations.

There are two main approaches in call graph-based integration:

• Pair-wise integration

• Neighbourhood integration

In pair-wise integration, we restrict a test session with only a pair of modules, whereas in neighbourhood integrationwe group the modules around the target node as a subsystem to be tested.

2.1 Pair-wise Integration

In pair-wise integration, we eliminate the need of stub and driver, by using the real code instead. This is similar to bigbang where has problem isolation problem due to the large amount of modules we are testing at once. To resolve this,we restrict a test session to only a pair of units in the call graph (Figure 2.1). In case problem occurs then we just needto look into the vertices (the modules) and the edge (the function) in question, thus problem isolation is no longer aproblem.

By pairing up the modules using the edges, we will have a number of test sessions equal to the number of edgesthat exist in the call graph. Since the edges correspond to functions or procedures invocated, in a standard system, thisimplies many test sessions.

For pair-wise integration the number of integration testing sessions is the number of edges.

6 3

2 1

4 5

7 8

10

9

Figure 2.1: Pair-wise integration, some pairs that are usedfor different testing sessions.

15

2.1.1 Case Study

By going to a call graph-based strategy, we will be able to eliminate the problem of empty test sessions. This isbecause the edges actually represent the calls and functional dependencies of the modules.

To use pair-wise integration in this example, we pair-up themodules according to the edges of the call graph(Figure 2.2). For instance,lastDayOfMonthwith isLeapand Main with printDate. Each of these pairs will thenconstitute a subsystem for a test session. With this call graph, there will be seven test sessions because there are sevenedges.

M ain

isLeaplastD ayO fM onth

getD ate

va lidD ate

increm entD ateprin tD ate

Figure 2.2: Two pairs used in pairwise integration of thenextDateprogram.

2.1.2 Summary

In summary, pair-wise integration has the following characteristics:

• Each test session is restricted to only a pair of modules

• Module pairing is based on the edges of the call graph

• The number of integration testing sessions is the number of edges

And the following advantages and disadvantages:

Advantages:

• Stub and driver need is eliminated

• Use of real code

Disadvantages:

• Many test sessions

16

2.2 Neighbourhood Integration

While pair-wise integration eliminates the need of stub anddriver, it still requires many test cases. As an attempt ofimproving from pair-wise, neighbourhood requires fewer test cases.

In neighbourhood integration, we create a subsystem for a test session by having a target node and grouping all thenodes near it (Figure 2.3. Near is defined as nodes that are linked to the target node that is an immediate predecessoror successor of it.

By doing this we will be able to reduce considerably the amount of test sessions required. Notice that the predeces-sors and successors correspond to the modules that are replaced by stubs and drivers in the functional decompositiontree. This means that we are doing something what sandwich integration does, smaller version of big bang integrationin the tree. Consequently, this causes difficulty in isolating faults.

For neighbourhood integration the number of integration testing sessions isnnodes− nsinknodes.

6 3

2 1

4 5

7 8

10

9

6 3

2 1

4 5

7 8

10

9

6 3

2 1

4 5

7 8

10

9

Figure 2.3: Neighbourhood integration, neighbourhood of nodes1, 2 and 3respectively.

2.2.1 Case Study

Applying this approach to the example we will have:

Neighbourhood of validDate: getDate, lastDayOfMonth

Neighbourhood of lastDayOfMonth: validDate, incrementDate, isLeap

Neighbourhood of getDate:Main, validDate

Neighbourhood of incrementDate: Main, lastDayOfMonth

Neighbourhood of Main: getDate, incrementDate, printDate

M ain

isLeaplastD ayO fM onth

getD ate

va lidD ate

increm entD ateprin tD ate

Figure 2.4: Neighbourhood oflastDayOfMonthin neighbourhood integration of thenextDateprogram.

As we can see, we group the modules such that all the modules that are immediate successors or predecessor of thetarget node are within the same subsystem. Each of these subsystems will be used for separate test session. Figure 2.4show the neighbourhood for the modulelastDayOfMonth. While this approach decreases the number of test sessionsneeded, it also makes it harder to isolate faults.

17

2.2.2 Summary

In summary, neighbourhood integration has the following characteristics:

• Modules are grouped as neighbourhoods

• A neighbour module is a module that is the immediate successor or predecessor of another unit

• The number of integration testing sessions isnnodes− nsinknodes

And the following advantages and disadvantages:

Advantages:

• Stub and driver need is eliminated

• Use of real code

• Reduction of test sessions

Disadvantages:

• Hard to isolate faults

18

Chapter 3

Path-based Integration

By moving to path-based integration we will be approaching integration testing from a new direction. In decomposition-based testing we use a structural approach and in call-graphbase testing we use a functional approach. Here we will tryto combine both structural and functional approach in path-base integration. Finally, instead of testing the interfaces(which are structural), we will be testing the interactions(which are behavioural).

Here, when a unit is executed certain path of source statements is traversed. When this unit calls source statementsfrom another unit, the control is passed from the calling unit to the called unit. For integration testing we treat theseunit calls as an exit followed by an entry.

The main path-based integration approach that we will be discussing is:

• MM-Path1 based integration

In MM-path based integration we track all the modules execution paths2 and messages3 used in the system. Themodules traversed by this path is then used as a subsystem to be tested.

Note that in path-based integration we will introduce two new types of nodes:

• Source node4

• Sink node5

3.1 MM-Path based Integration

MM-Paths represent actions with inputs and outputs thus showing us the system in a functional level. The way theMM-Path graph is constructed, how the modules and messages are identified allow us to work on the system in astructural level.

A MM-Path graph is represented by a directed graph in which nodes are module execution paths and edges tomessages and returns from one unit to another. These cross unit paths show the possible executions paths, allowing usto examine more closely the behaviour of the system (Figure 3.1).

To be able to apply MM-Path based integration, we must first find guidelines which allow us to limit us the depthof an MM-Path. Two possible behavioural criteria are:

• Message Quiescence

• Data Quiescence

1Method/Message-Path – an interleaved sequence of module executionpaths and messages.2Module Execution Path – a sequence of statements that beginswith a source node and ends with a sink node.3Message – a programming language mechanism by which one unittransfers control to another unit.4Source Node – a statement fragment at which program execution begins or resumes.5Sink Node – a statement fragment at which program execution terminates.

19

It is said to be message quiescence when we arrive to a unit that sends no messages. Data quiescence is reachedwhen we terminate a set of execution with the creation of datathat is stored but not used immediately.

With these definitions, we will be able to do an integration testing that combines both structural and functionalapproach, allowing a closer coupling with the actual systembehaviour. Avoiding the drawbacks from a structuralbased approach and allowing us to have a smoother transitionfrom integration testing to system testing (where the useof behavioural threads is desired).

However, to use a MM-Path based integration we must put extraeffort to identify the MM-Paths; which may becompensated by the elimination of stub and driver.

For MM-Path based integration the number of integration testing sessions is dependant to the system in question.

1

2

3

4

5

1

2

3

4

1

2 3

4

5

A B C

Figure 3.1: A MM-Path (dotted arrows) across unitsA, B and C.

3.1.1 Case Study

The example is a data-driven program; the MM-Paths begin in and return to the main program. The following fragmentrepresent the first MM-Path for “5/27/2002”:

Main(1,2)message1getDate(36,37,38,39,40,41,42,43,44,45,46,47)

message7validDate(24,25,26,27,28)

message6lastDayOfMonth(14,15,16,23); /* Point of quiescence */

validDate(28,30,31,33,35)getDate(48)

Main(3)

A main problem when using this approach is knowing how many MM-Paths are required to complete the integra-tion test. The set of MM-Paths should traverse all source-to-sink paths.

A large number of paths (or even infinite) caused by loops can be reduced by condensation graphs6 of the directedacyclic graphs7.

6Condensation Graph – a graphCG = (CV,CE) based on the graphG = (V,E) where each vertex inCV corresponds to a strongly connectedcomponent inG and edge (u, v) is in CE if and only if there exists an edge inE connecting any of the vertices in the component ofu to any of thevertices in the component ofv.

7Directed Acyclic Graph – a directed graph with no directed cycles.

20

3.1.2 Summary

In summary, MM-Path integration has the following characteristics:

• Messages sent between modules are tracked

• The set of MM-Paths should cover all source-to-sink paths

• Points of quiescence are natural endpoints for an MM-Path

• The number of integration testing sessions is dependant to the system in question

And the following advantages and disadvantages:

Advantages:

• Hybrid of functional and structural testing

• Closely coupled with actual system behaviour

• Does not require stub or driver

Disadvantages:

• Extra effort required to identify the MM-Paths

21

Conclusion

It is important to note again that all the strategies and approaches mentioned are targeted for the traditional proceduralprogramming languages. Still, the foundation and many basic concepts can be applied to object-oriented programminglanguages by undergoing some changes.

For instance, in many situations we have assumed that the units are able to behave correctly independently fromother units of the same system. This assumption will not be able to be applied in object-oriented applications ascommonly as procedural ones because the units are tightly coupled; making the use of stub and driver more frequent.

It is also important to remind that at first sight, integration testing and system testing seems to be testing the systemas a whole (all units are present and complete) but their goals are different. In integration testing we are more concernedin finding faults when the units are integrated. While in system testing we are more concerned in demonstrating theperformance of the system.

With all these testing strategies and approaches we are ableto test different type of systems more effectively andmore systematically. For each system there may be an optimaltesting strategy and approach that should be used.However, due to the demands in the software development industry and limiting factors like lack of time, capital andtechnical expertise such choices are not always possible oras complete as desired.

Yet, I believe that we should not put testing as something to be avoided or just for the completeness of the soft-ware development cycle. In fact, it is one of the most important stages and more effort should be put in it. Generally,the loss caused by a deployed faulty system is greater than the amount of investment needed to do a more completetesting to avoid such problem. This is especially true in critical systems where the damage is irreversible, like loss ofhuman lives and environmental damages.

22

Bibliography

[Jor02] Paul C. Jorgensen.Software Testing A Craftsman’s Approach. CRC Press, second edition, 2002.

[KFN99] Cem Kaner, Jack Falk, and Hung Quoc Nguyen.Testing Computer Software. John Wiley & Sons Inc,second edition, 1999.

[Pez05] Mauro Pezze. Quality control of softwares. Chapter 18: Integration and Component-based Software Testing,2005.

[Sch06] Holger Schlingloff, 2006. Handouts for Advanced Topics in Computer Science: Testing.

23