Goal-Centric Traceabilityfor Managing Non-Functional Requirements

Jane Cleland-Huang, Raffaella Settimi, Oussama BenKhadra,Eugenia Berezhanskaya, Selvia Christina

Center for Requirements EngineerngDePaul University243 S. Wabash

Chicago, IL 60604. USA.1-312-362-8863

{jhuang,rsettimi}@cs.depaul.edu

ABSTRACTThis paper describes a Goal Centric approach for effectivelymaintaining critical system qualities such as security,performance, and usability throughout the lifetime of a softwaresystem. In Goal Centric Traceability (GCT) non-functionalrequirements and their interdependencies are modeled as softgoalsin a Softgoal Interdependency Graph (SIG). A probabilisticnetwork model is then used to dynamically retrieve links betweenclasses affected by a functional change and elements within theSIG. These links enable developers to identify potentiallyimpacted goals; to analyze the level of impact on those goals; tomake informed decisions conceming the implementation of theproposed change; and finally to develop appropriate riskmitigating strategies. This paper also reports experimental resultsfor the link retrieval and illustrates the GCT process through anexample of a change applied to a road management system.

Categories and Subject DescriptorsD.2.7 [Distribution, Maintenance, and Enhancement]:Extensibility

General TermsManagement

KeywordsTraceability, link retrieval, impact analysis, non-functionalrequirements, system quality.

1. INTRODUCTIONSoftware products are increasingly deployed within criticalapplications in which failure of the system could lead to loss oflife, environmental damage, or significant financial loss. Thereare numerous accounts of catastrophic software failures such asthe Therac 25 incidents [16], Ariane 5 launch failure [17], the

Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and thatcopies bear this notice and the full citation on the first page. To copyotherwise, or republish, to post on servers or to redistribute to lists,requires prior specific permission and/or a fee.ICSE'05, May 15-21, 2005, St. Louis, Missouri, USA.Copyright 2004 ACM 1-581 13-963-2/05/0005...$5.00.

London Ambulance System [11], and more recently problemswith the British Air Control System that introduced over $1M inoverruns and subsequently led to major flight downtime atHeathrow International airport [9]. Post mortem analyses havelaid at least partial blame on the incorrect implementation andmanagement of non-functional requirements (NFRs). Whereasfunctional requirements describe what the system needs to do,NFRs describe constraints on the solution space, and capture abroad spectrum of properties such as reliability, portability,maintainability, usability, safety, and security [22]. These NFRsare also known as 'ilities' or quality requirements. Several studieshave shown that failure to effectively manage these requirementscan have devastating effects on system quality, and can result inexpensive downtime or even complete failure of the system [5].Traceability, which has been defined as "the ability to describeand follow the life of a requirement in both a forwards andbackwards direction" from inception throughout the entiresystem's lifecycle provides useful support mechanisms formanaging requirements during the ongoing change process [13].However, NFRs are difficult to trace because they tend to have aglobal impact upon a software system [10,14], and because of theextensive network of interdependencies and trade-offs that existbetween them [7,10]. Due to these difficulties many organizationsentirely fail to trace NFRs, exposing themselves to huge riskswhen change is introduced or the software is redeployed in a newcontext.

This paper introduces a goal-centric approach to managing theimpact of change upon the NFRs of a software system. GoalCentric Traceability (GCT) enables developers to understand andassess the impact of functional changes upon NFRs in order tomaintain the quality of the system throughout its operationallifetime. In the GCT approach, NFRs are first modeled as goalsand operationalizations within a Softgoal interdependency graph(SIG) [7,10]. During the change process, traces are thendynamically established from impacted elements of the functionaldesign to potentially impacted elements of the SIG. The traces aregenerated using a probabilistic network model. They enabledevelopers to first identify the direct impact of the change andthen subsequently to evaluate its broader impact on related NFRgoals.

The remainder of this paper is laid out as follows. Section 2provides a brief introduction to Softgoal Interdependency graphsand explains their use for modeling non-functional requirements.Section 3 then introduces the GCT model and its major

362

Good Performance

/ Satisficed goal or Op Store data in Index the datanon- according to

X Non-Satisficed goal or Op compesdcomm ocormpressed commonformat. retrieval keys

Figure 1. A softgoal interdependency graph

components. Section 4 discusses the role played by traceability inimpact analysis activities, describes the probabilistic approachadopted in GCT, and reports on an experiment conducted toevaluate its effectiveness. Section 5 then discusses goal re-evaluation during a GCT change event, and section 6 provides anexample in which GCT is utilized to evaluate the impact of aproposed change. Finally, section 7 concludes with an analysis ofthe applicability and limitations of the approach and somesuggestions for future work.

2. MODELING NFRS AS SOFTGOALSGCT utilizes SIGs to model non functional goals and their varioustradeoffs [7,10]. Figure 1 provides an example of a SIG depictingthe goal to achieve goodperformance for an inventory system. Ina SIG, goals are decomposed into subgoals describing qualities ofthe system desired by one or more of its stakeholders. Forexample in this case good performance is decomposed into thesubgoals of space and response time. In turn, each subgoal couldbe further refined into lower-level subgoals, such as the subgoal toretrieve data in < 3 seconds.

The term softgoal reflects the fact that extensiveinterdependencies exist between various NFRs, and it is ofteninfeasible to entirely fulfill each and every system goal. Tradeoffsmust therefore be made [2]. To understand these tradeoffs, thesubgoals are decomposed into 'operationalizations', whichprovide candidate design solutions for achieving the goal andprovide the basis for reasoning about potential tradeoffs. Forexample the subgoal shown in Figure 1 to retrieve data in < 3seconds could be helped through storing data in non-compressedformat, or indexing the data. However, the first option negativelyimpacts the performance related space subgoal and provides anexample of a tradeoff between competing subgoals. Standard

NFR catalogues can be used to support the task of goaldecomposition [7].

An operationalization can contribute either positively ornegatively to its parent goal, or sometimes have no effect on it atall. Contribution relationships are therefore measuredqualitatively as: '+' helps, '++" makes, '-' hurts, '- -' breaks, or'?' unknown. SIG analysis includes determining whether inputcontributions from lower level operationalizations or subgoals,will "satisfice " the goal (ie fulfill at least its minimumrequirements) [7]. Each node is then analyzed to determine if it issatisficed, based upon the satisficing of its children nodes, and ontheir individual contributions. The SIG's ability to relateimplementation details to system goals, creates the opportunity toprovide effective support for the long-term management of NFRs.The challenge is to create a feasible traceability infrastructurecapable of identifying impacted regions of the SIG during achange event.

3. THE GCT MODELGCT is implemented through the four distinct phases of goalmodeling, impact detection, goal analysis, and decision making.These phases are depicted in Figure 2.

Goal modeling primarily occurs during the elicitation,specification, and architectural design of the system. It is duringthis phase that non-functional goals are initially modeled assoftgoals, decomposed into operationalizations, and negotiatedand agreed upon between various stakeholders. The SIG ismaintained throughout the life of the software system in order toprovide the underlying mechanism for reasoning about the impactof change on NFRs.

During impact detection traceability links are established betweenthe functional model of the system and a set of potentiallyimpacted SIG elements. For the purposes of this project UMLclass diagrams were selected as the primary representation of thefunctional model. This decision was made for several reasons andin light of the fact that functional change is implemented at eitherthe code or design level, depending upon the selecteddevelopment environment. Many UML case tools such asTogether®, by TogetherSoftTm [25] provide capabilities formanaging traces and consistency between design models andcode, meaning that establishing links to the design model impliestraceability to code and vice versa. Furthermore the ability tounderstand the impact of a change on the underlying UML model,provides developers with a useful and high-level perspective ofthe change, and enables them to evaluate its impact prior toimplementing it in the code [4]. As numerous papers havediscussed the topic of functional impact analysis [3], the startingpoint of the GCT analysis described in this paper is the point atwhich a set of classes, or methods within those classes, have beenidentified as a target of the proposed change.

GCT implements a dynamic approach to trace retrieval whichsupports the extensive network of links needed between thefunctional and non-functional models of the system. Duringimpact detection the retrieval algorithm returns a set of potentiallyimpacted SIG elements. These links are then evaluated by theuser and any incorrectly retrieved links are discarded. Thisprocess is very similar to issuing a query on a web-based searchengine and then selecting which of the returned links arepotentially valid ones and worth further exploration. The outputs

363

Figure 2. Goal-centric traceability

of this phase are a set of operationalizations and goals, confirmedby the user to be relevant.

During the goal analysis phase, the impact of the change ispropagated throughout related regions of the SIG in order toevaluate its effect on system wide goals. Goal re-evaluation is arecursive activity in which each retrieved operationalization isexamined to determine how the proposed change impacts thesatisficing of its parent's goals. In turn, any parent goal that is nolonger satisficed is also re-evaluated to determine how the changewill impact its own parents. This continues until all potentiallyimpacted goals have been re-evaluated. In order to fullyunderstand the benefits of the change, a similar process isexecuted for operationalizations that strengthen their parent goals.The primary output of this phase is an impact analysis reportidentifying all goals that are either negatively or positivelyaffected by the proposed change.

Finally, during the decision making phase, stakeholders examinethe impact report and determine if the change should beimplemented or not. Alternate design solutions can be proposedand explored in order to mitigate or remove potential negativeimpacts of the change.

4. IMPACT DETECTIONImpact detection is dependent upon the effectiveness of thetraceability mechanism to establish correct links between thefunctional and non-functional models. In current practice,traceability links are implemented through matrices, hypertext

links, graphs, or more formal methods [13]. However practice hasshown that it is hard to maintain links in a constantly evolvingsystem [2,24], and the problem is exacerbated for NFRs becauseof their tendency for broad ranging impact which often results inthe need for an excessive number of links.

To address problems of trace maintenance, several researchershave investigated the use of dynamic schemes for runtime linkgeneration. These include information retrieval methods [1,15,20]and heuristic approaches [23]. The advantages of dynamicmethods are that they eliminate the need for long-termmaintenance of traceability links and subsequent problems of linkdegradation. However, these approaches also suffer from acertain level of imprecision. Despite this precision problem, theextensive network of traceability links needed to support impactanalysis of NFRs, indicates the impracticality of establishing andmaintaining explicit links. GCT therefore utilizes a dynamicapproach to link generation in which loss ofprecision is counteredby the user's role in evaluating the set of generated links.

The problem of linking SIGs to UML artifacts was directlyaddressed by Cysneiros et al [10] who proposed an ontologicalapproach in which specific keywords were embedded in both theoperationalizations of the SIGs and as notations within UMLdiagrams. This approach has the benefit of explicitly and preciselydefining relationships between two entities in which the samekeyword is embedded. It does however require the formalconstruction of a system-wide ontology and the formality ofcorrectly using it throughout the development process. In contrast

364

ConsiElicit, analy2specify NFduring s5

truct SIG MODELINGze, negotiate, andRs using a SIGystem design.

ntain SIGelements and

ions to reflectnted change. modeled

withinSIG4

MainUpdate SI(contributiimpleme

Link RetrievalProbabilistic retrieval algorithm

dynamically returns linksW between impacted classes andelements in the SIG.

2IMPACT

DETECTION

directly

impactedSIGelements

I

User EvaluationUser assesses the set of

returned links and accepts orrejects each one.

4Impact Evaluation DECISION

Stakeholders evaluate the MAKINGimpact of the proposed changeupon NFR goals, & identify risk

mitigation strategies Gono-go

l decisionDecision 1 isk

Stakeholders determine mtgtowhether to proceed with the

proposed change.

3. 1Contribution Re-Analysis GOALUser modifies contributions ANALYSIS

from impacted SIG elements totheir parents as necessary.

Goal Re-evaluationFor each impacted element, Goalchanges are propagated evaluationthroughout the SIG to identify report

potentially impacted goals.

T r11

11

I I

the more relaxed retrieval approach adopted by GCT returns lessprecise results but also offers several of its own advantages. First,this approach does not require the systematic construction ofkeywords into the SIG and UML artifacts, but allows developersto use their own words. As a result, GCT can be appliedretrospectively to a project and still return meaningful results.More importantly the GCT retrieval algorithm returns a broaderset of results to the developer, thereby providing a richer contextfor understanding the ramifications of a proposed change.

The dynamic approach adopted in GCT falls under the generalcategory of an information retrieval (IR) method. In IR a userrequests information using either keywords or a natural text basedquery, and a search engine retrieves a set of candidate objectsfrom which the user selects the relevant ones [12]. In mostretrieval algorithms a similarity index is calculated that measuresthe degree of similarity between two documents. A thresholdvalue is then established so that any index value above thethreshold indicates that the document should be retrieved.

IR algorithms are then typically evaluated using the metrics ofrecall and precision [ 12] where:

Recall = Number of relevant documents retrievedNumber of relevant documents

Precision = Number of relevant documents retrievedTotal number of documents retrieved

Maximum recall can be trivially achieved by retrieving alldocuments in the collection, and maximum precision by retrievinga single document that is known to be correct. For this reason, anIR system normally attempts to maximize recall and precisionsimultaneously.

In existing traceability-related retrieval methods, a query istypically formulated from the words in a requirement, and asearch algorithm executes the query within a search space of otherartifacts such as other requirements, code, test cases, or designmodels. Results from previous IR retrieval methods have metlimited success with precision rates typically at 10-60% when thethreshold level is set to achieve 90-100% recall [1,15,20,23].

4.1 Impact Detection in GCTThe GCT retrieval algorithm was implemented using aprobabilistic network model. Our approach follows previous workof Wong and Yao who adopted a probabilistic inference model to

q

Q)t2 t3 t4 _f)tk

Figure 3. Probabilistic network model for informationretrieval

support information retrieval [26,27]. We selected this methodbecause our intention to incorporate additional factors such ashierarchical information into future retrieval algorithms issupported by the expressiveness of the approach. Wong andYao's probabilistic retrieval model is based on an epistemologicalview of probability for which probabilities are regarded as degreesof belief, and may not be necessarily learned from statistical data.The model assumes that the relevance relationship between adocument and a user's query cannot be determined with certainty.Both documents (d1,d2,...,d, and queries (q1,q2,...,)q arerepresented as propositions within a concept space U, called theUniverse of Discourse. Singletons in the model space U areelementary concepts, so propositions are subsets of U. Aprobability function is defined over the model space, so that theprobability pr(d) of a proposition can be interpreted as the degreeof coverage of Uby the knowledge contained in proposition d.

In a preliminary phase of the information retrieval algorithm,documents and queries are reduced to sets of index terms. Thisgreatly diminishes the computational effort, and improves theretrieval performance. In this preliminary phase, stop words whichare not informative words such as conjunctions and adverbs areeliminated, and words are stemmed to their common grammaticalroot by eliminating prefixes and suffixes [12]. The elementaryconcepts in U are the index terms extracted from the collection ofdocuments. Thus documents and queries are modeled as subsets inU.

In the probabilistic network model documents, queries, and indexterms are network nodes, and each node is associated to a binaryrandom variable. As depicted in the independence graph of Figure3, index term nodes point to the document nodes and query nodesin which they are contained. The main model assumptions amongthe variables in the graph are that documents are conditionallyindependent on the query given the index terms, and that the indexterms are pair-wise disjoint.

For each document, the probability pr(d) is computed aspr(d1) = T pr(d1 ti)pr(ti) The epistemological approach

interprets pr(djlti) as the user's degree of belief that the elementaryconcept ti supports the knowledge in dj. It seems natural to use thedocument semantics to estimate the probability values, assuggested in [27]. The conditional probability pr(djltd is assumedto be proportional to the frequency of term ti in dj and is estimatedas follows

pr(dj I ts) freq(dj t,)Efreq(di Itk)k

The probability of each index term pr(tJ is estimated as a functionof the inverse term frequency

pr(t,) = q -, where q is a normalizing constant and ni is theni

number of documents containing the term ti. This expressionassumes that the probability of each index term is inverselyproportional to the number of documents containing the indexterm. Thus less frequent index terms have higher probabilityvalues, since words that appear in fewer documents are consideredto be more informative in representing a concept in U. This is in

365

line with other information retrieval algorithms such as the vectorspace model algorithm that use tf-idf ranking strategies [ 12].

The relevance of a document dj to a query q is evaluated asthe conditional probability pr(djlq), which can be regarded as thedegree of coverage for document dj provided by q. Theconditional independence assumption in the model allows us tocompute pr(djlq) using the conditional probabilities pr(djltd andpr(qltd as follows:

pr(dj q) = FEpr(dj ti)pr(q t,)pr(t1) /pr(q)

where pr(q t ) = freq(q,t;) is computed analogously toI freq(q, tk)k

pr(djl tjd and the probability of a query q ispr(q) = , pr(q ti)pr(t1)

Documents dj are ranked according to their degree of relevance tothe query q, computed as pr(djlq). A threshold value is typicallyestablished so that all documents with probability of relevancehigher than the threshold will be retrieved. In this paper, thethreshold values were selected by optimizing the objectivefunction "maximize Recall + Precision, where recall > 85%".This is equivalent to finding the threshold value which maximizesboth recall and precision while maintaining a sufficiently highrecall level to effectively implement GCT. A training set can beused to select the threshold values.

4.2 Experimental evaluationAn experiment was designed to measure the effectiveness of theimpact detection method. This experiment was based on artifactsfrom the Ice Breaker System, which is described in [19] andenhanced with requirements mined from documents obtained fromthe public work departments of Charlotte, Colorado [6]; Greeley,Colorado [8]; and the Region of Peel, Ontario [18]. The IceBreaker System manages de-icing services to prevent iceformation on roads. It receives inputs from a series of weatherstations and road sensors within a specified district, and uses thisinformation to forecast freezing conditions and scheduledispersion of salt and other de-icing materials. The systemmaintains maps of the district in order to plan de-icing routes andto ensure complete coverage of all roads. It also manages theinventory of de-icing materials; maintains, dispatches, and trackstrucks in real time; and issues and tracks work orders. The IceBreaker system consists of 180 functional requirements, and ninedistinct SIGS representing NFRs of accuracy, availability,completeness, cost, extensibility, operational security,performance, safety, and usability. The Usability SIG from theIce Breaker System is illustrated in Figure 4. These SIGs werecomposed of 82 goals and subgoals, and 98 operationalizations.The functional model of the system was constructed by a team ofstudents and faculty at DePaul University, and was representedthrough nineteen use cases, and 75 classes bundled into sixteenpackages. A project glossary was constructed at an early stageand developers were encouraged to utilize standard termswherever possible.

The Ice Breaker System was selected as opposed to a largerindustrial sized data set, due to the sheer enormity of the task ofconstructing an accurate traceability matrix against which tocompare experimental retrieval results. In this system alone, the75 classes and 180 SIG elements introduced 13,500 potential

+

AutomatedStandard

Online Onlinehelp tutodals

Style guidIdentfy AllocatEroads icng tato de- to Workice Orders

tnuckI

Calculate number ofdrvers needed on duty.

Thermal Zoomablemaps maps

Figure 4. The Usability SIG from the Ice-Breaker system(Shaded goals and operationalizations are incorporated into the change impact analysis of Figure 6)

366

sections ofroute

Trucks GUI

stationGUI

s GUI

Table 1. Results of link retrieval

l______ |______ Training Set Full data set

Threshold 0.015 |0.02 0.025 0.030 0.04 0.05 0.02Actual 90 90 90 90 90 90 619Retrieved 226 205 190 180 154 135 1052Correctly retrieved | 87 86 | 80 78 76 70 540

Recall 0.9667 0.9556 0.8889 0.8667 0.8444 0.7778 0.8724Precision 0.3850 0.4195 0.4210 0.4333 0.4935 0.5185 0.5133

Objeca+eFuncion) 1.3517 1.3751 1.3099 1.3000 Recall

a. SIG prior to change

3

b. SIG showing change impact

Figure 5. Goal re-evaluation

5. GOAL ANALYSISOnce the retrieval algorithm has returned a set of potentiallyimpacted SIG elements, and the user has filtered these to removenon-relevant ones, the goal analysis phase can commence. Duringthe goal analysis phase ofGCT, the potential impact ofthe changeis propagated throughout the SIG to determine its broader effecton non-functional goals. Goal evaluation follows the standardSIG methodology in which the levels of contribution from agoal's children are evaluated in order to determine if the goal canbe fulfilled by the current operationalizations [7]. The goal re-evaluation process implemented in GCT is depicted through theexample of Figure 5, in which a change is introduced that directlyimpacts Operationalization (Op) 2.

To support reasoning about the impact of change upon the SIGelements two additional symbols are introduced. The"strengthens" (1 ) symbol is used to depict contributions that havebeen strengthened through a change in the impactedoperationalization or subgoals, while the "weakens" (4-) symboldepicts contributions that have been weakened. In this example,Op2's contribution to Goal 3 is weakened and developers musttherefore determine if the helps contribution from Opl and thereduced help provided by Op2, is sufficient to satisfice Goal 3.Even if it is still satisficed, it is likely to be satisficed at a lower

level than it previously was, and so repercussions on its higherlevel goals should also be evaluated.

Goal evaluation within a SIG is by nature a qualitative andsubjective process. However its usefulness has been clearlydemonstrated through numerous examples and case studies [7,10].GCT extends the application of the SIG analysis to moreeffectively support goal re-evaluation during the ongoing changeprocess.

6. GCT IN ACTIONIn this section the GCT process is illustrated through a proposedenhancement to the Ice Breaker System intended to provide moreaccurate predictions of road freezing conditions. The currentsystem predicts freezing conditions based solely on temperaturereadings from the road sensors and data received from localweather stations, whereas the enhanced version would incorporateelevation and exposure information for each road section into theprediction algorithms.

During functional impact analysis the sequence diagram entitled"Predict freezing conditions" was analyzed to determine how toaccommodate the proposed changes. It was determined thatchanges should be made in the road sensor and road classes. Theroad sensor class would be extended to store elevation andexposure data for the location of the sensor, while the methodcontained in the road class for predicting freezing conditions foreach road section would be modified. Using the threshold valueof 0.02 identified from the training set, a query was thereforeissued for each of these classes against the Ice Breaker SIGelements. Combined results from these queries resulted in areturn of seventeen SIG elements shown in Table 3. Followinguser evaluation of the links, eight were accepted as correct, fourwere labeled as indirectly related, and the remaining five were outrightly rejected.

All of the directly impacted SIG elements, their potentiallyimpacted goals, and additional elements that contributed topotentially impacted goals, were combined into an integrated SIG.This is shown in Figure 6. For example, the Operationalization"Analyze district freezing conditions < 10 minutes" which wasidentified as an impacted element, contributes a helps relationshipto its parent goal of "Real-time processing". A siblingoperationalization of "Generate weather forecast in < 10minutes" was also included in the SIG because both contributionsneeded to be considered in order to determine if the parent goalwas still fulfilled. Depending upon events during the analysisprocess, additional elements may be included at a later time.

Affected operationalizations and goals identified in the impactdetection phase are shaded gray. Impacted elements are firstcategorized according to whether the proposed change enhancesor detracts from the original contribution to the parent goal (ie tor 'I contribution). In this example the operationalizations"analyze district freezing conditions < 10 minutes" and "assignsensors to roads" both exhibit weakened contributions to theirparent goal, whereas the other impacted components either havestrengthened contributions or no observed change.

The performance goal to "analyze district freezing conditions <10 minutes" is re-evaluated according to normal practices. Forexample, if during initial system development the response timehad been validated through the application of a softwareperformance graph [21], then the graph could be updated to

368

Table 3. Combined results for a query against road and road sensor classes

incorporate additional processing of elevation and exposure datafor each road sensor in order to determnine if the performance goalcould still be met. In this case it was determined that theadditional processing in larger districts could result in failure tomeet the performance goal. This failure would have negativeimplications on the satisficing of real-time processing, responsetime, andperformance goals.

The road safety goal to "modify road sensors" could also benegatively impacted by the impact of the proposed change uponthe "assign sensors to roads" operationalization. This is becauseunder the proposed enhancement, sensors could only be assignedto road sections for which additional elevation and exposure datais available. Goal analysis indicates that this problem couldnegatively impact the goals of modifiable maps and extensibility.

It was determined that all other impacted operationalizations andgoals would contribute positively to the system goals, and wouldresult in an improvement in road safety, more cost efficient use ofde-icing materials, and improvements to the automated schedulingof de-icing tasks.

During the final stage of Decision Making, the tradeoffs wereanalyzed and risk mitigation strategies were considered. It wasdetermined that the performance issue could be mitigated throughthe use of multiple processors and that the modifiability issuecould be easily resolved through the use of default elevation andexposure values. Without the benefit of GCT analysis theperformance issue may have only been discovered followingdeployment of the new system, and modifiability of the systemcould have been negatively impacted by the change. Thisexample therefore demonstrates that GCT can provide a usefulapproach to discovering these potential impacts prior todeployment of the change. Once a decision has been made toimplement the change all impacted operationalizations are re-evaluated to determine if any contributions to their parents havechanged sufficiently in strength to merit a formal change in theSIG. Contributions are then updated in order to maintain the SIGin an accurate state to support future change events.

369

0.07669 Remove road sections Modify map Extensibility 0.07669 0.14333 MaybeAnalyze freezing

0.05795 conditions Real time processing Performance 0.05795 0.10857 Accept

0.05752 Add new road sections Modify map Extensibility 0.05752 0.10750 MaybeMonitoring of road

0.05286 conditions Road safety Safety 0.05286 AcceptDisplay de-iced

0.04967 sections of route Onboard directions Usability 0.04967 0.06786 MaybeRecord road freezing Accurate weather

0.04510 events predictions Accuracy 0.04510 0.08036 AcceptBroadcast road Monitoring ofroad

0.03964 conditions to contacts conditions Safety 0.03964 MaybeReal time communication

Update truck status < through onboard0.03725 1 minute. computer Performnance 0.03725 Reject

Remove weather Network ofweather0.02714 station stations Extensibility 0.02714 Reiect

Assign roads to0.02571 sensors Modify road sensors Extensibility 0.02571 Accept

Identify roads to de-0.02571 ice Automated scheduler Usability 0.02571 Accept0.02571 Modify road sensors Modifiable maps Extensibility 0.02571 Accept

Validation by road0.02571 engineer Accuracy of maps Accuracy 0.02571 Accept

Add new weather Network of weather0.02036 station stations Extensibility 0.02036 Reject

Compare actualconditions to Accurate weather

0.02036 predictions predictions Accuracy 0.02036 AcceptEnter weather

0.02036 conditions manually Failure mode operations Availability 0.02036 Reject

Ice BreakerSystem ,

New Notation

t Enhanced ability to flilfill goalDecreased ability to fiilfill goal

Record road Cornparefreezing actualevents conditions

topredictions

Figure 6. Integrated SIG showing potentially impacted elements of the Ice Breaker System.

7. CONCLUSIONSGoal Centric Traceability provides developers with the means ofmanaging the impact of functional change upon non-functionalrequirements. This is crucial to the long-term maintenance ofcritical systemic qualities such as safety, security, reliability,usability, and performance.

The experimental results reported in this paper indicate thefeasibility of using a probabilistic approach to dynamicallyretrieve traceability links for non-functional requirements. Theimprecision problems introduced through use of this method arelargely mitigated through user inspection of retrieved links andthrough establishing a sufficiently low threshold that minimizesthe number of omission errors. Although users' feedback isrequired to filter out unwanted links, the effort is only a smallfraction of that which would be required to perform the tracemanually. For example, in the query described in this paperagainst the Road and Road sensor classes, the user had to examine22 candidate links. However, without the benefit of the tool, amanual impact analysis would require the user to consider 180SIG elements for each of the two classes being traced - a total of360 possible comparisons.

Although dynamic traceability methods do not offer a silver-bullet, they do provide a very practical effort-reducing approachto link retrieval. Furthermore, in any non-trivial system, the costand effort required to construct and maintain a traceability matrixthat would capture the extensive network of relationships betweenthe NFRs and other downstream artifacts is often infeasible.

Nevertheless, future work will focus on improving both recall andprecision metrics through incorporating additional factors, such ashierarchical information, into the retrieval process. Additionalwork is also needed to analyze the effectiveness of the approachfor various SIG types and even for different types ofoperationalizations within a single SIG.

8. ACKNOWLEDGMENTSThe work described in this paper was partially funded by NSFgrant CCR-0306303. We also acknowledge the contributions ofDePaul students Jigar Mody, Chris DePalma, Wiktor Lukasik, andJulie Zhang in collecting and managing data, and in contributingto the development of earlier versions of the Poirot tool.

9. REFERENCES[1] Antoniol, G., Canfora, G., Casazza, G., De Lucia, A., and

Merlo, E., "Recovering Traceability Links between Code andDocumentation", IEEE Transactions on SoftwareEngineering. 28,10 (Oct. 2002), 970-983.

[2] Boehm, B., and In, H., "Identifying Quality-RequirementConflicts (Extended Abstract)," Proc. ofIEEE InternationalConference Requirements Engineering, (Apr. 1996) 218.

[3] Bohner, S. and Arnold, R., Software Change ImpactAnalysis, IEEE Comp. Soc. Pub., Tutorial Series, 1996.

370

[4] Briand, L.C., Labiche, Y., O'Sullivan, L., "Impact Analysisand Change Management ofUML Models", IEEE Intn '7Conf: on Software Maint, (Sept. 2003) 256-265.

[5] Brooks Jr., F.P., "No Silver Bullet: Essences and Accidentsof Software Engineering," IEEE Computer., 4, (Apr. 1987),10-19.

[6] Charlotte Dept of Transportation, N.C, http://www.charneck.org/Departments/Transportation/Home.htm

[7] Chung, L., Nixon, B., Yu, E., and Mylopoulos, J., Non-Functional Requirements in Software Engineering, KluwerAcademic, 2000.

[8] City of Greeley, Colorado, http://www.ci.greeley.co.us[9] CNN News, "Air travel chaos hits Britain"

http://www.cnn.com/2004/WORLD/europe/06/03/britain.flightlindex.html

[10] Cysneiros, L., Sampaio de Prado Leite, J., "NonfunctionalRequirements: From Elicitation to Conceptual Models",IEEE Transactions on Software Engineering, 30,5, (May2004) 328-350.

[ 1] Finkelstein, A., and Dowell, J., "A Comedy of Errors: TheLondon Ambulance Service Case Study," Proc. Eighth Intn '7Workshop Software Spec and Design, (1996), 2-5.

[12] Frakes W.B., and Baeza-Yates, R., Information retrieval:Data structures and Algorithms, Prentice-Hall, EnglewoodCliffs, NJ, 1992.

[13] Gotel, O., and Finkelstein, A., "An Analysis of theRequirements Traceability Problem," 1st InternationalConference on Requirements Eng., (1994), 94-101.

[14] Gross, D., and Yu, E., "From Non-Functional Requirementsto Design through Pattems", Requirements EngineeringJournal, 6,1 (2001), 18-36.

[15] Huffman Hayes, J., Dekhtyar, A., and Osbome, J.,"Improving Requirements Tracing via InformationRetrieval", IEEE International Requirements EngineeringConference, (Monterey, CA, Sept. 2003), 138-150.

[16] Leveson, L., and Turner, C. S. "An Investigation of theTherac-25 Accidents" IEEE Computer., 26,7, (July 1993),18-41.

[17] Nuseibeh, B. Ariane 5: Who Dunnit?, IEEE Software, 14,3(May 1997), 15-16.

[18] Region of Peel, http://www.region.peel.on.ca

[19] Robertson, S., Robertson, J., "Mastering the RequirementsProcess'" Addison-Wesley, 1999.

[20] Settimi, R., Cleland-Huang, J., BenKhadra, O., Mody, J.,Lukasik, W., and DePalma, C., "Supporting Change inEvolving Software Systems through Dynamic Traces toUML", IEEE International Workshop on Principles ofSoftware Evolution, Kyoto, Japan, (Sept. 2004), 49-54.

[21] Smith, C., and Williams, L., Performance Solutions: APractical Guide to Creating Responsive, Scalable Software.Addison-Wesley, 2002

[22] Sommerville, I., and Sawyer, P., "Viewpoints: Principles,Problems, and a Practical Approach to Requirements Eng.,"Annals ofSoftware Eng., 3, N. Mead, ed., (1997), 101-130.

[23] Spanoudakis, G., "Plausible and Adaptive RequirementTraceability Structures", 14th International Conference onSoftware Engineering and Knowledge Engineering, (Ischia,Italy, 2002), 135-142.

[24] Sugden, S.C., and Strens, M.R., "Strategies, Tactics, andMethods for Handling Change", IEEE Symposium andWorkshop on Eng. ofComputer Based Systems,(Fredrichshafen, Germany, Mar. 1996), 457-462.

[25] TogetherSofttm, http://www.togethersoft.com[26] Wong, S.K.M., and Yao, Y.Y., "A probabilistic inference

model for information retrieval" Information Systems, 16,3,(1991), 301-321.

[27] Wong, S.K.M. and Yao, Y.Y. "On Modeling InformationRetrieval with Probabilistic Inference", ACM Transactionson Information Systems, 13,1 (1995), 38-68.

[28] X. Zou., Settimi, R., Cleland-Huang, J., Duan,C., "EmpiricalStudy of a Probabilistic Network Approach for RetrievingTraceability Links", Technical Report # TR05-003, School ofComputer Science, Telecommunications, and InformationSystems, DePaul University, (March 2005).

371