4 isabelmargarido-2confcmmiportugal-v1-0-split

Portugal

A Method to Improve the Classification of Requirements Defects

Isabel Margarido ([email protected])Ph.D. Student Researcher

Faculty of Engineering, University of Porto

06-07-2012, Coimbra

João Pascoal FariaFEUP/INESC

22/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra

agenda

introduction

literature review

proposal

assessment

conclusion

33/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra

introduction review proposal assessment conclusion

Explained Understood Designed Coded Sold

Installed Billed Supported NeededDocumented

44/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra

v


55/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra

v


66/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra

v


77/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra


88/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra

related CMMI practices


Isabel Margarido

Colocar uma pirâmide do cmmi e acender só as áreas que são afectadas

99/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra


maturity level 3 (engineering process areas)


Isabel Margarido


1010/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra


maturity level 3 (engineering process areas)Requirements Development (RD)


Isabel Margarido


1111/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra



Specific Practice (SP) 3.3 “Analyse Requirements”


Isabel Margarido


1212/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra



Specific Practice (SP) 3.3 “Analyse Requirements”ensure that they are necessary and sufficient


Isabel Margarido


1313/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra




Verification (VER)


Isabel Margarido


1414/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra




Verification (VER) SP 1.3 “Establish Verification Procedures and Criteria”


Isabel Margarido


1515/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra




Verification (VER) SP 1.3 “Establish Verification Procedures and Criteria” SP 2.1 “Prepare for Peer Reviews”


Isabel Margarido


1616/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra




Verification (VER) SP 1.3 “Establish Verification Procedures and Criteria” SP 2.1 “Prepare for Peer Reviews” SP 2.2 “Conduct Peer Reviews”


Isabel Margarido


1717/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra




Verification (VER) SP 1.3 “Establish Verification Procedures and Criteria” SP 2.1 “Prepare for Peer Reviews” SP 2.2 “Conduct Peer Reviews” SP 2.3 “Analyse Peer Review Data”


Isabel Margarido


1818/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra





maturity level 5


Isabel Margarido


1919/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra





maturity level 5Causal Analysis and Resolution (CAR)


Isabel Margarido


2020/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra

Higher-severity Problem Factors

7 – continually changing requirements

2 – documentation obscure/untrustworthy

3 – changes not adequately documented

4 – lack of traceability

(Chen and Huang, 2009)


10 – lack of consideration for sw quality requirements

motivation

2121/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra

(Hamill and Goseva-Popstojanova, 2009)

Higher-severity Problem Factors

7 – continually changing requirements

2 – documentation obscure/untrustworthy

3 – changes not adequately documented

4 – lack of traceability

(Chen and Huang, 2009)


10 – lack of consideration for sw quality requirements

motivation

2222/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra

“classifying or grouping problems helps to indentify clusters in which systematic errors are likely to be found” (Card, 1998)

our goal is to define classification scheme for requirements defects that facilitates

identification of more frequent defects with higher impact analysis of root causes preparation of reviews checklists reduction of risks (bad communication, incomplete requirements,

final acceptance difficulties)


motivation

2323/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra





HP

ODC(Chillarege et al., 1992)

(Grady, 1976)


motivation

2424/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra





more adequate for other phases than the requirements phase

HP

ODC(Chillarege et al., 1992)

(Grady, 1976)


motivation

2525/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra

requirements review


2626/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra

quality requirements for classification schemes


2727/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra


clearly and meaningfully define attributes


2828/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra


complete: every defect is classifiable using the scheme


2929/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra


attributes values: • clear and meaningful definition• small number (5-9)• aggregate to reduce ambiguity(Freimut et al., 2005)• unambiguous


3030/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra

review literature to compile list of existent classifiers and remove

the ones that do not apply to the phase or document vague and generic classifiers overdetailed duplicates (classifiers with same meaning)

define each classifier and give examples, eliminate ambiguity through definition

phase 1 – assemble classifiers list


3131/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra

type of defect


3232/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra

type of defect

• Not in current baseline• Out of scope• Missing/Omission• Incomplete• Inadequate• Incorrect• Inconsistent• Incompatible• New• Changed Requirement• Typos/Clerical• Unclear

(Bell and Thayer, 1976)


3333/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra

type of defect


• Ambiguity• Wrong Section/Misplaced• Other


(Basilli and Weiss, 1981)


3434/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra

type of defect


• Ambiguity• Wrong Section/Misplaced• Other• Infeasible• Untestable/Non-verifiable• Redundant/Duplicate• General



(Walia and Craver, 2007)


3535/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra

type of defect



• Missing Interface• Missing Function/Description




(Ackerman et al., 1989)


3636/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra

type of defect




• Missing/Incorrect Checking• Missing/Incorrect

Assignment• Missing/Incorrect

Timing/Serialization• Missing/Incorrect

Build/Package/Merge• Missing/Incorrect

Documentation• Missing/Incorrect Algorithm





(Chillarege et al., 1992)


3737/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra

type of defect








Documentation• Missing/Incorrect Algorithm• Missing

Functionality/Feature• Missing Software Interface• Missing Hardware Interface• Missing User Interface• Missing

Requirement/Specification






(Grady, 1992)


3838/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra

type of defect











• Incorrect or Extra Functionality

• Data Type Consistency






(Grady, 1992)

(Porter et al., 1992)


3939/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra

type of defect












• Data Type Consistency• Over-specification

• Not Traceable• Unachievable• Intentional Deviation






(Grady, 1992)


(Hayes et al., 2003/6)


4040/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra

type of defect













• Not Traceable• Unachievable• Intentional Deviation• Extraneous

Information(Bell and Thayer, 1976)





(Grady, 1992)


(Hayes et al., 2003/6)

(Kalinowski et al., 2010)


4141/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra














Information

type of defect


4242/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra














Information

change management

type of defect


4343/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra

• Out of scope• Missing/Omission• Incomplete• Inadequate• Incorrect• Inconsistent• Incompatible • Typos/Clerical• Unclear












• Unachievable• Intentional Deviation• Extraneous

Information

change management

type of defect


4444/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra

• Out of scope• Missing/Omission• Incomplete• Inadequate• Incorrect• Inconsistent• Incompatible • Typos/Clerical• Unclear













Information

vague

type of defect


4545/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra

• Out of scope• Missing/Omission• Incomplete • Incorrect• Inconsistent• Incompatible • Typos/Clerical• Unclear

• Ambiguity• Wrong Section/Misplaced • Infeasible• Untestable/Non-verifiable• Redundant/Duplicate












Information

vague

type of defect


4646/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra

• Out of scope• Missing/Omission• Incomplete • Incorrect• Inconsistent• Incompatible • Typos/Clerical• Unclear













Information

subsumed

type of defect


4747/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra

• Out of scope• Missing/Omission• Incomplete • Incorrect• Inconsistent • Typos/Clerical• Unclear













Information

subsumed

type of defect


4848/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra














Information

generic

type of defect


4949/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra













Information

generic

type of defect


5050/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra













Information

over detailed

type of defect


5151/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra



• Over-specification • Unachievable• Intentional Deviation• Extraneous

Information

over detailed

type of defect


5252/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra

• Out of scope• Missing/Omission• Incomplete• Incorrect• Inconsistent• Typos/Clerical• Unclear

• Ambiguity• Wrong Section/Misplaced

• Infeasible• Untestable/Non-verifiable• Redundant/Duplicate

• Over-specification• Unachievable• Intentional Deviation

• Extraneous Information


5353/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra

• Out of scope

• Missing/Omission

• Incomplete• Incorrect• Inconsistent• Typos/Clerical• Unclear• Ambiguity

• Wrong Section/Misplaced





5454/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra

• Out of scope

• Incorrect• Inconsistent• Typos/Clerical• Unclear• Ambiguity





• Missing or Incomplete


5555/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra

• Out of scope

• Incorrect• Inconsistent• Typos/Clerical• Unclear• Ambiguity



• Over-specification

• Unachievable

• Intentional Deviation


• Missing or Incomplete


5656/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra

• Incorrect• Inconsistent• Typos/Clerical

• Unclear• Ambiguity• Wrong Section/Misplaced


• Unachievable

• Missing or Incomplete• Not Relevant or Extraneous


5757/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra


• Unclear• Ambiguity• Wrong Section/Misplaced• Infeasible• Untestable/Non-verifiable• Redundant/Duplicate

• Unachievable

• Missing or Incomplete• Not Relevant or Extraneous


5858/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra


• Wrong Section/Misplaced• Infeasible• Untestable/Non-verifiable• Redundant/Duplicate

• Unachievable

• Missing or Incomplete• Not Relevant or Extraneous• Ambiguous or Unclear


5959/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra


• Wrong Section/Misplaced• Infeasible• Untestable/Non-verifiable• Redundant/Duplicate

• Unachievable



6060/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra


• Wrong Section/Misplaced• Infeasible• Untestable/Non-

verifiable• Redundant/Duplicate

• Unachievable



6161/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra


• Wrong Section/Misplaced• Infeasible• Untestable/Non-

verifiable• Redundant/Duplicate

• Unachievable



6262/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra



• Redundant/Duplicate

• Missing or Incomplete• Not Relevant or Extraneous• Ambiguous or Unclear• Infeasible or Non-verifiable


6363/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra Missing or Incomplete

Incorrect InformationInconsistentAmbiguous or UnclearMisplacedInfeasible or Non-verifiableRedundant or DuplicateTypo or FormattingNot Relevant or Extraneous


6464/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra

train pilots understand classifiers, distinguish them, can use them in

practice use examples (apply classification scheme)

conduct experiment same document, team A trained and using new

classifiers, team B not trained nor using new classifiers

analyse results number detected defects number similar classifications to same defect description classifiers systematically confounded

phase 2 – validate classification list


6565/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra

adjust definitions, examples, words

repeat phase 2

repeat phase 3 if necessary

phase 3 – readjust classifiers


6666/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra

experiments

group A:19 MSc students

group B:6 undergrad students


6767/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra

results

H0 - all subjects use the same value to classify the type of a defectH1 - not all subjects use the same value to classify the type of a defect


6868/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra

results


Fleiss Kappa: moderate (0,46 and 0,44)


6969/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra

results


Cochran: 0,60 and 0,63 p-value > 0,05

H0 cannot be rejected



7070/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra

results


Cochran: 0,60 and 0,63 p-value > 0,05

H0 cannot be rejected


McNemar: similar results


7171/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra

in pilot teams use the least in a full development cylce

control number of defects from requirements detected on

subsequent phases compare with number of defects before using classifiers

list

make adjustments to the classifiers list as necessary (as in phase 3) and repeat phase 4

phase 4 – pilot


7272/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra

update tools include tool tips so people can remember definitions include definitions and examples on help

test tools use them in practice to detect bugs and necessary

improvementstrain all teams (phase 2)deploy tools

control number of defects from requirements detected on

subsequent phases

phase 5 – deploy


7373/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra

people should be trained in the usage of the defects classification focusing on

distinction of classifiers clarification of definitions practical examples and exercises

avoid choosing a classifier based on its name only definition easily available e.g., as a tool tip in a tool

recommendations


7474/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra

summary


7575/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra

proposed and assessed a requirements defects classification scheme based on a literature review and quality guidelines

summary


7676/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra

proposed and assessed a requirements defects classification scheme based on a literature review and quality guidelinesdefect taxonomy needs to support specific organisation needs (Card)

summary


7777/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra

proposed and assessed a requirements defects classification scheme based on a literature review and quality guidelinesdefect taxonomy needs to support specific organisation needs (Card)organisations should follow proposed methodology when deciding on classification schemes

summary


7878/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra

proposed and assessed a requirements defects classification scheme based on a literature review and quality guidelinesdefect taxonomy needs to support specific organisation needs (Card)organisations should follow proposed methodology when deciding on classification schemestype of defect should follow quality properties and support

summary


7979/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra


CAR

summary


8080/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra


CAR creation of checklists

summary


8181/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra


CAR creation of checklists prevent risks resulting from requirements defects

summary


8282/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra



different people may classify same defect in a different way

summary


8383/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra



different people may classify same defect in a different wayproblems in defects classification

summary


8484/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra




different interpretations

summary


8585/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra




different interpretations misleading analysis

summary


8686/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra

questions

8787/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra

http://softwareandme.files.wordpress.com/2009/10/3260585819-project_management.jpg – 21-04-2010http://www.screenhog.com/sketch/LightbulbIdea.jpg – 21-04-2010http://igraduatedwhatnow.files.wordpress.com/2009/11/thank_you_small.jpg – 02-05-2010http://cartoontester.blogspot.com/2010/01/big-bugs.html – 01-06-2011http://4.bp.blogspot.com/_eCShgHga-_g/TPA5KRac8_I/AAAAAAAAH3o/mgw0g75jOus/s400/disagreement.jpg – 13-06-2011http://chaospet.com/comics/2008-06-16-90.png – adapted, 13-06-2011http://3.bp.blogspot.com/-_Z2dYcXxMmA/TbLat4c6i_I/AAAAAAAAAnk/KlLdgG-dgtw/s1600/whereamigoing.jpg – adapted, 25-05-20111http://www.veryhappypig.com/blog/results.jpg -06-06-2011http://www.stampa.unibocconi.it/immagini/LA4_economiaq20100603145905.jpg - 06-06-2011

images

8888/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra

J.-C. Chen and S.-J. Huang, "An empirical analysis of the impact of software development problem factors on software maintainability," Journal of Systems and Software, vol. 82, pp. 981-992 June 2009.

M. Hamill and G.-P. Katerina, "Common Trends in Software Fault and Failure Data," IEEE Trans. Softw. Eng., vol. 35, pp. 484-496, 2009.

R. Chillarege, et al., "Orthogonal Defect Classification - A Concept for In-Process Measurements," IEEE Transactions on Software Engineering, vol. 18, pp. 943-956, November 1992.

R. B. Grady, Practical software metrics for project management and process improvement: Prentice-Hall, Inc., 1992.

T. E. Bell and T. A. Thayer, "Software requirements: Are they really a problem?," presented at the Proceedings of the 2nd international conference on Software engineering, San Francisco, California, United States, 1976.

V. R. Basili and D. M. Weiss, "Evaluation of a software requirements document by analysis of change data," presented at the Proceedings of the 5th international conference on Software engineering, San Diego, California, United States, 1981.

references

8989/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra

G. S. Walia and J. C. Carver, "Development of Requirement Error Taxonomy as a Quality Improvement Approach: A Systematic Literature Review," Department of Computer Science and Engineering, 2007.

A. F. Ackerman, et al., "Software Inspections: An Effective Verification Process," IEEE Software, vol. 6, pp. 31-36, May 1989.

A. A. Porter, et al., "Comparing Detection Methods for Software Requirements Inspections: A Replicated Experiment," IEEE Transactions on Software Engineering, vol. 21, pp. 563-575, June 1995.

J. H. Hayes, et al., "Case History of International Space Station Requirement Faults," presented at the Proceedings of the 11th IEEE International Conference on Engineering of Complex Computer Systems, Standford, California, 2006.

M. Kalinowski, et al., "Applying DPPI: A Defect Causal Analysis Approach Using Bayesian Networks," in Product-Focused Software Process Improvement. vol. 6156, M. Ali Babar, et al., Eds., ed: Springer Berlin / Heidelberg, 2010, pp. 92-106.

M. Kalinowski, et al., "Guidance for Efficiently Implementing Defect Causal Analysis," presented at the Brazilian Software Quality Symposium,VII SBSQ Florianópolis, Brazil, 2008.

references

9090/27

Isa

bel L

opes

Mar

garid

o,

6th o

f Ju

ne 2

012

, C

oim

bra

published on IEEE: http://ieeexplore.ieee.org/xpl/articleDetails.jsp?

tp=&arnumber=5974237&contentType=Conference+Publications&queryText%3DLopes+Margarido

author for correspondence: Isabel Lopes Margarido, [email protected] http://paginas.fe.up.pt/~pro09003/

affiliation mui nobre

partially sponsored by:

interested in our research?

mailto:[email protected]

http://paginas.fe.up.pt/~pro09003/

4 isabelmargarido-2confcmmiportugal-v1-0-split

Technology

Transcript of 4 isabelmargarido-2confcmmiportugal-v1-0-split