Traceability - Structural Conformance Checking with Design Tests: An Evaluation of Usability and...
-
Upload
icsm-2011 -
Category
Technology
-
view
555 -
download
2
description
Transcript of Traceability - Structural Conformance Checking with Design Tests: An Evaluation of Usability and...
![Page 1: Traceability - Structural Conformance Checking with Design Tests: An Evaluation of Usability and Scalability](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b4a8a64a7959a0168b4610/html5/thumbnails/1.jpg)
MotivationDesign Test
DesignWizardEvaluation
Conclusion
Design Tests
João Brunet
Dalton Guerrero and Jorge Figueiredo
Federal University of Campina Grande (UFCG), Brazil.
09/28/2011
João Brunet 1/ 26
![Page 2: Traceability - Structural Conformance Checking with Design Tests: An Evaluation of Usability and Scalability](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b4a8a64a7959a0168b4610/html5/thumbnails/2.jpg)
MotivationDesign Test
DesignWizardEvaluation
Conclusion
Agenda
1 Motivation
2 Design Test
3 DesignWizard
4 EvaluationAPI Usability
5 Conclusion
João Brunet 2/ 26
![Page 3: Traceability - Structural Conformance Checking with Design Tests: An Evaluation of Usability and Scalability](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b4a8a64a7959a0168b4610/html5/thumbnails/3.jpg)
MotivationDesign Test
DesignWizardEvaluation
Conclusion
Motivation
Check code against design rules is an important activity toguarantee quality on source code
Design Review
Manual ProcessError-proneDoes not scale
João Brunet 3/ 26
![Page 4: Traceability - Structural Conformance Checking with Design Tests: An Evaluation of Usability and Scalability](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b4a8a64a7959a0168b4610/html5/thumbnails/4.jpg)
MotivationDesign Test
DesignWizardEvaluation
Conclusion
Motivation
Check code against design rules is an important activity toguarantee quality on source code
Design Review
Manual ProcessError-proneDoes not scale
João Brunet 3/ 26
![Page 5: Traceability - Structural Conformance Checking with Design Tests: An Evaluation of Usability and Scalability](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b4a8a64a7959a0168b4610/html5/thumbnails/5.jpg)
MotivationDesign Test
DesignWizardEvaluation
Conclusion
The Problem
Gap between state-of-the-art and the state-of-the-practiceApproaches from the state-of-the-art require learning differentlanguages to express design rules
João Brunet 4/ 26
![Page 6: Traceability - Structural Conformance Checking with Design Tests: An Evaluation of Usability and Scalability](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b4a8a64a7959a0168b4610/html5/thumbnails/6.jpg)
MotivationDesign Test
DesignWizardEvaluation
Conclusion
The Problem
Lack of structural conformance checking
Design decisions are violated
João Brunet 5/ 26
![Page 7: Traceability - Structural Conformance Checking with Design Tests: An Evaluation of Usability and Scalability](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b4a8a64a7959a0168b4610/html5/thumbnails/7.jpg)
MotivationDesign Test
DesignWizardEvaluation
Conclusion
Design Test
Design Test
A test that checks whether an implementation complies with a givendesign rule expressed as an algorithm.
João Brunet 6/ 26
![Page 8: Traceability - Structural Conformance Checking with Design Tests: An Evaluation of Usability and Scalability](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b4a8a64a7959a0168b4610/html5/thumbnails/8.jpg)
MotivationDesign Test
DesignWizardEvaluation
Conclusion
What’s the big deal?
It is written in the target programming language
Automated test
João Brunet 7/ 26
![Page 9: Traceability - Structural Conformance Checking with Design Tests: An Evaluation of Usability and Scalability](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b4a8a64a7959a0168b4610/html5/thumbnails/9.jpg)
MotivationDesign Test
DesignWizardEvaluation
Conclusion
Example of Design Rule
Only controller package shall access dao package of a givencomponent.
João Brunet 8/ 26
![Page 10: Traceability - Structural Conformance Checking with Design Tests: An Evaluation of Usability and Scalability](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b4a8a64a7959a0168b4610/html5/thumbnails/10.jpg)
MotivationDesign Test
DesignWizardEvaluation
Conclusion
Example of Design Test
João Brunet 9/ 26
![Page 11: Traceability - Structural Conformance Checking with Design Tests: An Evaluation of Usability and Scalability](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b4a8a64a7959a0168b4610/html5/thumbnails/11.jpg)
MotivationDesign Test
DesignWizardEvaluation
Conclusion
Example of Design Test
Facts about the code (an API)
João Brunet 10/ 26
![Page 12: Traceability - Structural Conformance Checking with Design Tests: An Evaluation of Usability and Scalability](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b4a8a64a7959a0168b4610/html5/thumbnails/12.jpg)
MotivationDesign Test
DesignWizardEvaluation
Conclusion
Example of Design Test
Facts about the code (an API)
Assertion routines
João Brunet 11/ 26
![Page 13: Traceability - Structural Conformance Checking with Design Tests: An Evaluation of Usability and Scalability](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b4a8a64a7959a0168b4610/html5/thumbnails/13.jpg)
MotivationDesign Test
DesignWizardEvaluation
Conclusion
DesignWizard
João Brunet 12/ 26
![Page 14: Traceability - Structural Conformance Checking with Design Tests: An Evaluation of Usability and Scalability](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b4a8a64a7959a0168b4610/html5/thumbnails/14.jpg)
MotivationDesign Test
DesignWizardEvaluation
Conclusion
Some methods of DesignWizard’s API
getAllPackages()
getAllSubClasses()
getStaticMethods()
getImplementedInterfaces()
...
João Brunet 13/ 26
![Page 15: Traceability - Structural Conformance Checking with Design Tests: An Evaluation of Usability and Scalability](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b4a8a64a7959a0168b4610/html5/thumbnails/15.jpg)
MotivationDesign Test
DesignWizardEvaluation
Conclusion
Process
João Brunet 14/ 26
![Page 16: Traceability - Structural Conformance Checking with Design Tests: An Evaluation of Usability and Scalability](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b4a8a64a7959a0168b4610/html5/thumbnails/16.jpg)
MotivationDesign Test
DesignWizardEvaluation
Conclusion
Process
João Brunet 15/ 26
![Page 17: Traceability - Structural Conformance Checking with Design Tests: An Evaluation of Usability and Scalability](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b4a8a64a7959a0168b4610/html5/thumbnails/17.jpg)
MotivationDesign Test
DesignWizardEvaluation
Conclusion
API Usability
Methodology
“It is the comparison between what developers expect and what theAPI provides that is interesting when evaluating the usability of an API(Steven Clarke)”
Think Aloud Protocol For APIs to capture expectations, goals,difficulties and strategies
João Brunet 16/ 26
![Page 18: Traceability - Structural Conformance Checking with Design Tests: An Evaluation of Usability and Scalability](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b4a8a64a7959a0168b4610/html5/thumbnails/18.jpg)
MotivationDesign Test
DesignWizardEvaluation
Conclusion
API Usability
Experiment
11 developers
5 real design tests (selected by the designer of the OurGrid)
OurGrid
Eclipse
API
Pseudocode vs. Code
João Brunet 17/ 26
![Page 19: Traceability - Structural Conformance Checking with Design Tests: An Evaluation of Usability and Scalability](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b4a8a64a7959a0168b4610/html5/thumbnails/19.jpg)
MotivationDesign Test
DesignWizardEvaluation
Conclusion
API Usability
Experiment
João Brunet 18/ 26
![Page 20: Traceability - Structural Conformance Checking with Design Tests: An Evaluation of Usability and Scalability](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b4a8a64a7959a0168b4610/html5/thumbnails/20.jpg)
MotivationDesign Test
DesignWizardEvaluation
Conclusion
API Usability
Experiment
Only one developer did not write one instance of design test.
João Brunet 19/ 26
![Page 21: Traceability - Structural Conformance Checking with Design Tests: An Evaluation of Usability and Scalability](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b4a8a64a7959a0168b4610/html5/thumbnails/21.jpg)
MotivationDesign Test
DesignWizardEvaluation
Conclusion
API Usability
Experiment
...but, 10 tests were incorrectly written. The good news is that alldevelopers found the right method, but used it incorrectly.
João Brunet 20/ 26
![Page 22: Traceability - Structural Conformance Checking with Design Tests: An Evaluation of Usability and Scalability](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b4a8a64a7959a0168b4610/html5/thumbnails/22.jpg)
MotivationDesign Test
DesignWizardEvaluation
Conclusion
API Usability
Results
Two weaknesses of API:
getAllClasses()
We decided to follow the Java Reflection API pattern
getDeclaredMethod (String name)
An appropriated name of the parameter would be signature
João Brunet 21/ 26
![Page 23: Traceability - Structural Conformance Checking with Design Tests: An Evaluation of Usability and Scalability](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b4a8a64a7959a0168b4610/html5/thumbnails/23.jpg)
MotivationDesign Test
DesignWizardEvaluation
Conclusion
API Usability
Results
Think Aloud Protocol revealed that devolopers had no difficultiesto find the right method to achieve a given task
Results suggested changes to DesignWizard API anddocumentation.
João Brunet 22/ 26
![Page 24: Traceability - Structural Conformance Checking with Design Tests: An Evaluation of Usability and Scalability](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b4a8a64a7959a0168b4610/html5/thumbnails/24.jpg)
MotivationDesign Test
DesignWizardEvaluation
Conclusion
API Usability
Threats to validity
Coding and designing expertise of developers
More complex design rules may produce different observations
The API was not fully evaluated
João Brunet 23/ 26
![Page 25: Traceability - Structural Conformance Checking with Design Tests: An Evaluation of Usability and Scalability](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b4a8a64a7959a0168b4610/html5/thumbnails/25.jpg)
MotivationDesign Test
DesignWizardEvaluation
Conclusion
Final Remarks
Approach to check conformance between code and design rules
Evaluation regarding its usability
Design Tests and maintenance
Executable documentation of design rulesEnables developers changing functionality keeping in mindstructural constraints
João Brunet 24/ 26
![Page 26: Traceability - Structural Conformance Checking with Design Tests: An Evaluation of Usability and Scalability](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b4a8a64a7959a0168b4610/html5/thumbnails/26.jpg)
MotivationDesign Test
DesignWizardEvaluation
Conclusion
Thank you
www.designwizard.org
João Brunet 25/ 26
![Page 27: Traceability - Structural Conformance Checking with Design Tests: An Evaluation of Usability and Scalability](https://reader034.fdocuments.in/reader034/viewer/2022042623/54b4a8a64a7959a0168b4610/html5/thumbnails/27.jpg)
MotivationDesign Test
DesignWizardEvaluation
Conclusion
Related Work
ArchJava
Reflexion Models
FindBugs
...
João Brunet 26/ 26