Research Heaven, West Virginia FY2003 Initiative: Hany Ammar, Mark Shereshevsky, Walid AbdelMoez,...
-
Upload
prudence-sherman -
Category
Documents
-
view
215 -
download
0
Transcript of Research Heaven, West Virginia FY2003 Initiative: Hany Ammar, Mark Shereshevsky, Walid AbdelMoez,...
![Page 1: Research Heaven, West Virginia FY2003 Initiative: Hany Ammar, Mark Shereshevsky, Walid AbdelMoez, Rajesh Gunnalan, and Ahmad Hassan LANE Department of.](https://reader036.fdocuments.in/reader036/viewer/2022070403/56649f325503460f94c4dead/html5/thumbnails/1.jpg)
Research Heaven,West Virginia
FY2003 Initiative:
Hany Ammar, Mark Shereshevsky,Walid AbdelMoez, Rajesh Gunnalan, and Ahmad Hassan
LANE Department of Computer Science and Electrical EngineeringWest Virginia University
The OSMA Software Assurance Symposium July 30, 2003Lakeview, WV
WVU UI: WVU UI: Quantitative Relations Between Static and Dynamic
Software Metrics
![Page 2: Research Heaven, West Virginia FY2003 Initiative: Hany Ammar, Mark Shereshevsky, Walid AbdelMoez, Rajesh Gunnalan, and Ahmad Hassan LANE Department of.](https://reader036.fdocuments.in/reader036/viewer/2022070403/56649f325503460f94c4dead/html5/thumbnails/2.jpg)
NASA OSMA SAS’03 2
Research Heaven,West Virginia
Two main views of software systems: The static view of the structure and composition,
The dynamic view during execution
• Static and dynamic properties of software have been widely studied in the literature.
![Page 3: Research Heaven, West Virginia FY2003 Initiative: Hany Ammar, Mark Shereshevsky, Walid AbdelMoez, Rajesh Gunnalan, and Ahmad Hassan LANE Department of.](https://reader036.fdocuments.in/reader036/viewer/2022070403/56649f325503460f94c4dead/html5/thumbnails/3.jpg)
NASA OSMA SAS’03 3
Research Heaven,West Virginia
So how good is static analysis?
• IV&V testing is mostly focused of static analysis– Software rarely executed at Fairmont.
• Why?– NASA software supports NASA hardware
– So, some software can’t run without its associated hardware
• Traditional view:– Dynamic execution more informative than
static analysis
• This research:– To what extent are static measure
surrogate for dynamic measures?
– Compare insights gained from static and dynamic measures
![Page 4: Research Heaven, West Virginia FY2003 Initiative: Hany Ammar, Mark Shereshevsky, Walid AbdelMoez, Rajesh Gunnalan, and Ahmad Hassan LANE Department of.](https://reader036.fdocuments.in/reader036/viewer/2022070403/56649f325503460f94c4dead/html5/thumbnails/4.jpg)
NASA OSMA SAS’03 4
Research Heaven,West Virginia
This talk
• Propose three hypotheses of possible correlations between selected static and dynamic metrics– Hypothesis I: Static coupling
metrics correlate with error propagation in software architectures.
– Hypothesis II: Static error propagation correlates with dynamic error propagation
– Hypothesis III: Change proneness” correlates with dynamic coupling of components
• Experiments to measure – static metrics
– dynamic metrics
– for selected case studies
• Statistical analysis of the data:– computing correlations
– developing linear or non-linear regression models.
• In summary:– Hypothesis I: not quite
supported– Hypothesis II: supported– Hypothesis III: rejected
![Page 5: Research Heaven, West Virginia FY2003 Initiative: Hany Ammar, Mark Shereshevsky, Walid AbdelMoez, Rajesh Gunnalan, and Ahmad Hassan LANE Department of.](https://reader036.fdocuments.in/reader036/viewer/2022070403/56649f325503460f94c4dead/html5/thumbnails/5.jpg)
NASA OSMA SAS’03 5
Research Heaven,West Virginia
Project Overview
FY03
• Conduct empirical study of the relationship between the static and dynamic metrics.
• Measure Static Metrics • Measure related Dynamic Metrics • Conduct statistical Analysis of the
Data
FY04
• Establish Relationships between Static and Dynamic Metrics
![Page 6: Research Heaven, West Virginia FY2003 Initiative: Hany Ammar, Mark Shereshevsky, Walid AbdelMoez, Rajesh Gunnalan, and Ahmad Hassan LANE Department of.](https://reader036.fdocuments.in/reader036/viewer/2022070403/56649f325503460f94c4dead/html5/thumbnails/6.jpg)
Research Heaven,West Virginia
Tools
![Page 7: Research Heaven, West Virginia FY2003 Initiative: Hany Ammar, Mark Shereshevsky, Walid AbdelMoez, Rajesh Gunnalan, and Ahmad Hassan LANE Department of.](https://reader036.fdocuments.in/reader036/viewer/2022070403/56649f325503460f94c4dead/html5/thumbnails/7.jpg)
NASA OSMA SAS’03 7
Research Heaven,West VirginiaTools for Static Metrics
• We can automatically collect several static metrics (OO metrics,Complexity metrics and Size/Volume metrics).
• We are currently using Understand for Java and C++ by Scientific Toolworks for collecting static metrics.
![Page 8: Research Heaven, West Virginia FY2003 Initiative: Hany Ammar, Mark Shereshevsky, Walid AbdelMoez, Rajesh Gunnalan, and Ahmad Hassan LANE Department of.](https://reader036.fdocuments.in/reader036/viewer/2022070403/56649f325503460f94c4dead/html5/thumbnails/8.jpg)
NASA OSMA SAS’03 8
Research Heaven,West Virginia
Tools for Dynamic Metrics
• Rational Rose-Realtime simulation facility is used to simulate UML models
• At the code level, Rational Purifyplus produces dynamic sequence diagrams at run-time
• JProbe profiler by Quest Software serves a similar purpose for Java code but does not produce sequence diagrams.
![Page 9: Research Heaven, West Virginia FY2003 Initiative: Hany Ammar, Mark Shereshevsky, Walid AbdelMoez, Rajesh Gunnalan, and Ahmad Hassan LANE Department of.](https://reader036.fdocuments.in/reader036/viewer/2022070403/56649f325503460f94c4dead/html5/thumbnails/9.jpg)
Research Heaven,West Virginia
Hypothesis I
Static coupling metrics correlate with error propagation in software architectures.
![Page 10: Research Heaven, West Virginia FY2003 Initiative: Hany Ammar, Mark Shereshevsky, Walid AbdelMoez, Rajesh Gunnalan, and Ahmad Hassan LANE Department of.](https://reader036.fdocuments.in/reader036/viewer/2022070403/56649f325503460f94c4dead/html5/thumbnails/10.jpg)
NASA OSMA SAS’03 10
Research Heaven,West Virginia
Study of Static Coupling Metrics and Error Propagation
• Correlate the experimental data on – error propagation – static information coupling (connector based).
• Correlate:– experimental error propagation – and static NAS (number of associations) coupling measure
(connector Based).
• Correlate:– experimental error propagation – and CBO (connector based).
![Page 11: Research Heaven, West Virginia FY2003 Initiative: Hany Ammar, Mark Shereshevsky, Walid AbdelMoez, Rajesh Gunnalan, and Ahmad Hassan LANE Department of.](https://reader036.fdocuments.in/reader036/viewer/2022070403/56649f325503460f94c4dead/html5/thumbnails/11.jpg)
NASA OSMA SAS’03 11
Research Heaven,West Virginia
![Page 12: Research Heaven, West Virginia FY2003 Initiative: Hany Ammar, Mark Shereshevsky, Walid AbdelMoez, Rajesh Gunnalan, and Ahmad Hassan LANE Department of.](https://reader036.fdocuments.in/reader036/viewer/2022070403/56649f325503460f94c4dead/html5/thumbnails/12.jpg)
NASA OSMA SAS’03 12
Research Heaven,West Virginia
Table Highlighting Non-zero Values Information Flow Vs Ep
From To1 1 5 0.1331 2.3219 10672 1 6 0.128 2.3219 10553 1 3 0.0557 2 5924 3 1 0.0161 1.585 5595 7 1 0.75 2.3219 646 1 4 0.4912 2 577 2 1 0.7838 2.8074 378 1 2 0.5 2.585 369 4 2 0.6429 2 28
10 4 1 0.7083 2 2411 3 2 0.7917 1.585 2412 2 8 0.4286 2 713 10 1 1 1 414 6 2 1 1 415 5 2 1 1 4
EntryConnector EP
(Dynamic)# Fault
InjectionsInformation Flow(Stat)
![Page 13: Research Heaven, West Virginia FY2003 Initiative: Hany Ammar, Mark Shereshevsky, Walid AbdelMoez, Rajesh Gunnalan, and Ahmad Hassan LANE Department of.](https://reader036.fdocuments.in/reader036/viewer/2022070403/56649f325503460f94c4dead/html5/thumbnails/13.jpg)
NASA OSMA SAS’03 13
Research Heaven,West Virginia
Table Highlighting Non-zero Values CBO Vs Ep
From To1 1 5 0.1331 16 10672 1 6 0.128 16 10553 1 3 0.0557 15 5924 3 1 0.0161 3 5595 7 1 0.75 5 646 1 4 0.4912 15 577 2 1 0.7838 31 378 1 2 0.5 6 369 4 2 0.6429 4 28
10 4 1 0.7083 3 2411 3 2 0.7917 4 2412 2 8 0.4286 14 713 10 1 1 3 414 6 2 1 2 415 5 2 1 2 4
EntryConnector EP
(Dynamic)# Fault
InjectionsCBO
![Page 14: Research Heaven, West Virginia FY2003 Initiative: Hany Ammar, Mark Shereshevsky, Walid AbdelMoez, Rajesh Gunnalan, and Ahmad Hassan LANE Department of.](https://reader036.fdocuments.in/reader036/viewer/2022070403/56649f325503460f94c4dead/html5/thumbnails/14.jpg)
NASA OSMA SAS’03 14
Research Heaven,West Virginia
Correlation Values
• Correlation between Information Flow Coupling and Error Propagation is 0.545657
• Correlation between CBO and Error Propagation is 0.130478
• Correlation between NAS and Error Propagation is 0.0• The correlations are only for the values where the # fault
injections >25.
![Page 15: Research Heaven, West Virginia FY2003 Initiative: Hany Ammar, Mark Shereshevsky, Walid AbdelMoez, Rajesh Gunnalan, and Ahmad Hassan LANE Department of.](https://reader036.fdocuments.in/reader036/viewer/2022070403/56649f325503460f94c4dead/html5/thumbnails/15.jpg)
NASA OSMA SAS’03 15
Research Heaven,West Virginia
Discussion of Results
• Information flow coupling metric shows a higher correlation with dynamic error propagation than CBO.
• On further analysis the R2 correlation between Information Flow and EP was 0.297742.
• This low correlation value could be partly due to the small size of our sample (9 data points)
• This result is not sufficient to validate the Hypothesis I
![Page 16: Research Heaven, West Virginia FY2003 Initiative: Hany Ammar, Mark Shereshevsky, Walid AbdelMoez, Rajesh Gunnalan, and Ahmad Hassan LANE Department of.](https://reader036.fdocuments.in/reader036/viewer/2022070403/56649f325503460f94c4dead/html5/thumbnails/16.jpg)
Research Heaven,West Virginia
Hypothesis II:
Static error propagation correlates with dynamic error propagation
![Page 17: Research Heaven, West Virginia FY2003 Initiative: Hany Ammar, Mark Shereshevsky, Walid AbdelMoez, Rajesh Gunnalan, and Ahmad Hassan LANE Department of.](https://reader036.fdocuments.in/reader036/viewer/2022070403/56649f325503460f94c4dead/html5/thumbnails/17.jpg)
NASA OSMA SAS’03 17
Research Heaven,West Virginia
Static and Dynamic Error Propagation
• We will use the experimental data on error propagation and the static error propagation measure(connector Based).
• The static error propagation measure is based on the relation developed in the software architecture metrics USIP using the information flow between components.
![Page 18: Research Heaven, West Virginia FY2003 Initiative: Hany Ammar, Mark Shereshevsky, Walid AbdelMoez, Rajesh Gunnalan, and Ahmad Hassan LANE Department of.](https://reader036.fdocuments.in/reader036/viewer/2022070403/56649f325503460f94c4dead/html5/thumbnails/18.jpg)
NASA OSMA SAS’03 18
Research Heaven,West Virginia
Experimental (Dynamic) Error Propagation Matrix of HCS
C1 C2 C3 C4 C5 C6 C7 C8 C9 C10C1 1 0.11 0.15 0.3 0.76 0.73 0 0 0 0C2 0.37 1 0 0 0 0 0 0.52 0 0C3 0.03 0.32 1 0 0 0 0 0 0 0C4 0.03 0.19 0 1 0 0 0 0 0 0C5 0 0.67 0 0 1 0 0 0 0 0C6 0 0.28 0 0 0 1 0 0 0 0C7 0.32 0 0 0 0 0 1 0 0 0C8 0 0 0 0 0 0 0 1 0 0C9 0 0 0 0 0 0 0 0 1 0
C10 0 0 0 0 0 0 0 0 0 1
B
A
![Page 19: Research Heaven, West Virginia FY2003 Initiative: Hany Ammar, Mark Shereshevsky, Walid AbdelMoez, Rajesh Gunnalan, and Ahmad Hassan LANE Department of.](https://reader036.fdocuments.in/reader036/viewer/2022070403/56649f325503460f94c4dead/html5/thumbnails/19.jpg)
NASA OSMA SAS’03 19
Research Heaven,West Virginia
Static Error Propagation Matrix of HCS
C1 C2 C3 C4 C5 C6 C7 C8 C9 C10C1 1 0.11 0.38 0.44 0.58 0.59 0 0 0 0C2 0.18 1 0 0 0 0 0 0.5 0 0C3 0.11 0.26 1 0 0 0 0 0 0 0C4 0.17 0.15 0 1 0 0 0 0 0 0C5 0 0.57 0 0 1 0 0 0 0 0C6 0 0.18 0 0 0 1 0 0 0 0C7 0.23 0 0 0 0 0 1 0 0 0C8 0 0 0 0 0 0 0 1 0 0C9 0 0 0 0 0 0 0 0 1 0
C10 0.08 0 0 0 0 0 0 0 0 1
B
A
![Page 20: Research Heaven, West Virginia FY2003 Initiative: Hany Ammar, Mark Shereshevsky, Walid AbdelMoez, Rajesh Gunnalan, and Ahmad Hassan LANE Department of.](https://reader036.fdocuments.in/reader036/viewer/2022070403/56649f325503460f94c4dead/html5/thumbnails/20.jpg)
NASA OSMA SAS’03 20
Research Heaven,West Virginia
Correlation Results
• There is a strong correlation between static error propagation and dynamic error propagation.
• Correlation = 0.875098• R2 = 0.765797• This supports Hypothesis II and our conjecture that
some static metrics correlate with related dynamic metrics.
![Page 21: Research Heaven, West Virginia FY2003 Initiative: Hany Ammar, Mark Shereshevsky, Walid AbdelMoez, Rajesh Gunnalan, and Ahmad Hassan LANE Department of.](https://reader036.fdocuments.in/reader036/viewer/2022070403/56649f325503460f94c4dead/html5/thumbnails/21.jpg)
Research Heaven,West Virginia
Hypothesis III:
“Change proneness” correlates with dynamic coupling of components
![Page 22: Research Heaven, West Virginia FY2003 Initiative: Hany Ammar, Mark Shereshevsky, Walid AbdelMoez, Rajesh Gunnalan, and Ahmad Hassan LANE Department of.](https://reader036.fdocuments.in/reader036/viewer/2022070403/56649f325503460f94c4dead/html5/thumbnails/22.jpg)
NASA OSMA SAS’03 22
Research Heaven,West Virginia
Change Proneness and Change Propagation
• Hypothesis III “change proneness correlates with dynamic coupling of components”– from the study of Erik Arisholm June 2002, – Dynamic Coupling Measures for Object-Oriented Software,
– Eighth IEEE Symposium on Software Metrics
• Correlate – dynamic import coupling, – dynamic export coupling metrics – with change proneness measured – based on the software architecture metrics USIP using interface
change propagation probabilities.
![Page 23: Research Heaven, West Virginia FY2003 Initiative: Hany Ammar, Mark Shereshevsky, Walid AbdelMoez, Rajesh Gunnalan, and Ahmad Hassan LANE Department of.](https://reader036.fdocuments.in/reader036/viewer/2022070403/56649f325503460f94c4dead/html5/thumbnails/23.jpg)
NASA OSMA SAS’03 23
Research Heaven,West Virginia
Dynamic Coupling Metrics
• The dynamic coupling metrics we focus on are as follows1. Dynamic Export Coupling
2. Dynamic Import Coupling
3. Total Dynamic Coupling sum of both import and export coupling.
![Page 24: Research Heaven, West Virginia FY2003 Initiative: Hany Ammar, Mark Shereshevsky, Walid AbdelMoez, Rajesh Gunnalan, and Ahmad Hassan LANE Department of.](https://reader036.fdocuments.in/reader036/viewer/2022070403/56649f325503460f94c4dead/html5/thumbnails/24.jpg)
NASA OSMA SAS’03 24
Research Heaven,West Virginia
Experiment
• The experiment was conducted on the SharpTool case study.with 32 components.
• Dynamic coupling was obtained by counting the object interactions dynamically at run time. During the run all functionalities of the tool were executed.
![Page 25: Research Heaven, West Virginia FY2003 Initiative: Hany Ammar, Mark Shereshevsky, Walid AbdelMoez, Rajesh Gunnalan, and Ahmad Hassan LANE Department of.](https://reader036.fdocuments.in/reader036/viewer/2022070403/56649f325503460f94c4dead/html5/thumbnails/25.jpg)
NASA OSMA SAS’03 25
Research Heaven,West Virginia
Experiment Results
Export Import Proneness18 14 0.2
39466 0 0.3757317 20 07182 7179 3.4
75 2 10 0 10 0 1
78 0 2.523 97 0.510 48 08 3 19 25 0.9591 1 03 5 0
23 78 0.85718 14 0.734
123 151 0.56 9 0.151
92 0 0.677 0 25 0 19 5 01 0 0
374 32893 1.541 95 053 3 0.5411 44 0.603
153 15208 0.75181 215 013 5 0.42 158 0.8889 60 1.142
![Page 26: Research Heaven, West Virginia FY2003 Initiative: Hany Ammar, Mark Shereshevsky, Walid AbdelMoez, Rajesh Gunnalan, and Ahmad Hassan LANE Department of.](https://reader036.fdocuments.in/reader036/viewer/2022070403/56649f325503460f94c4dead/html5/thumbnails/26.jpg)
NASA OSMA SAS’03 26
Research Heaven,West Virginia
Discussion of Results
• Zero correlation between – change proneness – and dynamic coupling.
• We also computed CBO and found zero correlation between change proneness and CBO
• This negates Hypothesis III.• The study in the paper by Erik Arisholm was done
using actual project changes data .• In our experiment we used static change proneness data
using our interface change propagation probabilities defined in the architecture metrics USIP
![Page 27: Research Heaven, West Virginia FY2003 Initiative: Hany Ammar, Mark Shereshevsky, Walid AbdelMoez, Rajesh Gunnalan, and Ahmad Hassan LANE Department of.](https://reader036.fdocuments.in/reader036/viewer/2022070403/56649f325503460f94c4dead/html5/thumbnails/27.jpg)
Research Heaven,West Virginia
![Page 28: Research Heaven, West Virginia FY2003 Initiative: Hany Ammar, Mark Shereshevsky, Walid AbdelMoez, Rajesh Gunnalan, and Ahmad Hassan LANE Department of.](https://reader036.fdocuments.in/reader036/viewer/2022070403/56649f325503460f94c4dead/html5/thumbnails/28.jpg)
NASA OSMA SAS’03 28
Research Heaven,West Virginia
So how good is static analysis?
• Hypothesis I: Static coupling metrics correlate with error propagation in software architectures.– Not Sure
• Hypothesis II: Static error propagation correlates with dynamic error propagation– Yes
• Hypothesis III: Change proneness” correlates with dynamic coupling of components– No
• And so, what does all this mean? – IV&V can not run code – And still offer value added to NASA
systems