Software Metrics and Measurements
description
Transcript of Software Metrics and Measurements
![Page 1: Software Metrics and Measurements](https://reader035.fdocuments.in/reader035/viewer/2022062501/568168cd550346895ddfbae2/html5/thumbnails/1.jpg)
University of Southern California
Center for Systems and Software Engineering
Software Metrics and Measurements
Supannika KoolmanojwongCS577
1
![Page 2: Software Metrics and Measurements](https://reader035.fdocuments.in/reader035/viewer/2022062501/568168cd550346895ddfbae2/html5/thumbnails/2.jpg)
University of Southern California
Center for Systems and Software Engineering
Outline
• General Concepts about Metrics• Example of Metrics• Agile Metrics• Case Studies
– Metrics from Empirical Data – Northrop Grumman
– Metrics in IT services – Lockheed Martin
2
![Page 3: Software Metrics and Measurements](https://reader035.fdocuments.in/reader035/viewer/2022062501/568168cd550346895ddfbae2/html5/thumbnails/3.jpg)
University of Southern California
Center for Systems and Software Engineering
Measurements in daily life
3
![Page 4: Software Metrics and Measurements](https://reader035.fdocuments.in/reader035/viewer/2022062501/568168cd550346895ddfbae2/html5/thumbnails/4.jpg)
University of Southern California
Center for Systems and Software Engineering
Why do we measure?
4
![Page 5: Software Metrics and Measurements](https://reader035.fdocuments.in/reader035/viewer/2022062501/568168cd550346895ddfbae2/html5/thumbnails/5.jpg)
University of Southern California
Center for Systems and Software Engineering
5http://www.teamqualitypro.com/software-metrics/do-you-know-your-abcs-of-software-metrics/
![Page 6: Software Metrics and Measurements](https://reader035.fdocuments.in/reader035/viewer/2022062501/568168cd550346895ddfbae2/html5/thumbnails/6.jpg)
University of Southern California
Center for Systems and Software Engineering
Objectives of software measurement
• “You can not control what you cannot measure.” – Tom DeMarco
• “Not everything that counts can be counted. Not everything that is counted counts.” – Albert Einstein
6
![Page 7: Software Metrics and Measurements](https://reader035.fdocuments.in/reader035/viewer/2022062501/568168cd550346895ddfbae2/html5/thumbnails/7.jpg)
University of Southern California
Center for Systems and Software Engineering
Software Metrics
• Numerical data related to software development
• Strongly support software project management activities
• Can be directly observable quantities or can be derived from one
7
![Page 8: Software Metrics and Measurements](https://reader035.fdocuments.in/reader035/viewer/2022062501/568168cd550346895ddfbae2/html5/thumbnails/8.jpg)
University of Southern California
Center for Systems and Software Engineering
A simplified measurement information model
8Ref: Ebert and Dumke 2007
Decisions / Actions Measurements
Process Work Products
Information products
Information needs
Attributes
Results
Information Needs,
Objectives, Control
![Page 9: Software Metrics and Measurements](https://reader035.fdocuments.in/reader035/viewer/2022062501/568168cd550346895ddfbae2/html5/thumbnails/9.jpg)
University of Southern California
Center for Systems and Software Engineering
How the software measurements are used?
• Understand and communicate• Specify and achieve objectives• Identify and resolve problems• Decide and Improve
9
![Page 10: Software Metrics and Measurements](https://reader035.fdocuments.in/reader035/viewer/2022062501/568168cd550346895ddfbae2/html5/thumbnails/10.jpg)
University of Southern California
Center for Systems and Software Engineering
Measurement Standard
10
ISO/IEC 12207 Software Life Cycle Processes
ISO/IEC 15288 System Life Cycle processes
SWEBOKSoftware Engineering Body of Knowledge
PMBOKProject Management Body of Knowledge
CMMICapability Maturity Model Integration
ISO 15504Software Process Capability Determination
ISO 9001Quality Management System
ISO/IEC 9126Software Product QualityTL 9000, AS 9100, etc.Objectives adaptations
How to do How to do better
ISO/IEC 15939:2002Software Measurement Process
How to measure what you are doing
![Page 11: Software Metrics and Measurements](https://reader035.fdocuments.in/reader035/viewer/2022062501/568168cd550346895ddfbae2/html5/thumbnails/11.jpg)
University of Southern California
Center for Systems and Software Engineering
Ground rules for a Metrics• Metrics must be
– Understandable to be useful– Economical– Field tested– Highly leveraged– Timely– Must give proper incentives for process improvement– Evenly spaced throughout all phases of development– Useful at multiple levels
11http://www.stsc.hill.af.mil/resources/tech_docs/gsam3/chap13.pdf
![Page 12: Software Metrics and Measurements](https://reader035.fdocuments.in/reader035/viewer/2022062501/568168cd550346895ddfbae2/html5/thumbnails/12.jpg)
University of Southern California
Center for Systems and Software Engineering
Measurements for Senior Management
• Easy and reliable visibility of business performance
• Forecasts and indicators where action is needed
• Drill-down into underlying information and commitments
• Flexible resource refocus
12
![Page 13: Software Metrics and Measurements](https://reader035.fdocuments.in/reader035/viewer/2022062501/568168cd550346895ddfbae2/html5/thumbnails/13.jpg)
University of Southern California
Center for Systems and Software Engineering
13
![Page 14: Software Metrics and Measurements](https://reader035.fdocuments.in/reader035/viewer/2022062501/568168cd550346895ddfbae2/html5/thumbnails/14.jpg)
University of Southern California
Center for Systems and Software Engineering
Measurements for Project Management
• Immediate project reviews• Status and forecasts for quality, schedule,
and budget• Follow-up action points• Report based on consistent raw data
14
![Page 15: Software Metrics and Measurements](https://reader035.fdocuments.in/reader035/viewer/2022062501/568168cd550346895ddfbae2/html5/thumbnails/15.jpg)
University of Southern California
Center for Systems and Software Engineering
Metrics for Senior Management
15
![Page 16: Software Metrics and Measurements](https://reader035.fdocuments.in/reader035/viewer/2022062501/568168cd550346895ddfbae2/html5/thumbnails/16.jpg)
University of Southern California
Center for Systems and Software Engineering
Measurements for Engineers
• Immediate access to team planning and progress
• Get visibility into own performance and how it can be improved
• Indicators that show weak spots in deliverables
• Focus energy on software development
17
![Page 17: Software Metrics and Measurements](https://reader035.fdocuments.in/reader035/viewer/2022062501/568168cd550346895ddfbae2/html5/thumbnails/17.jpg)
University of Southern California
Center for Systems and Software Engineering
18http://www.ibm.com/developerworks/rational/library/customized-reports-rational-team-concert/
![Page 18: Software Metrics and Measurements](https://reader035.fdocuments.in/reader035/viewer/2022062501/568168cd550346895ddfbae2/html5/thumbnails/18.jpg)
University of Southern California
Center for Systems and Software Engineering
The E4-Measurement Process
19
Objectives, needs
Business Process
Environment, resources
1. Establish 2. Extract 3. Evaluate 4. Execute
Decisions, re-direction,
updated plans
Ref: Ebert and Dumke 2007
![Page 19: Software Metrics and Measurements](https://reader035.fdocuments.in/reader035/viewer/2022062501/568168cd550346895ddfbae2/html5/thumbnails/19.jpg)
University of Southern California
Center for Systems and Software Engineering
Aggregation of informationEnterprise
Division
Product Line/ Department
Projects
20
Cycle time, quality, cost, productivity, customer
satisfaction, resources, skills
Sales, cost reduction, innovative products, level of
customization
Cost reduction, Sales, margins, Customer Service
Cash flow, Shareholder value, Operations cost
![Page 20: Software Metrics and Measurements](https://reader035.fdocuments.in/reader035/viewer/2022062501/568168cd550346895ddfbae2/html5/thumbnails/20.jpg)
University of Southern California
Center for Systems and Software Engineering
SMART goals
• Specific - precise• Measurable - tangible• Accountable – in line with individual responsibilities
• Realistic - achievable• Timely – suitable for the current needs
21
![Page 21: Software Metrics and Measurements](https://reader035.fdocuments.in/reader035/viewer/2022062501/568168cd550346895ddfbae2/html5/thumbnails/21.jpg)
University of Southern California
Center for Systems and Software Engineering
Components of software measurements
23
![Page 22: Software Metrics and Measurements](https://reader035.fdocuments.in/reader035/viewer/2022062501/568168cd550346895ddfbae2/html5/thumbnails/22.jpg)
University of Southern California
Center for Systems and Software Engineering
Example of Metrics
• Progress / Effort / Cost Indicator• Earned value management• Requirements / Code Churn• Defect-related metrics• Test-related metrics
24
![Page 23: Software Metrics and Measurements](https://reader035.fdocuments.in/reader035/viewer/2022062501/568168cd550346895ddfbae2/html5/thumbnails/23.jpg)
University of Southern California
Center for Systems and Software Engineering
Size
• How big is the healthcare.gov website? – http://www.informationisbeautiful.net/
visualizations/million-lines-of-code/
25
![Page 24: Software Metrics and Measurements](https://reader035.fdocuments.in/reader035/viewer/2022062501/568168cd550346895ddfbae2/html5/thumbnails/24.jpg)
University of Southern California
Center for Systems and Software Engineering
Size• Earth System Modeling Framework Project
26http://www.earthsystemmodeling.org/metrics/sloc.shtml
![Page 25: Software Metrics and Measurements](https://reader035.fdocuments.in/reader035/viewer/2022062501/568168cd550346895ddfbae2/html5/thumbnails/25.jpg)
University of Southern California
Center for Systems and Software Engineering
Progress Indicator
27
![Page 26: Software Metrics and Measurements](https://reader035.fdocuments.in/reader035/viewer/2022062501/568168cd550346895ddfbae2/html5/thumbnails/26.jpg)
University of Southern California
Center for Systems and Software Engineering
Effort Indicator
28
![Page 27: Software Metrics and Measurements](https://reader035.fdocuments.in/reader035/viewer/2022062501/568168cd550346895ddfbae2/html5/thumbnails/27.jpg)
University of Southern California
Center for Systems and Software Engineering
Cost Indicator
29
![Page 28: Software Metrics and Measurements](https://reader035.fdocuments.in/reader035/viewer/2022062501/568168cd550346895ddfbae2/html5/thumbnails/28.jpg)
University of Southern California
Center for Systems and Software Engineering
Earned value management• Planned Value (PV) or Budgeted Cost of Work Scheduled (BCWS)• Earned Value (EV) or Budgeted Cost of Work Performed (BCWP)
30http://en.wikipedia.org/wiki/Earned_value_management
![Page 29: Software Metrics and Measurements](https://reader035.fdocuments.in/reader035/viewer/2022062501/568168cd550346895ddfbae2/html5/thumbnails/29.jpg)
University of Southern California
Center for Systems and Software Engineering
Burndown Chart
31http://en.wikipedia.org/wiki/Burn_down_chart
![Page 30: Software Metrics and Measurements](https://reader035.fdocuments.in/reader035/viewer/2022062501/568168cd550346895ddfbae2/html5/thumbnails/30.jpg)
University of Southern California
Center for Systems and Software Engineering
Requirements Churn/ Requirements Creep/
Requirements Volatility• number of changes to system requirements
in each phase/week/increment
32
![Page 31: Software Metrics and Measurements](https://reader035.fdocuments.in/reader035/viewer/2022062501/568168cd550346895ddfbae2/html5/thumbnails/31.jpg)
University of Southern California
Center for Systems and Software Engineering
Code Churn• Software change history• Large / recent changes• Total added, modified and deleted LOC• Number of times that a binary was edited• Number of consecutive edits
33
![Page 32: Software Metrics and Measurements](https://reader035.fdocuments.in/reader035/viewer/2022062501/568168cd550346895ddfbae2/html5/thumbnails/32.jpg)
University of Southern California
Center for Systems and Software Engineering
Code Complexity
34
• Gathered from code itself• Multiple complexity values• Cyclomatic complexity• Fan-In / Fan-Out of functions• Lines of Code• Weighted methods per class• Depth of Inheritance• Coupling between objects• Number of subclasses• Total global variables
![Page 33: Software Metrics and Measurements](https://reader035.fdocuments.in/reader035/viewer/2022062501/568168cd550346895ddfbae2/html5/thumbnails/33.jpg)
University of Southern California
Center for Systems and Software Engineering
Code coverage
• Degree to which the source code is tested• Statement coverage
– Has each node in the program been executed?• Branch coverage
– Has each control structure been evaluated both to true and false?
35
![Page 34: Software Metrics and Measurements](https://reader035.fdocuments.in/reader035/viewer/2022062501/568168cd550346895ddfbae2/html5/thumbnails/34.jpg)
University of Southern California
Center for Systems and Software Engineering
Code Coverage
36http://www.firstlinesoftware.com/metrics_group2.html
![Page 35: Software Metrics and Measurements](https://reader035.fdocuments.in/reader035/viewer/2022062501/568168cd550346895ddfbae2/html5/thumbnails/35.jpg)
University of Southern California
Center for Systems and Software Engineering
JUnit Code CoverageTool instruments byte code with extra code to measure
which statements are and are not reached.
37
Package Level Converage
A Code Coverage Report
Line Level Coverage
http://www.cafeaulait.org/slides/albany/codecoverage/Measuring_JUnit_Code_Coverage.html
![Page 36: Software Metrics and Measurements](https://reader035.fdocuments.in/reader035/viewer/2022062501/568168cd550346895ddfbae2/html5/thumbnails/36.jpg)
University of Southern California
Center for Systems and Software Engineering
Defect reporting metric
• Can be categorized by – Status
• Remaining / Resolved / Found– Defect Sources
• Requirements / Design / Development– Defect found
• Peer review / unit testing / sanity check– Time
• Defect arrival rate / Defect age
38
![Page 37: Software Metrics and Measurements](https://reader035.fdocuments.in/reader035/viewer/2022062501/568168cd550346895ddfbae2/html5/thumbnails/37.jpg)
University of Southern California
Center for Systems and Software Engineering
Defect Status
39
![Page 38: Software Metrics and Measurements](https://reader035.fdocuments.in/reader035/viewer/2022062501/568168cd550346895ddfbae2/html5/thumbnails/38.jpg)
University of Southern California
Center for Systems and Software Engineering
Defect Density
40
![Page 39: Software Metrics and Measurements](https://reader035.fdocuments.in/reader035/viewer/2022062501/568168cd550346895ddfbae2/html5/thumbnails/39.jpg)
University of Southern California
Center for Systems and Software Engineering
Test Pass Coverage
41http://www.jrothman.com/Papers/QW96.html
![Page 40: Software Metrics and Measurements](https://reader035.fdocuments.in/reader035/viewer/2022062501/568168cd550346895ddfbae2/html5/thumbnails/40.jpg)
University of Southern California
Center for Systems and Software Engineering
Defect Density
42
![Page 41: Software Metrics and Measurements](https://reader035.fdocuments.in/reader035/viewer/2022062501/568168cd550346895ddfbae2/html5/thumbnails/41.jpg)
University of Southern California
Center for Systems and Software Engineering
Defect Per LOC
43
![Page 42: Software Metrics and Measurements](https://reader035.fdocuments.in/reader035/viewer/2022062501/568168cd550346895ddfbae2/html5/thumbnails/42.jpg)
University of Southern California
Center for Systems and Software Engineering
Developer Code Review
44
After 60‒90 minutes, our ability to find defects drops off precipitously
http://answers.oreilly.com/topic/2265-best-practices-for-developer-code-review/
![Page 43: Software Metrics and Measurements](https://reader035.fdocuments.in/reader035/viewer/2022062501/568168cd550346895ddfbae2/html5/thumbnails/43.jpg)
University of Southern California
Center for Systems and Software Engineering
45
As the size of the code under review increases, our ability to find all the defects decreases. Don’t review more than 400 lines of code at a time.
http://answers.oreilly.com/topic/2265-best-practices-for-developer-code-review/
![Page 44: Software Metrics and Measurements](https://reader035.fdocuments.in/reader035/viewer/2022062501/568168cd550346895ddfbae2/html5/thumbnails/44.jpg)
University of Southern California
Center for Systems and Software Engineering
Top 6 Agile Metrics
46Ref: Measuring Agility, Peter Behrens
![Page 45: Software Metrics and Measurements](https://reader035.fdocuments.in/reader035/viewer/2022062501/568168cd550346895ddfbae2/html5/thumbnails/45.jpg)
University of Southern California
Center for Systems and Software Engineering
47Ref: Measuring Agility, Peter Behrens
Velocity = Work Completed per sprint
![Page 46: Software Metrics and Measurements](https://reader035.fdocuments.in/reader035/viewer/2022062501/568168cd550346895ddfbae2/html5/thumbnails/46.jpg)
University of Southern California
Center for Systems and Software Engineering
48
![Page 47: Software Metrics and Measurements](https://reader035.fdocuments.in/reader035/viewer/2022062501/568168cd550346895ddfbae2/html5/thumbnails/47.jpg)
University of Southern California
Center for Systems and Software Engineering
Case studies
• Metrics from Empirical Data – Northrop Grumman
• Metrics in IT services – Lockheed Martin
49
![Page 48: Software Metrics and Measurements](https://reader035.fdocuments.in/reader035/viewer/2022062501/568168cd550346895ddfbae2/html5/thumbnails/48.jpg)
University of Southern California
Center for Systems and Software Engineering
50Richard W. Selby, Northrop Grumman Space Technology, ICSP '09Title: "Synthesis, Analysis, and Modeling of Large-Scale Mission-Critical Embedded Software Systems"
![Page 49: Software Metrics and Measurements](https://reader035.fdocuments.in/reader035/viewer/2022062501/568168cd550346895ddfbae2/html5/thumbnails/49.jpg)
University of Southern California
Center for Systems and Software Engineering
51
![Page 50: Software Metrics and Measurements](https://reader035.fdocuments.in/reader035/viewer/2022062501/568168cd550346895ddfbae2/html5/thumbnails/50.jpg)
University of Southern California
Center for Systems and Software Engineering
52
![Page 51: Software Metrics and Measurements](https://reader035.fdocuments.in/reader035/viewer/2022062501/568168cd550346895ddfbae2/html5/thumbnails/51.jpg)
University of Southern California
Center for Systems and Software Engineering
Case studiesLockheed Martin Consolidated InformationTechnology Infrastructure Contract (CITIC)
• Customer: Centers for Medicare & Medicaid Services (CMS)– day-to-day IT Operations & Maintenance (O&M) functions
(high availability – 24/7)• Service Desk, Mainframe Support (Tier 1), Mid-Tier
Support (Tier 2), Desktop Support (Tier 3), Voice, Data, NOC, etc.
• Performance, Request, Incident, Configuration, Asset, and Change management
• Security and Disaster Recovery– modernization of the IT infrastructure
53http://www.psmsc.com/UsersGroup2012.asp
![Page 52: Software Metrics and Measurements](https://reader035.fdocuments.in/reader035/viewer/2022062501/568168cd550346895ddfbae2/html5/thumbnails/52.jpg)
University of Southern California
Center for Systems and Software Engineering
Service Measurement Plans & SLAs
54http://www.psmsc.com/UsersGroup2012.asp
![Page 53: Software Metrics and Measurements](https://reader035.fdocuments.in/reader035/viewer/2022062501/568168cd550346895ddfbae2/html5/thumbnails/53.jpg)
University of Southern California
Center for Systems and Software Engineering
55
![Page 54: Software Metrics and Measurements](https://reader035.fdocuments.in/reader035/viewer/2022062501/568168cd550346895ddfbae2/html5/thumbnails/54.jpg)
University of Southern California
Center for Systems and Software Engineering
56
![Page 55: Software Metrics and Measurements](https://reader035.fdocuments.in/reader035/viewer/2022062501/568168cd550346895ddfbae2/html5/thumbnails/55.jpg)
University of Southern California
Center for Systems and Software Engineering
57
![Page 56: Software Metrics and Measurements](https://reader035.fdocuments.in/reader035/viewer/2022062501/568168cd550346895ddfbae2/html5/thumbnails/56.jpg)
University of Southern California
Center for Systems and Software Engineering
58
![Page 57: Software Metrics and Measurements](https://reader035.fdocuments.in/reader035/viewer/2022062501/568168cd550346895ddfbae2/html5/thumbnails/57.jpg)
University of Southern California
Center for Systems and Software Engineering
Measurements for progress vs predictionsProject Phase For Measurements For PredictionsProject Management
-Effort and Budget Tracking- Requirements Status-Task Status
-Top 10 risks- Cost to complete- Schedule evolution
Quality Management
-Code Stability-Open defects-Review status and follow up
-Residual defects-Reliability-Customer satisfaction
Requirements Management
-Analysis status -Specification progress
-Requirements volatility / completeness
Construction -Status of documents- Change requests-Review status
-Design progress of reqm- Cost to complete-Time to complete
Test Test progress (defects, coverage, efficiency, stability
- Residual defects- reliability
Transition, deployment
-Field performance (failure, corrections) - maintenance effort
-Reliability-Maintenance effort
59Ref: Ebert and Dumke, 2007
![Page 58: Software Metrics and Measurements](https://reader035.fdocuments.in/reader035/viewer/2022062501/568168cd550346895ddfbae2/html5/thumbnails/58.jpg)
University of Southern California
Center for Systems and Software Engineering
Recommended books
60
Software Measurement: Establish - Extract - Evaluate – Execute by Christof Ebert, Reiner Dumke (2010)
Practical Software Measurement: Objective Information for Decision Makers by John McGarry, David Card, Cheryl Jones and Beth Layman (Oct 27, 2001)
![Page 59: Software Metrics and Measurements](https://reader035.fdocuments.in/reader035/viewer/2022062501/568168cd550346895ddfbae2/html5/thumbnails/59.jpg)
University of Southern California
Center for Systems and Software Engineering
References• http://sunset.usc.edu/classes/cs577b_2001/metricsguide/metrics.html• Fenton NE, Software Metrics: A Rigorous Approach, Chapman and Hall,
1991.• http://www.stsc.hill.af.mil/resources/tech_docs/gsam3/chap13.pdf• Christof Ebert, Reiner Dumke, Software measurement: establish, extract,
evaluate, execute, Springer 2007• http://se.inf.ethz.ch/old/teaching/2010-S/0276/slides/kissling.pdf• Richard W. Selby, Northrop Grumman Space Technology, ICSP '09
Title: "Synthesis, Analysis, and Modeling of Large-Scale Mission-Critical Embedded Software Systems“
• Measuring Agility, Peter Behrens• [Nikora 91] Nikora, Allen P. Error Discovery Rate by Severity Category and
Time to Repair Software Failures for Three JPL Flight Projects. Software Product Assurance Section, Jet Propulsion Laboratory, 4800 Oak Grove Drive, Pasadena, CA 91109-8099, November 5, 1991.
61
![Page 60: Software Metrics and Measurements](https://reader035.fdocuments.in/reader035/viewer/2022062501/568168cd550346895ddfbae2/html5/thumbnails/60.jpg)
University of Southern California
Center for Systems and Software Engineering
62
Today’s in-class assignment
• Individual in-class assignment• Pick any 2 measurements that would be
useful to your team• After the class, talk to your team members,
pick 2 measurements that will be used in your team
• Start collecting data, you will have to present them during the DCR ARB /TRR ARB(Transition Readiness Review)