1 Software Testing Strategies CIS 375 Bruce R. Maxim UM-Dearborn.
Process Improvement CIS 376 Bruce R. Maxim UM-Dearborn.
-
Upload
branden-lime -
Category
Documents
-
view
218 -
download
2
Transcript of Process Improvement CIS 376 Bruce R. Maxim UM-Dearborn.
![Page 1: Process Improvement CIS 376 Bruce R. Maxim UM-Dearborn.](https://reader035.fdocuments.in/reader035/viewer/2022081515/551a99065503466b3a8b5313/html5/thumbnails/1.jpg)
Process Improvement
CIS 376
Bruce R. Maxim
UM-Dearborn
![Page 2: Process Improvement CIS 376 Bruce R. Maxim UM-Dearborn.](https://reader035.fdocuments.in/reader035/viewer/2022081515/551a99065503466b3a8b5313/html5/thumbnails/2.jpg)
Process Improvement Goals
• Understanding existing processes• Introduce process changes to improve quality,
reduce costs, or accelerate schedules• Industry is demanding increased attention to
quality in general• Most process improvement work focuses on defect
reduction and prevention• There are other process attributes that deserve our
attention
![Page 3: Process Improvement CIS 376 Bruce R. Maxim UM-Dearborn.](https://reader035.fdocuments.in/reader035/viewer/2022081515/551a99065503466b3a8b5313/html5/thumbnails/3.jpg)
Process Improvement Attributes - part 1
• Understandability - degree to which a process is well defined and understood
• Visibility - process activities have results that are externally recognizable
• Supportability - process activities supported by CASE tools
• Acceptability - defined processes are used and accepted by software engineers
![Page 4: Process Improvement CIS 376 Bruce R. Maxim UM-Dearborn.](https://reader035.fdocuments.in/reader035/viewer/2022081515/551a99065503466b3a8b5313/html5/thumbnails/4.jpg)
Process Improvement Attributes - part 2
• Reliability - process is defined so that errors are avoided or trapped before product errors result
• Robustness - process can continue despite unexpected problems
• Maintainability - process can evolve to reflect changing organizational requirements or identified process improvements
• Rapidity - the time required to complete a system from specification to delivery
![Page 5: Process Improvement CIS 376 Bruce R. Maxim UM-Dearborn.](https://reader035.fdocuments.in/reader035/viewer/2022081515/551a99065503466b3a8b5313/html5/thumbnails/5.jpg)
Process Improvement Stages
• Process analysis– modeling and quantitative analysis of existing processes
• Improvement identification– quality, cost, and scheduling bottlenecks located
• Process change introduction– modify process to remove bottlenecks
• Process change training– train staff involved in process revision proposals
• Change tuning– process improvements are revised and allowed to evolve
![Page 6: Process Improvement CIS 376 Bruce R. Maxim UM-Dearborn.](https://reader035.fdocuments.in/reader035/viewer/2022081515/551a99065503466b3a8b5313/html5/thumbnails/6.jpg)
Process Improvement Activities
Processmodel
Process changeplan
Trainingplan
Feedback onimprovements
Revised processmodel
Analyseprocess
Identifyimprovements
Tuneprocess changes
Introduceprocess change
Trainengineers
![Page 7: Process Improvement CIS 376 Bruce R. Maxim UM-Dearborn.](https://reader035.fdocuments.in/reader035/viewer/2022081515/551a99065503466b3a8b5313/html5/thumbnails/7.jpg)
Process and Product Quality
• Closely related to one another
• Good processes are usually required to produce good products
• In manufacturing applications, process is principle determinant of quality
• For design-based activities, the capabilities of the designers are also important
![Page 8: Process Improvement CIS 376 Bruce R. Maxim UM-Dearborn.](https://reader035.fdocuments.in/reader035/viewer/2022081515/551a99065503466b3a8b5313/html5/thumbnails/8.jpg)
Product Quality Factors
• Development technology– for large projects with average capability this is the main
determinant of product quality
• Quality of people involved– for small projects the developer capability is the main determinant
of product quality
• Process quality– significant for both small and large projects
• Cost, time, and schedule constraints– unrealistic schedules can doom the quality of most products
![Page 9: Process Improvement CIS 376 Bruce R. Maxim UM-Dearborn.](https://reader035.fdocuments.in/reader035/viewer/2022081515/551a99065503466b3a8b5313/html5/thumbnails/9.jpg)
Process Analysis and Modeling
• Process analysis– study of existing processes to understand relationships among
process components
– allows comparisons with other processes
• Process modeling– documentation of process in which the tasks, roles, and entities
used are recorded
– best to represent models graphically
– several different perspectives may be used (e.g. activities, deliverables, etc.)
– model should be examined for weaknesses, this involves discussion with stakeholders
![Page 10: Process Improvement CIS 376 Bruce R. Maxim UM-Dearborn.](https://reader035.fdocuments.in/reader035/viewer/2022081515/551a99065503466b3a8b5313/html5/thumbnails/10.jpg)
Process Model Elements - part 1
• Activity - (round edged rectangle)– has clearly defined objective, entry, and exit conditions
• Process - (round edged rectangle with shadow)– set of coherent activities with agreed upon objective
• Deliverable - (rectangle with shadow)– tangible output of an activity predicted by project plan
• Condition - (parallelogram)– process or activity pre- or post-conditions
![Page 11: Process Improvement CIS 376 Bruce R. Maxim UM-Dearborn.](https://reader035.fdocuments.in/reader035/viewer/2022081515/551a99065503466b3a8b5313/html5/thumbnails/11.jpg)
Process Model Elements - part 2
• Role - (circle with shadow)– defined and bounded area of responsibility
• Exception - (double edged box))– description of how to modify the process if anticipated
or unanticipated events occur
• Communication - (arrow)– exchange of information between people and/or
machines
![Page 12: Process Improvement CIS 376 Bruce R. Maxim UM-Dearborn.](https://reader035.fdocuments.in/reader035/viewer/2022081515/551a99065503466b3a8b5313/html5/thumbnails/12.jpg)
Process Model Example
Testmodule
Signed-of f testrecord
Module testdata
Modulespecification
Module compileswithout syntax
errors
All defined testsrun on module
Testengineer
Pre-condition
InputProcess
RôlePost-condition
OutputsResponsible
for
![Page 13: Process Improvement CIS 376 Bruce R. Maxim UM-Dearborn.](https://reader035.fdocuments.in/reader035/viewer/2022081515/551a99065503466b3a8b5313/html5/thumbnails/13.jpg)
Process Exceptions
• Process models can’t represent how to handle exceptions– key people are lost prior to a critical review
– failure of e-mail server for several days
– organizational reorganization
– request to respond to change requests
• General procedure is to suspend the process model and follow RMMM plans augmented with the managers own initiatives
![Page 14: Process Improvement CIS 376 Bruce R. Maxim UM-Dearborn.](https://reader035.fdocuments.in/reader035/viewer/2022081515/551a99065503466b3a8b5313/html5/thumbnails/14.jpg)
Process Measurement
• Wherever possible quantitative process data should be collected
• Organizations without process standards may have to be define processes before measurements can be made (since they won’t know what to measure)
• Process measurements should be used to assess process improvements
• Organization objectives drive process improvement, not measurements
![Page 15: Process Improvement CIS 376 Bruce R. Maxim UM-Dearborn.](https://reader035.fdocuments.in/reader035/viewer/2022081515/551a99065503466b3a8b5313/html5/thumbnails/15.jpg)
Process Measurement Classes
• Time taken to complete process activities– e.g. calendar time to complete a milestone
• Resources required to complete processes or activities– e.g. person months
• Number of event occurrences – e.g. number of defects found
![Page 16: Process Improvement CIS 376 Bruce R. Maxim UM-Dearborn.](https://reader035.fdocuments.in/reader035/viewer/2022081515/551a99065503466b3a8b5313/html5/thumbnails/16.jpg)
Goal Question Metric Paradigm
• Goals– What is the organization trying to achieve?
– Process improvement deals with goal satisfaction.
• Questions– Concerned with areas of uncertainty related to goals.
– You need process knowledge to derive questions.
• Metrics– Measurements collected to answer questions
![Page 17: Process Improvement CIS 376 Bruce R. Maxim UM-Dearborn.](https://reader035.fdocuments.in/reader035/viewer/2022081515/551a99065503466b3a8b5313/html5/thumbnails/17.jpg)
SEI Process Maturity Model
• Level 1 - Initial– essentially uncontrolled
• Level 2 - Repeatable– project management procedures defined and used
• Level 3 - Defined– process management strategies defined and used
• Level 4 - Managed– quality management strategies defined and used
• Level 5 - Optimizing– process improvement strategies defined and used
![Page 18: Process Improvement CIS 376 Bruce R. Maxim UM-Dearborn.](https://reader035.fdocuments.in/reader035/viewer/2022081515/551a99065503466b3a8b5313/html5/thumbnails/18.jpg)
SEI Process Model Problems• Focuses on project management rather than project
development• Ignores the use of strategies like rapid prototyping• Model is intended to represent organizational
capability and not practices used on particular projects
• There may be wide variation in the practices used in a single organization
• Capability assessment is questionnaire-based
![Page 19: Process Improvement CIS 376 Bruce R. Maxim UM-Dearborn.](https://reader035.fdocuments.in/reader035/viewer/2022081515/551a99065503466b3a8b5313/html5/thumbnails/19.jpg)
Capability Assessment Process
Select projectsfor assessment
Distributequestionnaires
Analyseresponses
Clarifyresponses
Identify issuesfor discussion
Interviewproject managers
Interviewengineers
Interviewmanagers
Brief managersand engineers
Presentassessment
Write report
![Page 20: Process Improvement CIS 376 Bruce R. Maxim UM-Dearborn.](https://reader035.fdocuments.in/reader035/viewer/2022081515/551a99065503466b3a8b5313/html5/thumbnails/20.jpg)
Process Classification
• Informal– No detailed process model, developers created their
own way of doing things
• Managed– defined model drive development process
• Methodical– processes supported by standard development method
• Supported– processes supported by automated CASE tools
![Page 21: Process Improvement CIS 376 Bruce R. Maxim UM-Dearborn.](https://reader035.fdocuments.in/reader035/viewer/2022081515/551a99065503466b3a8b5313/html5/thumbnails/21.jpg)
Process Tool Support
Informalprocess
Managedprocess
Methodicalprocess
Improvingprocess
Specializedtools
Analysis anddesign
workbenches
Projectmanagement
tools
Configurationmanagement
tools
Generictools
![Page 22: Process Improvement CIS 376 Bruce R. Maxim UM-Dearborn.](https://reader035.fdocuments.in/reader035/viewer/2022081515/551a99065503466b3a8b5313/html5/thumbnails/22.jpg)
Defect Removal Effectiveness
• Defect removal is central to software development
• One of the top expense items
• Affects project scheduling
• Improves product quality
![Page 23: Process Improvement CIS 376 Bruce R. Maxim UM-Dearborn.](https://reader035.fdocuments.in/reader035/viewer/2022081515/551a99065503466b3a8b5313/html5/thumbnails/23.jpg)
PSP - Defect Density• This is the primary defect measure used in
PSP
• Dd = 1000 * D/N
• D = total number of defects found in all phases of the process
• N = number of new and changed lines of code in the program
![Page 24: Process Improvement CIS 376 Bruce R. Maxim UM-Dearborn.](https://reader035.fdocuments.in/reader035/viewer/2022081515/551a99065503466b3a8b5313/html5/thumbnails/24.jpg)
Defect Density Example
• For a program with 96 new or changed lines of code and 14 defects
• Dd = 1000 * (14/96) = 145.83 defects/KLOC
![Page 25: Process Improvement CIS 376 Bruce R. Maxim UM-Dearborn.](https://reader035.fdocuments.in/reader035/viewer/2022081515/551a99065503466b3a8b5313/html5/thumbnails/25.jpg)
Defect Metrics - part 1
• Error Detection Efficiency100%*(#errors found in 1 inspection)/(#errors in product before inspection)
• Defect Removal Efficiency100%*(#defects found now)/(#defects found now + #defects found later)
• Error Detection Percentage100%*(#inspection errors)/(#inspection errors + #valid discrepancy reports)
![Page 26: Process Improvement CIS 376 Bruce R. Maxim UM-Dearborn.](https://reader035.fdocuments.in/reader035/viewer/2022081515/551a99065503466b3a8b5313/html5/thumbnails/26.jpg)
Defect Metrics - part 2
• Total Defect Containment Effectiveness (TDCE)(#prerelease defects)/(#prerelease defects + #post-release defects)
• Phase Containment Effectiveness (PCE)(#phase(i) defects)/(#phase(i) defects + #phase(i+x) defects)
• Effectiveness (E)100%*N/(N + S)
N = #defects found by an activity
S = #defects found in subsequent activities
![Page 27: Process Improvement CIS 376 Bruce R. Maxim UM-Dearborn.](https://reader035.fdocuments.in/reader035/viewer/2022081515/551a99065503466b3a8b5313/html5/thumbnails/27.jpg)
Phase-based Defect Removal Model
• Defects present at exit of each development phase are estimated
• This allows us to set realistic targets and assess the costs of reducing error injection rates
• This is a quality management tool and not a device for estimation of software reliability
• How would this work in practice?
![Page 28: Process Improvement CIS 376 Bruce R. Maxim UM-Dearborn.](https://reader035.fdocuments.in/reader035/viewer/2022081515/551a99065503466b3a8b5313/html5/thumbnails/28.jpg)
Assumptions
• Suppose we decide to create two broad defect removal classes– activities that handle defects before code is
integrated into the system library (design reviews, inspections, unit testing)
– formal machine tests after code integration
• Also assume the same defect removal effectiveness for each phase
![Page 29: Process Improvement CIS 376 Bruce R. Maxim UM-Dearborn.](https://reader035.fdocuments.in/reader035/viewer/2022081515/551a99065503466b3a8b5313/html5/thumbnails/29.jpg)
Example - part 1
• MP = major problems found in before integration• PTR = errors found during formal machine tests• mu = MP/PTR
– the higher the value of mu the better• Q = defects found after release to customer• TD = (MP + PTR + Q)
– total defects for life of software
![Page 30: Process Improvement CIS 376 Bruce R. Maxim UM-Dearborn.](https://reader035.fdocuments.in/reader035/viewer/2022081515/551a99065503466b3a8b5313/html5/thumbnails/30.jpg)
Example - part 2
• Phase 1 effectiveness
E1 = MP/TD
MP = E1 * TD• Phase 2 effectiveness
E2 = PTR/(TD - MP)
PTR = E2 * (TD - MP)
![Page 31: Process Improvement CIS 376 Bruce R. Maxim UM-Dearborn.](https://reader035.fdocuments.in/reader035/viewer/2022081515/551a99065503466b3a8b5313/html5/thumbnails/31.jpg)
Example - part 3
• Some equations that can be useful in quality planning (assuming that E1 = E2)
Q = PTR /(mu - 1)
Q = MP / [mu * (mu - 1)]
Q = TD / (mu * mu)
• These equations work with either raw or normalized defect values
![Page 32: Process Improvement CIS 376 Bruce R. Maxim UM-Dearborn.](https://reader035.fdocuments.in/reader035/viewer/2022081515/551a99065503466b3a8b5313/html5/thumbnails/32.jpg)
PSP – Phase Yield
Phase yield =
100 * (defects removed during phase)/
(defects in product at phase entry)
Note: cannot be computed until project is completed
![Page 33: Process Improvement CIS 376 Bruce R. Maxim UM-Dearborn.](https://reader035.fdocuments.in/reader035/viewer/2022081515/551a99065503466b3a8b5313/html5/thumbnails/33.jpg)
Phase Yield - Example
• 5 defects found during code review• 3 defects found during compile• 2 defects found during unit testing• 2 defects found during integration testing
• Phase yield for compile =100 * 3 / (3 + 2 + 2) = 42.9 %
• Phase yield for code review =100 * 5 /(5 + 3 + 2 + 2) = 41.7 %
![Page 34: Process Improvement CIS 376 Bruce R. Maxim UM-Dearborn.](https://reader035.fdocuments.in/reader035/viewer/2022081515/551a99065503466b3a8b5313/html5/thumbnails/34.jpg)
Seven Basic Software Quality Tools
• Checklists (paper forms)– used to gather data for later analysis– used to confirm that process tasks are complete– both simple yes/no and branching questions
![Page 35: Process Improvement CIS 376 Bruce R. Maxim UM-Dearborn.](https://reader035.fdocuments.in/reader035/viewer/2022081515/551a99065503466b3a8b5313/html5/thumbnails/35.jpg)
![Page 36: Process Improvement CIS 376 Bruce R. Maxim UM-Dearborn.](https://reader035.fdocuments.in/reader035/viewer/2022081515/551a99065503466b3a8b5313/html5/thumbnails/36.jpg)
Seven Basic Software Quality Tools
• Pareto Diagram– bar chart sorted in descending height order– vertical axis labeled with # defects– horizontal axis (nominal) labeled with defect
cause types– software defects tends cluster near related
causes
![Page 37: Process Improvement CIS 376 Bruce R. Maxim UM-Dearborn.](https://reader035.fdocuments.in/reader035/viewer/2022081515/551a99065503466b3a8b5313/html5/thumbnails/37.jpg)
![Page 38: Process Improvement CIS 376 Bruce R. Maxim UM-Dearborn.](https://reader035.fdocuments.in/reader035/viewer/2022081515/551a99065503466b3a8b5313/html5/thumbnails/38.jpg)
Seven Basic Software Quality Tools
• Histogram– frequency bar graph – vertical axis is # defects– horizontal axis has ordinal or interval type
labels
![Page 39: Process Improvement CIS 376 Bruce R. Maxim UM-Dearborn.](https://reader035.fdocuments.in/reader035/viewer/2022081515/551a99065503466b3a8b5313/html5/thumbnails/39.jpg)
![Page 40: Process Improvement CIS 376 Bruce R. Maxim UM-Dearborn.](https://reader035.fdocuments.in/reader035/viewer/2022081515/551a99065503466b3a8b5313/html5/thumbnails/40.jpg)
Seven Basic Software Quality Tools
• Flowchart– pictorial representation of a process– breaks down process into its constituent steps– can be useful in identifying were errors are
likely to be found in the system
![Page 41: Process Improvement CIS 376 Bruce R. Maxim UM-Dearborn.](https://reader035.fdocuments.in/reader035/viewer/2022081515/551a99065503466b3a8b5313/html5/thumbnails/41.jpg)
![Page 42: Process Improvement CIS 376 Bruce R. Maxim UM-Dearborn.](https://reader035.fdocuments.in/reader035/viewer/2022081515/551a99065503466b3a8b5313/html5/thumbnails/42.jpg)
Seven Basic Software Quality Tools
• Scatter diagram (point plots)– used with correlation, regression, or statistical
modeling – vertical axis is # defects– horizontal axis some metric (e.g. McCabe’s
index)
![Page 43: Process Improvement CIS 376 Bruce R. Maxim UM-Dearborn.](https://reader035.fdocuments.in/reader035/viewer/2022081515/551a99065503466b3a8b5313/html5/thumbnails/43.jpg)
![Page 44: Process Improvement CIS 376 Bruce R. Maxim UM-Dearborn.](https://reader035.fdocuments.in/reader035/viewer/2022081515/551a99065503466b3a8b5313/html5/thumbnails/44.jpg)
Seven Basic Software Quality Tools
• Run chart– line graph showing performance of dependent
variable (y) over time (x)– best used for trend analysis (e.g. arrival of
defects during formal machine testing)– can plot cumulative dependent variables (S
curves)
![Page 45: Process Improvement CIS 376 Bruce R. Maxim UM-Dearborn.](https://reader035.fdocuments.in/reader035/viewer/2022081515/551a99065503466b3a8b5313/html5/thumbnails/45.jpg)
![Page 46: Process Improvement CIS 376 Bruce R. Maxim UM-Dearborn.](https://reader035.fdocuments.in/reader035/viewer/2022081515/551a99065503466b3a8b5313/html5/thumbnails/46.jpg)
![Page 47: Process Improvement CIS 376 Bruce R. Maxim UM-Dearborn.](https://reader035.fdocuments.in/reader035/viewer/2022081515/551a99065503466b3a8b5313/html5/thumbnails/47.jpg)
Seven Basic Software Quality Tools
• Control chart– advanced form of run chart where capability is defined
– upper and lower control limits (dashed lines) are drawn to alert the user when dependent measure is out of control
– can plot cumulative dependent variables (S curves)
– C chart based on # conforming or not
– R chart based on subgroup ranges (max – min)
– X bar chart based on subgroup means
![Page 48: Process Improvement CIS 376 Bruce R. Maxim UM-Dearborn.](https://reader035.fdocuments.in/reader035/viewer/2022081515/551a99065503466b3a8b5313/html5/thumbnails/48.jpg)
Control Chart (C)
![Page 49: Process Improvement CIS 376 Bruce R. Maxim UM-Dearborn.](https://reader035.fdocuments.in/reader035/viewer/2022081515/551a99065503466b3a8b5313/html5/thumbnails/49.jpg)
![Page 50: Process Improvement CIS 376 Bruce R. Maxim UM-Dearborn.](https://reader035.fdocuments.in/reader035/viewer/2022081515/551a99065503466b3a8b5313/html5/thumbnails/50.jpg)
![Page 51: Process Improvement CIS 376 Bruce R. Maxim UM-Dearborn.](https://reader035.fdocuments.in/reader035/viewer/2022081515/551a99065503466b3a8b5313/html5/thumbnails/51.jpg)
Seven Basic Software Quality Tools• Cause and effect (fish bone) diagram
– not widely used in software development, but can be useful
– shows effect between quality variable and the factors affecting it
![Page 52: Process Improvement CIS 376 Bruce R. Maxim UM-Dearborn.](https://reader035.fdocuments.in/reader035/viewer/2022081515/551a99065503466b3a8b5313/html5/thumbnails/52.jpg)
Fishbone Diagram