310414 SOFTWARE QUALITY ASSURANCE 1 310414 SOFTWARE ENGINEERING 310414 SOFTWARE ENGINEERING.
-
Upload
myrtle-roberts -
Category
Documents
-
view
238 -
download
1
Transcript of 310414 SOFTWARE QUALITY ASSURANCE 1 310414 SOFTWARE ENGINEERING 310414 SOFTWARE ENGINEERING.
310414310414 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE1
SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE
310414310414SOFTWARE ENGINEERINGSOFTWARE ENGINEERING
310414310414SOFTWARE ENGINEERINGSOFTWARE ENGINEERING
310414310414 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE3
THE FOUR P’s IN SOFTWARE DEVELOPMENTTHE FOUR P’s IN SOFTWARE DEVELOPMENT
ProductProjectPeople
template
participate result
software engineers set of artifacts
set of activities
Process
education&
training
management&
monitoring
testing&
measurement
How canHow canwe achievewe achieve
software quality?software quality?
measurement&
feedback
310414310414 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE4
SOFTWARE QUALITY ASSURANCE (SQA)SOFTWARE QUALITY ASSURANCE (SQA)
Quality assurance consists of those procedures, techniques, and tools applied by professionals to ensure that a product meets or exceeds pre-specified standards during it’s development cycle. E.H. Bersoff
Quality assurance consists of those procedures, techniques, and tools applied by professionals to ensure that a product meets or exceeds pre-specified standards during it’s development cycle. E.H. Bersoff
It is an essential activity for any business thatproduces products used by others.
It needs to be planned and systematic.(It does not just happen)
It needs to be built into the development process.(A natural outcome of software engineering)
Continuous improvement is the overall goal.
310414310414 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE5
SQA — AN UMBRELLA ACTIVITYSQA — AN UMBRELLA ACTIVITY
QualityQualityQualityQuality
Standardsand
Procedures
Standardsand
Procedures
Metricsand
Measurement
Metricsand
Measurement
SoftwareConfigurationManagement
SoftwareConfigurationManagement
TestingTesting
FormalTechnicalReviews
FormalTechnicalReviews
Methodsand
Tools
Methodsand
Tools
310414310414 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE7
WHY SQA ACTIVITIES PAY OFFWHY SQA ACTIVITIES PAY OFF
cost to findand fix a defect
logscale
1
10
100
Reqmts
1.00
Design
1.30
Code
2.00
Test
4.00
Systemtest
13.00
Fielduse
80-130
SQA pays off here becausedefects are cheap to fix
310414310414 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE8
SOFTWARE QUALITYSOFTWARE QUALITY
What is software quality and how do we measure it?
customer’s viewpoint meets specifications
developer’s viewpoint easy to maintain, test, . . .
Other attributes of software that affect its quality:– safety – understandability – portability– security – testability – usability– reliability – adaptability – reusability– resilience – modularity – efficiency– robustness – complexity – learnability
Software quality is not just about meetingspecifications and removing defects!
We need to select critical quality attributes early in the development process and plan for how to achieve them.
310414310414 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE9
PRINCIPLES OF SOFTWARE QUALITY ASSURANCEPRINCIPLES OF SOFTWARE QUALITY ASSURANCE
1. We have a set of standards and quality attributes that a software product must meet. There is a goal to achieve.
2. We can measure the quality of a software product. There is a way to determine how well the product
conforms to the standards and the quality attributes.
3. We track the values of the quality attributes. It is possible to assess how well we are doing.
4. We use information about software quality to improve the quality of future software products. There is feedback into the software development process.
keypoint
310414310414 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE10
1. encapsulate best (or most appropriate) practices acquired after much trial and error helps avoid previous mistakes
2. provide a framework around which to implement SQA process ensures that best practices are properly followed
3. assist in ensuring continuity of project work reduces learning effort when starting new work
product standards: define the characteristics all product artifacts should exhibit so as to have quality
process standards: define how the software process should be conducted to ensure quality software
SOFTWARE STANDARDSSOFTWARE STANDARDS
Why are software standards important?
Standards
Standards
Handbook
Handbook
needed!
needed!
Each project needs to decide which standards should be: ignored; used as is; modified; created.
310414310414 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE11
SOFTWARE METRICSSOFTWARE METRICS
metric:metric: any any type of measurementtype of measurement that relates to a that relates to asoftware system, process or related artifactsoftware system, process or related artifact
metric:metric: any any type of measurementtype of measurement that relates to a that relates to asoftware system, process or related artifactsoftware system, process or related artifact
control metrics - used to plan, manage and control the development process
(e.g., effort expended, elapsed time, disk usage, etc.)
predictor metrics - used to predict an associated product quality
(e.g., cyclomatic complexity can predict ease of maintenance)
external attribute: something we can only discover after the software has been put into use (e.g.,ease of maintenance)
internal attribute: something we can measure directly from the software itself (e.g., cyclomatic complexity)
310414310414 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE12
SOFTWARE METRICS (cont’d)SOFTWARE METRICS (cont’d)
We want to use internal attributes to predict the value of external attributes.
ProblemsProblems: 1. hard to formulate and validate relationships between internal and external attributes
2. software metrics must be collected, calibrated, interpreted
external
maintainability
reliability
portability
usability
internal
# of parameters
cyclomatic complexity
lines of code
# of error messages
length of user manual
310414310414 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE13
PRODUCT QUALITY: PRODUCT QUALITY: DESIGN QUALITY METRICSDESIGN QUALITY METRICS
For a For a design componentdesign component, the , the key key quality attributequality attribute is is maintainabilitymaintainability..
For a For a design componentdesign component, the , the key key quality attributequality attribute is is maintainabilitymaintainability..
For design components maintainability is related to:– cohesion - how closely related is the functionality of the component?
– coupling - how independent is the component?
– understandability - how easy is it to understand what the component does?
– adaptability - how easy is it to change the component?
Problem: most of these cannot be measured directly, but it is reasonable to infer that there is a relationship between these attributes and the “complexity” of a component
measure complexity How?How?
310414310414 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE14
PRODUCT QUALITY: PRODUCT QUALITY: DESIGN QUALITY METRICSDESIGN QUALITY METRICS
a) Structural fan-in/fan-out
fan-in – number of calls to a component by other components
fan-out – number of components called by a component
high fan-in => high coupling
high fan-out => calling component has high complexity
b) Informational fan-in/fan-out– consider also the number of parameters passed plus access to
shared data structures
complexity = component-length x (fan-in x fan-out)2
It has been validated using the Unix system
It is a useful predictor of effort required for implementation
310414310414 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE15
PRODUCT QUALITY: PRODUCT QUALITY: DESIGN QUALITY METRICSDESIGN QUALITY METRICS
c) IEEE Standard 982.1-1988
looks at: subsystem properties (number of subsystems and degree of coupling)
database properties (number of attributes and classes)
compute a design structure quality index—DSQI (0-1)
used to compare with past designs; if DSQI is too low,further design work and review may be required
we can also consider changes made throughout the lifetime of the software and compute how stable the product is (i.e., how many changes have been made in subsystems in the current release)
define a software maturity index—SMI (0-1)
as SMI approaches 1, the product begins to stabilize
310414310414 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE19
PRODUCT QUALITY: PRODUCT QUALITY: PROGRAM QUALITY METRICSPROGRAM QUALITY METRICS
a) Halstead’s Software Science
Looks at operators and operands in an implementation component and calculates values for component volume, V, component difficulty, D, and effort, E, required to implement the component.
n1 = number of unique operators in a componentn2 = number of unique operands in a componentN1 = the total number of operatorsN2 = the total number of operandsL = N1+ N2 (component length)
V = L * log2(n1 + n2) (component volume in bits)
D = (n1/2) * (N2/n2) (difficulty of implementing the component)
E = V * D (effort required to implement the component)
For an For an implementation componentimplementation component, the key quality , the key quality attributes are attributes are reliabilityreliability and and difficulty of implementationdifficulty of implementation..
For an For an implementation componentimplementation component, the key quality , the key quality attributes are attributes are reliabilityreliability and and difficulty of implementationdifficulty of implementation..
310414310414 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE20
PRODUCT QUALITY: PRODUCT QUALITY: PROGRAM QUALITY METRICSPROGRAM QUALITY METRICS
b) McCabe’s Complexity Metric
Looks at control flow in a component.
Cyclomatic Complexity measures component’s logical complexity
an indication of the testing difficulty of a component
Studies have shown a distinct relationship between the Cyclomatic Complexity of a component and the number of errors found in the source code, as well as the time required to find and fix errors.
c) Other program quality metrics– length of code – length of identifiers– depth of conditional nesting
Standards can be established to avoid complex components and/or highlight problem components.
310414310414 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE21
PRODUCT QUALITY: PRODUCT QUALITY: FORMAL APPROACHESFORMAL APPROACHES
a) Proving programs/specifications correct– logically prove that requirements have been correctly transformed
into programs
(e.g., prove assertions about programs)
b) Statistical Quality Assurance– categorize and determine cause of software defects
– 80-20 rule 80% of defects can be traced to 20% of causes– isolate and correct 20% of the causes
effort directed to things that cause the majority of defects
c) The Cleanroom Process– combination of above two approaches
310414310414 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE22
PROJECT QUALITY: PROJECT QUALITY: REVIEWSREVIEWS
Requirementscapture
Analysis
Design
Implementation
Testing
requirementswalkthroughs
analysiswalkthroughs
designwalkthroughs
codewalkthroughs
test planreview
Leads to Leads to earlyearly
discovery of discovery of defectsdefects
Requirements, Requirements, Analysis and DesignAnalysis and Designintroduce 50-60%introduce 50-60%
of all defects.of all defects.
Formal technical Formal technical reviews can uncover reviews can uncover
75% of these!75% of these!
The principal method of The principal method of validating the qualityvalidating the quality of a project. of a project.The principal method of The principal method of validating the qualityvalidating the quality of a project. of a project.
310414310414 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE23
SOFTWARE CONFIGURATION MANAGEMENT (SCM)SOFTWARE CONFIGURATION MANAGEMENT (SCM)
An activity executed throughout the system life cycle An activity executed throughout the system life cycle to to control changecontrol change of of productsproducts and life cycle and life cycle artifactsartifacts..
An activity executed throughout the system life cycle An activity executed throughout the system life cycle to to control changecontrol change of of productsproducts and life cycle and life cycle artifactsartifacts..
auditcontrol report
identify support
To what do we want to control changes?To what do we want to control changes?
configuration item: an artifact to which we want to control changes
10.2
Software Configuration Management• plans • programs• specifications • documents/manuals• procedures • data
310414310414 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE24
CONFIGURATION ITEM IDENTIFCATION AND CONFIGURATION ITEM IDENTIFCATION AND DESCRIPTIONDESCRIPTION
each configuration item must be identified and described– a unique name– configuration item type– project identifier– change and/or version information– resources the configuration item provides or requires– pointer to actual configuration item
the configuration items also must be organized (i.e., need to define the relationships that exist between configuration items)– object diagram <part-of> domain model– domain model <related-to> use-case model
This isThis ismetadatametadata
aboutaboutconfigurationconfiguration
itemsitems
Define and construct a software library (database) that stores, manages and tracks configuration items.
10.3.1
310414310414 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE25
CONTROLLING CONFIGURATION ITEMS: CONTROLLING CONFIGURATION ITEMS: BASELINESBASELINES
A A baselinebaseline is a time/phase in the software development (usually is a time/phase in the software development (usually aa project milestone project milestone) after which any ) after which any changes must be formalizedchanges must be formalized
(e.g., go through a formal change control procedure).(e.g., go through a formal change control procedure).
A A baselinebaseline is a time/phase in the software development (usually is a time/phase in the software development (usually aa project milestone project milestone) after which any ) after which any changes must be formalizedchanges must be formalized
(e.g., go through a formal change control procedure).(e.g., go through a formal change control procedure).
10.3.2
In order to become a baseline, the configuration item must first pass a set of formal review procedures (e.g., formal code review, documentation review, etc.).
It then becomes part of the project software library.
After this a “check-out” procedure is applied to the item(i.e., access to and change of the configuration item is controlled)
Any modified configuration item must again go through a formal review process before it can replace the original (baselined) item.
310414310414 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE27
CONTROLLING CONFIGURATION ITEMS:CONTROLLING CONFIGURATION ITEMS:VERSION CONTROLVERSION CONTROL a configuration item usually evolves throughout the software
engineering process (i.e., it will have several versions) An evolution graph can be used to describe a
configuration item’s change history
version– configuration itemk is obtained by modifying configuration itemi
– usually a result of bug fixes, enhanced system functionality, etc.– itemk supercedes original itemi; created in a linear order
branch– a concurrent development path requiring independent configuration
management
variant– different configurations that are intended to coexist– e.g., different configurations depending on operating system type
Oracle for Windows Oracle for Linux
310414310414 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE29
Uncontrolledchange rapidlyleads to chaos !
Uncontrolledchange rapidlyleads to chaos !
SCM CHANGE CONTROLSCM CHANGE CONTROL
STOP
CH
AN
GE
change request submitted byusers/developers
change control authority evaluates,decides and issues an engineeringchange order if approved
configuration item is checked-out,changed, checked-in after SQA,and version controlled
configuration item made availablefor use (promotion; release)
change request submitted byusers/developers
change control authority evaluates,decides and issues an engineeringchange order if approved
configuration item is checked-out,changed, checked-in after SQA,and version controlled
configuration item made availablefor use (promotion; release)
310414310414 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE31
SCM AUDIT AND STATUS REPORTINGSCM AUDIT AND STATUS REPORTING
Audit: ensures that changes have been properly implemented.
– Have the proper steps and procedures been followed? checklist
– Usually done by Quality Assurance (QA) group if SCM is a formal activity.
Status reporting: keeps all parties informed and up-to-date on the status of a change.
– A communication mechanism among project members to help keep them coordinated.
– Can determine who made what changes, when and why.
310414310414 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE32
SCM SUPPORTSCM SUPPORT
a software library provides facilities to store, label, identify versions and track the status of the configuration items
softwarerepository
developer’sworkspace
masterdirectory
promotions
releases
everydaydevelopment
used by asingle developer
used by other developers
for userstracks
tracks
check-in; check-out
after SQA
10.3.5
310414310414 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE33
SCM BENEFITSSCM BENEFITS
reduces the effort required to manage and effect change
improved productivity
leads to better software integrity and security
increased quality
generates information about the process
enhanced management control
maintains a software development database
better record keeping and tracking
310414310414 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE34
Does Does process quality process quality product qualityproduct quality??Does Does process quality process quality product qualityproduct quality??
PROCESS QUALITYPROCESS QUALITY
unlike manufactured products, software development has some unique factors that affect its quality:– software is designed, not manufactured
– software development is creative, not mechanical
– individual skills and experience have a significant influence
– external factors (application novelty, commercial pressure)
software development processes are organization specific
people, technology may be more important than process
insufficient resources will always adversely affect quality
310414310414 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE35
PROCESS QUALITY: PROCESS QUALITY: ISO 9001/ 9000-3ISO 9001/ 9000-3
. . .
ISO 9000quality model
Organizationquality model
Project 1quality plan
Project 2quality plan
Organizationalquality process
Project qualitymanagement
instantiated as
instantiated as
documents
supports
is usedto develop
certification is easy; can be a marketing ploy
genericquality model
customized fora particularorganization
Focus is onFocus is onprocessprocess
managementmanagement
310414310414 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE39
PROCESS QUALITY:PROCESS QUALITY:THE SEI CAPABILITY MATURITY MODEL (CMM)THE SEI CAPABILITY MATURITY MODEL (CMM)
Intended to help a software organizationIntended to help a software organizationimproveimprove their their software development processessoftware development processes..
Intended to help a software organizationIntended to help a software organizationimproveimprove their their software development processessoftware development processes..
Level 1 Organization: Initial process (ad hoc)– no formal procedures, no cost estimates, no project plans,
no management mechanism to ensure procedures are followed
Level 2 Organization: Repeatable process (intuitive)– basic project controls; intuitive methods used
Level 3 Organization: Defined process (qualitative)– development process defined and institutionalized
Level 4 Organization: Managed process (quantitative)– measured process; process database established
Level 5 Organization: Optimizing process– improvement feedback; rigorous defect-cause analysis and prevention
Focus is onFocus is onprocessprocess
improvementimprovement
12.3
310414310414 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE41
PROCESS QUALITY — NOTESPROCESS QUALITY — NOTES
It is possible for a Level 1 organization to receive ISO 9000 certification!
A Level 3 organization would have little difficulty in obtaining ISO 9000 certification.
A Level 2 organization would have significant advantage in obtaining ISO 9000 certification.
excellent software developers (e.g., space shuttle) attain around level 3-4 only
310414310414 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE42
PEOPLE QUALITY:PEOPLE QUALITY:PEOPLE CAPABILITY MATURITY MODEL (PCMM)PEOPLE CAPABILITY MATURITY MODEL (PCMM)
Intended to Intended to improve knowledgeimprove knowledge and and skillskill of people. of people.Intended to Intended to improve knowledgeimprove knowledge and and skillskill of people. of people.
Level 1 – Initial– no technical or management training provided;
staff talent not a critical resource; no organizational loyalty
Level 2 – Repeatable– focus on developing basic work practices; staff recruiting, growth and
development important; training to fill skill “gaps”; performance evaluated
Level 3 – Defined– focus on tailoring work practices to organization’s business; strategic plan to
locate and develop required talent; skills-based compensation
Level 4 – Managed– focus on increasing competence in critical skills; mentoring; team-building;
quantitative competence goals; evaluation of effectiveness of work practices
Level 5 – Optimizing– focus on improving team and individual skills; use of best practices
Focus is onFocus is onpeoplepeople
improvementimprovement
310414310414 SOFTWARE QUALITY ASSURANCESOFTWARE QUALITY ASSURANCE44
SUMMARY — SOFTWARE QUALITYSUMMARY — SOFTWARE QUALITY
Quality software does not just happen!Quality software does not just happen!Quality software does not just happen!Quality software does not just happen!
Quality assurance mechanisms should be built into the software development process
Developing quality software requires:– Management support and involvement
– Gathering and use of software metrics
– Policies and procedures that everyone follows
– Commitment to following the policies and procedures even when things get rough!
Testing is an important part of quality assurance, but its not all there is to obtaining a quality software product.