Thomas L. Gilchrist Testing Basics Set 5: CM & Peer Reviews By Thomas L. Gilchrist CSQE,CSQA 2009.
-
Upload
caitlin-farmer -
Category
Documents
-
view
217 -
download
1
description
Transcript of Thomas L. Gilchrist Testing Basics Set 5: CM & Peer Reviews By Thomas L. Gilchrist CSQE,CSQA 2009.
Thomas L. Gilchrist [email protected]
Testing BasicsSet 5: CM & Peer Reviews
By
Thomas L. GilchristCSQE,CSQA
2009
Thomas L. Gilchrist [email protected]
SCCM (software change and configuration management) Basics
• Identification: How does and organization identify and manage the many existing versions of a program and its documentation in a manner that will enable change to be accommodated efficiently?
• Control: How does an organization control changes before and after the software (and supporting documentation) is released to the customer?
• Status Accounting: Who has responsibility for approving and ranking changes
• Verification: How can we ensure that changes have been made properly?
• Reporting: What mechanism is used to appraise others of changes that have been made (or being made).
Roger Pressman…”Software Engineering, Fifth Edition”
Thomas L. Gilchrist [email protected]
CM High Level View
Project CM Repository
SW EngTasks
NewSCI
TechReview
ApprovedSCI
SCMControls
SCI = Software Configuration Item
Thomas L. Gilchrist [email protected]
CM High Level View
Project CM Repository
SW EngTasks
NewSCI
TechReview
ApprovedSCI
SCMControls Read
OnlyCopyVer
1SCI
SCI = Software Configuration Item
Ver1
SCI
Project CM Repository
TechReview
Thomas L. Gilchrist [email protected]
CM High Level View
Project CM Repository
SCMControls
RequestFor
Change
Read OnlyCopyVer
1Ver1
SCI Ver1
SCI
Project CM Repository
Thomas L. Gilchrist [email protected]
CM High Level View
Project CM Repository
SW EngTasks
CurrentBaseline SCIApproved to
Change
SCMControls
RequestFor
Change
Read OnlyCopyVer
1Ver1
SCI Ver1
SCI
Project CM Repository
Thomas L. Gilchrist [email protected]
CM High Level View
Project CM Repository
SW EngTasks
SCMControls
RequestFor
Change
Read OnlyCopyVer
1Ver1
SCI Ver1
SCI
CurrentBaseline SCIApproved to
Change
Project CM Repository
Thomas L. Gilchrist [email protected]
CM High Level View
Project CM Repository
SW EngTasks
ModifiedSCI
TechReview
SCMControls
SCMControls
RequestFor
Change
Read OnlyCopyVer
1
ApprovedSCI
Ver1
SCI Ver1
SCI
CurrentBaseline SCIApproved to
Change
Project CM Repository
TechReview
Thomas L. Gilchrist [email protected]
CM High Level View
Project CM Repository
SW EngTasks
TechReview
SCMControls
SCMControls
RequestFor
Change
Read OnlyCopy
ModifiedSCI
ApprovedSCI
?
Ver2
SCI
CurrentBaseline SCIApproved to
Change
Project CM Repository
TechReview
Thomas L. Gilchrist [email protected]
Peer Reviews
Thomas L. Gilchrist [email protected]
Reviews Vs. Testing
• Dynamic Vs. Static Testing• Peer Reviews can be done
before code can be tested.• Testing evaluates product while
performing function in target environment.
• Not mutually exclusive
Thomas L. Gilchrist [email protected]
Peer Reviews
• Intended as a method of static verification for software by the producers’ peers to identify defects and areas where changes are needed.
• Method is not intended to replace dynamic testing, but rather to augment it.
Thomas L. Gilchrist [email protected]
What Peer Review Is Not
• A “silver bullet.”• A tool for personnel evaluation.• A quality assurance program.
Thomas L. Gilchrist [email protected]
Static Verification Techniques
• Desk Checking• Walkthroughs• Formal Technical Reviews• Inspection (Fagan)• Management Reviews• Audits• Static Program Analysis• Formal (Mathematical) Verification
Thomas L. Gilchrist [email protected]
Peer Review Methods
Walkthroughs
Methods Typical Goals Typical Attributes
• Minimal overhead• Developer training• Quick turnaround
•Little/no preparation•Informal process•Meetings•No measurement•Not Inspection!
Inspections•Detect and report all defects efficiently and effectively.
•Formal process•Checklists•Structured Meetings•Measurements•Verify phase
Desk Checks• Minimal overhead• Quick turnaround
•Little/no preparation•Informal process•No measurement•No Meetings•Not Inspection!
Copyright © 1998 Philip M. Johnsonwww.ics-hawaii.edu/~johnsonused with permission
Thomas L. Gilchrist [email protected]
Initial Level
InformalChecking
DoWork
Down-stream
Customers Re-Work
InputSource
Materials
ErrorsFound
ProcessProduct
EntryCriteriaLooselyDefined
ExitCriteriaForQualityNotDefined
Work Process
ENTRY
EXIT
Thomas L. Gilchrist [email protected]
Software Inspections
Thomas L. Gilchrist [email protected]
Measure and Improve Product Quality
DoWork
Down-stream
CustomersRe-
Work
ErrorsFound
ProcessProduct
CrossfunctionalChecklists
SoftwareInspection
InputSource
Materials
EntryCriteriaLooselyDefined
ExitCriteriaForQualityDefined
ENTRY
EXIT
Work Process
Thomas L. Gilchrist [email protected]
Improve Product Quality and Reduce Variation
DoWork
Down-stream
CustomersRe-
Work
ErrorsFound
ProcessProduct
CrossfunctionalChecklists
InputSource
Materials
EntryCriteriaDefined
ExitCriteriaForQualityDefined
ENTRY
EXIT
SoftwareInspection
Work Process
Thomas L. Gilchrist [email protected]
To Improve Product AND Process
(EntryCriteria
Met)
DoWork
Down-stream
CustomersRe-
Work
InputSource
Materials
ErrorsFound
ProcessProduct
CrossfunctionalChecklists
ExitCriteriaMet
SoftwareInspection
Repeatable Work Process
Thomas L. Gilchrist [email protected]
To Improve Product AND Process
(EntryCriteria
Met)
DoWork
Down-stream
CustomersRe-
Work
InputSource
Materials
ErrorsFound
ProcessProduct
CrossfunctionalChecklists
ExitCriteriaMet
Statistical DataAnalysis and
Causal Analysis
SoftwareInspection
Repeatable Work Process
Thomas L. Gilchrist [email protected]
To Improve Product AND Process
(EntryCriteria
Met)
DoWork
Down-stream
CustomersRe-
Work
InputSource
Materials
ErrorsFound
ProcessProduct
CrossfunctionalChecklists
ExitCriteriaMet
Statistical DataAnalysis and
Causal Analysis
PD
CA
DemingCycle
(Defect Prevention Process)
SoftwareInspection
Repeatable Work Process
Thomas L. Gilchrist [email protected]
Inspection Characteristics• Works for any kind of technical material -
plans, specifications, reports, design representations, code, etc.
• Structured, rigorous, repeatable method of crossfunctional review.
• Works in "low process maturity" environments: Product Improvement
• Works in "high process maturity" environments: Process Improvement
• Indicated when the material is important
Thomas L. Gilchrist [email protected]
Software Inspections
Software Inspection is a method for the improvement of technical materials with an eye to the identification and reporting of defects as early as possible.
50
Thomas L. Gilchrist [email protected]
Swimlane Process Tool
ProcessParticipant 1
ProcessParticipant 2
Process AreaProcess
Participant 3Process
Participant 4
Task
Tasks that span multilple participants
Meeting
Report or Document
Y
N Decision?
Milestone
Another Task
The Team
The Process
Flow
Solid flow lines go down. Dashed flow lines go up.
R
TeamFlow Process Tool
Thomas L. Gilchrist [email protected]
ProcessParticipant 1
ProcessParticipant 2
Process AreaProcess
Participant 3Process
Participant 4
Task
Tasks that span multilple participants
Meeting
Report or Document
Y
N Decision?
Milestone
Another Task
Swimlane Process Tool
Thomas L. Gilchrist [email protected]
ProcessParticipant 1
ProcessParticipant 2
Process AreaProcess
Participant 3Process
Participant 4
Task
Tasks that span multilple participants
Meeting
Report or Document
Y
N Decision?
Milestone
Another Task
Swimlane Process Tool
Thomas L. Gilchrist [email protected]
ProcessParticipant 1
ProcessParticipant 2
Process AreaProcess
Participant 3Process
Participant 4
Task
Tasks that span multilple participants
Meeting
Report or Document
Y
N Decision?
Milestone
Another Task
Swimlane Process Tool
Thomas L. Gilchrist [email protected]
Inspection Process Flow
See Handout
SW ProcessOwner & Manager
AuthoringGroup
Scribe Moderator Author Representative(1-reviewer)
Downstream Customers(2-5 reviewers)
Inspection Team Members
Sample Checklists
High Level Documents and Standards
Finished Product, Ready for Inspection
Planning
Y
NReady to Inspect?
Inspection Materials- Crossfunctional Checklist- Target Document, Chunked & Zoned- Forms & Instructions- Logistics-High Level Documents
Overview Meeting
Inspection Materials and Instructions
Indiv idual Preparation
Errors & Questions
Error Logging Meeting
Error Logs and Meeting Summary
Y
NAnnother ChunK?
Address Errors
Addressed Product
Verify & Approve
Y
NReady to Exit Process?
Verif iedProduct
InspectionData
Thomas L. Gilchrist [email protected]
Entry/Exit Materials
Checklists– A list of criteria to facilitate the identification of major
errors.– Crossfunctional in nature because contents of deliverable
must address the entry-level materials as well as satisfy all the downstream customers.
High-level Materials– The source materials entered into the process in which
the deliverable was produced.
Standards– Rules and / or guidelines established to improve the
quality of a broad range of customer sets.
Thomas L. Gilchrist [email protected]
Inspection Materials
InspectionProcess
Checklists
Target Document
Thomas L. Gilchrist [email protected]
Inspection Materials
InspectionProcess
High Level Documents
Checklists
Target Document
Thomas L. Gilchrist [email protected]
Inspection Materials
High Level DocumentsStandards
Checklists
Target Document InspectionProcess
Thomas L. Gilchrist [email protected]
Inspection Materials
High Level DocumentsStandards
Checklists
Target Document InspectionProcess
Metrics
Thomas L. Gilchrist [email protected]
Inspection Process
[process owner]
• Knowledge and expertise in software inspections
• Expertise in finding errors which will cause future rework in the deliverable being inspected.
Inspectors. (“Users" or “customers” of thesoftware programelement (product)being inspected.)
Author orauthoring grouprepresentative.
• Knowledge and expertise in developing high quality software.
Trained Moderator • Deliverable with all known errors removed.
• Informed author(s)
Software processowner.
All individuals whoare affected bythe quality of theinspecteddeliverable
• Inspection metrics• Error logs• Inspection checklists• Problem reports for
high level documents and standards
Software Inspection
ver3
1 2
ProjectManagement
AuthoringGroup Moderator Author
Representative Reader DownstreamCustomers Scribe
Inspection Team Members
Sample Checklists
High Level Documents and Standards
Produce Product Specific Checklists
High Level Documents
Product Specific Checklist
Produce Product
Finished Product
Planning
Inspection Materials
N
YReady to Inspect
Overview Meeting
Inspection Materials and Instructions
Individual Preparation
Errors & Questions
Error Logging Meeting
Error Logs and Meeting Summary
N
YAnnother ChunK?
Address Errors
Addressed Product
Verify & Approve
N
YReady to Exit Process?
VerifiedProduct
InspectionData
[A]
[A,B,C]
[B,C]
Gilchrist 3/95
Process Name:
Suppliers Inputs Process Flow Outputs Customers
Process Owner: Date/Ver:Phone: M/S:
Process ID: Page: of
See Handout
Thomas L. Gilchrist [email protected]
Inspection Process
Software Inspection
Has a moderator/team leader been identified and formally trained in the SW inspection technique?
Has all exit criteriadescription (process) been satisfied?
Have the necessary upstream documents and standards been identified and are they available?
Is the author confident that the deliverable is of high quality (he knows of no errors in the deliverable to be inspected).
Are there qualified people available to serve on the inspection team?
Does the authoring group supervision and management aware that they will not be able to participate in the inspection logging meetings (SW inspections are not to be used for evaluating people)?
Is there the possibility that there are major errors? Is there a written process and rules for conducting the
software inspection?
Did the inspection team spend sufficient time finding major errors?
Did the inspection team spend sufficientreporting and logging errors found (but not more that 2 hours per session)?
Were concerns and questions documented? Was an individual or resource identified as a contact
point for each question and/or issue? Were action items documented and assigned? Was the current health of the inspection process
indicated? Have Action Items been resolved? Have all errors been addressed by the author? Has the inspection team decided on the disposition
of the inspected deliverable (release or re-inspect)?
No meeting will last over 2 hours. Checklist will be used to stimulate The emphasis is on finding and reporting major errors. One and only one person from the authoring group is on the inspection team. Focus is on team effort Inspection team management/supervision does not participate in the inspection processes. A trained team leader (moderator) is used to facilitate the process and the meetings. Team members are trained and assigned specific “roles”. Rationale (i.e. Error detection is maximized by using optimum material coverage (i.e.
chunk). If there are more than 10-15 type written pages to be inspected, then the inspection will be divided into “chunks”. The number of major and minor errors, number of questions/concerns, the number of pages (or LOC) inspected
number of team members, and the time spent finding, reporting, and addressing errors will be collected for each chunk.
Documented inspection rules are followed.
To find and remove errors which would cost substantiallymigrate to downstream. To create a quality baseline measure for process improvement.
A Process checks of meetings.B Inspection process key indicators.C Product/process quality.
The authoring group considers all process exit criteria met (the deliverable being produced is ready to be used for it’s intended purpose).
22Ver 3
Gilchrist 3/95
Entry Checklists Exit Checklists
Measures
Triggers
Continued on page:
Process Objectives
Process Scope
Continued on page: Continued on page: Continued on page:
Continued on page:
Continued on page:
Process ID: Page: ofProcess Name:
Rules/Standards Continued on page:
See Handout
Thomas L. Gilchrist [email protected]
Software Inspection Entry Criteria Has a moderator/team leader been
identified and formally trained in the SW inspection technique?
Has all exit criteria found in the activity task description (process) been satisfied?
Have the necessary upstream documents and standards been identified and are they available?
Is the author confident that the deliverable is of high quality (he knows of no errors in the deliverable to be inspected).
Thomas L. Gilchrist [email protected]
Are there qualified people available to serve on the inspection team?
Does the authoring group supervision and management aware that they will not be able to participate in the inspection logging meetings (SW inspections are not to be used for evaluating people)?
Is there the possibility that there are major errors?
Is there a written process and rules for conducting the software inspection?
Software Inspection Entry Criteria
Thomas L. Gilchrist [email protected]
Software Inspection Exit Criteria
Did the inspection team spend sufficient time finding major errors?
Did the inspection team spend sufficient time reporting and logging errors found (but not more that 2 hours per session)?
Were concerns and questions documented? Was an individual or resource identified as a
contact point for each question and/or issue?
Were action items documented and assigned?
Thomas L. Gilchrist [email protected]
Was the current health of the inspection process indicated?
Have Action Items been resolved? Have all errors been addressed by the
author? Has the inspection team decided on the
disposition of the inspected deliverable (release or re-inspect)?
Software Inspection Exit Criteria
Thomas L. Gilchrist [email protected]
Basic Rules of Software Inspection No meeting will last over 2 hours. Checklist will be used to stimulate error
detection. The emphasis is on finding and reporting
major errors. One and only one person from the
authoring group is on the inspection team.
Focus is on team effort in assisting the author to improve product (attacking the author is not permitted).
Thomas L. Gilchrist [email protected]
Basic Rules of Software Inspection Inspection team management/supervision
does not participate in the inspection processes (inspections are not used to evaluate personnel).
A trained team leader (moderator) is used to facilitate the process and the meetings.
Team members are trained and assigned specific “roles”.
Rationale (i.e.,design/style) is not questioned or discussed during the logging meetings.
Thomas L. Gilchrist [email protected]
Basic Rules of Software Inspection Error detection is maximized by using
optimum material coverage (i.e.,10-15 typewritten pages per 2 hour chunk).
The number of major and minor errors, number of questions/concerns, the number of pages (or LOC) inspected, number of team members, and the time spent finding, reporting, and addressing errors will be collected for each chunk.
Documented inspection rules are followed.
Thomas L. Gilchrist [email protected]
If there are more than 10-15 type written pages to be inspected, then the inspection will be divided into “chunks”.
Basic Rules of Software Inspection
Thomas L. Gilchrist [email protected]
Roles and Responsibilities• Moderator• Author/Developer/Engineer/Producer• Inspector(s)
– Key inspector: if absent or unprepared, inspection meeting will be rescheduled.
– Optional inspector: if absent or unprepared, inspection meeting will be held as scheduled.
• Scribe• Reader
– Key inspector
Supervisors of authors are specifically excluded from participating in inspections but are kept informed of
results.
Thomas L. Gilchrist [email protected]
Overview Meeting
• Distribution of Target Document
• Distribution of Checklists, Standards, High Level Documents
• Assignment and Clarification of Roles
• Special Instructions
• Understand Expectations of Moderator/Process
• Acceptance of Materials
• Agreement to Proceed
Moderator Team
Thomas L. Gilchrist [email protected]
Individual Preparation
Moderator Team
• Available for Questions
• Self-Study of Documentation (Up to 2 hours)
• Identify and Describe Errors and Questions (Individually)
Thomas L. Gilchrist [email protected]
Error Logging Meeting
Moderator/Scribe* Team• Record Prep Times*• Facilitate Reporting of Errors• Log Errors/Questions*• "Playback" Error Log*• Record Logging Times*• Determine Disposition of Errors• Kickoff Next Chunk• Process Check*
• Report Prep Time
• Report and Classify Errors
• Build on Synergy
• Process Feedback