Effective Test Planning: Scope, Estimates, and Schedule
description
Transcript of Effective Test Planning: Scope, Estimates, and Schedule
Effective Test Planning:Scope, Estimates, and Schedule
Presented By: Shaun [email protected]
Agenda• Why Plan?• What’s in a test plan?• Define– Objectives– Scope– Priorities
• Estimate work effort• Develop schedule
Software Peace of Mind in ActionSlide 2
Why Plan?
Software Peace of Mind in ActionSlide 3
What’s in a Test Plan?• Most test plan templates available are
derived from IEEE standards• The template presented here is a
minimalist version of that template• Generally, a test plan includes:– Objectives– Scope– Risks & Contingencies– Focus / Priorities – Work Effort Estimation– Go/No Go Criteria
Software Peace of Mind in ActionSlide 4
Sections of the Document
Software Peace of Mind in ActionSlide 5
Software Peace of Mind in ActionSlide 6
Sections of the Document
Software Peace of Mind in ActionSlide 7
Sections of the Document
Software Peace of Mind in ActionSlide 8
Sections of the Document
Software Peace of Mind in ActionSlide 9
Sections of the Document
Test Objectives• Test Objectives should be based on the
needs of the business and established organizational processes & goals– Determine short-term and long-term business
needs– Is the business most concerned about
qualitative or quantitative objectives– How can our test objectives help the business
achieve its objectives– Test plan objectives should tie into business
objectives
Software Peace of Mind in ActionSlide 10
Let’s identify test plan objectives related to each of the following business objectives
Software Peace of Mind in ActionSlide 11
Test Objectives
Scope• Identify the scope and how the test effort
will allow the business to meet its objectives– Determine functionality, processes, and
systems to be validated– Establish types of testing necessary to perform
the validation– Identify supporting systems that should be in
scope – if not in scope then document it– Read available documentation – identify
potential anomalies– Examine existing applications & processes– Get involved EARLY!!
Software Peace of Mind in ActionSlide 12
Risk• Risk is the probability of an undesirable
event occurring and its subsequent impact to a project
Software Peace of Mind in ActionSlide 13
Types of Risk
Software Peace of Mind in ActionSlide 14
Contingency Planning• Because risks tend to have a domino
effect, develop contingency plans for each
• Contingency should align with theprobability and cost of the risk
• Contingency should address the impact of the risk
Software Peace of Mind in ActionSlide 15
Prioritization• Prioritization of application components
assists in determining what functionality has precedence when organizing the test effort.
• Creates a systematic approach for identifying what to test first, what can be minimally tested, and what can be eliminated from testing, if necessary
• Determine criticality and impact of application components based on discussions with SMEs
Software Peace of Mind in ActionSlide 16
Prioritization
Software Peace of Mind in ActionSlide 17
Risk/Impact Analysis• Use a Prioritization Table to determine
which components have the highest priority– Set Risk value (probability of failure)– Set Impact value (impact of failure)– Calculate Total and sort highest to lowest
Component Risk Impact Total PriorityRegister 3 5 15 2
Search 2 2 4 4
Add to Cart 2 5 10 3
Checkout 4 5 20 1
Software Peace of Mind in ActionSlide 18
Estimating the Work Effort• Producing estimates using a defined
process provides greater confidence in its potential accuracy– Is an “accurate estimate” an oxymoron?– Think about The Price Is Right
• Following a process for estimation allows us to review and identify information necessary to complete the project – it is NOT wasted time
• The problem is not in doing an estimate, but in the assumption that an estimate is automatically accurate and therefore a promise!! Software Peace of Mind in
ActionSlide 19
Estimating Size
• Provides direct correlation to the number of tests necessary to validate the component
• Use existing documentation, conversations, and historical data to create estimate Software Peace of Mind in
ActionSlide 20
Estimating Complexity
The more complex an application, the longer it will take to analyze, create, and execute the tests.
Software Peace of Mind in ActionSlide 21
Estimating Reliability
Use historical analysis to evaluate the likelihood that the development process produces complete, stable, high-quality
applications.
Software Peace of Mind in ActionSlide 22
Estimation ExampleStep 1
Estimate size by counting the number of business rules, logical branches, inputs and outputsComponent BusRule LogBranch Inputs Outputs Totals
Register 5 4 6 4 19
Search 15 25 8 6 54
Add to Cart 13 8 3 25 49
Checkout 54 36 22 8 120
Software Peace of Mind in ActionSlide 23
Estimation Example
Component ComplexityRegister .5
Search 1.5
Add to Cart 1
Checkout 1
Component ReliabilityRegister .5
Search .5
Add to Cart 1.5
Checkout 1
Step 2Estimate complexity & reliability for each component using a 5-point scale of 0.25, 0.5, 1.0, 1.5, and 2.0.
Note that the scales for complexity and reliability are reversed, i.e. a 0.25 for complexity indicates very low complexity, but a 0.25 for reliability indicates a highly reliable component.
Software Peace of Mind in ActionSlide 24
Estimation Example
Step 3Based on historical information or industry standards, determine:
• Average TC Analysis & Creation Time 1.5 hours• Average TC Execution Time 1.0 hours• Average Defect Rate
35%• Productive Hours/Day
6.5 hours
Software Peace of Mind in ActionSlide 25
Estimation Example
Step 4Calculate component estimates
Calculated Factor (CF)= Size * Complexity * Reliability
Analysis & Creation Time (ACT)= CF * Avg TC Analysis & Creation Time ÷ Productive
Hrs/Day Execution Time (ET)
= CF * Avg Time to Execute a TC ÷ Productive Hrs/DayRe-execution Time (RT)
= ET * Avg Defect RateComponent ACT + ET + RT = Total
Register 1.25 0.75 0.25 2.25 days
Search 9.50 6.50 2.25 18.25 days
Add to Cart 17.00 11.50 4.00 32.50 days
Checkout 27.75 18.50 6.50 52.75 days
Test Schedule• Develop a work schedule that assigns
testing resources and timeframes to each test activity and identifies relationships/dependencies between them– Schedules help leads determine if the project is
adequately staffed– Remember to match the experience of your
resources to the level of work necessary, but look for opportunities to challenge them and cultivate their skills
Software Peace of Mind in ActionSlide 27
Contingency Time
Software Peace of Mind in ActionSlide 28
Primary Test Activities• There are 5 primary test activities– Test planning– Test analysis– Test creation– Test execution (includes defect tracking)– Test summary
• Each of these activities may be combined or duplicated for each type of test performed
• Each activity should have associated deliverables identified in the schedule
Software Peace of Mind in ActionSlide 29
Sample Gantt Chart
Software Peace of Mind in ActionSlide 30
Conclusion• Careful planning lays the groundwork for a
successful test effort by providing opportunities to:– Learn the scope of the project and the testing
needs– Research and analyze, early in the project, the
business needs and environment– Identify the relative size, complexity, and
reliability of the application
Software Peace of Mind in ActionSlide 31
Questions
Software Peace of Mind in ActionSlide 32