Testing Planning with

Test Plan Templates

HCA / Nashville, TNMarch 9, 2010

Sponsored by: NASQP Board

Nashville Association Software Quality Professionals

Where IT-QA professionals come to network, to learn, and to exchange ideas.

Web addresshttp://nasqpros.ning.com

Agenda

5:30 PM     Registration and Networking

6:00 PM     Presentation Testing Planning with Test Plan Templates

• Test Planning within SDLC - Katherine McFarland (Vanderbilt)• Traditional and COTS Test Plan Templates – John Bell ( Nissan)• Test Planning for Agile – Charlie Williams (Ingenix Technology Engr.)• Questions / Open discussion

7:00 PM     Program Closure:  NASQP Group Sign-up and Surveys

Overview: Test Planning within SDLC

Test Planning

Project Rarities…NOT!

Test Phases: Review

• Unit (Component, Module, or Class) Testing• This type of testing is defined as the testing of individual

software components after changes have been made • “White-box” testing

•Integration Testing• At its lowest level, it is testing done by developers to check the

interactions between components or modules. • On the other extreme, system integration testing tests

interactions between different systems and can be done after system testing.

Testing Definitions (cont)

• System (Functional) Testing• Verifies that the integrated system satisfies the specified

business requirements. • Investigates both functional and non-functional (commonly

known as the “-ilities” – reliability, portability, maintainability, usability, and efficiency)

• “Black-box” testing

• Acceptance Testing (User Acceptance Testing)• Ensures that the system satisfies the pre-defined acceptance

criteria and to permit the customers to determine whether or not to accept the system.

• Determines whether or not the system is fit for purpose and ready for deployment.

V-Model Diagram

User Requirements

System Requirements

Global Design

Detailed Design

Implementation

Component Test Execution

Integration Test Execution

System Test Execution

Acceptance Test Execution

Operational System

Need, wish, policy, law

Preparation Acceptance test

Preparation System test

Preparation Integration test

QA and Test Planning

• Early involvement• If you relegate QA responsibilities to just testing the

end product, you overlook an opportunity to integrate quality into the entire software development life cycle.

• By getting involved upfront, the QA Specialist also has time to prepare for the actual testing. 

•Testing and requirements are synergistic • The “yin and yang” of the system

Traditional Test Planning

Traditional Test Planning Traditional test plans are those which are following the waterfall, waterscrum, or the “pitch it over

the wall” methodologies.

Traditional Test Plan A test plan/strategy is a document describes the

testing portion of the software development cycle. It is created to inform project managers, testers, and developers about some key issues of the testing process.

In the test strategy is described how the product risks are mitigated via testing, which test types are performed, and which entry and exit criteria apply.

The QA Master Test Plan defines what testing is to be conducted, who will do it and when.

Traditional Test Plan Templates

Major elements of the traditional test plan are: Introduction Project Administration Resource Requirements In Scope and Out of Scope Requirements Test Approach Test Deliverables Dependencies/Risks Approval Log

Traditional Test Plan Templates

Reason for the test plan: Provide clear and concise

information to all stakeholders on the testing approach

Defines the deliverables of the project

Differentiates between “Optional” and “Required” testing

Defines what “done” looks like

COTS Test Planning

COTS Test Planning Organizations, large and small, use

Commercial-Off-The-Shelf software

Testing COTS provides it’s own challenges especially due to unknowns and constraints

Testing templates for COTS testing are the same as used for traditional testing

Testing requires a slightly different mind-set since all application functionality may not be used

Does COTS software have defects? Is there a gap between marketing

promises, management expectations and software functionality?

How should we plan for installing and testing COTS software releases?

What about software security, back-doors, hacks and Easter eggs?

COTS Test Planning

Template for Testing COTS software The QA Master Test Plan - Identify

testing to be conducted

Identify testers

Document test steps

Document expected / acceptable results

COTS Testing

Steps to conduct COTS Testing1. Plan for install / upgrade2. Build QA environment3. Test the environment (smoke testing)4. Automated testing for upgrades5. Stress testing upgrades6. UAT testing7. Reporting results

COTS Testing

Agile Test Planning

Agile Scrum Tester

Charlie WilliamsFebruary18, 2010

©2006 Ingenix, Inc

©2006 Ingenix, Inc

©2006 Ingenix, Inc

Planning Stages

20 Points

10 Points

3 Points

Scrum — Project Vision Drives the Features

Fix These

Estimate These

Features

ScheduleCost

ScheduleCost

Features

Plan

Driven

Value / Vision

Driven

The Plan creates

cost/schedule estimates

The Vision creates

feature estimates

Waterfall Scrum

©2006 Ingenix, Inc

Centralized or Distributed Scrum Teams

©2006 Ingenix, Inc

The Advantages of Scrum

The Scrum Framework 3 Roles:Scrum MasterProduct Owner Delivery Team

3 Artifacts:Product BacklogSprint BacklogBurndown Charts

5 Meetings:Release PlanningSprint PlanningDaily ScrumReview / DemoRetrospective

©2006 Ingenix, Inc

©2006 Ingenix, Inc

Story Points

Story points are often used on agile teams to determine the complexity of the work being done. The number of points completed each sprint determines their velocity ( points per sprint ) and gives management an approximation on how much work can be completed in a given sprint.

©2006 Ingenix, Inc

Estimation

• In an agile software team you don't estimate your work till right before you begin. And you only estimate, in the case of Scrum, the next sprints work. So how do you know how long it will take? You don't. Furthermore, you really don't care. You'll continue to deliver functionality every sprint. As soon as product management and QA say you have a good enough product; it'll be released as a production version. You might have a guess, but until the team estimates it....you really don't know long it will take.

©2006 Ingenix, Inc

©2006 Ingenix, Inc

©2006 Ingenix, Inc

Tester Activities

©2006 Ingenix, Inc

©2006 Ingenix, Inc

©2006 Ingenix, Inc

©2006 Ingenix, Inc

©2006 Ingenix, Inc

Exploratory Testing

Learning Areas for Story Testing :

1. Usage Scenarios2. Capabilities / Confirming3. Across Stories Relationships4. Creative Ideas5. Failure Modes6. Quality Factors

“As a Tester I want to confirm that …, So that …”Type of Test Case Stories

©2006 Ingenix, Inc

Test Automation Pyramid

©2006 Ingenix, Inc

Automate Non-Functional Testing

• Performance Testing

• Load / Stress Testing

• Security Testing

• Scalability Testing

• Data Migration

• Infrastructure Testing

©2006 Ingenix, Inc

©2006 Ingenix, Inc

Testing/Coding: Don’t sit and wait!

• Is any testable part of a story ready? - Test with behind-the-GUI tool such as FIT? - Or other harness to bypass GUI

• Pair with programmers - Test together before check-in - Show them issues - Bugs found here are cheap & easy to fix

Copyright 2007: Lisa Crispin and Janet Gregory

Automation Tools Automation can be accomplished by a broad

range of technologies. Effective automation starts by focusing on what is to be automated/tested and then applies the appropriate tools and approaches

– Common automation options include:• Automated acceptance test frameworks (e.g. FIT,

StoryTeller, or Cucumber, Business Process Test)• Technically oriented automation tools (e.g. SoapUI,

SOATest, STaF)• Functional Automation tools (QuickTest Pro) • Scripting Languages (Ruby, Python/IronPython, Perl,

VBScript)• Performance Testing Tools (e.g. JMeter, The Grinder,

LoadRunner)© Ingenix, Inc. 44

©2006 Ingenix, Inc

Sprint Documentation

The Stories, Code, and Test Cases that are written in the Sprint are the source of documentation.

It is NOT true that Scrum has no documentation.

All stories within a sprint must be defect free in order to be considered "done"

• Any defects that are found during a sprint for a previously completed story are logged and put into the backlog.

• They're estimated just like anything else and prioritized by the product owner.

• If a product owner prioritizes new features over defects, they're choosing functionality over quality and vice-versa.

Sample Definition of DoneWorking Software

• Unit tested• Code Review

Tests (Test Strategy/Tollgate)• Functional Testing• Product Owner Acceptance• Performance/Load Tests• Regression tests (Automated preferably)

Documentation• User documentation/Help• Design• Release Notes• API documentation• Statement of work

Sprint Demo and Review• Team reviews what it accomplished during the Sprint• Typically takes the form of a demo of delivered

product backlog items and/or underlying architecture and a review of metrics and data

• Informal – 2 hour prep time rule• Attendees

– Team (Customer Representative, Developers, Testers, Architect, etc.)– ScrumMaster– Product Owner, Customers– Stakeholders– All other interested parties

• A PowerPoint is not a demo!

Sprint Retrospective

Inspection of process and team practices at the end of every Sprint

Attended only by the delivery team Facilitated by ScrumMaster or third party What went well, what could be

improved ScrumMaster prioritizes improvements

based on team direction Team devises solution to most vexing

problems If there are many ideas, create an

“Implementation Backlog” with ranked ordering of recommendations

©2006 Ingenix, Inc

Ready ? Jump In

Thank You

Open forum for additional questions, comments, etc.

Next Event :Test Planning Workshop

April 2010

Thank you for coming.In an effort to make these

meeting useful and beneficial to all, please complete the survey on our home page.http://nasqpros.ning.com