Good luck at your hunt!

Hunt on the white rabbits

By Andrew Issaenko ©May, 2010

Do We test good?

• Kan we predict the end quality?

• Are our test results 100% reliable?

• Is our test planning 100% reliable?

• Do we test as efficient as possible?

Do We Know

For Sure?

Do We Want to

Know for Sure?

Yes!

Test Overview

Qua

lity o

f del

ivera

bles

:

Prod

uct d

efec

ts an

d PC

Rs

Achievement of desired goals:

Coverage of the test scope by testing

Qualit

y of t

he p

roje

ct:

Issue

s ont

he b

usin

ess g

oals

of th

e pro

ject

The real test progress vs planned realization:

Realization and spent time of each planned test task

Test Manager for You?As Project Management

• Progress Tracking: Current situation and work left

• Predictability: Impact on the future test executions by latest test results

• Test Budget Control: Control over spent test time and test budget

Test Manager for You? As Test Engineer/Application Administrator

• Test-Progress: Always up-to-date info about the test progress

• Team work facilitation: Distributing manual test execution work between several persons

• Flexibility: Flexible way of planning test execution

Test Requirements (Representation of the test scope)

Level Name Priority Risk Dependenties Resources

1 Applications (Domains, Sources)

HighMediumLow

High

2 Services InheritedHighMediumLow

InheritedHighMediumLow3 Sub-Services

4 Functions

5 Platforms or configurations

Test Requirements Realization

Level Description of the category

1 Consists of two items for different products under test. Those products are groups of features.

2 Each feature is the group of user functions.

3 Each sub-feature is one user function.

Experience of LO 3.7

Level Description of the category

1 Test scope is separated between new and existing functionality.

2 Each group of features is a functionality related to a particular domain/application.

3 Each feature is a service offered by application to the end user.

4 Each sub-feature is a function that supports this service.

Experience of PZMO Mei 2010

Level Description of the category

1 Each group of features is the type of available business process.

2 Each feature is the business process for the certain type of data.

Experience of dKOC

• Groups of product/features, certain features or subfeatures (product specifications)

• WBS en Project Initiation Document (PID) (project documents)

• Business Importance (define priority)

• Risk expectations from project management and development team (define risks)

Test Requirements Sources

Test Requirements FillingManual Synchronization with external sources

In all cases of filling up test requirements, it is possible to select synchronized test scope manually or automatically by using requirements filters.

Test BridgeFrom/To the test scope

• Automatic generation of test cases for chosen part of Test Scope

• Test Scope on the basis of existing Test Design

• Features Based generation of Test Design

• Sub Features Based generation of Test Design

Filling Test Design • Predefined Attributes for test steps: Name, Action and Expected Result

• Custom Attributes for Test Cases/Steps

• Synchronization with Microsoft sources

• Manual and automated test cases

• Planned Test Execution Time per test caseManual filling up test case steps

Synchronisation with external sources: MS Excel/Word

Test Design Linking • Internal project issue at a test case

• PCR en defects of Issue Manager (Star Team) can be linked to a test case(n) en direct entered within Test Manager on the basis of test results

• Test cases can make use of automated test scripts en test data in Star Team.

Planning Test ExecutionWhat, by Whom

• Each Test Execution in Test Manager is a planned task in project planning

• Planned Test Cases can be defined manually or selected by filter

• Test Execution work distribution between more persons

• Manual and automated tests can be combined but only from the same test container

Planning Test Execution Dependencies

• Startup en Cleanup test procedures (also from another test container)

• Conditional dependencies

• Test deployment on the basis of dependencies

Start Test ExecutionAdapt Test Planning

• Executions Management

• Continue with existing test execution tasks

• Start Test Execution on de basic of previous test results

Test Management Planning and Progress Tracking

• Automatic e-mail notification of involved persons

• Auto-measurement of time spent for test execution

• Manual/planned start of test execution

• Overview of all test execution activities

Test ResultsEnter and Overview

• Latest result and the base for test conclusion per each test sep

• Passed/Failed/Unresolved/Unsupported

• Overview of the current test results during test execution

• Attach any file to clarify test results

Automatische Test• Test Execution Servers define where

application(s) under test runs

• SilkTest AUT define where SilkTest Agent runs

• Run Analyses define from which host(s) dynamic test execution information is collected

• Collect Product code coverage information

Deployment Options

Product Code Coverage

Trophies Room-10• Test Requirements:

– Borland Caliber Synchronization– MS Word/Excel Synchronization

• Test Design: – Automatic generation of test cases– MS Word/Excel Synchronization– Linking with met internal and external

issues– Use of test data/scripts in Star Team

• Test Execution:– Work distribution and Team work

facilitation– Automatic measurement of execution

time– Control and Overview per each test

case and each test step– Integration with code coverage tools

Overal Test ReportsWhich information:

• The quality of deliverables:

• Conformance of deliverables to the desired goals (feat for business)

• The latest test progress vs. planned test realization:

• The quality of project:

Where do you get this information from?:

•Issue Manager: Reports on all deliverables related issues

•Test Manager: Reports on the latest coverage of test scope by test activities

•Test Manager & Planning Tool: One test execution is a task of project planning

•Test Manager: Reports on all project related issues

So.. What Now?• Use of test manager within LG projects is a

goal confirmed and supported by management!

• Import Test Scope from existing sources

• Import existing test cases in Test Manager

• Pink Roccade LG is a learning organization!

Camp Fire• Questions and Discussion