Just in Time Testing Course Materials

7/17/2019 Just in Time Testing Course Materials

1/108

AmiBug.Com,Inc.

Just-In-Time Testing

www.amibug.com

Robert Sabourin, 2010

Robert Sabourin

[email protected]

AmiBug.Com, Inc.

61 8thavenue

Roxboro, QuebecCanada, H8Y 2W7

514-916-0440


2/108


3/108

AmiBug.Com, Inc.

June 16, 2010 Robert Sabourin, 2010 Slide 1


Robert Sabourin

President

AmiBug.Com, Inc.

Montreal, Canada

[email protected]

AmiBug.Com, Inc.


Overview

Welcome

Some Philosophy

Context Drivers

Turbulence

Skills

Bug Flow

Testing Ideas

Test Triage

Exploratory Testing


4/108

AmiBug.Com, Inc.


Just In Time Testing Robert Sabourin ,

Software Evangelist

President

AmiBug.Com Inc.

Montreal, Quebec,

Canada

[email protected]

AmiBug.Com, Inc.



5/108

AmiBug.Com, Inc.



Welcome

AmiBug.Com, Inc.



Pain points?

What hurts? How Much?


6/108

AmiBug.Com, Inc.



Just In Time For ________?

AmiBug.Com, Inc.



7/108

AmiBug.Com, Inc.



Some Philosophy

AmiBug.Com, Inc.


Fundamental Question

How do you know when you are finished?


8/108

AmiBug.Com, Inc.


Crosby on Quality Quality is defined as conformance to

requirements

Quality is not a measure of

GOODNESS

Phil B. Crosby, Quality is Free

AmiBug.Com, Inc.


Quality is fitness for use

Joseph Juran

Quality ControlHandbook


9/108

AmiBug.Com, Inc.


Gerald M. WeinbergQuality is value to some person

Exploring RequirementsQuality Before Design

Dorset House

AmiBug.Com, Inc.


Edsger W. Dijkstra

Program testing can be used to show

the presence of bugs, but never to show

their absence


10/108

AmiBug.Com, Inc.


Watts S. Humphrey

even the most

experienced software

engineer injects about one

defect for ten lines of code

AmiBug.Com, Inc.


C. Northcote Parkinson

Parkinsons Law:

work expands so as to fill thetime available for itscompletion


11/108

AmiBug.Com, Inc.


James Bach

The classical approach to test design is like playing20 Questions by writing out all the questions in

advance.

AmiBug.Com, Inc.


Purpose of Testing

Common definition: To find bugs before our customers do!

Broader definition: The role of testing is to provide objective input to

facilitate business decisions!

Keeps stakeholders aware of all issues or

concerns that relate to shipping a product!


12/108

AmiBug.Com, Inc.


Bug Defined

To make our job more fun, whenever

we have a concern with software, we

call it a bug.

AmiBug.Com, Inc.


Its all about people! (and the occasional

bug too)



13/108

AmiBug.Com, Inc.



Context Drivers

AmiBug.Com, Inc.


Context Drivers - BTO

Business Value

To whom?

Why?

Technology Solutions

Organization Corporate Structure

Team Structure

Roles and Responsibilities


14/108

AmiBug.Com, Inc.


AmiBug.Com, Inc.


Context Listeners

Find Sources

Monitor Drivers

Anticipate Change

React


15/108

AmiBug.Com, Inc.


Just In Time Testing

Get Ready, Get Set,

Cause here it comes

AmiBug.Com, Inc.


Turbulence

(.wav)



16/108

AmiBug.Com, Inc.


Unprepared


AmiBug.Com, Inc.



Sharpen Testing Skills

Thinker

Detective

Reporter

Diplomat

Negotiator

Cheer Leader

Pragmatist


17/108

AmiBug.Com, Inc.


Philosophy

We have precious little time to run tests!

We must always be prepared!

AmiBug.Com, Inc.


Time


18/108

AmiBug.Com, Inc.


Getting

Things Done

DevelopmentREQFLOW

BUGFLOW

ReleaseCycle

- Who manages them?

- What are they?

- Where can I find them?

- When are they updated?- Why are they changing?

- How are they evolving?

- Do we observe turbulence?

- Who manages them?

- How are they prioritized?

- Where can I find them?

- Are the communicated?

- Do they get reprioritized?- Are business drivers known?

- Are technical risks known?

- Are builds delivered?

- Where do developers work?

- Configuration management?

- Source control? Baseline?

- Transition? Periodic?

- Smoke tests?

- Owners:Dev IT DBA SQA?

AmiBug.Com, Inc.


Getting Things DoneAdapt to change

Triage Testing

Prioritize Bugs

Track Progress

Revised risks?

New test objectives?

Assign testing team members.

Analysis and exploration to more senior team members

Relative importance of testing objective?

Any test objective more important than any other?

Total budget effort

Spread across testing objectives

DAILY

Smoke TestShould the new build be tested at all?

On failure continue with previous build in test.

Confirmation TestHave bugs really been fixed?

Double check in test lab for each bug!

Stress TestingHow well does the application behave in harsh conditions?

Treated as an experiment.

BUILD

FAST TestEach area of functionality has a simple test.

Is functional area stable enough to test

Regression TestDoes application still work as expected?

Did we accidentally break something?


19/108

AmiBug.Com, Inc.



Test Triage

AmiBug.Com, Inc.


"No! Try not, Do. Or do not.

There is no try."

Yoda

Plantosupport

change


20/108

AmiBug.Com, Inc.


Testing Ideas Collect all testing ideas you can find!

List

Sort

Organize

Shuffle

Plantosup

portchange

AmiBug.Com, Inc.


Testing Ideas

How to find them?

Does system do what it is suppose to do?

Does the system do things it is not supposed to?

How can the system break?

How does the system react to its environment?

What characteristics must the system have?

Why have similar systems failed?

How have previous projects failed?

Plantosupport

change


21/108

AmiBug.Com, Inc.


Testing Ideas

Collect testing ideas

From testing ideas build a

series of testing objectives

Each can be assigned as

workto testers

Each can include all, part of,

or multiple testing ideas

Capturetes

tingideas

AmiBug.Com, Inc.


Testing Ideas

I often use Index Cards

Unique id

One testing idea per card

Colour indicates source

Shuffled and reviewed

Organized and reorganized

Sorted, grouped, prioritized and collected

Capturetesting

ideas


22/108

AmiBug.Com, Inc.


Test Idea Sources Capabilities

Failure Modes

Quality Factors

Usage Scenarios

Creative Ideas

States

Data

Environments

White Box

Taxonomies

Capturetes

tingideas

AmiBug.Com, Inc.


Testing Ideas

Investigative

approaches

We become truffle

snorting pigs and try to

find useful information in

all evidence we discover

We can even get goodideas from out of date

sources

Capturetesting

ideas


23/108

AmiBug.Com, Inc.


Testing Ideas Capabilities

Use cases

Functionalrequirements

Documentedrequirements

Implicit requirements

Capturetes

tingideas

AmiBug.Com, Inc.


Testing Ideas

Failure Modes What can break?

Reaction to invalid input?

How does softwarebehave in constrainedenvironment?

Memory

Disk Space

Network Bandwidth

CPU capacity

Shared resources

Stress, Load, Volume

Capturetesting

ideas


24/108

AmiBug.Com, Inc.


Adaptabili

ty

Accessibi

lity

Auditabilit

y

Availability

Continuity

Dependab

ility

Expandab

ility

Functiona

lity

Integrity

Interopera

bility

Maintainability

Operability

Portability

Reliability

Re-usabili

ty

Scalability

Security

Serviceability

Testability

Usability

Application service provider

Automatic content generator

Customized access

Database access

Delivery

Document access

File sharing

Informational

Interactive

Transaction oriented

User-provided content

Workflow oriented High Focus

Medium Focus

Low Focus

Quality Factors ImportanceDifferent Application Types

Capturetes

tingideas

AmiBug.Com, Inc.


Testing Ideas

Usage Scenarios

Identify classes of

users

Identify how users

will use system

Describe scenarios

Use Story board orsimilar approaches

Identify variations

Capturetesting

ideas


25/108

AmiBug.Com, Inc.


Testing Ideas Creative

approaches

Action verbs

Mind Maps

Soap Operas

Lateral Thinking

Capturetes

tingideas

AmiBug.Com, Inc.


State Models

power up

idleinserting

coins

user

choose

make

coffee

service

needed

coin insertedreset button

coin return

right amount

enteredcoin return

button pushed

no cups

OR no coffee

OR sensor jam

cup removed

Testing Ideas

Capturetesting

ideas


26/108

AmiBug.Com, Inc.


Testing Ideas Data

Flow

Structure

Create

Update

Change

Capturetes

tingideas

AmiBug.Com, Inc.


Testing Ideas

Environment

Hardware

Software

Operating systems

Locales

Browsers

Plug-ins

Co-dependent software

Capturetesting

ideas


27/108

AmiBug.Com, Inc.


Testing Ideas

White Box

Design

Internal structure

Code

Capturetes

tingideas

AmiBug.Com, Inc.


Testing Ideas

Bug taxonomies Collections of possible bugs

Appendix A of TestingComputer Software, Kaner,Falk, Nguyen

Boris Biezer Taxonomy OttoVinter manages

Shopping cart taxonomy GiriVijayaraghavan

Capturetesting

ideas


28/108

AmiBug.Com, Inc.


Which test?Impact estimation

For each test idea guesstimate:

benefit of implementation

consequence of implementation

benefit for not implementing

consequence of not implementing

How credible is the information?

Triagetesti

ngideas

AmiBug.Com, Inc.


Which test?

Test Idea Rejection What If?

If the cost/benefit does not make business

sense then consider implementing:

part of the test, could that lead to part of the

benefit at a more reasonable cost?

more than the stated test, would that generate

more benefit?

a different test than the stated idea, could that

generate more benefit for less cost?

Triagetestingid

eas


29/108

AmiBug.Com, Inc.


Test Triage Test Triage Meeting

Review Context

Business

Technical

Information since last

triage

Test results

Bug results

New testing ideas

Triagetesti

ngideas

AmiBug.Com, Inc.


Test Triage

Allocate Testing Assignments to Testers

Make sure testers know context

Best thing to test

Best person to test it

Best people to explore it

Best lead

Are subject matter experts required

Triagetestingid

eas


30/108

AmiBug.Com, Inc.


Test TriageLife of a test idea

a. Comes into existence

b. Clarified

c. Prioritized

a. Test Now (before further testing)

b. Test before shipping

c. Nice to have

d. May be of interest in some future release

e. Not of interest in current form

f. Will never be of interest

d. Integrate into a testing objective

Triagetesti

ngideas

AmiBug.Com, Inc.


Which test is next?

Questions

Given state of project, state of business, state of

technology, our abilities, our experience and our

history, what we know and what we do not know,

what should we test next?

How much effort are we willing to spend

continuing to test this project? Can we ship yet?

Triagetestingid

eas


31/108

AmiBug.Com, Inc.


Which test is next?

Magic crystal ball If it existed how would you use it?

What question would you ask?

What question would it ask?

Triagetesti

ngideas

AmiBug.Com, Inc.


Deciding what not totest?

Time pressure

Should we skip a test?

If test failed could system still be of

value to some stakeholder?

If test was skipped could important

bugs have been otherwise found?

Triagetestingid

eas


32/108

AmiBug.Com, Inc.


Guidelines and

Decisions

To each stakeholder

risk of failure

consequence of failure

value of success

how much certainty do we have

is it a wild guess or an absolute

truth?

GetStarted

Right

AmiBug.Com, Inc.


Bottom Line

My experience is that it is better to

omit a test on purpose than to skip it

because you ran out of time or

forgot about it!

GetStartedRig

ht

Systematically collecting, evaluatingand triaging testing ideas helps me

decide what not to test - at least for

now?


33/108

AmiBug.Com, Inc.



Exploratory Testing

AmiBug.Com, Inc.


Mandate to explore

The object of your mission is to explore the Missouri river,

& such principle streams of it, as, by its course and communicationwith the waters of the Pacific ocean...may offer the most direct &

practicable water communication across this continent for the

purposes of commerce.

- Thomas Jefferson's letter to Meriwether Lewis, June 1803

William Clark Meriwether Lewis


34/108

AmiBug.Com, Inc.


Make intelligent

decisionsTake notes

about your

decisions

Map out

where you

have been

Others can

use the

result

AmiBug.Com, Inc.


Chart as you explore

Further

exploration

yields a

good idea of

the state of

the world!

One bit at a

time


35/108

AmiBug.Com, Inc.


Exploration Notes

- Tabular

- Chronological

- Schematic

- Point form

- Concise

AmiBug.Com, Inc.


Exploratory Testing

Test Cases Not known in advance

Defined & executed on the fly while you learn

about the product

Map Making Skills

Consistent note taking style Practice


36/108

AmiBug.Com, Inc.


Exploratory Testing During test we must capture

Function, options or sub-functions being explored

Test cases attempted

Comments, notes, images or attachments

Hints, reminders and observations which may be useful to

future testers

Date, Platform, Build or Configuration under test

Name of person running test

Oracles, strategy to assess correctness

Other relevant details

AmiBug.Com, Inc.


An Exploratory TestProcess

Kick Off

Prepare

Run

CompleteReview

Follow Up

Confirm Test Objective

Ensure context known

Ensure HW and SW OK

All tools available

Chunk of 90 to 120 min

Test, Plan, Discover

Wrap up

Collect all notes data

Review results with

Test Lead

Reassess goals

Piece together map


37/108

AmiBug.Com, Inc.


Light Test IdeaWorkflow

AmiBug.Com, Inc.


Entered

Test Now

Elaborated

Test Ran

Elaborate

Run

Light Test Idea Workflow

Rejected

Test Later

Do Not

Test

Enter

Idea

Prioritize

Reconsider

Prioritize

Prioritize

Reorganize

Reject

Change


38/108

AmiBug.Com, Inc.


Comprehensive TestIdea Workflow

AmiBug.Com, Inc.


Captured

Exists

Test Now

Open

To Be

Tested

Clarified

Elaborated

Test Ran

Get

Idea

Capture

Gist

Clarify

Accept

Prioritize

Decide

Approach

Assign to

Elaborate

Assign to

Run

Assigned

to

Elaborate

Assigned

to

Run

Elaborate

Run

Comprehensive Test Idea

Workflow

RedundantIdentify

Duplicate

Punted

Punt

ChangedRefactor

Recapture

Test Soon

Test Later

Do Not

Test

Prioritize

Prioritize

Prioritize

Reconsider

Reorganize


39/108

AmiBug.Com, Inc.


Finished?

How do you know you are finished?

AmiBug.Com, Inc.


You know you arefinished when

the only bugs left are the ones are

acceptable (based on your objective

test team input) ...


40/108

AmiBug.Com, Inc.


the only bugs left are the ones are

acceptable (based on your objective

test team input) ...

At least for now!

You know you are

finished when

AmiBug.Com, Inc.


Thank You

Questions?


41/108

Just-In-Time

Testing Skills Exercise


42/108


43/108

Data looks through meat pie cartWear cool hats

Smoke a pipeCOP OPENS SCENE - AUTHORITY FIGURE

Cool hand signalsWe will see whos cicuits will short out

Wait a minuteIt's elementary dear data

Truly original mysteryCircuits would short out with an original mystery

Give credit for vaste knowledgeIt is not possible

True strengths of HolmesOriginal thought Inspiration

All he knows is stored in his memory bankNow do you see my point

Variation on a themeIs that not the way Sherlock Holmes worked

Definition of deductionFrom general to specific

ReasoningRecognize elements from two different stories

You didn't deduce anythingFraud

Unexpected snakesMeet a most distasteful and untimely demise

DeduceEnabled me to deduce

Put plaque and rope together to derive conclusionSaw rope hanging from bell

Saw plaque read the signDupe of a gang of crooks

Know victim

The real crimeRunning youth was a ploy

Question what is over hereQuestion what you are doing?

Holmes knows which was to goNo - it is a ruse

He stole my goodsKid runs through be chased "Stop It" distracton

I say Holmes were should we head?Computer fools you in other ways

Images so perfectSimulate environment

Watson takes notesDistant perspective

Computer illusionThe real London was ...

How does it work?Tools to see - Visor

Period CostumesGet into PERSONA

Dress the partBackground noise confusion

ImpressiveStereotypes

RolesTouch to Know

Level of SophisticationHolodeck

First Visit

Observations


44/108


45/108

Just-In-Time

Testing Ideas Exercise


46/108


47/108

Id

Date

Focus

Source

Description

Business

Importance

H

M

L

BusImportance

Credibility

-

TechnicalRisk

M

N

S

TechnicalRisk

Credibility

-

Priority

1

2

3

Name

www.amibug.com

Notes

Id

Date

Focus

Source

Description

Business

Importance

H

M

L

BusImportance

Credibility

-

TechnicalRisk

M

N

S

TechnicalRisk

Credibility

-

Priority

1

2

3

Name

www.amibug.com

Notes

Id

Date

Focus

Source

Description

Business

Importanc

e

H

M

L

BusImpo

rtance

Credibility

-

TechnicalRisk

M

N

S

TechnicalRisk

Credibility

-

Priority

1

2

3

Name

www.amibug

.com

Notes

Id

Date

Focus

Source

Description

Business

Importanc

e

H

M

L

BusImpo

rtance

Credibility

-

TechnicalRisk

M

N

S

TechnicalRisk

Credibility

-

Priority

1

2

3

Name

www.amibug

.com

Notes


48/108


49/108

Cleaning

Upgrades

Updates

Periodic maintenance

Emergency repair

Maintainer

Peanuts

Ingredients match labels

Contamination

Health Inspector

Rejected chocolates

Boxed chocolatesUnloader

Box wrapping materials

Empty Boxes

Ribbons

Chocolates

Chocolate Wrapping Paper

Loader

Monthly reports

Daily reports

Batch reports

Auditor

Stops

Starts

Enter manifests

Monitor status

Operator

Large

Medium

Small

Size

Multiple layer

Single layerConfiguration

Bag

OEM Custom

Heart Shaped

Circular

Rectangular

Type

Empty Boxes

Wide

Wire

Cord

String

Thread

None

Ribbons

Tissue

Foil

Wax

Thin

None

Paper

Dense

SparceFrequency

Random

Regular

Distribution

Very Fast

High

Medium

Slow

Conveyor Speed

Input Stream

Hollow

Semi Solid

Jelly

Solid

Viscosity

Filled

Bon Bon

Turtle

Traditional Chocolate

Bar

Praline

Truffle

Configuration

Belge

Swiss

Semisweet

Bitter

White

Hard Candy

Fudge

Dark

Milk

Composition

Three

Two

One

Size

Too Heavy

In Range

Too Light

Weight

Chocolates

Manifest describing box chololate layout selection

Inputs UsersWrap-O-Matic


50/108


51/108

Cleaning

Upgrades

Updates

Periodic maintenance

Emergency repair

Maintainer

Peanuts

Ingredients match labels

Contamination

Health Inspector

Rejected chocolates

Boxed chocolatesUnloader

Box wrapping materials

Empty Boxes

Ribbons

Chocolates

Chocolate Wrapping Paper

Loader

Monthly reports

Daily reports

Batch reports

Auditor

Enter manifests

Monitor statusOperator

UsersWrap-O-Matic


52/108


53/108

The Wrap-O-Matic


54/108

Contents

Section I: Introduction ............................................................................................................. 3

1.1 Purpose ........................................................................................................................ 3

1.2 Scope ........................................................................................................................... 3

1.3 Definitions, Acronyms & Abbreviations ................................................................... 3

1.4 References .................................................................................................................. 3

1.5 Overview ...................................................................................................................... 3

Section II: Overall Description ................................................................................................. 3

2.1 Product Perspective ................................................................................................... 42.2 Product Functions ....................................................................................................... 5

Section III: Specific Requirements ........................................................................................ 5

3.1 External Interface Requirements .............................................................................. 6

3.2 System Features .......................................................................................................... 6

3.3 Performance Requirements ...................................................................................... 7

3.4 Design Constraints ...................................................................................................... 7

3.5 Software System Attributes ........................................................................................ 7


55/108

Section I: Introduction

1.1 PurposeThe purpose of this Software Requirement Specifications document is to outline the requirements and

functionality of the Wrap-o-Matic, a machine designed to wrap chocolates.

1.2 ScopeThe software this document describes is hereto referred to as WrapSoft, the software to be installed on the

embedded system that controls the physical hardware of the Wrap-o-Matic.

WrapSoft will control all functionality, timing, and reporting concerning the Wrap-o-Matic machine.WrapSoft will allow operator manifest data entry describing a production run.

1.3 Definitions, Acronyms & AbbreviationsInputs to the Wrap-o-Matic include the following:

Chocolates: Specification of input chocolates including:

Composition

Configuration

Viscosity

Weight

Size

Input Stream Distribution Frequency and PatternManifest: Description of a production run including the box chocolate layout selection.

Paper: Refers to the type of paper used to wrap chocolatesRibbons: Refers to the type of ribbon used to wrap chocolates

Empty Boxes: Refers to the type of boxes wrapped chocolates are placed in including:

Box Shape

Box Size

Box Configuration

Wrap-o-Matic Users include:

Operator: Refers to the operator of the Wrap-o-Matic, whose tasks include monitoring status,

Entering manifests, Starts, Stops

Auditor: Refers to the auditor of the Wrap-o-Matic, whose tasks include reviewing Batch reports,

Daily reports, and Monthly reports

Loader: Refers to the loader of the Wrap-o-Matic, whose tasks include loading materials into the

Wrap-O-Matic including wrapping paper, chocolates and ribbons.Unloader: Refers to the unloader of the Wrap-o-Matic, who is responsible for unloading Boxed

chocolates and Rejected Chocolates.

Health Inspector: Refers to the health inspector, who inspects the Wrap-O-Matic for Contamination, checks

whether Ingredient match labels, and for the presence of nuts.

Maintainer: Refers to the maintenance personnel responsible for Emergency repair, Periodic

maintenance, Updates, Upgrades, and Cleaning

WrapSoft: The software that will control the Wrap-o-Matic

Wrap-o-Matic: The physical machine designed to wrap chocolates

1.4 References

This document complies with IEEE Standard 830-1998 for Software Requirement Specifications.

1.5 OverviewThe remainder of this document contains a full description of the functionality of the Wrap-o-Matic

followed by specific requirements of the system.


56/108

Section II: Overall Description

2.1 Product Perspective

The Wrap-o-Matic performs the packaging step of a system designed to create boxes of chocolates. The

inputs to the Wrap-o-Matic (including chocolates, manifest, papers, ribbons, and empty boxes) can be

created using any method or machinery desired, subject to the size and shape constraints of the configured

inputs to the Wrap-o-Matic. The Loader loads these inputs into the Wrap-o-Matic. The output, packagedchocolates can then be further boxed and shipped to the appropriate location using any machinery desired.

The Unloader is responsible for unloading the boxed chocolates for further processing.

2.1.1 System Interfaces

The Wrap-o-Matic interfaces with the remainder of the chocolate box creation system at the loading and the

unloading area.

Loading Area: The system external interfaces to the Wrap-o-Matic providing chocolates, papers,

ribbons, and boxes.

Unloading Area: The system external interfaces allowing for unloading boxed and rejected chocolates.

2.1.2 User Interfaces

Users interact with the Wrap-o-Matic through different interfaces depending on their roles:

Operator: The Operator will interact with the console. WrapSoft provides functionality allowing the

operator to monitor the status of the Wrap-o-Matic, enter manifests, and start/stop the

machine.

Auditor: The Auditor will interact with the console to request production reports. WrapSoft must

therefore provide functionality to create Daily, Monthly, and Batch reports.

Loader: The Loader interacts with the loading areas of the Wrap-o-Matic.

Unloader: The Unloader will interact with the unloading area of the Wrap-o-Matic.

Health Inspector: The Health Inspector will interact with the console. The Health Inspector can pause

the Wrap-o-Matic and remove and inspect completed boxes of chocolates.

Maintainer: The Maintaner will interact with the Wrap-o-Matic to ensure proper functionality and

maintenance. The Wrap-O-Matic console enables the Maintainer to update or upgrade Wrap-

O-Matic firmware.

2.1.3 Operations

Input Manifest:

In this mode, the Wrap-o-Matic will enable the operator to input the Manifest information.

Running:

In this mode, the Wrap-o-Matic is packaging the chocolates.

Stopped:

In this mode, the Wrap-o-Matic has stopped packaging chocolates.

Data Logging:


57/108

In the background at all times the reporting functions of WrapSoft are monitoring and storing data.

Backup

All stored data in the Wrap-O-Matic may be backed up on a third party storage device at user-configurable

intervals.

Restore

Wrap-O-Matic data may be restored from previously prepared backups.

RestartIn the case of a malfunction, the system can be restarted after error conditions or malfunctions are cleared.

2.2 Product Functions

2.2.1 Wrapping & Ribboning Chocolates

The Wrap-o-Matic may takes individual chocolates and wraps them in paper and ties them with a ribbon.

The wrapped chocolates are then inserted into a box.

Paper available to wrap the chocolates: None, Thin, Wax, Foil, Tissue

Ribbon available to tie the chocolates: None, Thread, String, Cord, Wire, Wide

2.2.2 Boxing Chocolates

Once the individual chocolates have been wrapped and/or ribboned (if specified), the Wrap-o-Matic inserts

the finished chocolates into the selected box. The Wrap-o-Matic is designed to box chocolates with the

following parameters:

Type: Rectangular, Circular, Heart Shaped, OEM Custom, Bag

Size: Small, Medium, Large

Configuration: Single Layer, Multiple Layer

2.2.3 Reporting

WrapSoft automatically performs all data logging and report creation in the background. This includes

Monthly, Weekly, and Batch Reports.


58/108

Section III: Specific Requirements

3.1 External Interface Requirements3.1.1 User interfaces

3.1.1.1 The Wrap-O-Matic console consists of a display, keyboard and mouse.

3.1.1.2 The display is a 14 inch LCD monitor with a resolution of 1366x768 pixels.

3.1.1.3 The keyboard is a US 102 Key QWERTY Layout Keyboard are available for operator

data entry.3.1.1.4 The mouse is PC style with two buttons.

3.1.1.2 The operator can use the console to stop and start the Wrap-O-Matic, and enter &

configure manifests (see 3.2.4.2 to 3.2.4.5).

3.1.1.3 The auditor can use the console to display and print Daily, Monthly, and Batch reports

(see 3.2.4.x).

3.1.1.4 There are twenty inbound conveyor belts to accept the chocolates loaded by the loader

(see 3.2.1.1).

3.1.1.5 There are four loading slots to accept paper

3.1.1.6 There are four spools to accept ribbons

3.1.1.7 There is one bin to accept empty boxes

3.1.1.6 There is an output conveyor belt that transfers the completed boxes of chocolates to the

unloader (see 3.2.3.1).

3.1.2 Hardware interfaces3.1.2.1 No additional hardware interfaces are required.

3.1.3 Software interfaces

3.1.3.1 No additional software interfaces are required.

3.1.4 Communications interfaces

3.1.3.2 No additional communication interfaces are required.

3.2 System Features3.2.1 Loading of Chocolates & Additional Material

3.2.1.1 There are twenty inbound conveyor belts.

3.2.1.2 There are four input paper trays.

3.2.1.3 There are four input ribbon spools.

3.2.1.4 There is a loading area where the empty boxes are loaded by the loader.

3.2.2 Wrapping & Boxing of Chocolates

3.2.2.1 Chocolates can be wrapped in Thin, Foil, Wax or Tissue paper, or not wrapped at all.

3.2.2.2 Chocolates can be tied in Thread, String, Cord, Wire or Wide ribbon, or not tied at all.

3.2.2.3 Chocolates can be placed in boxes in the pattern as designated in the manifest.

3.2.2.4 Chocolates can be placed in Rectangular, Circular, Heart shaped or OEM Custom

boxes, or in a bag.

3.2.2.5 Chocolates can be placed in Small, Medium, or Large boxes.

3.2.2.6 Chocolates can be placed in boxes with Single or Multiple layers.

3.2.3 Output of Chocolates

3.2.3.1 Completed boxes of chocolates are transferred to the unloader.

3.2.4 Monitor Control of the Wrap-o-Matic

3.2.4.1 The monitor has a machine status panel providing information on the current state of the

Wrap-o-Matic: current manifest information, rate and frequency of chocolate

processing, and a constantly updated batch report. This is the main display tab.3.2.4.2 The monitor has a button to start the machine (commence processing with current

settings).

3.2.4.3 The monitor has a button to stop the machine (immediately cease processing with

current settings).

3.2.4.4 If the monitor is stopped and subsequently started, processing resumes as previously.

3.2.4.5 If the monitor is stopped and subsequently reconfigured, the Wrap-o-Matic will clear all

other chocolates from the system prior to starting the process with the new manifest.


59/108

3.2.4.6 The monitor has a secondary tab for all reporting purposes. From this tab, the auditor

can select a date range, the type of report (Weekly, Monthly or Batch reports), and

generate the reports.

3.2.4.7 When a report is requested, it is displayed on the screen. On this report display there is a

button offering to email or print the report.

3.2.4.8 The monitor has a reset button that will completely purge all materials currently being

processed by the Wrap-o-Matic, and reset WrapSoft to the default manifest.

3.3 Performance Requirements3.3.1 The Wrap-o-Matic supports and requires only one terminal and a single user.

3.3.2 There will not be user accounts. WrapSoft will have a simple password protected interface (see

3.5.3).

3.3.3 There are no strict timing requirements on the system, however all user actions must be realized

within 10 seconds of instruction initiation.

3.3.4 The Wrap-o-Matic should complete one box of chocolates per ten seconds.

3.4 Design Constraints3.4.1 Standards Compliance

3.4.1.1 FDA Food processing standards conformance.

3.5 Software System Attributes3.5.3 Security

3.5.3.1 The Wrap-o-Matic security will be controlled by WrapSoft, which has a simple

password protected interface known only to personnel with the appropriate privileges.

Operator passwords allow only login and operation of the Wrap-o-Matic. Administrator

passwords allow the password(s) to be changed, in addition to the privileges above.


60/108

Decision Table

Wrapping & Ribboning Chocolates

Wrap-O-Matic Decision Table

Wrapping Rules:

o The Wrap-O-Matic:

Disallows ribbons applied to unwrapped chocolates.

Disallows hollow chocolates tied with metallic ribbon.

Uses the gentle wrapping algorithm with tissues wrappers.

Uses the rapid wrapping algorithm whenever chocolates do not have

ribbons and do not have tissue wrappers.

Uses the gentle algorithm whenever hollow chocolates are tied with

ribbons.

Uses the normal algorithm for all other cases.

Wrap-O-Matic Decision Table

Rules

R01 R02 R03 R04 R05 R06 R07 R08 R09 R10 R11 R12 R13 R14

Viscosity Hollow Not Hollow

Ribbon Metallic Other None Metallic or Other None

WrapperMetallic

orPaper

Tissue NoneMetallic

orPaper

Tissue NoneMetallic

orPaper

Tissue NoneMetallic

orPaper

Tissue NoneMetallic

orPaper

Tissue

Disallow x x x . . x . . . . . x . .

RapidAlgorithm

. . . - . - X . x . . . x .

NormalAlgorithm

. . . . . . . . . x . . . .

GentleAlgorithm

. - . x x x . x . . x . . x


61/108

StateDiagram

TheWrappingandBoxin

gChocolateProcess


62/108

UseCaseSample

Operation:InputManifest

udPrimaryUseCases

EnterManifest

Operator

ConfigureManif

est

Loader

InputMaterials

InputPaper

InputChocolates

InputRibbons

InputBoxes

InputManifestInformation


63/108

Creating a New ManifestThis use case allows a new manifest to be entered into WrapSoft.

Use Case Name: InputManifestInformation

Participating Actors: Initiated by OperatorCommunicates with Loader

Flow of Events:

1. The Operator activates the Input Manifest Information function of the console2. WrapSoft responds by presenting an options page. The page includes the choice of

configuring or creating a manifest

3. The Operator chooses the option to create a manifest and choose a name for it4. Wrapsoft responds by presenting a form to the Operator. The form includes the following

options: chocolate, paper, ribbons, boxes.

5. The Operator chooses the materials needed by scrolling through the scroll down menu ofeach option. Once completed the Operator submits the form.

6. Wrapsoft receives the form, saves it in the database and notifies the Loader by a pop-updialog

7. The Loader reviews the submitted information and makes sure all the materials are in stockThe loader then sends a confirmation to the Operator to begin.

Entry Conditions: The Operator is logged into Wrapsoft

The Operator stopped the Wrap-o-Matic

Exit Conditions: The Operator receives acknowledgement OR The Operator receives an explanation why transaction cannot be completed

Configuring an Existing ManifestThis use case allows an existing manifested to be configured/altered and then stored into

WrapSoft.

Alternate Flow of Events:

Configuring an existing

Manifest



3. The Operator chooses the option to configure a manifest4. Wrapsoft responds by presenting a page of existing manifests. The page includes all

existing manifest by their respective names.

5. The Operator chooses the manifest needed.6. Wrapsoft responds by presenting a form with the existing selected options of the manifes

chosen. The Operator chooses the materials to be configured by scrolling through the

scroll down menu of the option needed to be changed. Once completed the Operator

submits the form


8. The Loader reviews the submitted information and makes sure all the materials are instock. The loader then sends a confirmation to the Operator to begin.


64/108

Creating a Manifest with Insufficient MaterialThis use case indicates to the operator that the created manifest does not have the materialneeded to start packaging chocolates.

Error Flow:

Material needed out of stock



3. The Operator chooses the option to create a manifest and choose a name for it4. Wrapsoft responds by presenting a form to the Operator. The form includes the followin

options: chocolate, paper, ribbons, boxes.

5. The Operator chooses the materials needed by scrolling through the scroll down menu oeach option. Once completed the Operator submits the form.


7. The Loader reviews the submitted information and makes sure all the materials are instock. One of the materials is out of stock. Therefore the loader sends an explanation

why the manifest cannot be completed at this time


65/108

User (WrapSoft) stories for the Wrap-o-Matic

Identifier Description

IDK00001 WrapSoft can allow operators to login based on a password protected system.

IDK00002 WrapSoft can allow administrators to login based on a password protected system.

IDK00003 WrapSoft can allow administrators to change password(s) and permissions on the system.

IDK00004 WrapSoft can perform a system check.

IDK00005

WrapSoft can deliver messages to the operator based on system checks (these messages include

maintenance warnings, insufficient inputs (paper/ribbon/boxes), etc)

IDK00006 WrapSoft can accept new manifests.

IDK00007 WrapSoft can run a previously saved manifest.

IDK00008

WrapSoft can reconfigure a previously saved manifest, while overwriting or preserving the old

manifest as desired.

IDK00009 WrapSoft can start the processing of chocolates corresponding to a new manifest.

IDK00010

WrapSoft can continue processing an existing manifest while a new manifest is entered or

configured.

IDK00011

WrapSoft can alter the chocolate input speed and frequency by changing the speed of the inbound

conveyor belts.

IDK00012 WrapSoft can determine if a box of chocolates does not meet quality standards and reject the box.

IDK00013 WrapSoft can wrap individual chocolates with paper of type Thin, Foil, Wax or Tissue.

IDK00014 WrapSoft can place a printed manifest describing box configuration into each box of chocolates.

IDK00015

WrapSoft can box chocolates in boxes of type Rectangular, Circular, Heart Shaped, OEM Custom,

or Bag

IDK00016 WrapSoft can box chocolates in boxes of size small, medium, or large.

IDK00017 WrapSoft can box chocolates in boxes of configuraion single layer or multiple layer.

IDK00018 WrapSoft can tie boxes of chocolates with ribbon of type Thread, String, Cord, Wire, or Wide.

IDK00019

WrapSoft can accept and package inbound chocolates of composition Milk, Dark, Fudge, Hard

Candy, White, Bitter, Semisweet, Swiss, or Belge.

IDK00020

WrapSoft can accept and package inbound chocolates of configuration Truffle, Praline, Bar,

Traditional Chocolate, Turtle, Bon Bon, or Filled.

IDK00021WrapSoft can accept and package inbound chocolates of viscosity Solid, Jelly, Semi Solid, orHollow.

IDK00022 WrapSoft can accept and package inbound chocolates of weight Too Light, In Range, or Too Heavy.

IDK00023 WrapSoft can accept and package inbound chocolates of size One, Two, or Three.

IDK00024 WrapSoft can turn off the Wrap-o-Matic.

IDK00025 WrapSoft can reset the Wrap-o-Matic.

IDK00026 WrapSoft can recover reports and manifests stored on a backup disk.

IDK00027 WrapSoft can backup reports and manifests.

IDK00028 WrapSoft can create Monthly, Weekly, and Batch Reports.

IDK00029 WrapSoft can continually monitor and store data for use in reporting functions.

IDK00030 WrapSoft will complete the packaging of one box of chocolates every 10 seconds.

IDK00031 WrapSoft will detect and report paper, ribbon, or box jams.

IDK00032 WrapSoft will monitor temperature levels in the machine and report a warning if temperatures riseabove 35 degrees.

IDK00033 WrapSoft can deliver finished boxes of chocolates to the unloader via the outbound conveyor belt.

IDK00034 WrapSoft can print reports.

IDK00035 WrapSoft can enter idle mode when no manifest is being processed.

IDK00036

WrapSoft can be connected to a network for purposes of global control, emergency shutoff, or

system override.

IDK00037

WrapSoft can continually monitor current status of the Wrap-o-Matic and display this to the operator

or administrator.


66/108


67/108

Just-In-Time

Purpose Exercise


68/108


69/108

AmiBug.Com, Inc.1.1

Robert Sabourin

President

AmiBug.Com, Inc.

Montreal, Canada

[email protected]

Wrap-O-Matic

AmiBug.Com, Inc.1.2

Robert Sabourin ,Software Evangelist

President

AmiBug.Com Inc.

Montreal, Quebec,

Canada [email protected]

www.amibug.com

Wrap-O-Matic


70/108

AmiBug.Com, Inc.1.3

Wrap-O-Matic

New Product

First commercial release

Development is feature complete

Independent system test team

AmiBug.Com, Inc.1.4

Wrap-O-Matic

New Business Model

We will be leasing the Wrap-O-Matic

During rush seasons customers will be

able to use multiple concurrent Wrap-O-

Matics No features have been added

Input and output streams are shared


71/108

AmiBug.Com, Inc.1.5

Wrap-O-Matic

Trade Show Demo

In advance of an upcoming trade show the

testing team is asked to identify possible

problems with the Wrap-O-Matic in order to

avoid them in a trade show context

Focus will be on showing off product toGodiva

AmiBug.Com, Inc.1.6

Wrap-O-Matic

New Feature

Ability to wrap individual chocolates with

ribbon has been added to the Wrap-O-

Matic.

Nothing else was supposed to havechanged.


72/108

AmiBug.Com, Inc.1.7

Wrap-O-Matic

Bug Fix

A bug has been corrected in the paperwrapping modules. The bug was related tohaving multiple chocolates with differentpaper types adjacent to each other in a boxof chocolates.

Nothing else was supposed to havechanged.

AmiBug.Com, Inc.1.8

Wrap-O-Matic

Anticipated Health Inspection

A health inspector will be coming to audit

the Wrap-O-Matic. Special concern is

related to whether the machines

contaminate products. We want to identify possible concerns in

advance.


73/108

AmiBug.Com, Inc.1.9

Wrap-O-Matic

Performance Tuning

The Wrap-O-Matic has been modified to

improve throughput for the most commonly

occurring operational profiles.

No features of the product are supposed to

have changed.

AmiBug.Com, Inc.1.10

Wrap-O-Matic

Competitive Product Comparison A major competitor is releasing a product

(the Wrap-Zapper) designed to replace theWrap-O-Matic. They have great marketingand aggressive sales teams but no trackrecord.

The testing team is challenged to findproblems, concerns, strengths andweakness in the Wrap-Zapper.


74/108


Wrap-O-Matic

Localized Wrap-O-Matic

The new release of the Wrap-O-Matic has

been localized to allow operators to work in

their native languages and with locale

specific units of measure.

No features of the product are supposed tohave changed.


Wrap-O-Matic

Customer Acceptance Testing

A customer intends to purchase a Wrap-O-Matic (as an off the shelf product).

The customer will run the acceptance testsin a real production environment to confirm

the Wrap-O-Matic is acceptable.

Customer will not buy the Wrap-O-Matic ifthe tests expose important concerns.


75/108


Wrap-O-Matic

Knowledge of Project context

Influences testing

ideas

priorities

oracles

Helps to focus testers


Wrap-O-Matic

It is important to understand WHY a

product is being tested.


76/108


77/108

Just-In-Time

Testing Articles


78/108


79/108


80/108


81/108


82/108


83/108


84/108


85/108


86/108


87/108


88/108


89/108


90/108


91/108


92/108


93/108


94/108


95/108


96/108


97/108


98/108


99/108


100/108


101/108


102/108


103/108


104/108


105/108


106/108


107/108

by Robert Sabourin

I am always on the lookout for new testing ideas. Some of the most valuable test ideas come to mind when I am

actually testing. Here is a list of ten sources of test ideas I use on Just-in-Time (JIT) testing projects.

CAPABILITIES.Capability-based test ideas focus on confirming that an application does what it is supposed to do.Requirement and functional specifications can be used as a source of capability-based testing ideas.

FAILUREMODES. Failure mode-based test ideas are what if questions that often are inspired by how a system is

designed. I look at all of the objects, components, and interfaces in a system and ask myself, what if they break or

exhibit some sort of unanticipated failure. Failure modes can be the result of harshly constrained system resources

or forced error conditions.

QUALITYFACTORS. Quality factors are characteristics of a system that must be present for the project to be suc-

cessful. Quality factors are the ilities including: usability, reliability, availability, scalability, and maintainability.

Quality factor test ideas often involve experiments to determine if a quality factor is present. Examples include

performance, load, and stress testing.

USAGESCENARIOS. A usage scenario test idea challenges whether a user can achieve his tasks with the software

under test. To paraphrase the Kennedy inaugural addresswe ask not what the software does for the user, but

rather we ask what the user does with the software. Usage scenario-based test ideas involve identifying who is

using the system, what he is trying to achieve, and in what context.

CREATIVEIDEAS. Creative test ideas come from many sources. I often use deliberate lateral thinking techniques (for

example, Edward De Bonos Six Thinking Hats) to come up with cool and effective tests. I also use metaphors tocome up with testing ideas. I wonder what would happen if the Tasmanian Devil used the system. Perhaps a Dr.

Seuss story inspires some testing ideas. Perhaps great detectives or movies. Pretty much anything goes here.

STATES. When testing an application, a state model helps me come up with test ideas. For example, a transaction

goes through many states of existence from creation through approval, payment, and delivery. I use state models

to identify test ideas such as getting to states, exercising state transitions, and navigating paths through the

system.

DATA. Data is a rich source of testing ideas. Data flow paths can be exercised, different data sets can be used, data

can be cooked up and built from combinations of different data types, and stored procedures can be verified. Test

ideas can be developed to create, update, and remove any persistent data.

ENVIRONMENTS.Exploring how the application behaves in different operating environments is a rich source of test-ing ideas. Environment test ideas can relate to varying the platform, hardware, software, operating system, co-

resident third-party software, and locales.

WHITEBOX. White box test ideas come from reviewing low-level design, code, and data schemas. White box test

ideas include exercising the code, decisions, and paths through the program. White box test ideas look at the work

products of developers to find ways to challenge the application under test.

TAXONOMIES. One of my favorite sources of test ideas is bug taxonomies. These are organized, documented collec-

tions of bugs. I peruse taxonomies randomly in search of interesting bugs. If the bug could happen in my applica-

tion, I come up with a test idea to expose it. Three taxonomies I often use are:

Appendix A of Testing Computer Software(Kaner, Falk, and Nguyen)

Boris Beizer Taxonomy (Otto Vinter)

Shopping Cart Taxonomy (Giri Vijayaraghavan)

46 BETTER SOFTWARE APRIL Sti k Mi d


108/108

Just in Time Testing Course Materials

Documents

Transcript of Just in Time Testing Course Materials