Tests and Requirements: Like Ham and Eggs, Sugar and Spice, Lucy and Desi

T12 Concurrent Class

10/3/2013 11:15:00 AM

"Tests and Requirements: Like

Ham and Eggs, Sugar and Spice,

Lucy and Desi"

Presented by:

Ken Pugh

Net Objectives

Brought to you by:

340 Corporate Way, Suite 300, Orange Park, FL 32073

888-268-8770 ∙ 904-278-0524 ∙ [email protected] ∙ www.sqe.com

Ken Pugh

Net Objectives

A fellow consultant with Net Objectives, Ken Pugh helps companies transform into lean-agile

organizations through training and coaching. His special interests are in communication

(particularly effectively communicating requirements), delivering business value, and using lean

principles to deliver high quality quickly. Ken trains, mentors, and testifies on technology topics

from object-oriented design to Linux/Unix.

9/19/2013

1

[email protected]

www.netobjectives.com

1 Copyright © 2007 Net Objectives. All Rights Reserved. 19 September 2013

Tests and

Requirements:

You Can’t Have

One without the

Other

Ken Pugh

Fellow Consultant

Net Objectives


Ken Pugh

ken.pugh

@netobjectives.com

Photo

Size:

Height: 2.25

Position:

from top left corner

Horizontal 0.75

Vertical 1.

Picture Style: Simple Black

Frame

No code goes in till the test goes on.

A journey of two thousand miles begins with a single step.

� Fellow Consultant

� OOA&D, Design Patterns, Lean, Agile, Test-Driven

Development

� Over 2/5 century of software development

experience

� Author of seven books, including:

– Prefactoring: Extreme Abstraction, Extreme

Separation, Extreme Readability (2006 Jolt Award)

– Interface Oriented Design

– Lean Agile Acceptance Test-Driven Development:

Better Software Through Collaboration

9/19/2013

2


Objectives

� Show how requirements and tests are complementary

facets

� How to create requirements and tests collaboratively


Overall Rule

� There are exceptions to every statement, except this

one

4

9/19/2013

3


Fast Car

� Who wants a fast car?

6 19 September 2013

RequirementsRequirements

9/19/2013

4


Requirement Definition

� Requirement is:

– Condition or capability needed by user to solve problem or

achieve objective

– Condition or capability met or possessed by system

component to satisfy contract, standard, specification or

regulation

7


Requirements

� Requirements should include

– Functions – what the software should do

– Constraints – limitations on the manifestation

� Requirements should NOT include (usually)

– Ways to implement the requirements

– Specific manifestations – exactly what you want

8

9/19/2013

5


An Brownie Specification - MIL-C-44072C (1)

� 3.3.1 Brownie formula. The formula for the brownie shall be as

follows:

Ingredient Parts by weight

Sugar 1/ 23.0

Flour 2/ 21.0

Shortening 16.8

Nuts 3/ 16.0

whole eggs 4/ 5/ 13.0

Cocoa 5.5

Dextrose, anhydrous 4.4

Salt .03

3/ U.S. Standards for Shelled English Walnuts (Fresh Products Branch, Fruit and

Vegetable Division, Agricultural Marketing Service, U.S. Department of

Agriculture)


An Brownie Specification - MIL-C-44072C (2)

� 3.3.2 Brownie preparation.

a. Whip eggs in large bowl on high speed until light and fluffy.

b. Combine sugars, cocoa, salt, and leavening; add to beaten eggs, and whip

on high speed until thick.

c. Add shortening slowly while mixing on low speed.

d. Scrape bowl and whip on high speed until thick.

e. Mix flour, nuts, and flavors together and fold into batter; mix until

uniform.

f. Pour batter into pan at a rate that will yield uncoated brownies which,

when cut such as to meet the dimension requirements specified in 3.4f, will

weigh approximately 35 grams each.

g. Bake at 3500F until done (30 to 45 minutes).

9/19/2013

6


Characteristics of Requirements

� Specific

� Unambiguous

� Testable/Verifiable

� Singular

� Design Independent

� Necessary

� Correct

� Consistent

� Complete

� Documented

� Prioritized

� Modifiable

11

These characteristics often applied to requirements. Bold indicates characteristics emphasized by agility


Question

� How do you know a requirement is testable?

9/19/2013

7

13 19 September 2013

Software Software

DevelopmentDevelopment


The Team

� Customer Unit - develops the requirements

– Product Owner

– Business analysts

– Users

– Quality Assurance

� Developer / Tester Unit – develops the implementation

– Programmers

– Testers / Quality Assurance

� The Triad

– Customer, developer, tester

9/19/2013

8


Value Stream Map – Classical Development

Elicit RequirementsAnalyze

Requirements Design

Code Test

Why go back?

Deploy


Why Mistakes?

� Misunderstandings, missed requirements, mis-other

� Feedback helps to correct misunderstandings

� Quick feedback better than slow feedbackDesired ActualActual

9/19/2013

9


Value Stream Map – Agile Development

Elicit Requirements

Analyze

Requirements With

TestsDesign

Code With Tests Deploy


Move Testing Forward

� Two types of testing

� Attempting to find defects – is WASTE

� Attempting to prevent defects – is ESSENTIAL

� When are defects found?

– Prevention is just early detection

9/19/2013

10


Steps

� Author the tests (write)

– Customer, tester, developer together

� Connect tests to system (bind)

– Developer unit

� Run the tests (run)

– Developers, testers, customers

– Automated – part of build

Test write bind run


Usually the Case

Codewrite run

bind

9/19/2013

11


A Better Way

Codewrite

run

bind


Testing Strategies

Meszaros, XUNIT TEST PATTERNS: REFACTORING TEST CODE,

Fig 6.1 "Purpose of Tests" p. 51, © 2007 Pearson Education, Inc

9/19/2013

12


Another Look at Testing Strategies

Customer Tests Exploratory Testing

Expected Results Unexpected Results

Valid Input Specified effects Unspecified effects

Invalid input Specified error message Any side effects

Focus of the tests (not exclusive)


Verification and Validation

� Verification

– Have we got the requirements right?

– Focus of acceptance tests

� Validation

– Have we got the right requirements?

� Customer really does it this way

� Environment is what we believe it to be

– How do you test this?

9/19/2013

13


Acceptance Testing

Application /

Middle tier

Core Business Logic

UI

Persistence

Acceptance

Tests


Tests all the way down

Requirement Requirement

Test Test

Requirement

Test

Test

Service

Implementation

Test

Customer

Acceptance

Tests

Developer

Acceptance

Tests

Service

Implementation

Requires services

to implement

9/19/2013

14


Definitions

� Acceptance criteria

– A statement of how something will be tested

� Acceptance test outline

– Steps in an acceptance test

� Acceptance test

– A test with all information for clear pass/fail repeatability


Requirements and Tests

� Requirements and tests are inter-related

– You can’t have one without the other

� Failing test is a requirement

– Passing test denotes specification on how system works

9/19/2013

15


ATDD as a Prism

RequirementsATDD


Acceptance Test Acceptance Test

ExamplesExamples

9/19/2013

16


A First ExampleA First Example


First Example

� As a user, I want to convert a temperature in Fahrenheit to a

temperature in Celsius.

� What is an acceptance criteria for this?

� What’s your estimate?

9/19/2013

17


Acceptance Test Outline

� Given:

– A temperature in Fahrenheit

� When

– The user converts it

� Then

– The result should be the corresponding temperature in

Celsius

� Exercise: What specific tests should there be?


Acceptance Tests

Fahrenheit (Input) Celsius

(Output)

Notes

32 0

212 100

-459.7 -273.15 Precision

Error -273.151 Below 0 Kelvin

1000 550 Maximum – Needed?

9/19/2013

18


StoriesStories


Context

� Sam’s Lawn Mower Repair and CD Rental Store

– A custom software package for CD rentals.

9/19/2013

19


User Story Examples

� As the clerk, I want to check out a CD for a customer.

� As the clerk, I want to check in a CD for a customer.

� As the inventory maintainer, I want to know where every CD is—in the store or rented.

� As the finance manager, I want to submit a credit card charge every time a CD is rented so that the store does not have to handle cash.

37


Story Acceptance Criteria

� Check Out CD

– Check out a CD. Check to see that it is recorded as rented.

� Check In CD

– Check in a CD. Check to see that it is recorded as returned.

– Check in a CD that is late. Check to see that it is noted as late.

� Report Inventory

– Check out a few CDs. See if the report shows them as rented.

– Check in a few CDs. See if the report shows them as in the store.

� Charge Rental

– Check in a CD. See if the rental charge is correct. See if the credit charge matches the rental charge. See if the charge is made to the credit card company. Check that the bank account receives money from the charge.

9/19/2013

20


Developable Stories

� To become developable, stories require– More detailed requirements

�Workflow or use cases

� Possibly mock-up user interfaces

– Acceptance test outline against those details

39


Detailing Stories

� Use Cases

� Event / Response Tables (if there’s time)

9/19/2013

21


Use Case Template – Part One

� Name—Identifier to easily reference it by

� Description—Brief note

� Actor—Who initiates the use case

� Pre-conditions—What must be true before the use

case is initiated

� Post-conditions—What is true if the use case

successfully executes

� Main course—Steps that show the sequence of

interactions

41


Use Case Example – Part One

� Name—Check Out CD.

� Description—Check out a CD for a customer.

� Actor—Clerk.

� Pre-conditions—The customer has an identification. The CD has an identity.

� Post-conditions—The CD is recorded as rented. The rental contract is printed.

� Main Course: 1. The clerk enters the customer identification and CD identifier into

the system.

2. The system records the information.

3. The system prints a contract that the customer signs.

42

9/19/2013

22


Use Case Template – Part Two

� Alternatives: flow that allows the use case to be successful even if some condition occurs

� Exceptions: conditions can occur that may not allow it to reach its post-conditions.

� Business Rules: rules that must be followed regardless of technology

� Note: Alternatives and Exceptions could become separate story(ies) if requires significant effort

� Exceptions and Alternatives numbered with reference to Main Course steps

43


Use Case Example – Part Two

Exceptions:1a. Customer identification is not recognized.

Clerk repeats step 1.

1b. The customer violates the CD Rental Limit business rule.

The clerk notifies the customer of the violation.

The use case is abandoned.

Business Rule:CD Rental Limit

A customer can rent only three CDs at any one time.

Alternatives

3a. The printer jams.

The clerk fills out the contract by hand.

The use case exits.

44

9/19/2013

23


Acceptance Tests

� List of Acceptance Tests– Rent a CD—This is the main course.

– Bad Customer ID—Enter the customer ID wrong.

– CD Rental Limit—A customer has three CDs and rents

another one.

– Printer Jam—Simulate a printer jam (maybe out of paper).

45



OutlineOutline

9/19/2013

24


Test Flow

Initial

SystemState

Test

Setup

(Given))

Trigger

(When)

Assert

(Then)

Final

System

State and

Output

Action or Event

Expected

System

State and

Output


Given / When / Then Outline

� Given (Setup)

Customer has ID (initial system state)

CD has ID (initial system state)

CD is not currently rented (initial system state)

� When (Trigger)

Clerk checks out CD (action)

� Then (Verify)

CD recorded as rented (final system state)

Rental contract printed (output)

9/19/2013

25



Using Tables Using Tables


Table Types

� Calculation Table

� Action Table

� Data Table

9/19/2013

26


Calculation Table

Title

Input Name 1 Input Name 2 Result Name? Notes

Value for input 1 Value for input 2 Expected output

Anything that

describes

scenario

Another value

for input 1

Another value

for input 2

Another

expected output


Calculation Table Example

Discount Calculation

Item Total Customer Rating Discount Percentage?

$10.00 Good 0%

Given

When

Then

9/19/2013

27


Data Table

� Exists (or should exist) – for Given and Then

Title Data

Value Name 1 Value Name 2

Value for 1 Value for 2

Another value for 1 Another value for 2


Data Table Example

Customer Data

Name ID

James 007

Maxwell 86

9/19/2013

28


Action Table

� Enter enters data into an entry field

� Press initiates a process, such as a Submit button on a dialog box

� Check sees if a result is equal to an expected value.

Action Name

Enter Value Name 1 Value for 1

Enter Value Name 2 Value for 2

Press Submit

Check Value Name 3 Expected value for 3


Action Table Example

Check Out CD

Enter Customer ID 007

Enter CD ID CD2

Press Rent

Check Rented True

Can be just “When” or a stand-alone Given-When-Then

Given

When

Then

9/19/2013

29


Full Full

Requirements Requirements

and Tests and Tests


Given / When / Then Example Outline

� Given (Setup)

Customer has ID (initial system state)

CD has ID (initial system state)

CD is not currently rented (initial system state)

� When (Trigger)

Clerk checks out CD (action)

� Then (Verify)

CD recorded as rented (final system state)

Rental contract printed (output)

9/19/2013

30


Full Test

Check Out CD

� Given Customer has ID

and CD has ID

and CD is not currently rented

Customer Data

Name ID

James 007

CD Data

ID Title Rented

CD2 Beatles Greatest Hits No


Full Test (2)

� When a clerk checks out a CD:

Check Out CD


Enter CD ID CD2

Press Rent

9/19/2013

31


Full Test (3)

� Then the CD is recorded as rented and a rental contract

is printed:

CD Data

ID Title Rented Customer ID

CD2 Beatles Greatest Hits Yes 007

Rental Contract

Customer ID Customer Name CD ID CD Title

007 James CD2 Beatles Greatest Hits


The Action

� Can drive a GUI

� Or a method Rent(CustomerID aCustomer, CDID, aCD);

9/19/2013

32


Full Test - Exception

Check Out CD with CD Rental Limit

� Given Customer has ID and CD has ID

� and the Customer has three rentals already

Customer Data

Name ID

James 007

CD Data

ID Rented Customer ID

CD2 Yes 007

CD3 Yes 007

CD4 Yes 007

CD5 No


Full Test Exception (2)

� When a clerk checks out a CD

� Then CD Rental Limit occurs

Check Out CD


Enter CD ID CD5

Press Rent

Check Status CD_Rental_Limit_Exceeded

9/19/2013

33


The Other Tests

� To be developable

– All tests should be complete

� You are going to need them all anyway

� The tests detail the requirement

� Pragmatically

– Sufficient tests need to be complete

– Defining “sufficient” is the hard part


When To Write When To Write

Story TestsStory Tests

9/19/2013

34


Iteration Based

Product

Backlog

Acceptance

Criteria

Iteration

Backlog

Acceptance

Tests (or

Outline)

Make Developable


Make Developable

Codewrite

run

bind

Codewrite

run

bind

Previous iteration

Within iteration

9/19/2013

35


Iteration-less (Kanban)

Idea

Queue

Prioritize Ready to

Analyze

Analyze Ready to

Detail

Create

accept-

ance

tests

Ready to

Develop

Develop Ready

for

“Release

Set”

testing

Idea Story Story Story Done


Not An Ending, Not An Ending,

But A Beginning But A Beginning

9/19/2013

36


Outlines

� Objectives were:

– Show how requirements and tests are complementary facets

– How to create requirements and tests collaboratively


Other Things Tests Are Good For

� Acceptance tests can also be used as:

– Estimation aid

� To break stories down if they are too big

– Degree of doneness

� Percentage of passing acceptance tests is “rough” indication of story

completion

– Regression test suite

� particularly if automated

9/19/2013

37


Why Acceptance Tests?

� Rework Down from 60% to 20%

� Workflows Working First Time

� Little Room for Miscommunication

� Saving Time

� Getting Business Rules Right

� Game Changing

� Tighter Cross-Functional Team Integration

� Crisp Visible Story Completion Criteria

� Automation Yields Reduced Testing Time


� Now, if there’s time, an exercise

� If not:

9/19/2013

38


� Thank you.

� Please fill out the evaluations


An ExerciseAn Exercise

9/19/2013

39


The Story

� As a computer user, I want to copy a file from one

directory to another.

� What’s your estimate?


The Interface

� GUI

� Command line

CopyFile <SourceDirectory>

<TargetDirectory> <Filename>

9/19/2013

40


Exercise

� Write tests for:

“copying a file from one directory to another”

� Describe in terms of:

– Given: some state

– When: copy (file, source_directory, destination_directory)

– Then: expected state

Source

Directory

Contents

Destination

Directory

Contents

Copy

File

Destination

Directory

Contents?

Anything else?

X X X

Given When Then (Expected)


Copying Table Example

Source

Directory

Contents

Destination

Directory

Contents

Copy

File

Destination

Directory

Contents?

Source

Directory

Contents?

Content

Comparison?

Notes

X, Y, Z D,E,f X D,E,F,X X,Y,Z (Destination.X

== Source.X)

Standard

copy

X, Y, Z D,E,F,X X D,E,F,X X,Y,Z (Destination.X

== Source.X)

X

overwritte

n

X, Y, Z D,E,F,X X D,E,F,X X,Y,Z (Destination.X

==

Destination.X)

X not

overwritte

n

What about modification date and other attributes?

Make more columns

Given When Then (Expected)

9/19/2013

41


Copy File Example

Copy a file (happy path) � Given a source

� and target directory

File system

Directory Name Directory Contents

A X, Y, Z

B D,E,F

Copy File

Enter Source Directory A

Enter Target Directory B

Enter File name X

Press Copy

� When copy request is

made

� Then file is copiedFile system


A X, Y, Z

B D,E,F,X


Copy File Example

Non-existent file Given a source and target directory

File system


A X, Y, Z

B D,E,F

Copy File

Enter Source Directory A

Enter Target Directory B

Enter File name Q

Press Copy Error_file_does_not_exist

� When copy request is made for a non-existent file , an error is generated

� Anything else?

9/19/2013

42


Implementing Implementing

Acceptance TestsAcceptance Tests


A Business Rule Example

If Customer Rating is Good and the Order Total is less

than or equal $10.00,

Then do not give a discount,

Otherwise give a 1% discount.

If Customer Rating is Excellent,

Then give a discount of 1% for any order.

If the Order Total is greater than $50.00,

Then give a discount of 5%.

9/19/2013

43


Business Rule Table = Test

Discount

Order total Customer rating Discount percentage?

$10.00 Good 0%

$10.01 Good 1%

$50.01 Good 1%

$.01 Excellent 1%

$50.00 Excellent 1%

$50.01 Excellent 5%


Ways To Implement Test

� Testing script

� Program interface

� Xunit framework

� ATDD framework

9/19/2013

44


Testing script

� Tester creates script (usually GUI based).

� Example:

– Logon as Customer who is rated Good

– Start order

– Put items in the order until the total is exactly $10.01

– Complete order

– Check it shows a $.10 discount

� Repeat for other five cases


Program interface

� Create a command line or graphic user interface

Screen

Discount

Method

Discount Percentage Screen

Customer Type: Good

Order Total: 10.01

Percentage: 1 %

C:>DiscountPercentage Good 10.01

Percentage: 1%

9/19/2013

45


XUnit Test

class TestCase {

testDiscountPercentageForCustomer() {

SomeClass o = new SomeClass()

assertEquals(0, o.computeDiscount(10.0, Good));



assertEquals(1, o.computeDiscount(.01, Excellent));

assertEquals(1, o.computeDiscount(50.0, Excellent));

assertEquals(5, o.computeDiscount(50.01, Excellent));

}

}


Fit (Table = Test)

Discount

Order total Customer rating Discount percentage?

10.00 Good 0

10.01 Good 1

50.01 Good 1

.01 Excellent 1

50.00 Excellent 1

50.01 Excellent 5

9/19/2013

46


ExampleExample

ContributionsContributions

With StatesWith States


Retirement Contribution Example

� Requirement

The system shall match incoming retirement contribution files

with bank deposits. If the total on the contribution files

matches the bank deposit, the corresponding funds shall be

purchased.

� Exceptions ?

Contribution

File

Bank DepositRetirement Plan

Funding Process

9/19/2013

47


Given (Setup)

PlanID = XYZ

Name Fund

George WildEyedStocks

Sam GovernmentBonds

Bill UnderTheMattress

PlanID Bank Routing

Number

Account Number

XYZ 111-11-11111 12345678


When (Trigger) File Received

401K Contributions PlanID=XYZ

Name Amount

George 5000

Sam 1000

Bill 500

Bank Deposit

Routing Number Account Number Amount

111-11-11111 12345678 6500

9/19/2013

48


Then (Expected)

Expected Output

Funding PlanID = XYZ

Name Fund Amount

George WildEyedStocks 5000

Sam GovernmentBonds 1000

Bill UnderTheMattress 500


Contribution States

Contribution State Meaning

Received Contribution File received

Awaiting Match Waiting for Bank Deposit

Purchase Funds Matched Bank Deposit, buy

funds

Begin Exception Processing Bank Deposit received, but

incorrect

Edit Processing Contribution File bad

format

9/19/2013

49


State Transitions

Contribution State Event New State Notes

Initial Received

data

Received

Received Data check is

bad

EditProcessing

Received Data check is

good

Awaiting Match

Awaiting Match Matched

exactly

Purchase Funds Matching on entry and ½

hour before close

Awaiting Match Discrepancy

in match

BeginExceptionProcessing

AwaitingMatch Matching

time

exceeded

BeginExceptionProcessing

Purchase Funds One minute

prior to

market close

Awaiting Confirmation Enter order for funds


Deposit States

Deposit State Event New State

Initial Data received Received

Received Data check is bad Edit Processing

Received Data check is good Awaiting Match

Awaiting Match MatchedExactly Matched

9/19/2013

50

[email protected]

www.netobjectives.com


A Few ProblemsA Few Problems


A Hotel Display

9/19/2013

51


Another Hotel Display (1)


Another Hotel Display (2)

Tests and Requirements: Like Ham and Eggs, Sugar and Spice, Lucy and Desi

Technology

Transcript of Tests and Requirements: Like Ham and Eggs, Sugar and Spice, Lucy and Desi