Test environment

Achilles heel of the future?

Michiel Vroon

Test environment

Sounds familiar?

• I thought we solved this defect before?

• This part worked fine some moment ago!

• Who knows the correct configuration?

• What do you mean, first production??!!

• This looks different from production!

• Do you know a user-id we can use?

• What is in the current release?

• I cannot reproduce the defect!

Some causes

• No dedicated competence

• Difficult to define requirements for test

• There are no processes, plan or strategy

• Design is not clear

• Test data is not maintained

• Too many people involved

• Responsibilities are vague

• Manageable versus flexible

• …

Some solutions

• Define single point of contact

• Set up processes

• Arrange test data

• Work with environment types

• Use master/copy

• Install shared environments

• Have an authorization strategy

Define single point of contact

• Test environment coordinator

– Responsible for setting up and maintaining environment

– Single gateway to all parties involved

– Combination of technical and testing skills

– Control function

– Communicator and project leader

– In bigger organizations:

dedicated department of

test environments

Set up processes - 1

• Define what to deliver when and by whom

• Connection with other processes: design, build,

test and implementation

• Simple process:

1. Specifying the infrastructure

2. Realising the infrastructure

3. Specifying the infrastructure intake

4. Intake of the infrastructure

5. Maintaining the infrastructure

6. Preserving the infrastructure

2 3

5

Start

4

1

6

End

Set up processes - 2

Maintaining the environment

– Configuration management

– Change management

– Release management

Wish Use

Dev Test Acc Prod

Configuration management

Change

Release

Arrange test data

• Define test data

– Production data (real or scrambled)

– Test data (regular system functions or separate)

• Naming test data

– Real names

– Functional names

– Test related names

• Maintain test data

– Cumulative

– Periodic restore

– Several versions

Work with environment types

• DTAP –Model, 4 types of environments

– Development

– Test

– Acceptance

– Production

• Not a technical solution!

• Each type has it’s own characteristics on:

– Maintenance and ownership

– Authorization

– Flexibility

Example

Local development

STenvironment

UATenvironment

Development Test Acceptance Production

Environment in project

Environment type acc. to DTAP

Central development

PATenvironment

Productionenvironment

Shadowenvironment

Use master/copy

• Principle for setting up environments

Master

CopyCopy

Copy

• What is the master?

• What is in the master?

• Who owns the master?

Install shared environments

• An end-to-end environment

• Consistent end-to-end test data

• Shared use by different projects

• Well defined user processes

• Always up and running

• Continuous maintenance

Have an authorization strategy

• How to deal with authorization in the

environment?

• Test users or real life users?

• Impersonal users (roles like “guest”) or

personal users?

The case!

ITO ProjectIntegrale Test Omgeving(Integral Test Environment)

The situation

• Dutch bank

• Department Environments

– Responsible for all test environments

Single point of contact

Department environments

Coordinator environments

Maintenance&

Exploitation

Outsourceparties

Projects

External partners

Deliveryunit

Unix

Local

Central

Test data

Processes

Current situation

• Availability environments

– Demand > supply

– Temporary solutions

• Maintainability environments

– Versions

• Test data environments

– End-to-end consistency

• Complexity

– Replacement old client systems (OLI > SIEBEL)

– Connections

The environments

OLI (25)

X (25)

Y (20)

Z (15)

A (10)

B (5)

Siebel (2)

Replacement old client systems

OLI (25)

X (25)

Y (20)

Z (15)

A (10)

B (5)

Siebel (25)

end-to-end shared environments

OLI (5)

X (5)

Y (5)

Z (5)

A (5)

B (5)

Siebel (5) Connected systemsConsistent test dataMaintainedShared useRules

Goals

• Reduction of systems

• Transparency

– You know what’s in the environment (data, software,

hardware, connections etc)

– It’s consistent

– All users are known

• Shorten setup time

– It’s already there

– Only adjustments of connections

Results

Quaboo

Ludwigstraat 39

4701 NE Roosendaal

The Netherlands

E. michiel@quaboo.eu

M. +31(0)6 21 489 775

I. www.quaboo.eu