ISEB6

8/18/2019 ISEB6

1/26

Tool support for testing (CAST)

Software Testing

ISEB Foundation Certificate Course

1 Principles 2 Lifecycle

4 Dynamic test

techniques

Static testing

! "anagement # Tools

8/18/2019 ISEB6

2/26

Types of CAST tool

Tool selection an$ implementation

Contents


Tool support

1 2

4 !

#

8/18/2019 ISEB6

3/26

Testing tool classification

%equirements testing tools

Static analysis tools

Test $esign tools

Test $ata preparation tools

Test running tools & character&'ase$ *+

Comparison tools

Test harnesses an$ $ri,ers

Performance test tools

Dynamic analysis tools

De'ugging tools

Test management tools

Co,erage measurement

8/18/2019 ISEB6

4/26

Static analysis


Test design

Test data preparation

Coveragemeasures

Test running

Dynamic analysis

Debug

Performancemeasurement

Comp- Test

Where tools fit

%eq Anal

Co$e

.unction

Design

Sys Test

+nt Test

Acc Test

Reuirementstesting

Test !arness" drivers

Comparison

8/18/2019 ISEB6

5/26

Requirements testing tools

Automate$ support for ,erification an$

,ali$ation of requirements mo$els

- consistency checkingconsistency checking

- animationanimation

Tool information a,aila'le from/

0,um ,aluates Softare Testing Tools (su'scription ser,ice)

CAST %eport 1333

orl$ i$e e'

8/18/2019 ISEB6

6/26

Static analysis tools

Pro,i$e information a'out the quality of

softare

Co$e is e5amine$ not e5ecute$

0'6ecti,e measures

- cyclomatic complexitycyclomatic complexity

- others: nesting levels, sizeothers: nesting levels, size

8/18/2019 ISEB6

7/26

Test design tools

enerate test inputs

- from a formal specification or CASE repositoryfrom a formal specification or CASE repository

- from code (e.g. code not covered yetfrom code (e.g. code not covered yet

8/18/2019 ISEB6

8/26

Test data preparation tools

Data manipulation

- selected from existing data!ases or filesselected from existing data!ases or files

- created according to some r"lescreated according to some r"les

- edited from other so"rcesedited from other so"rces

8/18/2019 ISEB6

9/26

Test running tools 1

+nterface to the softare 'eing teste$

%un tests as though run 'y a human tester

Test scripts in a programma'le language

Data test inputs an$ e5pecte$ results hel$ in

test repositories

"ost often use$ to automate regression

testing

8/18/2019 ISEB6

10/26

Test running tools 2

Character&'ase$

- sim"lates "ser interaction from d"m! terminalssim"lates "ser interaction from d"m! terminals

- capt"re keystrokes and screen responsescapt"re keystrokes and screen responses

*+ (raphical *ser +nterface)- sim"lates "ser interaction for #$%& applicationssim"lates "ser interaction for #$%& applications

(#indo's, $cons, %o"se, &ointer(#indo's, $cons, %o"se, &ointer

- capt"re mo"se movement, !"tton clicks, and key!oardcapt"re mo"se movement, !"tton clicks, and key!oard

inp"tsinp"ts

- capt"re screens, !itmaps, characters, o!ect statescapt"re screens, !itmaps, characters, o!ect states

8/18/2019 ISEB6

11/26

Comparison tools

Detect $ifferences 'eteen actual test results

an$ e5pecte$ results

- screens, characters, !itmapsscreens, characters, !itmaps

- masking and filteringmasking and filtering Test running tools normally inclu$e

comparison capa'ility

Stan$&alone comparison tools for files or

$ata'ases

8/18/2019 ISEB6

12/26

Test harnesses and drivers

*se$ to e5ercise softare hich $oes not

ha,e a user interface (yet)

*se$ to run groups of automate$ tests or

comparisons 0ften custom&'uil$

Simulators (here testing in real en,ironment

oul$ 'e too costly or $angerous)

8/18/2019 ISEB6

13/26

Performance testing tools

Loa$ generation

- drive application via "ser interface or test harnessdrive application via "ser interface or test harness

- sim"lates realistic load on the system ) logs thesim"lates realistic load on the system ) logs the

n"m!er of transactionsn"m!er of transactions Transaction measurement

- response times for selected transactions via "serresponse times for selected transactions via "ser

interfaceinterface

%eports 'ase$ on logs graphs of loa$ ,ersus

response times

8/18/2019 ISEB6

14/26

Dynamic analysis tools

Pro,i$e run&time information on softare

(hile tests are run)

- allocation, "se and de-allocation of reso"rces, e.g.allocation, "se and de-allocation of reso"rces, e.g.

memory leaksmemory leaks- flag "nassigned pointers or pointer arithmetic fa"ltsflag "nassigned pointers or pointer arithmetic fa"lts

8/18/2019 ISEB6

15/26

Debugging tools

*se$ 'y programmers hen in,estigatingfi5ing an$ testing faults

*se$ to repro$uce faults an$ e5amine

program e5ecution in $etail

- single-steppingsingle-stepping- !reakpoints or 'atchpoints at any statement !reakpoints or 'atchpoints at any statement

- examine contents of varia!les and other dataexamine contents of varia!les and other data

8/18/2019 ISEB6

16/26


"anagement of testare/ test plans

specifications results

Pro6ect management of the test process e-g-

estimation sche$ule tests log results +nci$ent management tools (may inclu$e

or7flo facilities to trac7 allocation

correction an$ retesting)

Tracea'ility (of tests to requirements $esigns)

8/18/2019 ISEB6

17/26

Coverage measurement tools

0'6ecti,e measure of hat parts of the

softare structure as e5ecute$ 'y tests

Co$e is instrumente$ in a static analysis pass

Tests are run through the instrumente$ co$e Tool reports hat has an$ has not 'een

co,ere$ 'y those tests line 'y line an$

summary statistics

Different types of co,erage/ statement

'ranch con$ition LCSA8 et al

8/18/2019 ISEB6

18/26

Types of CAST tool

Tool selection an$ implementation

Contents


Tool support

1 2

4 !

#

8/18/2019 ISEB6

19/26

Tool Selection and mplementation

Select

,aluate

Lots of tools 0ne tool Tool use$ 'y lots of people

+mplementation processSelection process

8/18/2019 ISEB6

20/26

The Tool Selection Process

Analyse nee$sDefine pro'lem

Consi$er tool supportas a solution

"a7e

'usinesscase

Definerequire$features

Constraints

Short list

,aluate

Demo

9Trial:

,aluators

Deci$eLonglist

8/18/2019 ISEB6

21/26

Where to start !ith tools"

Do not start

- 'ith a vendor visit ) tool demonstration'ith a vendor visit ) tool demonstration

- 'ith a list of tool feat"res and f"nctions'ith a list of tool feat"res and f"nctions

-'hile yo"r testing process is chaotic (good testing is more'hile yo"r testing process is chaotic (good testing is moreimportant than tools (*CAS+ readinessimportant than tools (*CAS+ readiness

Do start

- !y identifying yo"r needs - 'hich test activities have the !y identifying yo"r needs - 'hich test activities have the

'orst pro!lems, prioritise'orst pro!lems, prioritise

- consider constraints, e.g. hard'are, S, integration 'ithconsider constraints, e.g. hard'are, S, integration 'ith

other tools (cosmetic onlyother tools (cosmetic only

8/18/2019 ISEB6

22/26

Tool selection process

After automation requirements are agree$/

- create shortlist of candidate toolscreate shortlist of candidate tools

- arrange demosarrange demos

- eval"ate selected tool(seval"ate selected tool(s

- revie' and select toolrevie' and select tool

Don;t un$erestimate

8/18/2019 ISEB6

23/26

The Tool mplementation Process

Assem'leteam

"anagementcommitment

Pu'licity

+nternalmar7et

research

Pilot

Pilote,aluation

Phase$implementation

Postimplementation

re,ie

8/18/2019 ISEB6

24/26

Pilot pro#ect and implementation

0'6ecti,es of the pilot pro6ect

- gain experience in the "se of the toolgain experience in the "se of the tool

- identify changes in test processidentify changes in test process

- set internal standards and conventionsset internal standards and conventions- assess costs and achieva!le !enefitsassess costs and achieva!le !enefits

+mplementation

- !ased on s"ccessf"l pilot !ased on s"ccessf"l pilot

- needs strong commitment from tool "sers ) managersneeds strong commitment from tool "sers ) managers

(overcome resistance, overheads for learning c"rve(overcome resistance, overheads for learning c"rve

8/18/2019 ISEB6

25/26

Buy

Sell #internally$

Support

Infrastructure

Tool implementation iceberg

8/18/2019 ISEB6

26/26

Summary/ >ey Points


There are many $ifferent types of tool support fortesting co,ering all areas of the life cycle-

Selecting an$ implementing tools nee$s attention an$effort if 'enefits are to 'e realise$-

Tool support

1 2

4 !

#

ISEB6

Documents

Transcript of ISEB6