Testing Techniques

35
Testing Techniques Equivalence class analysis Cause effect graphs All pair testing

description

Testing Techniques. Equivalence class analysis Cause effect graphs All pair testing. Equivalence classes. Equivalence classes. Equivalence classes. Triangle Problem. Read 3 integer values in range [1..200]. - PowerPoint PPT Presentation

Transcript of Testing Techniques

Page 1: Testing Techniques

Testing Techniques

Equivalence class analysis Cause effect graphs All pair testing

Page 2: Testing Techniques

Equivalence classes

Page 3: Testing Techniques

Equivalence classes

Page 4: Testing Techniques

Equivalence classes

Page 5: Testing Techniques

Triangle Problem

• Read 3 integer values in range [1..200]. • These 3 values represent the length of the

sides of a triangle. The programme displays a message which establishes that the triangle is isosceles, equilateral or scalene.

Page 6: Testing Techniques

Triangle Problem• we are interested in 4 questions:

– Is it a triangle?– Is it an isosceles?– Is it a scalene?– Is it an equilateral?

• We may define the input test data by defining the equivalence class through “viewing” the 4 output groups:– input sides <a, b, c> do not form a triangle– input sides <a, b ,c> form an isosceles triangle– input sides <a, b, c> form a scalene triangle– input sides <a, b, c> form an equilateral triangle

Page 7: Testing Techniques

Valid Classes

Page 8: Testing Techniques

Non Valid Classes

Page 9: Testing Techniques

Non Valid Classes

Could you find other non valid classes ?

Page 10: Testing Techniques

Cause effect graph

Page 11: Testing Techniques

Methodology

Page 12: Testing Techniques

Triangle example

Page 13: Testing Techniques

Triangle Problem

Page 14: Testing Techniques

Cause Effect Graph

Page 15: Testing Techniques

Decision Table

Page 16: Testing Techniques

Test cases

Page 17: Testing Techniques

All pairs testing

• Based on orthogonal Array• Drastically reduces number of test cases• A combination of worst cases and equivalence

class testing• Well supported in the agile methods

community• Supported by commercial and non-

commercial products

Page 18: Testing Techniques

What Does All Pairs Do?

• Input is a set of equivalence classes for each variable.• Sometimes the equivalence classes are those used in

Boundary Value testing (min, min+, nominal, max-, max)• Output is a set of (partial) test cases that approximate an

orthogonal array, plus some pairing information.• Why partial? No expected outputs are provided. natural

enough, but a tester still needs to do this step.

Page 19: Testing Techniques

All Pairs Assumptions

• Variables have clear equivalence classes.

• Variables are independent.

• Failures are the result of the interaction of a pair of variable values.

Page 20: Testing Techniques

Example

• Suppose that we have three parameters.• For each parameter, there are two possible values.

– Values are :• A, B for parameter 1.• J, K for parameter 2.• Y, Z for parameter 3.

• Degree of interaction coverage is 2.– We want to cover all potential 2-way interactions

among parameter values.

Page 21: Testing Techniques

Set of all possible test configurations

A J Y

A K Z

A K Y

A J Z

K YB

B ZJ

B YJ

K ZB

There are 23 = 8 possible test configurations.

Three parameters, twovalues for each.

Page 22: Testing Techniques

Set of all possible degree 2interaction elements

• Coverage measure:– Percentage of interaction elements covered.

There are ( ) 2 2 = 12 possible interaction elements. 32

A J

B K

B J

A K

A Y

A Z

B Z

B Y

J Y

K Z

K Y

J Z

Page 23: Testing Techniques

Test configurations assets of interactions

A J Y

A J

A Y

J Y

One test configuration...

… covers 3 possibleinteraction elements.

Page 24: Testing Techniques

…using a subset ofall test configurations.

Interaction test coverage goal

Goal: cover all interactionelements…

A J Y

K YB

A K ZA K YA J Z

B ZJB YJ

K ZB

A J

B KB JA K

A YA Z

B ZB Y

J Y

K ZK YJ Z

Page 25: Testing Techniques

A J Y

K YB

A K ZA K YA J Z

B ZJB YJ

K ZB

A J

B KB JA K

A YA Z

B ZB Y

J Y

K ZK YJ Z

Selection of test configurations forcoverage of interaction elements

Interaction elements Test configurations

A J A Y J YA J Y

Degree 2 coverage: 3 / 12 = 25%

Degree 3 coverage: 1 / 8 = 12.5%

Page 26: Testing Techniques

A KA K

A J Y

K YB

A K ZA K YA J Z

B ZJB YJ

K ZB

A J

B KB J

A YA Z

B ZB Y

J Y

K ZK YJ Z

Selection of test configurations forcoverage of interaction elements

Interaction elements Test configurations

A K Z

A Z

K Z

Degree 2 coverage: 6 / 12 = 50%

Degree 3 coverage: 2 / 8 = 25%

Page 27: Testing Techniques

A J Y

K YB

A K ZA K YA J Z

B ZJB YJ

K ZB

A J Y

A K Z

A J

B KB JA K

A YA Z

B ZB Y

J Y

K ZK YJ Z

Selection of test configurations forcoverage of interaction elements

Interaction elements Test configurations

B ZJ

B JB Z

J Z

Degree 2 coverage: 9 / 12 = 75%

Degree 3 coverage: 3 / 8 = 37.5%

Page 28: Testing Techniques

A J Y

K YB

A K ZA K YA J Z

B ZJB YJ

K ZB

A J Y

A K Z

B ZJ

A J

B KB JA K

A YA Z

B ZB Y

J Y

K ZK YJ Z

Selection of test configurations forcoverage of interaction elements

Interaction elements Test configurations

B KB Y K Y

K YBDegree 2 coverage: 12 / 12 = 100%

Degree 3 coverage: 4 / 8 = 50%

Page 29: Testing Techniques

Choosing the degree of coverage• In one experiment, covering 2 way interactions resulted in the following

average code coverage:– 93% block coverage.– 83% decision coverage.– 76% c-use coverage.– 73% p-use coverage.– Source: Cohen, et al, “The combinatorial design approach to

automatic test generation”, IEEE Software, Sept. 1996.• Another experience report investigating interactions among 2-4

components:– Dunietz, et al, “Applying design of experiments to software testing”,

Proc. Of ICSE ‘97.

Page 30: Testing Techniques

Online Mortgage Application(Thanks, Bernie Berger, STAREast)

NY L 1 fam A+ Pri 80% Yes Yes Yes Cust Yes Yes

NJ M 2 fam A Vac 90% No No No Bank No No

FL H 3 fam A- Inv 100%

TX H+1 4 fam B

CA H+2 Coop <B

DC H+3 Condo

Other

Regi

on

tier

Prop

erty

Cred

it

Resid

ence

LTV

NIV

NAV

Refin

ance

Clos

ing

Cost

Intr

o Ra

te

Bank

Em

p

Page 31: Testing Techniques

Twelve variables, with varying numbers of values, have7 x 6 x 6 x 5 x 3 x 3 x 2 x 2 x 2 x 2 x 2 x 2 = 725,760combinations of values.

“All Pairs does it in 50.” (Bernie Berger, STAREast2003)

Total number of combinations

Page 32: Testing Techniques

1234

47484950

NYNYNJNJNY

OtherOtherOther

LMLM

H+1H

H+2H+1

1 fam2 fam2 fam1 fam

~Coop2 fam3 fam

Condo

A+A

A-BB

~B~A-<B

PriVacInvPri

~Inv~Vac~Pri

~Vac

80%90%

100%90%

~80%~100%~80%~90%

YesNo

YesNo

~No~No

~Yes~No

YesNoNo

Yes~No~No~No~No

YesNo

YesNo

~Yes~Yes~No

~Yes

CustBankBankCust

~Cust~Cust~Cust~Cust

YesNo

YesNo

~Yes~No~No~No

YesNoNo

Yes~Yes~No~No~No

666654461112

Selected All Pairs Test Cases (note fault resolution!)

Page 33: Testing Techniques

Mortgage Application Observations

• The example uses values, not equivalence classes.• Values are independent,• System level testing (apparently).• The efficiency comes at the expense of fault isolation.

There are 2C12 = 12!/(2! x (12 - 2)!) = 66 pairs in the first row of the table. If that test case fails, which of the 66 pairs caused the failure?

• NB: excellent candidate for regression testing

Page 34: Testing Techniques

What if the fault involves more than just a pair of variables?

pCn1

12

å =2^12 - 1=4095

pCn =n!

p! (n - p)!

Number of combinations of n things taken p at a time is

Worst case (for first test case):

Page 35: Testing Techniques