Orthogonal Array Testing Strategy (OATS)

1. Region Faults.2. Isolated Faults.

Orthogonal Array based testing is highly effective for the detection of region faults with a relatively small number of tests.

Software Faults

Why use OATS?

• Systematic, statistical way to test pair-wise interactions.

• Interactions and integration points are a major source of defects.

• Most defects arise from simple pair-wise interactions.

• Exhaustive testing is impossible.

• Execute a well-defined, concise set of tests that are likely to uncover most (not all) bugs.

• Orthogonal approach guarantees the pair-wise coverage of all variables.

Orthogonal Arrays (OA’s)• Originally discovered by monks.• Dr. Genichi Taguchi was one of the first proponents of OA’s in test design.• Two Dimensional arrays of numbers.• If we choose any 2 columns in the array, we find an even distribution of all pair-wise combinations of values in the array.

Terminology for working with OA’s

OA’s are commonly represented as :

LRuns(LevelsFactors) or OA(Runs, LevelsFactors, Strength)

• Runs – Number of rows in the array, which translates into the number of

Test Cases that will be generated.

• Factors – Number of columns in the array, which translates into the

maximum number of variables that can be handled by the array.

• Levels – Maximum number of values that can be taken on by any single

factor.

• Strength – The number of columns it takes to see all the possibilities equal

number of times.

Steps to be followed in OATS Technique• Decide the independent variables. Independent variables map to factors of the array.

• Decide the maximum number of values each independent variable takes. The values map to the levels of the array.

• From the Orthogonal Array Selector, find a suitable Orthogonal Array with the smallest number of Runs. ( Suitable array is one that has at least as many factors (from step 1) and as many levels (from step 2). )

• Map the factors and levels onto the array.

• Transcribe the Runs into test cases.

• Add any particularly suspicious combinations that aren’t generated.

Example 1

A Web Page has three distinct sections (Top, Middle, Bottom) that can be individually shown or hidden from user

No.of Factors=3 (Top,middle,Bottom)

No.of Levels =2 (Hidden or shown)

Array Type =L4(2 3) or OA(4,3,2)

If we go for exhaustive testing we need :2 x 2 x 2 = 8 Test Cases

• Fixed Level Array: L4(2 3)

Example 1

F1 F2 F3

Run1 0 0 0

Run 2 0 1 1

Run 3 1 0 1

Run 4 1 1 0

L423 – OA with 4 Runs3 factors with 2 levels

Top Middle Bottom

Test 1 Hidden Hidden Visible

Test 2 Hidden Visible Visible

Test 3 Visible Hidden Visible

Test 4 Visible Visible Hidden

The Four Test Scenarios (4 Vs. 8)

1 - Display the home page and hide all sections. 2 - Display the home page and show all but the Top section.3 - Display the home page and show all but the Middle section.4 - Display the home page and show all but the Bottom section.

Example 2

Example 2

Factors Level 1 Level 2 Level 3

Address Option Keep thedefault option (0)

Select fromAddress Book (1)

Enter NewAddress (2)

Save Changes toAddress Book

Yes (0) No (1)  

Add to AddressBook

Yes (0) No (1)  

Make DefaultAddress

Yes (0) No (1)  

If we go for exhaustive testing we would need : 3 x 2 x 2 x 2 = 24 Test Cases

What we have is 3 factors with 2 levels and 1 factor with 3 levels

Example 2

L9(34) – OA with 9 Runs.4 factors with 3 levels.

Array Mapped to actual values

• Check catalog for an array with at least 3 levels and at least 4 factors L9(34)

• Smallest array with consistent # of levels is L9(34)

• This would generate 9 test cases, but wait…..

L82441 – OA with 8 Runs4 factors with 2 levels and 1 factor with 4 level

• Mixed Level Array: L8(2441)

• Our problem ‘fits’ into this array

• This generates only 8 TC

Example 2

Array Mapped to actual values

Conclusion

• In a hyper-competitive environment with constrained resources, the OATS provides an innovative approach for doing high-value testing with reduced costs.

• Being able to deliver a system faster and cheaper enables our company’s management to beat its competitors in the market.

• This method is flexible and does not require training in statistics to use or understand.