Black Box Testing

127
WHAT IS BLACK BOX TESTING? box testing is done without the knowledge of the in system under test. ox testing involves looking at the specifications a t require examining the code of a program. box testing is done from the customer’s viewpoint.

Transcript of Black Box Testing

Page 1: Black Box Testing

WHAT IS BLACK BOX TESTING?

Black box testing is done without the knowledge of the internals of the system under test.

Black box testing involves looking at the specifications and does not require examining the code of a program.

Black box testing is done from the customer’s viewpoint.

Page 2: Black Box Testing
Page 3: Black Box Testing

Example: Lock and Key System

Functionality What you need to know to use

Features of a lock

Feature of a key It is made of metal and created to fit into a particular lock’s keyhole.

Action performed Key inserted and turned clockwise to lockKey inserted and turned anticlockwise to unlock

State

Inputs

Expected outcome

LockedUnlocked

Key turned clockwise or anticlockwise

LockedUnlocked

It is made of metal, has a hole provision to lock, has a facility to insert

Page 4: Black Box Testing

Black Box Testing:

• Required a functional knowledge of the product to be tested.

• It does not mandate the knowledge of the internal logic of the used to build the product.

Page 5: Black Box Testing

WHY BLACK BOX TESTING

Black box testing helps in the overall functional verification of the system under test:

• Black box testing is done based on requirements: It helps in identifying any incomplete, inconsistent requirement as well as any issue involved when the system is tested as a complete entity.

• Black box testing address the stated requirements as well as implied requirements: - E.g., inclusion of dates, page header, and footer may not be explicitly stated in the report generation requirements specification.

Page 6: Black Box Testing

• Black box testing encompasses the end user perspectives: as we test the behaviour of a product from an external perspective.

• Black box testing handles valid and invalid inputs: This ensures that the product behaves as expected in a valid situation and does not hang or crash when provided with an invalid input.

Page 7: Black Box Testing

HOW TO DO BLACK BOX TESTING?

The various techniques are:

1. Requirements based testing

2. Positive and negative testing

3. Boundary value analysis

4. Decision table

5. Equivalence partitioning

6. Cause Effect Graphing

7. Compatibility testing

8. User documentation testing

9. Domain testing

Page 8: Black Box Testing

I. REQUIREMENT BASED TESTING

Requirements testing deals with validating the requirements given in the Software Requirements Specification (SRS) of the software system.

A requirements specification for the lock and key example is documented as given in Table:

S.No

RequirementsIdentifier

Description Priority (High,med,low)

1 BR-01Inserting the key numbered 123-456 and turning it clockwise should facilitate locking

H

2 BR-02Inserting the key numbered 123-456 and turning it anticlockwise should facilitate unlocking

H

3 BR-03Only key number 123-456 can be used to lock and unlock

H

Page 9: Black Box Testing

4 BR-04No other object can be used to lock

M

5 BR-05No other object can be used to unlock

M

6 BR-06The lock must not open even when it is hit with a heavy object

M

7 BR-07The lock must be made of metal and must weigh approximately 150 grams

L

8 BR-08Lock unlock directions should be changeable for usability of left-handers

L

Page 10: Black Box Testing

Requirements are tracked by a Requirements Traceability matrix (RTM).

• This matrix evolves through the life cycle of the project.

• Each requirement is given a unique id along with a brief description. The requirement identifier and description can be taken from the requirements Specification.

• Each requirement is assigned a requirement priority, classified as high, medium or low.

• The “test conditions” column lists the different way of testing the requirement. These conditions can be grouped together to form a single test case or each test condition can be mapped to one test case.

Page 11: Black Box Testing

• The “test case IDs” column can be used to complete the mapping between test case and the requirement.

• As we move further down in the life cycle of the product, and testing phases, the cross-reference between requirements and the subsequent phases is recorded in the RTM.

• In a complete RTM, there are columns to reflect the mapping of requirements to design and code.

Page 12: Black Box Testing

Sample Requirements Traceability Matrix:

Page 13: Black Box Testing
Page 14: Black Box Testing

RTM helps in identifying the relationship between the requirements and test case. The following combinations are possible:

One to one – For each requirement there is one test case.(BR-01) One to many – For each requirement there is one test case. (BR-03) Many to one – A set of requirements can be tested by one test case. Many to many – Many requirements can be tested by many test case. One to none – The set of requirements can have no test cases. The test team can take a decision not to test a requirement due to non-implementation or the requirements being low priority (BR-08)

Page 15: Black Box Testing

Advantages of RTM:

An RTM plays a valuable role in requirements based testing.

1. The RTM provides a tool to track the testing status of each requirement, without missing any (key) requirements (not possible manually).

2. Prioritization: Use to find out whether there are adequate test cases for high-priority requirements and to reduce the number of test case for low-priority requirements.

3. Test conditions can be grouped to create test cases or can be represented as unique test cases.

4. Test cases can be used as input to arrive at a size/effort/schedule estimation of tests.

Page 16: Black Box Testing

Some of the metrics that can be collected or inferred from this matrix are follows:

• Requirements addressed priority wise – This metric helps in knowing the test coverage based on the requirements. Number of test that is covered for high-priority requirement versus test created for low-priority requirement.

• Number of test cases requirement wise – For each requirement, the total number of test cases created.

• Total number of test cases prepared – Total of all the test cases prepared for all requirements.

Page 17: Black Box Testing

The test result can be used to collect metrics such as:

• Total number of test cases (or requirements) passed –What percent of requirements they correspond.

• Total number of test cases (or requirements) failed –What percent of requirements they correspond.

• Total number of defect in requirements – List of defect reported for each requirement (defect density for requirements). A comparatively high-defect density in low-priority requirements is acceptable for a release & vice versa.

Page 18: Black Box Testing

Number of requirements completed – Total number of requirements successfully completed without any defects.

Number of requirements pending – Number of requirements that are pending due to defects.

Page 19: Black Box Testing

To illustrate the metrics analysis, let us assume the test execution data as given in table:

Sample test execution data:

S.No Reg. ID Priority Test casesTotal test

casesTest cases

passedTest cases

failed % PassNo. of defect

1 BR-01 H Lock_01 1 1 0 100 1

2 BR-02 H Lock_02 1 1 0 100 1

3 BR-03 H Lock_03,04 2 1 1 50 3

4 BR-04 M Lock_05,06,07 3 2 1 67 5

5 BR-05 M Lock_08,09,10 3 3 0 100 1

6 BR-06 L Lock_11 1 1 0 100 1

7 BR-07 L Lock_12 1 1 0 100 0

8 BR-08 L   0 0 0 0 1

Total 8     12 10 2 83 12

Page 20: Black Box Testing

Following observations can be made:

• 83 percent passed test cases correspond to 71 percent of requirements being met. Similarly, from the failed test cases, outstanding defects affect 29 percent (=100-71) of the requirements.

• There is a high-priority requirement, BR-03, which has failed.

• The requirement BR-08 is not met; however, this can be ignored for the release due to the low-priority nature of this requirement.

Page 21: Black Box Testing

The metrics discussed can be expressed graphically:

Requirement Coverage with Pass/Fail test Cases

0

0.5

1

1.5

2

2.5

3

3.5

1 2 3 4 5 6 7 8

Requirements

No.

of T

est C

ases

Test cases failed

Test cases passed

Page 22: Black Box Testing

Defects in Each Requirement

0

1

2

3

4

5

6

1 2 3 4 5 6 7 8

Requirements

Def

ecs

No. of defect

Page 23: Black Box Testing

II. POSITIVE AND NEGATIVE TESTING

The purpose of positive testing is to prove that the product works as per specification and expectations. A product delivering an error when it is expected to give an error is also a part of positive testing.

Example of positive test cases

Req. no. Input 1 Input 2 Current stateExpected output

BR-01 Key 123-456 Turn clockwise Unlocked Locked

BR-01 Key 123-456 Turn clockwise Locked No change

BR-02 Key 123-456Turn anticlockwise

Unlocked No change

BR-02 Key 123-456Turn anticlockwise

Locked Unlock

BR-04 Hairpin Turn clockwise Locked No change

Page 24: Black Box Testing

Negative testing is done to show that the product does not fail when an unexpected input is given. Negative testing covers scenarios for which the product is not designed and coded. Negative test cases

S. No.

Input 1 Input 2Current

stateExpected

output

1Some other lock’s Key

Turn clockwise

Lock Lock

2Some other lock’s Key

Turn anticlockwise

Unlock Unlock

3Thin piece of wire

Turn anticlockwise

Unlock Unlock

4 Hit with a stone Lock Lock

*there are no requirement numbers because test conditions lie outside the specification.

Page 25: Black Box Testing

Difference: Coverage.

Positive testing: if all documented requirements and test conditions are covered, then coverage can be considered to be 100 percent.

Negative Testing: No end.

Page 26: Black Box Testing

III. BOUNDARY VALUE ANALYSIS

Boundary value analysis (BVA) is useful for arriving at tests that are effective in catching defects that happen at boundaries. Boundary value analysis believes and extends the concept that the density of defect is more towards the boundaries.Example:

Number of unit bought Price per unit

First ten units (that is, from 1 to 10 units) $5.00

Next ten units (that is, from units 11 to 20 units) $4.75

Next ten units (that is, from units 21 to 30 units) $4.50

More than 30 units $4.00

Page 27: Black Box Testing

Possible reasons:

• Programmers’ tentativeness in using the right comparison operator, E.g. to use the <= operator or < operator when trying to make comparisons.

• Confusion : multiple way to implement loops and condition checking. E.g. in C, for, while and loops. Each have different terminating conditions.

• Requirements may not clearly understood, especially around the boundaries, thus causing even the correctly coded program to not perform the correct way.

Most defects around the boundaries –E.g. Buying 9,10,11,19,20,21,29,30,31 and similar number of items.

Page 28: Black Box Testing

Example:Boundary Value AnalysisConsider a program with two input variables x and y. These input variables have specified boundaries as:

a ≤ x ≤ bc ≤ y ≤ d

Input domain

y

d

c

a bx

Fig.4: Input domain for program having two input variables

Page 29: Black Box Testing

The boundary value analysis test cases for our program with two inputs variables (x and y) that may have any value from 100 to 300 are: (200,100), (200,101), (200,200), (200,299), (200,300), (100,200), (101,200), (299,200) and (300,200). This input domain is shown in Fig. 5. Each dot represent a test case and inner rectangle is the domain of legitimate inputs. Thus, for a program of n variables, boundary value analysis yield 4n + 1 test cases.

y

x

Input domain

300

200

100

400

0 300200100 400

Fig.5: Input domain of two variables x and y with boundaries [100,300] each

Page 30: Black Box Testing

Example- I

Consider a program for the determination of the nature of roots of a quadratic equation. Its input is a triple of positive integers (say a,b,c) and values may be from interval [0,100]. The program output may have one of the following words.

[Not a quadratic equation; Real roots; Imaginary roots; Equal roots]

Design the boundary value test cases.

Page 31: Black Box Testing

Solution:

Quadratic equation will be of type:

ax2+bx+c=0

Roots are real if (b2-4ac)>0

Roots are imaginary if (b2-4ac)<0

Roots are equal if (b2-4ac)=0

Equation is not quadratic if a=0

Page 32: Black Box Testing

The boundary value test cases are :

Page 33: Black Box Testing

Example – 2

Consider a program for determining the Previous date. Its input is a triple of day, month and year with the values in the range

1 ≤ month ≤ 12

1 ≤ day ≤ 31

1900 ≤ year ≤ 2025

The possible outputs would be Previous date or invalid input date. Design the boundary value test cases.

Solution:

The Previous date program takes a date as input and checks it for validity. If valid, it returns the previous date as its output.

With single fault assumption theory, 4n+1 test cases can be designed and which are equal to 13.

Page 34: Black Box Testing

The boundary value test cases are:

Page 35: Black Box Testing

Example – 3

Consider a simple program to classify a triangle. Its inputs is a triple of positive integers (say x, y, z) and the date type for input parameters ensures that these will be integers greater than 0 and less than or equal to 100. The program output may be one of the following words:

[Scalence; Isosceles; Equilateral; Not a triangle]

Design the boundary value test cases.

Page 36: Black Box Testing

Solution:

The boundary value test cases are shown below:

Page 37: Black Box Testing

Robustness testing

It is nothing but the extension of boundary value analysis. Here, we would like to see, what happens when the extreme values are exceeds with a value slightly greater than the maximum, and a value slightly less than minimum. It means, we want to go outside the legitimate boundary of input domain. This extended form of boundary value analysis is called robustness testing and shown in Fig.6

There are four additional test cases which are outside the legitimate input domain. Hence total test cases in robustness testing are 6n+1, where n is the number of input variables. So, 13 test cases are:

Page 38: Black Box Testing

(200,99), (200,100), (200,101), (200,200), (200,299), (200,300)(200,301), (99,200), (100,200), (101,200), (299,200), (300,200), (301,200)

y

x

300

200

100

400

0 300200100 400

Fig. 6: Robustness test cases for two variables x and y with range [100,300] each

Page 39: Black Box Testing

Worst-case testing

If we reject “single fault” assumption theory of reliability and may like to see what happens when more than one variable has an extreme value. In electronic circuits analysis, this is called “worst case analysis”. It is more through in the sense that boundary value test cases are a proper subset of worst case test cases. It requires more effort. Worst case testing for a function of n variables generate 5n test cases as opposed to 4n+1 test cases for boundary value analysis. Our two variables example will have 52=25 test cases and are given in table1.

Page 40: Black Box Testing

Table 1: Worst cases test inputs for two variables example

Page 41: Black Box Testing

Example – 4

Consider the program for the determination of nature of roots of a quadratic equation as explained in example 1. Design the Robust test case and worst test cases for this program.

Solution:

Robust test cases are 6n+1. Hence, in 3 variable input cases total number of test cases are 19 as given on next slide:

Page 42: Black Box Testing
Page 43: Black Box Testing

In case of worst test case total test cases are 5n. Hence, 125 test cases will be generated in worst test cases. The worst test cases are given below:

Page 44: Black Box Testing
Page 45: Black Box Testing
Page 46: Black Box Testing
Page 47: Black Box Testing
Page 48: Black Box Testing
Page 49: Black Box Testing
Page 50: Black Box Testing
Page 51: Black Box Testing

Example – 5

Consider the program for the determination of previous date in a calendar as explained in example 2. Design the robust and worst test cases for this program.

Solution:

Robust test cases are 6n+1. Hence total 19 robust test cases are designed and are given on next slide.

Page 52: Black Box Testing
Page 53: Black Box Testing

In case of worst test case total test cases are 5n. Hence, 125 test cases will be generated in worst test cases. The worst test cases are given below:

Page 54: Black Box Testing
Page 55: Black Box Testing
Page 56: Black Box Testing
Page 57: Black Box Testing
Page 58: Black Box Testing
Page 59: Black Box Testing
Page 60: Black Box Testing
Page 61: Black Box Testing

Example – 6

Consider the triangle problem as given in example 3. Generate robust and worst test cases for this problem.

Solution:

Robust test cases are given on next slide.

Page 62: Black Box Testing
Page 63: Black Box Testing

Worst test cases are 125 and are given below:

Page 64: Black Box Testing
Page 65: Black Box Testing
Page 66: Black Box Testing
Page 67: Black Box Testing
Page 68: Black Box Testing
Page 69: Black Box Testing
Page 70: Black Box Testing
Page 71: Black Box Testing

IV. DECISION TABLES

A decision table lists the various decision variables, the conditions assumed by each of the decision variables, and the actions to take in teach combination of conditions. The variables that contribute to the decision are listed as the columns of the table. The last column of the table is the action to be taken for the combination of values of the decision variables.

Decision table act as invaluable tools for designing black box tests to examine the behavior of the product under various logical conditions of input variables.

Page 72: Black Box Testing

The steps in forming a decision table are as follows:

Identify the decision variables.

Identify the possible values of each of the decision variables.

Enumerate the combinations of the allowed values of each of the variables.

Identify the cases when values assumed by a variable are immaterial for a given combination of other input variables. Represent such variables by the don’t care symbol.

For each combination of values of decision variables list out the action or expected result.

Page 73: Black Box Testing

Once a decision table is formed, each row of the table acts as the specification for one test case. Identification of the decision variables makes these test cases extensive, if not exhaustive. Pruning the table by using don’t cases minimize the number of test cases. Thus, decision table are usually effective in arriving at test cases in scenarios which depend on the values of the decision variables.

Page 74: Black Box Testing

Decision Table Based Testing

Decision table terminology

Page 75: Black Box Testing

Test case design

Decision table for triangle problem

Page 76: Black Box Testing

Modified decision table

Page 77: Black Box Testing

Example 10

Consider the triangle program specified in example 3. Identify the test cases using the decision table of table 4.

Solution:

There are eleven functional test cases, three to fail triangle property, three impossible cases, one each to get equilateral, scalene triangle cases, and three to get on isosceles triangle. The test cases are given in table 5.

Page 78: Black Box Testing

Test cases of triangle problem using decision table

Page 79: Black Box Testing

Example 11

Consider a program for the determination of Previous date. Its input is a triple of day, month and year with the values in the range

1 ≤ month ≤ 12 1 ≤ day ≤ 311900 ≤ year ≤ 2025

The possible outputs are “Previous date” and “Invalid date”. Design the test cases using decision table based testing.

Page 80: Black Box Testing

Solution:The input domain can be divided into following classes:

I1= {M1: month has 30 days}I2= {M2: month has 31 days except March, August and January}I3= {M3: month is March}I4= {M4: month is August}I5= {M5: month is January}I6= {M6: month is February}I7= {D1: day = 1}I8= {D2: 2 ≤ day ≤ 29}I9= {D3: day = 29}I10={D4: day = 30}I11={D5: day = 31}I12={Y1: year is a leap year}I13={Y2: year is a common year}

Page 81: Black Box Testing

The decision table is given below:

Page 82: Black Box Testing
Page 83: Black Box Testing
Page 84: Black Box Testing
Page 85: Black Box Testing
Page 86: Black Box Testing
Page 87: Black Box Testing
Page 88: Black Box Testing
Page 89: Black Box Testing

EQUIVALENCE PARTTITIONING

Equivalence partitioning is a software testing technique that involves identifying a small set of representative input values that produce as many different output conditions as possible. This reduces the number of permutations and combination of input, output values used for testing thereby increasing the coverage and reducing the effort involved in testing.

The set of input values that generate one single expected output is called a partition. When the behavior of the software is the same for a set of values, then the set is termed as an equivalence class or a partition. In this case, one representative sample from each partition is picked up for testing.

Page 90: Black Box Testing

Since all the values produce equal and same output they are termed as equivalence partition.

Testing by this technique involves identifying all partitions for the complete this technique extrapolates the expected results for all the values in that partition. We gain good coverage with a small number of test cases. Redundancy of tests is minimized by not repeating the same tests for multiple values in the same partition.

Page 91: Black Box Testing

The steps to prepare an equivalence partitions table are as follows.

Choose criteria for doing the equivalence partitioning

Identify the valid equivalence classes based on the above criteria

Select a sample data form the partition

Write the expected result based on the requirement given

Identify special values, if any, and include them in the table

Check to have expected results for all the cases prepared

If the expected result is not clear for any particular test case, mark appropriately and escalate for corrective actions. If you can not answer a question, or find an inappropriate answer, consider whether you want to record this issue on your log, and clarify with the team that arbitrates/dictates the requirements.

Testing done to ensure that the product feature work consistently with different infrastructure components is called compatibility testing.

Page 92: Black Box Testing

Equivalence Class Testing

In this method, input domain of a program is partitioned into a finite number of equivalence classes such that one can reasonably assume, but not be absolutely sure, that the test of a representative value of each class is equivalent to a test of any other value.

Page 93: Black Box Testing

Two steps are required to implementing this method:

1. The equivalence classes are identified by taking each input condition and partitioning it into valid and invalid classes. For example, if an input condition specifies a range of values from 1 to 999, we identify one valid equivalence class [1<item<999]; and two invalid equivalence classes [item<1] and [item>999].

2. Generate the test cases using the equivalence classes identified in the previous step. This is performed by writing test cases covering all the valid equivalence classes. Then a test case is written for each invalid equivalence class so that no test contains more than one invalid class. This is to ensure that no two invalid classes mask each other.

Page 94: Black Box Testing

System under test Outputs

Valid inputs

Valid input

Input domain Output domain

Most of the time, equivalence class testing defines classes of the input domain. However, equivalence classes should also be defined for output domain. Hence, we should design equivalence classes based on input and output domain.

Fig. : Equivalence partitioning

Page 95: Black Box Testing

Example 7

Consider the program for the determination of nature of roots of a quadratic equation as explained in example 1. Identify the equivalence class test cases for output and input domains.

Solution:

Output domain equivalence class test cases can be identified as follows:

O1={<1,b,c>:Not a quadratic equation if a = 0}O1={<1,b,c>:Real roots if (b2-4ac)>0}O1={<1,b,c>:Imaginary roots if (b2-4ac)<0}O1={<1,b,c>:Equal roots if (b2-4ac)=0}`

Page 96: Black Box Testing

The number of test cases can be derived form above relations and shown below:

Test case a b c Expected output

1 0 50 50 Not a quadratic equation

2 1 50 50 Real roots

3 50 50 50 Imaginary roots

4 50 100 50 Equal roots

Page 97: Black Box Testing

We may have another set of test cases based on input domain.

I1= {a: a = 0}

I2= {a: a < 0}

I3= {a: 1 ≤ a ≤ 100}

I4= {a: a > 100}

I5= {b: 0 ≤ b ≤ 100}

I6= {b: b < 0}

I7= {b: b > 100}

I8= {c: 0 ≤ c ≤ 100}

I9= {c: c < 0}

I10={c: c > 100}

Page 98: Black Box Testing

Here test cases 5 and 8 are redundant test cases. If we choose any value other than nominal, we may not have redundant test cases. Hence total test cases are 10+4=14 for this problem.

Page 99: Black Box Testing

Example 8

Consider the program for determining the previous date in a calendar as explained in example 3. Identify the equivalence class test cases for output & input domains.

Page 100: Black Box Testing

Solution:

Output domain equivalence class are:

O1={<D,M,Y>: Previous date if all are valid inputs}O1={<D,M,Y>: Invalid date if any input makes the date invalid}

Test case M D Y Expected output

1 6 15 1962 14 June, 1962

2 6 31 1962 Invalid date

Page 101: Black Box Testing

We may have another set of test cases which are based on input domain.

I1={month: 1 ≤ m ≤ 12}

I2={month: m < 1}

I3={month: m > 12}

I4={day: 1 ≤ D ≤ 31}

I5={day: D < 1}

I6={day: D > 31}

I7={year: 1900 ≤ Y ≤ 2025}

I8={year: Y < 1900}

I9={year: Y > 2025}

Page 102: Black Box Testing

Inputs domain test cases are :

Page 103: Black Box Testing

Example – 9

Consider the triangle problem specified in a example 3. Identify the equivalence class test cases for output and input domain.

Page 104: Black Box Testing

Solution:

Output domain equivalence classes are:O1={<x,y,z>: Equilateral triangle with sides x,y,z}O1={<x,y,z>: Isosceles triangle with sides x,y,z}O1={<x,y,z>: Scalene triangle with sides x,y,z}O1={<x,y,z>: Not a triangle with sides x,y,z}

The test cases are:

Page 105: Black Box Testing

Input domain based classes are:

I1={x: x < 1}

I2={x: x > 100}

I3={x: 1 ≤ x ≤ 100}

I4={y: y < 1}

I5={y: y > 100}

I6={y: 1 ≤ y ≤ 100}

I7={z: z < 1}

I8={z: z > 100}

I9={z: 1 ≤ z ≤ 100}

Page 106: Black Box Testing

Some inputs domain test cases can be obtained using the relationship amongst x,y and z.

I10={< x,y,z >: x = y = z}

I11={< x,y,z >: x = y, x ≠ z}

I12={< x,y,z >: x = z, x ≠ y}

I13={< x,y,z >: y = z, x ≠ y}

I14={< x,y,z >: x ≠ y, x ≠ z, y ≠ z}

I15={< x,y,z >: x = y + z}

I16={< x,y,z >: x > y +z}

I17={< x,y,z >: y = x +z}

I18={< x,y,z >: y > x + z}

I19={< x,y,z >: z = x + y}

I20={< x,y,z >: z > x +y}

Page 107: Black Box Testing

Test cases derived from input domain are:

Page 108: Black Box Testing
Page 109: Black Box Testing

CAUSE EFFECT GRAPHING TECHNIQUE

• Consider single input conditions

• do not explore combinations of input circumstances

Steps

1. Causes & effects in the specifications are identified.

A cause is a distinct input condition or an equivalence class of input conditions.

An effect is an output condition or a system transformation.

2. The semantic content of the specification is analysed and transformed into a boolean graph linking the causes & effects.

3. Constraints are imposed

4. graph – limited entry decision table

Each column in the table represent a test case.

5. The columns in the decision table are converted into test cases.

Page 110: Black Box Testing

The basic notation for the graph is shown in fig. 8

Fig. 8 : Basic cause effect graph symbols

Page 111: Black Box Testing

Myers explained this effectively with following example. “The characters in column 1 must be an A or B. The character in column 2 must be a digit. In this situation, the file update is made. If the character in column 1 is incorrect, message x is issued. If the character in column 2 is not a digit, message y is issued”.

Page 112: Black Box Testing

The causes arec1: character in column 1 is Ac2: character in column 1 is Bc3: character in column 2 is a digit

and the effects aree1: update madee2: message x is issuede3: message y is issued

Fig. 9: Sample cause effect graph

Page 113: Black Box Testing

The E constraint states that it must always be true that at most one of c1 or c2 can be 1 (c1 or c2 cannot be 1 simultaneously). The I constraint states that at least one of c1, c2 and c3 must always be 1 (c1, c2 and c3 cannot be 0 simultaneously). The O constraint states that one, and only one, of c1 and c2 must be 1. The constraint R states that, for c1 to be 1, c2 must be 1 (i.e. it is impossible for c1 to be 1 and c2 to be 0),

Page 114: Black Box Testing

Fig. 10: Constraint symbols

Page 115: Black Box Testing

Fig. 11: Symbol for masks constraint

Page 116: Black Box Testing

Fig. : Sample cause effect graph with exclusive constraint

Page 117: Black Box Testing

Example 12

Consider the triangle problem specified in the example 3. Draw the Cause effect graph and identify the test cases.

Page 118: Black Box Testing

SolutionThe causes are

c1: side x is less than sum of sides y and zc2: side y is less than sum of sides x and yc3: side z is less than sum of sides x and yc4: side x is equal to side yc5: side x is equal to side zc6: side y is equal to side z

and effects aree1: Not a trianglee2: Scalene trianglee3: Isosceles trianglee4: Equilateral trianglee5: Impossible stage

The cause effect graph is shown in fig. 13 and decision table is shown in table 6. The test cases for this problem are available in table 5.

Page 119: Black Box Testing

Table 6: Decisi}on table

Page 120: Black Box Testing

Fig. 13: Cause effect graph of triangle problem

Page 121: Black Box Testing

COMPATIBILITY TESTING

The test case result not only depend on the product for proper functioning they depend equally on the infrastructure for delivering is expected to still behave correctly and produce the desired or expected results. The infrastructure parameters could be fo hardware, software or other components.

A black box testing not considering the effect of these parameters on the test case results, will necessarily be incomplete and ineffective.

Page 122: Black Box Testing

The parameters that generally affect the compatibility of the product are:

• Processor (CPU)(PIII, PIV, Xeon, SPARC, and so on) and the number of processor in the machine

• Architecture and characteristics of the machine (32 bit, 64 bit and so on)

• Resource availability on the machine (RAM, disk space, network)

• Equipment that the product is expected to work with (printers, moderms and so on)

• Operating system (windows, linux and so on)

Page 123: Black Box Testing

• Middle tier infrastructure components such as web server, application server, network server

• Backend component such database servers (Oracle, Sybase, and so on)

• Services that require special hardware cum software solutions (cluster machines, load balancing, RAID array)

• Any software used to generate product binaries (compiler, linker and so on and their appropriate versions)

• Various technologies component used to generate components (SDK, JDK, and so on their appropriate different versions)

Page 124: Black Box Testing

In the above example, described ten parameters. If each of the parameters can take four values, then there are forty different values to be tested. The permutation and combination of the parameters. The total number of combinations to be tested is 410, which is a large number and impossible to test exhaustively.

In order to arrive at practical combination of the parameters to be tested, a compatibility matrix is created. A compatibility matrix has as its columns various parameters the combinations of which have to be tested. Each row represent a unique combination of a specific set of the parameters. A sample compatibility matrix for a mail application is given in table 11.

It does not cover all parameters and their combinations. Common techniques that are used for performing compatibility testing , using a compatibility table are

Page 125: Black Box Testing

Horizontal combination all values of parameters that can coexist with the product for executing the set test cases grouped together as a row in the compatibility matrix. Machine or environments are set up for each row and the set of product features are tested using each of these environments.

Intelligent sampling: combination of infrastructure parameters are combined with the set of features intelligently and tested. The selection of intelligent samples is based on information collected on the set of dependencies of the product with the parameters. If the product result are less dependent on a set of parameters, then they are removed from the list of intelligent samples. All other parameters are combined and tested. This method significantly reduces the number of permutations and combinations for test cases.

Page 126: Black Box Testing

USER DOCUMENTATION TESTING

User documentation covers all the manuals, user guides, installation guides, setup guides, read me file, software release notes, and online help that are provided along with the software to help the user to understand the software system.

User documentation testing should have two objectives.

1. To check if what is stated in the document is available in the product.2. To check if what is there in the product is explained correctly in the document.3. Checks for language specific aspect like spell check and grammar.

Page 127: Black Box Testing

Need:

1. When a product is upgraded the corresponding documentation may not be updated all the time.

2. Users refer to documentation (manuals, installation, setup guides) when they need it