Black Box Testing

Click here to load reader

  • date post

  • Category


  • view

  • download


Embed Size (px)

Transcript of Black Box Testing


Black box testing is done without the knowledge of the internals of the system under test. Black box testing involves looking at the specifications and does not require examining the code of a program. Black box testing is done from the customers viewpoint.

Example: Lock and Key System

Functionality Features of a lock Feature of a key Action performed

What you need to know to use It is made of metal, has a hole provision to lock, has a facility to insert It is made of metal and created to fit into a particular locks keyhole. Key inserted and turned clockwise to lock Key inserted and turned anticlockwise to unlock Locked Unlocked Key turned clockwise or anticlockwise Locked Unlocked

State Inputs Expected outcome

Black Box Testing: Required a functional knowledge of the product to be tested. It does not mandate the knowledge of the internal logic of the used to build the product.

WHY BLACK BOX TESTING Black box testing helps in the overall functional verification of the system under test: Black box testing is done based on requirements: It helps in identifying any incomplete, inconsistent requirement as well as any issue involved when the system is tested as a complete entity. Black box testing address the stated requirements as well as implied requirements: - E.g., inclusion of dates, page header, and footer may not be explicitly stated in the report generation requirements specification.

Black box testing encompasses the end user perspectives: as we test the behaviour of a product from an external perspective. Black box testing handles valid and invalid inputs: This ensures that the product behaves as expected in a valid situation and does not hang or crash when provided with an invalid input.

HOW TO DO BLACK BOX TESTING? The various techniques are: 1. 2. 3. 4. 5. 6. 7. 8. 9. Requirements based testing Positive and negative testing Boundary value analysis Decision table Equivalence partitioning Cause Effect Graphing Compatibility testing User documentation testing Domain testing



Requirements testing deals with validating the requirements given in the Software Requirements Specification (SRS) of the software system. A requirements specification for the lock and key example is documented as given in Table:S. Requirem No ents Identifier 1 BR-01 Description Priority (High,med,l ow) H

Inserting the key numbered 123456 and turning it clockwise should facilitate locking Inserting the key numbered 123456 and turning it anticlockwise should facilitate unlocking Only key number 123-456 can be used to lock and unlock







4 5 6

BR-04 BR-05 BR-06

No other object can be used to lock No other object can be used to unlock The lock must not open even when it is hit with a heavy object The lock must be made of metal and must weigh approximately 150 grams Lock unlock directions should be changeable for usability of lefthanders








Requirements are tracked by a Requirements Traceability matrix (RTM). This matrix evolves through the life cycle of the project.

Each requirement is given a unique id along with a brief description. The requirement identifier and description can be taken from the requirements Specification. Each requirement is assigned a requirement priority, classified as high, medium or low. The test conditions column lists the different way of testing the requirement. These conditions can be grouped together to form a single test case or each test condition can be mapped to one test case.

The test case IDs column can be used to complete the mapping between test case and the requirement. As we move further down in the life cycle of the product, and testing phases, the cross-reference between requirements and the subsequent phases is recorded in the RTM. In a complete RTM, there are columns to reflect the mapping of requirements to design and code.

Sample Requirements Traceability Matrix:

RTM helps in identifying the relationship between the requirements and test case. The following combinations are possible: One to one For each requirement there is one test case.(BR-01) One to many For each requirement there is one test case. (BR-03) Many to one A set of requirements can be tested by one test case. Many to many Many requirements can be tested by many test case. One to none The set of requirements can have no test cases. The test team can take a decision not to test a requirement due to non-implementation or the requirements being low priority (BR-08)

Advantages of RTM: An RTM plays a valuable role in requirements based testing. 1. The RTM provides a tool to track the testing status of each requirement, without missing any (key) requirements (not possible manually). Prioritization: Use to find out whether there are adequate test cases for high-priority requirements and to reduce the number of test case for low-priority requirements. Test conditions can be grouped to create test cases or can be represented as unique test cases. Test cases can be used as input to arrive at a size/effort/schedule estimation of tests.


3. 4.

Some of the metrics that can be collected or inferred from this matrix are follows: Requirements addressed priority wise This metric helps in knowing the test coverage based on the requirements. Number of test that is covered for high-priority requirement versus test created for low-priority requirement. Number of test cases requirement wise For each requirement, the total number of test cases created. Total number of test cases prepared Total of all the test cases prepared for all requirements.

The test result can be used to collect metrics such as: Total number of test cases (or requirements) passed What percent of requirements they correspond. Total number of test cases (or requirements) failed What percent of requirements they correspond. Total number of defect in requirements List of defect reported for each requirement (defect density for requirements). A comparatively high-defect density in lowpriority requirements is acceptable for a release & vice versa.

Number of requirements completed Total number of requirements successfully completed without any defects. Number of requirements pending Number of requirements that are pending due to defects.

To illustrate the metrics analysis, let us assume the test execution data as given in table: Sample test execution data:Total test cases 1 1 2 3 3 1 1 0 12 Test cases passed 1 1 1 2 3 1 1 0 10 Test cases failed 0 0 1 1 0 0 0 0 2 No. of defect 1 1 3 5 1 1 0 1 12

S.No 1 2 3 4 5 6 7 8 Total

Reg. ID BR-01 BR-02 BR-03 BR-04 BR-05 BR-06 BR-07 BR-08 8

Priority H H H M M L L L

Test cases Lock_01 Lock_02 Lock_03,04 Lock_05,06,07 Lock_08,09,10 Lock_11 Lock_12

% Pass 100 100 50 67 100 100 100 0 83

Following observations can be made: 83 percent passed test cases correspond to 71 percent of requirements being met. Similarly, from the failed test cases, outstanding defects affect 29 percent (=100-71) of the requirements. There is a high-priority requirement, BR-03, which has failed. The requirement BR-08 is not met; however, this can be ignored for the release due to the low-priority nature of this requirement.

The metrics discussed can be expressed graphically:Requirement Coverage with Pass/Fail test Cases

No. of Test Cases

2 2 Test cases ailed Test cases passed

2 Requirements

D6 5 4 3 D 2 1 0 1 2 3


h R qu



4 R qu

5 n






The purpose of positive testing is to prove that the product works as per specification and expectations. A product delivering an error when it is expected to give an error is also a part of positive testing. Example of positive test casesReq. no. BR-01 BR-01 BR-02 BR-02 BR-04 Input 1 Key 123-456 Key 123-456 Key 123-456 Key 123-456 Hairpin Input 2 Turn clockwise Turn clockwise Turn anticlockwise Turn anticlockwise Turn clockwise Current state Unlocked Locked Unlocked Locked Locked Expected output Locked No change No change Unlock No change

Negative testing is done to show that the product does not fail when an unexpected input is given. Negative testing covers scenarios for which the product is not designed and coded. Negative test casesS. No. 1 2 3 4 Input 1 Input 2 Current state Lock Unlock Unlock Lock Expected output Lock Unlock Unlock Lock

Some other locks Turn Key clockwise Some other locks Turn Key anticlockwise Thin piece of wire Hit with a stone Turn anticlockwise

*there are no requirement numbers because test conditions lie outside the specification.

Difference: Coverage. Positive testing: if all documented requirements and test conditions are covered, then coverage can be considered to be 100 percent. Negative Testing: No end.

III. BOUNDARY VALUE ANALYSIS Boundary value analysis (BVA) is useful for arriving at tests that are effective in catching defects that happen at boundaries. Boundary value analysis believes and extends the concept that the density of defect is more towards the boundaries. Example:Number of unit bought First ten units (that is, from 1 to 10 units) Next ten units (that is, from units 11 to 20 units) Next ten units (that is, from units 21 to 30 units) More than 30 units Price per unit $5.00 $4.75 $4.50 $4.00

Most defects around the boundaries E.g. Buying 9,10,11,19,20,21,29,30,31 and similar number of items. Possible reasons: Programmers tentativeness in using the right comparison operat