Black Box Testing

ByP.Vinitha

Release Testing (Black box testing)• Release testing is the process of testing a release

of the system that will be distributed to customers.

• The primary goal of this process is to increase the supplier's confidence that the system meets its requirements.

• To demonstrate that the system meets its requirements, it must be shown that it delivers the specified functionality, performance and dependability, and that it does not fail during normal use.

Black Box Testing

• Black box testing, also called functional testing and behavioral testing, focuses on determining whether or not a program does what it is supposed to do based on its functional requirements

Black Box Testing

• The tester presents inputs to the component or the system and examines the corresponding outputs.

• If the outputs are not those predicted (i.e., if the outputs are in set O,) then the test has detected a problem with the software.

Ie

Input test data

OeOutput test results

System

Inputs causinganomalousbehaviour

Outputs which revealthe presence ofdefects

Black Box Testing

• Black box testing attempts to find errors in the external behavior of the code in the following categories – Incorrect or missing functionality– Interface errors– Errors in data structures used by interfaces– Behavior or performance errors– Initialization and termination errors

Guidelines for Black Box testing

• Choose inputs that force the system to generate all error messages.

• Design inputs that cause input buffers to overflow.

• Repeat the same input or series of inputs numerous times.

• Force invalid outputs to be generated.• Force computation results to be too large or

too small.

Advantages of Black Box Testing• Efficient when used on Larger systems • As the tester and developer are independent of each other,

test is balanced and unprejudiced • Tester can be non-technical. • There is no need of having detailed functional knowledge of

system to the tester. • Tests will be done from a end user's point of view. Because

end user should accept the system. (This is reason, sometimes this testing technique is also called as Acceptance testing)

• Testing helps to identify the vagueness and contradiction in functional specifications.

• Test cases can be designed as soon as the functional specifications are complete

Disadvantages of Black Box Testing

• Test cases are tough and challenging to design, without having clear functional specifications

• It is difficult to identify tricky inputs, if the test cases are not developed based on specifications.

• It is difficult to identify all possible inputs in limited testing time. So writing test cases is slow and difficult

• Chances of having unidentified paths during this testing

• Chances of having repetition of tests that are already done by programmer.

Black-Box Testing Techniques

• Graph Based Testing Methods• Equivalence Partitioning• Boundary Value Analysis• Comparison Testing• Orthogonal Array Testing

Graph-based Testing

• Graph-based Testing is a black-box testing technique that uses objects that are modeled in software and the relationships among these objects.

• Understanding the dynamics on how these objects communicate and collaborate with one another can derive test cases.

Graph-based Testing

• Create a graph of software objects and identify the relationship of these objects.

• Using nodes and edges, create a graph of software objects.

• Nodes represent the software objects• Links represent the relationship between objects.• Node weights describes the properties of the node.

Graph-based Testing

• Graph Notation

Object #1

Object #2

Object #3

Directed Link

Undirected Link Parallel Link

Node Weight (Value)

Graph-based Testing

• Sample Example

New File

Menu Select

Document Window

Document Text

menu Select generates

Is reperesented as

Contains

Attributes:Start dimension : default Setting or PreferencesBackground color: WhiteText Color: default color or preferences

(generation time <1.0 Sec)

Graph-based Testing

• Guidelines for Graph-based Testing1. Identify the start and stop points of the graph.

There should be an entry and exit nodes.2. Name nodes and specify their properties.3. Establish their relationship through the use of

edges. Specify the properties.4. Derive test cases and ensure that there is node

and edge coverage.

Equivalence Partitioning

• Equivalence Testing is a black-box testing technique that uses the input domain of the program.

• It divides the input domain into sets of data from which test cases can be derived.

• Derived test cases are used to uncover errors that reflect a class of errors.

• Thus, reduces the effort in testing the software. • It makes use of equivalence classes, which are sets of

valid and invalid states• that an input may be in.

Equivalence Partitioning

• Guidelines in Identifying Equivalence Classes1. Input Condition is specified as a range of value. The

test case is one valid input, and two invalid equivalence classes.

2. Input Condition requires a specific value. The test case is one valid, and two invalid equivalence classes.

3. Input condition specifies a member of a set. The test case is one valid and one invalid.

4. Input condition is Boolean. The test case is one valid, and one invalid

Equivalence Partitioning

• consider a text message code of registering a mobile number to a text service of getting traffic reports. Assume that the message requires the following structure:

Boundary Value Testing

• Boundary Value Analysis (BVA) is a Functional Testing technique where the extreme boundary values are chosen.

• Boundary values include maximum, minimum, just inside/outside boundaries, typical values, and error values.

Boundary Value Testing

• Guidelines in Deriving Test Cases Using BVA– If input condition specifies range bounded by

values and b, the test cases should be designed with values and b and just above and just below a and b.

– If input condition specifies number of values, the test cases should be developed that exercise the minimum and maximum numbers. Values just above and below the minimum and maximum are also tested.

Comparison Testing

• Black-box testing for safety critical systems in which independently developed implementations of redundant systems are tested for conformance to specifications.

• When redundant systems are developed, separate software engineering teams develop independent versions of an application using the same specification.

• Each version is tested with the same test data to ensure that all provide the same output.

• Then all versions are executed in parallel with real-time comparison of results to ensure consistency.

Comparison Testing• Often equivalence class partitioning is used to

develop a common set of test cases for each implementation.

• If the output from each version is the same then it is assumed that all the implementations are correct.

• If the output is different then careful analysis is done.

• Even though more than one implementations are developed only a single version will be used in the delivered computer based system

Orthogonal Array Testing

• Orthogonal array testing is applied to problems where the input domain is relatively small but too large to accommodate exhaustive testing.

• This method is useful for finding errors associated with the software component.

Orthogonal Array Testing

• Orthogonal arrays are two dimensional arrays of numbers which possess the interesting quality that by choosing any two columns in the array you receive an even distribution of all the pair-wise combinations of values in the array

terminology for working with orthogonal arrays

• Runs: the number of rows in the array. This directly translates to the number of test cases that will be generated by the OATS technique.

• Factors: the number of columns in an array. This directly translates to the maximum number of variables that can be handled by this array.

• Levels: the maximum number of values that can be taken on by any single factor. An orthogonal array will contain values from 0 to Levels-1.

• Orthogonal arrays are most often named following the pattern LRuns(LevelsFactors).

Orthogonal Array Testing• How to use this technique

1. Decide how many independent variables will be tested for interaction. This will map to the Factors of the array.

2. Decide the maximum number of values that each independent variable will take on. This will map to the Levels of the array.

3. Find a suitable orthogonal array with the smallest number of Runs.3 A suitable array is one that has at least as many Factors as needed from Step 1 and has at least as many levels for each of those factors as decided in Step 2.

4. Map the Factors and values onto the array.5. Choose values for any "left over" Levels.6. Transcribe the Runs into test cases, adding any

particularly suspicious combinations that aren't generated.

Orthogonal Array Testing

• Send function of a fax application• Four parameters p1,p2,p3,p4• Each takes three discrete values• P1 takes on values

P1=1, Send it nowP1=2, Send it one hour laterP1=3, Send it after midnight.

• P2, P3 and P$ also takes values 1,2 and 3

Orthogonal Array Testing (L9 orthogonal Array for fax send)

Test CasesTest Parameters

P1 P2 P3 P41 1 1 1 12 1 2 2 23 1 3 3 34 2 1 2 35 2 2 3 16 2 3 1 27 3 1 3 28 3 2 1 39 3 3 2 1

Orthogonal Array Testing

• Can identify– Single mode fault– Double mode fault– Multimode fault