MUTATION TESTING

Mutation testing (or Mutation analysis or Program mutation) is used to design new software tests and evaluate the quality of existing software tests. Mutation testing was pioneered in the 1970s to locate and expose weaknesses in test suites. It is the process of "re-writing" source code (hence the name Mutation) in order to flush out ambiguities that may exist in the code. Such small changes are intended to model low level defects that arise in the process of coding systems.The process, given program P and test suite T, is as follows:

We systematically apply mutations to the program P to obtain a sequence P1, P2,... Pn of mutants of P. Each mutant is derived by applying a single mutation operation to P.

We run the test suite T on each of the mutants, T is said to kill mutant Pj if it detects an error. If we kill k out of n mutants the adequacy of T is measured by the quotient k/n. T is mutation adequate if k= n.

Kinds of Mutation

Value Mutations: these mutations involve changing the values of constants or parameters (by adding or subtracting values etc), Typical examples are: Changing values to one larger or smaller (or similar for real numbers)and Swapping values in initializations.

Decision Mutations: this involves modifying conditions to reflect potential slips and errors in the coding of conditions in programs, e.g. a typical mutation might be replacing a > by a < in a comparison.

Statement Mutations: these might involve deleting certain lines to reflect omissions in coding or swapping the order of lines of code. There are other operations, e.g. changing operations in arithmetic expressions. A typical omission might be to omit the increment on some variable in a while loop.

For example, consider the following C++ code fragment:

if (a && b) { c = 1;} else { c = 0;}

The condition mutation operator would replace && with || and produce the following mutant:

if (a || b) { c = 1;} else { c = 0;}

Now, for the test to kill this mutant, the following three conditions should be met:

1. A test must reach the mutated statement.2. Test input data should infect the program state by causing different program states for the mutant and the original program. For

example, a test with a = 1 and b = 0 would do this.3. The incorrect program state (the value of 'c') must propagate to the program's output and be checked by the test.

Mutation Testing ToolsThere is a range of possible mutation tools. Recently Offutt and others have created MuJava, a tool for creating Java mutants.

Documentation testing

Documentation testing is a non-functional type of software testing which involves testing the quality of the documentation, e.g. user guide or installation guide. Documentation is a very important part of a software product's success and effectiveness. If the documentation is poor, deficient, or defective, it may affect the quality of software or application. Documentation testing assures that the written explanations of how to use the system practically match what

the system does. Documentation testing can be performed in a lot of various methods and with many levels of complication: including testing the documents through a spelling and grammar checking device or manually inspecting the documentation to eliminate any ambiguous expressions or incompatibility.

Documentation can be tested by asking users try to use the new system using the documentation provided. After this you should choose those users who are conversant with their own domain (operation, application, administration and so on) but who are not conversant with the new system.

State based testing

State based testing (SBT) is Functional Testing technique. The boundary value and equivalence class partitioning technique is more related to the various input combinations and their result. But state based technique is different from this. 

In any system, it will have a set of inputs and corresponding actions. SBT is making use of this. You need to identify all possible valid and invalid transitions in the system and test them. 1. Identify various COMPONENTS in the system.

2. Identify the various possible STATES of these components.

3. Identify the possible ACTION (i.e.: action which causes a transition from one state to other) 

4. Draw a state diagram based on these.

5. Test all valid and invalid transitions.

Eg: Consider stack example.

Components: Stack 

States: Empty, Full, Holding 

Actions : Push, Pop