1

Software Testing Strategies: Approaches, Issues, Testing

Tools

2

Strategic Approach to Testing - 1

• Testing begins at the component level and works outward toward the integration of the entire computer-based system.

• Different testing techniques are appropriate at different points in time.

• The developer of the software conducts testing and may be assisted by independent test groups for large projects.

• The role of the independent tester is to remove the conflict of interest inherent when the builder is testing his or her own product.

3

Strategic Approach to Testing - 2

• Testing and debugging are different activities.

• Debugging must be accommodated in any testing strategy.

• Need to consider verification issues– are we building the product right?

• Need to Consider validation issues– are we building the right product?

4

Strategic Testing Issues - 1

• Specify product requirements in a quantifiable manner before testing starts.

• Specify testing objectives explicitly.• Identify the user classes of the software

and develop a profile for each.• Develop a test plan that emphasizes

rapid cycle testing.

5

Strategic Testing Issues - 2

• Build robust software that is designed to test itself (e.g. use anti-bugging).

• Use effective formal reviews as a filter prior to testing.

• Conduct formal technical reviews to assess the test strategy and test cases.

6

7

8

Stages of Testing

• Unit Testing,• Regression Testing,• Integration testing,• Incremental testing,• Validation testing,• Acceptance testing,• System Testing.

Unit Testing

• Unit testing -- focuses on the smallest element of software design viz. the module. – Corresponds to class testing in the OO

context. • Makes heavy use of white-box testing.

9

10

Regression Testing

• Check for defects propagated to other modules by changes made to existing program – Representative sample of existing test cases is

used to exercise all software functions.– Additional test cases focusing software functions

likely to be affected by the change.

11

Integration Testing

• Bottom - up testing• Top - down testing

12

Top-Down Integration Testing• Main program used as a test driver and stubs are

substitutes for components directly subordinate to it.• Subordinate stubs are replaced one at a time with real

components.• Tests are conducted as each component is

integrated.• On completion of each set of tests and other stub is

replaced with a real component.• Regression testing may be used to ensure that new

errors not introduced.

13

14

Bottom-Up Integration Testing

• Low level components are combined in clusters that perform a specific software function.

• A driver (control program) is written to coordinate test case input and output.

• The cluster is tested.• Drivers are removed and clusters are

combined moving upward in the program structure.

15

Incremental Testing

• A integration testing strategy in which you test subsystems in isolation, and then continue testing as you integrate more and more subsystems

16

17

Validation Testing

• Ensure that each function or performance characteristic conforms to its specification.

• Deviations (deficiencies) must be negotiated with the customer to establish a means for resolving the errors.

• Configuration review or audit is used to ensure that all elements of the software configuration have been properly developed, cataloged, and documented to allow its support during its maintenance phase.

18

Acceptance Testing• Making sure the software works correctly for

intended user in his or her normal work environment.

• Alpha test– version of the complete software is tested by

customer under the supervision of the developer at the developer’s site

• Beta test– version of the complete software is tested by

customer at his or her own site without the developer being present

19

System Testing

• Recovery testing– checks system’s ability to recover from failures

• Security testing– verifies that system protection mechanism prevents

improper penetration or data alteration• Stress testing

– program is checked to see how well it deals with abnormal resource demands

• Performance testing– tests the run-time performance of software

20

Debugging• Debugging (removal of a defect) occurs as a

consequence of successful testing.• Some people better at debugging than others.• Is the cause of the bug reproduced in another

part of the program?• What “next bug” might be introduced by the fix

that is being proposed?• What could have been done to prevent this bug

in the first place?

• Debugging is a consequence of successful testing -- when a test case uncovers an error, it is the debugging process that results in the removal of the error.

21

22

Debugging Approaches

• Brute force– memory dumps and run-time traces are examined

for clues to error causes• Backtracking

– source code is examined by looking backwards from symptom to potential causes of errors

• Cause elimination– uses binary partitioning to reduce the number of

locations potential where errors can exist

23

Testing Life Cycle• Establish test objectives.• Design criteria (review criteria).

– Correct.– Feasible.– Coverage.– Demonstrate functionality .

• Writing test cases.• Testing test cases.• Execute test cases.• Evaluate test results.

24

Test Team Members

• Professional testers.• Analysts.• System designers.• Configuration management specialists.• Users.

25

Test Documentation Needed

• Requirement being tested.• Design verification methodology.• Code verification methodology.

26

Document Each Test Case• Requirement tested.• feature / path tested.• Person & date.• Tools & code needed.• Test data & instructions.• Expected results.• Actual test results & analysis• Correction, schedule, and signoff.

Testing Tools

• Tools may be used for analyzing a test process as well as a product under testing.

• Tools may be used for testing, for document defects, or for preparing test artifacts.

27

Testing tools

• Tools used while testing artifacts such as checklists, check sheets, etc. These are useful for reviews, walkthroughs and inspection.

• Tools are used for test-process analysis such as Cause-and-effect charting, etc. These may be used for process and product analysis and also for devising corrective and preventing action.

28

Testing Tool

• Software tools used for various activities like regression testing, defect management, test management, etc.

29

Features of the test tool

• A test tool is a vehicle for performing test processes.

• Each tool has a specific purpose. Knowing its technique is very important while using a tool.

• Tools can be used in an entire SDLC for all verification and validation activities. A tool improves the efficiency and effectiveness of a testing process.

30

• Static Testing tools• Dynamic testing tools

31

Advantages

• Tool works faster than human.• Tools are consistent in behaviors and are not

prone to failure.• Some areas in software development can be

tested only be tools.•

32

Disadvantages• Wrong selection or wrong usage of a tool can lead

to disasters.• People using the tools must be properly trained.• Buying tool costs to an organization and one must

do a cost-benefit analysis before acquiring a tool.• Maintaining scripts in changing requirements may

be very difficult and if not done properly, a tool becomes useless as the requirements changes very frequently.

33

34

Testing Tools

• Simulators.• Monitors.• Analyzers.• Test data generators.