Chapter 9.3

OHT 9.1

Galin, SQA from theory to implementation © Pearson Education Limited 2004

Chapter 9.3

Software Testing Strategies

OHT 9.2


• Definitions and objectives• Software testing strategies• Software test classifications• White box testing

• Data processing and calculation correctness tests• Correctness tests and path coverage• Correctness tests and line coverage• McCabe’s cyclomatic complexity metrics• Software qualification and reusability testing• Advantages and disadvantages of white box testing

Black box testing• Equivalence classes for output correctness tests• Other operation factor testing classes• Revision factor testing classes• Transition factor testing classes• Advantages and disadvantages of black box testing

Chapter 9.3 – Software Testing Strategies

OHT 9.3


Introductory Thoughts• Black Box Testing: Use to perform ‘correctness tests’

and more• Some of these white box tests, such as maintainability

tests and calculation correctness testing.• Bottom line: BB testing canNOT substitute for WB

testing.

• Will look at:– Equivalence class testing and number of test cases required

– Performance methodology for other classes of BB tests

– Advantages and disadvantages of BB testing.

OHT 9.4


Equivalence Classes for Output Correctness

• Correctness tests take the greater part of testing resources.• One good approach to an improved choice of tests is the

equivalence class partitioning.– Increases testing efficiency

– An Equivalence Class (EC) is a carefully devised limited set of input variable values that essentially produce the same output results using many more ‘equivalent’ tests.

– We have single values, ranges, etc. to consider

– We have valid ECs and invalid ECs

OHT 9.5



• Design tests such that each valid EC and each invalid EC are included in at least one test case.

• Test cases are defined separately for each type of EC. • For EC class testing, we include ECs in EC Test Cases.

– Can have a number of valid EC tests in a single test case.

• For Invalid EC testing, we assign a single test case to each new invalid EC test.– Can, in this way, distinguish results of each invalid EC.

OHT 9.6



• Bottom line here: by using ECs, far fewer test cases needed than might otherwise be needed

• Test cases are based on the specifications not on code• Better results are obtained in testing boundary values

of ECs as well.

• Very important to test boundaries. Rich source of very common errors.

• More later on this.

OHT 9.7



• Boundary Class testing example: (home made)• Reading a file: (sequential)

– Boundary issues:• Can’t open the file• File is empty (no records)• Read beyond end of file• File has one record

– Standard EC test here: once you process two records, you are likely good to go.

• Stack– Stack empty; stack full. These are the boundary conditions.– Other processing - normal

• Always, always always test the boundaries!!!

OHT 9.8


Test Cases and Boundary Values• Formally, one test case per equivalence class should be

enough.• If, however, have range of values, border values are very

error prone.– Example: Range of acceptable values: 1-10.– Midrange test (no problemo) any number 2-9;– Boundary value tests: test 1 and test 10.

• Test for mid range, lower and upper bound values!!

• Let’s review the principles again for Equivalence Class partitioning…

OHT 9.9


A black box method aimed at increasing the efficiency of testing and, at the same time, improving coverage of potential error conditions.

OHT 9.10


• An equivalence class (EC) is a set of input variable values that produce the same output results or that are processed identically. – Again, consider a range of acceptable values: say, 1 – 10.– Tests with inputs 2 through 9 should produce the same result.– No need for more than one test with input value 2 to 9.

• EC boundaries are defined by a single numeric or alphabetic value, a group of numeric or alphabetic values, a range of values, and so on. – If range is 1-10, boundaries would be 1 and 10.

OHT 9.11


• An EC with only valid states is defined as a "valid EC," whereas an EC that contains only invalid states is defined as the "invalid EC." – By definition!

• In cases where a program's input is provided by several variables, valid and invalid ECs should be defined for each variable.

OHT 9.12


According to the equivalence class partitioning method: • Each valid EC and each invalid EC are included in at least

one test case.– Note: equivalence CLASS! – For an acceptable range of input integers is 1-10, we need at least

one test with a value in the range 1-10.– Boundaries: 1 and 10– Invalid EC classes might be EC with values < 1, another class > 10,

another one - negative numbers, ….

• Definition of test cases is done separately for the valid and invalid ECs. – Note: Run tests for invalid equivalence classes one at a time.

OHT 9.13


According to the equivalence class partitioning method: • In defining a test case for the valid ECs, we try to cover as

many as possible “new” ECs in that same test case. – Valid ranges: 1-10; 30-50.– Include both range EC values in same EC test.

• In defining invalid ECs, assign one test case to each “new” invalid EC.

• Run separately. A test case with more than one invalid EC may not allow the tester to distinguish between the program’s separate reactions to each of the invalid ECs.

• Test cases are added as long as there are uncovered ECs.

OHT 9.14


Operation Factor Testing Cases• Other kinds of tests include:

– Correctness • Documentation tests

• Availability (reaction time tests)

– Reliability• Reliability tests

– Efficiency• Stress tests (load and durability tests)

– Integrity• Software system security tests

– Usability• Training usability tests

• Operational usability tests

OHT 9.15


Operation Factor Testing Cases in more Detail

• Kinds of tests:– Correctness

• Documentation tests– So often neglected– Erroneous user manuals and operators manuals– Functional Descriptions

» Overview describes system. Useful for many purposes– Installation Manual

» How to install and interface with equipment; for COTS, contains customization instructions

– User Manual» How to use various functions, recovery instructions; documented help…

– Program Maintenance Manual» Bug corrections, changes, program structure, some algorithms…

• Availability (reaction time tests)– Critically important for online applications.– Response time– Critical for real time systems– Load and stress testing critical

OHT 9.16


More Black Box Tests– Reliability

• Reliability tests– Difficult to test

– Need to run under various and regular workload conditions

– Often simulated environment and loads

– Sometimes: MTBF; MTTR

– Efficiency• Stress tests (load and durability tests)

– Load tests are related to the functional performance under max load conditions

– E.g. Hits per minute, number of simultaneous users.

– Usually simulate more load than specifications cite.

• Durability tests – – extreme operating conditions, temperature, humidity, rough conditions, think

aircraft instrumentation and guidance…sudden electrical ‘jumps’ communications cutoff (battery backup? …)

– Can see we are not going ‘in the Box.’–

OHT 9.17


Still more BB Tests• Other kinds of tests include:

– Integrity – read this…• Software system security tests

– Usability• Training and Operational usability tests

– Test the operator’s productivity in using or trying to use the application.

» Observation and his/her productivity

» Can be quantitative and qualitative.

OHT 9.18


Other Tests

• Maintainability tests– Remember: More than 60% of a system’s

operational time is usually done in a maintenance mode!

– Other figures suggest higher rates!

• Overall manuals should include one that addresses: system structure, programmer maintenance and an operator’s manual.

OHT 9.19


Advantages: Allows us to carry out the majority of testing classes, most of which

can be implemented solely by black box tests, i.e. load tests and availability tests.

For testing classes that can be carried out by both white and black box tests, black box testing requires fewer resources.

Disadvantages: Possibility that coincidental aggregation of several errors will produce

the correct response for a test case, and prevent error detection. Absence of control of line coverage. There is no easy way to specify

the parameters of the test cases required to improve coverage. * Impossibility of testing the quality of coding and its strict adherence to

the coding standards.

Advantages and Disadvantages of Black Box Testing

OHT 9.20


Homework

• Questions 9.1 and 9.4– Due Wednesday, 13 Nov at 4pm

OHT 9.21


Team Presentation

• Discuss statement, path, and branch coverage measures.

• Discuss: static testing and dynamic testing with examples

• Discuss black box testing and white box testing with examples

• Discuss verification and validation and acceptance testing. Give examples

Chapter 9.3

Documents

Transcript of Chapter 9.3