Test Case Design: Strategies, Techniques & Models Robin Brennan, Senior Consultant Quality Assurance...
-
Upload
kayleigh-vinal -
Category
Documents
-
view
213 -
download
0
Transcript of Test Case Design: Strategies, Techniques & Models Robin Brennan, Senior Consultant Quality Assurance...
Test Case Design:Strategies, Techniques & Models
Robin Brennan, Senior ConsultantRobin Brennan, Senior ConsultantQuality Assurance InstituteQuality Assurance Institute
Quality Assurance Institute 2
Objectives
• Present test strategies, techniques and models to support test case design
• Particular focus on Equivalence Partitioning, Boundary Analysis and Decision Tables
Quality Assurance Institute 3
Why Test Case Design?• We cannot test everything• We need test strategies to design test cases for today’s
complex systems• We need systematic techniques to design input/output,
combinations, states, relationships and special testing• The design of the test models and test cases often
reveals defects from earlier phases• Models support test design at any level – method,
unit/class, integration, subsystem, system, user acceptance
Quality Assurance Institute 4
Designing Test Cases
• Has a high probability of detecting errors
• Reduces the number of necessary test cases by more than one
• Covers additional test case possibilities
A GOOD TEST CASE…
Quality Assurance Institute 5
Designing Test Cases
• Three components of a test case:
– Condition to be tested
– Expected result
– Procedure
Quality Assurance Institute 6
Test Case Design – Strategies, Techniques & Models
• Equivalence Partitioning– Represents the equivalent classes of input & tests
a representative of each class
• Boundary Analysis– Determines the boundary conditions on, above
and below the edges of inputs and outputs
• Decision Tables– Represents the relationship between input
conditions and output actions
Quality Assurance Institute 7
Quality Assurance Institute 8
What Is Equivalence Partitioning
• A black box testing technique– Originally from “The Art of Software Testing”
by Glenford Myers (1979)
• Determines categories of inputs
• Used for both valid and invalid variables
• Can be utilized at any level of testing
• Can be utilized early in the life cycle
Quality Assurance Institute 9
What is Equivalence Partitioning
• Incorporates, through inference, additional test cases– Reveals information beyond the set of input
values– Can reasonably assume that if the test uncovers
an error, the other items in the same equivalence class would result in the same error
Quality Assurance Institute 10
How to Perform Equivalence Partitioning
• Develop a set of conditions to be tested– Identify each input condition from requirement
or design specifications– Partition each into two or more groups
(equivalence classes)• One Valid
• One or more Invalid
Quality Assurance Institute 11
How to Perform Equivalence Partitioning (continued)
• Review all equivalence classes. If any components are not processed in the same manner, split the class into smaller classes
• Develop a minimal set of test cases– Test a representative of each category
Quality Assurance Institute 12
Create an Equivalence Class Table• External input condition on the left
• A column each for valid and invalid classes
• Unique number assigned to each equivalence class
EQUIVALENCE CLASSES
EXTERNAL INPUT CONDITION
VALID EQUIVALENCE
CLASSES
INVALID EQUIVALENCE CLASSES
Quality Assurance Institute 13
Defining the Equivalence Classes
1. Input descriptions that specify a range of values:
• Identify one valid equivalence class• Identify two invalid equivalence classes, one that
lies above the range and one that lies below the range
Quality Assurance Institute 14
Defining the Equivalence Classes
2. Input descriptions that define a number of values:
• Identify one valid equivalence class• Identify two invalid equivalence classes outside
each of the ends of the allowed number of values
Quality Assurance Institute 15
Defining the Equivalence Classes
3. Input descriptions that define a set of values and each is handled differently (discrete input):
• Identify one valid equivalence class for each discrete input in the set
• Identify one additional invalid equivalence class for a value not included in the set
Quality Assurance Institute 16
Defining the Equivalence Classes
4. Input descriptions that are “data typed”:• Identify one valid equivalence class for the
correct data type• Identify at least one invalid equivalence class
for a different data type
Quality Assurance Institute 17
Defining the Equivalence Classes
5. Input descriptions with specific mandatory conditions:
• Identify one valid equivalence class for the specific mandatory condition
• Identify one invalid equivalence class for a condition that is not mandatory
Quality Assurance Institute 18
EQUIVALENCE CLASS TABLE
EXTERNAL INPUT CONDITION
VALID EQUIVALENCE
CLASSES
INVALID EQUIVALENCE CLASSES
Job Code (1) 10-60 (2) <10 (3) >60 Office Location (4) Location GO1 (9) something else (5) Location S1 (6) Location S2 (7) Location S3 (8) Location S4 Employee Number (10) numeric (11) something other than numeric (12) special characters Telephone Number (13) one to five
telephone numbers (14) less than one telephone number
(15) more than five telephone numbers Catalog Number (16) first character
numeric (17) first character not numeric
SAMPLE EQUIVALENCE CLASS TABLE
Quality Assurance Institute 19
Construct the Test Cases
• Construct as few as possible while still maintaining a reasonable level of confidence
• Avoid “error masking” by not combining invalid values in one test case– If more than one invalid condition is in a test case, an
earlier value may result in never processing a subsequent value
Quality Assurance Institute 20
Guidelines
• Write a new test case including as many valid equivalence classes as possible, until all valid conditions have been covered
• Write a separate test case for each invalid equivalence class
Quality Assurance Institute 21
TEST CASES DERIVED FROM EQUIVALENCE CLASSES
EXTERNAL INPUT CONDITION
EQUIVALENCE CLASS
TEST CASE NO.
TEST CASE
Job Code, Office Location, Employee Number, Telephone Number, Catalog Number
1, 4, 10, 13, 16 1 41,GO1,53176,31037498457145639056,9A82437
Office Location 5 2 41,S1,53176,31037498457145639056,9A82437
Office Location 6 3 41,S2,53176,31037498457145639056,9A82437
Office Location 7 4 41,S3,53176,31037498457145639056,9A82437
Office Location 8 5 41,S4,53176,31037498457145639056,9A82437
Job Code 2 6 3,GO1,53176,31037498457145639056,9A82437
Job Code 3 7 87,GO1,53176,31037498457145639056,9A82437
Office Location 9 8 41,S9,53176,31037498457145639056,9A82437
Employee Number 11 9 41,S3,abcde,31037498457145639056,9A82437
Employee Number 12 10 41,S3,%&$#@,31037498457145639056,9A82437
Telephone Number 14 11 41,S3,53176,,9A82437 Telephone Number 15 12 41,S3,53176,31037498457145639056
714575493031035478922134347862213327903531037465897148725430,9A82437
Catalog Number 17 13 41,GO1,53176,31037498457145639056,A47236D
Quality Assurance Institute 22
Quality Assurance Institute 23
What Is Boundary Analysis• Another black box testing technique
• Complimentary with Equivalence Partitioning
• Higher return-on-investment than random selection
• Can be used at any level, early in the life cycle
• Sounds simple but is often used ineffectively
• Includes both input and output domains
• Includes values on, above and below the edges
Quality Assurance Institute 24
Guidelines for Input ValuesOne or more elements are selected from the
edges• For continuous input, select the highest and
lowest value within the range, and select the values just above and below the edges
• For discrete ordered sets of input, select the first and last elements in the set
• For a specific number of input values, select the minimum and maximum number of values, and select a number of values one below and one beyond the valid number of values
Quality Assurance Institute 25
Guidelines for Output ValuesOne or more elements are generated from the
edges• For continuous output, generate the highest and
lowest value within the range, and generate the values just above and below the edges
• For a discrete ordered set of output, ensure the first and last elements in the set are processed
• For a specific number of output values, generate the minimum and maximum number of values, and generate a number of values, one below and one beyond the valid number of values
Quality Assurance Institute 26
TEST CASES DERIVED FROM BOUNDARY ANALYSIS
EXTERNAL INPUT CONDITION
EQUIVALENCE CLASS & BOUNDARY
ANALYSIS
TEST CASE NO.
TEST CASE
Job Code, Office Location, Employee Number, Telephone Number, Catalog Number
1, 4, 10, 13, 16; highest valid Job Code, one Telephone No.
*1 60,GO1,53176,3103749845,9A82437
Job Code, Office Location, Employee Number, Telephone Number, Catalog Number
1, 4, 10, 13, 16, lowest valid Job Code, five Telephone No’s.
**2 10,GO1,53176,31037498457145639056714575493031035478922134347862,9A82437
Office Location 5 3 10,S1,53176,31037498457145639056,9A82437
Office Location 6 4 10,S2,53176,31037498457145639056,9A82437
Office Location 7 5 10,S3,53176,31037498457145639056,9A82437
Office Location 8 6 10,S4,53176,31037498457145639056,9A82437
Job Code 2; just below edge *7 9,GO1,53176,31037498457145639056,9A82437
Job Code 3; just above edge *8 61,GO1,53176,31037498457145639056,9A82437
Office Location 9 9 10,S9,53176,31037498457145639056,9A82437
Employee Number 11 10 10,S2,abcde,31037498457145639056,9A82437
Employee Number 12 11 10,S2,%&$#@,31037498457145639056,9A82437
Telephone Number 14 12 10,S2,53176,,9A82437 Telephone Number 15; one beyond valid
number of values *13 60,S2,53176,3103749845714
56390567145754930310354789221343478622133279035,9A82437
Catalog Number 17 14 41,GO1,53176,31037498457145639056,A47236D
* Changed values of Test Case ** Additional Test Case
Boundary
Analysis
Examples
Quality Assurance Institute 27
Quality Assurance Institute 28
Decision Tables• Graphical representation of conditional logic
• More effective than narratives for communicating information
• Can be used at any level
• Used in many software engineering methodologies as a design tool
• Constructed from specifications, use cases, class diagrams, behavior diagrams, source code
• Handles combinations of Equivalence Classes
Quality Assurance Institute 29
Decision Tables for Testing
• Forces consideration of all combinations of conditions and their resultant actions
• Helps develop all variations avoiding omission
• Helps determine irrelevant test conditions
• Results in fewer test cases while retaining good test coverage
Quality Assurance Institute 30
Use a Decision Table when…
• Distinct input conditions result in a distinct output action
• Conditions can be modeled by:– Mutually exclusive Boolean expressions– Binary response (Yes/No; True/False)
• Results do not depend on the order of input conditions
• Results do not depend on prior input or output
Quality Assurance Institute 31
Decision Table Format
TITLE RULES CONDITIONS
CONDITION ENTRIES
ACTIONS
ACTION ENTRIES
Quality Assurance Institute 32
Decision Table Format - Vertical
TITLE RULES CONDITIONS
CONDITION ENTRIES
ACTIONS
ACTION ENTRIES
TITLE: TITLE: Unique Name or NumberUnique Name or Number
to Identify the Tableto Identify the Table
RULES: RULES: Each Unique CombinationEach Unique Combination
of Conditions & Actionsof Conditions & Actions(single column for each Rule)(single column for each Rule)
CONDITIONS: CONDITIONS: All ConditionsAll Conditionsto be Examinedto be Examined
CONDITION ENTRIES: CONDITION ENTRIES: Decision Entry for the ConditionDecision Entry for the Condition
(Yes/No; True/False)(Yes/No; True/False)
ACTIONS: ACTIONS: All Actions that CanAll Actions that Can
Occur for the ConditionsOccur for the Conditions
ACTION ENTRIES: ACTION ENTRIES: Resultant ActionResultant Action
Based on the ConditionsBased on the Conditions
Quality Assurance Institute 33
Steps to Create Decision Tables
1. Identify the conditions and their entries
2. Identify the actions (results)
3. Determine the maximum number of Rules using the following formula:
2n …where n is the number of conditionse.g., 3 conditions: 23 = 8 rules
4 conditions: 24 = 16 rules
Quality Assurance Institute 34
Steps to Create Decision Tables4. Divide the “Entries” side into the # of Rules
Title
1 2 3 4 5 6 7 8
Conditions
Actions
Quality Assurance Institute 35
Steps to Create Decision Tables5. For maximum number of rules:
– The entries for condition 1 will be 50% “Yes” and 50% “No”
– For condition 2, the entries will be a pattern of 25% “Yes”and 25% “No”, repeated across the table
– This pattern continues for each subsequent condition…for example, condition 3 would have a pattern of 12.5% “Yes” and 12.5% “No”
6. Determine the action entry for each column
Quality Assurance Institute 36
Steps to Create Decision Tables
Title
1 2 3 4 5 6 7 8
Condition1 Y Y Y Y N N N N
Condition2 Y Y N N Y Y N N
Condition3 Y N Y N Y N Y N
Action1 x
Action2 x x x x x x x
Quality Assurance Institute 37
Steps to Create Decision Tables
7. Collapse the table by eliminating conditions that don’t need to be tested, as follows:
– When two rules have the same action entry and their condition entries only vary by one answer, eliminate the condition where it varies
– Next, analyze the table for redundancies and combine any rules that are now the same
– Repeat this process until all unnecessary tests have been eliminated
Quality Assurance Institute 38
Steps to Create Decision Tables
8. Confirm the accuracy of the table with the following calculation:
Number of Rules = n + 1
where n is the number of conditions
Quality Assurance Institute 39
Sample Decision Table
Automobile Product Reorder Criteria:
“Reorder the model if the current inventory of the model is less than 20 automobiles, the average sales is four automobiles a week or more, and the average sales profit on those sales is 7% or more”
Quality Assurance Institute 40
Sample Decision TableAutomobile Product Reorder:
1 2 3 4 5 6 7 8
Have 20 Units Y Y Y Y N N N N
Sold Average of 4/week
Y Y N N Y Y N N
Profit 7% Y N Y N Y N Y N
Reorder Auto X
No Reorder X X X X X X X
Quality Assurance Institute 41
Sample Decision TableAutomobile Product Reorder:
1 2 3 4 5 6 7 8
Have 20 Units Y Y Y Y N N N N
Sold Average of 4/week
Y Y N N Y Y N N
Profit 7% Y N Y N Y N Y N
Reorder Auto X
No Reorder X X X X X X X
Quality Assurance Institute 42
Sample Decision TableAutomobile Product Reorder:
1 2 3 5 7
Have 20 Units Y Y Y N N
Sold Average of 4/week
Y Y N Y N
Profit 7% Y N Y Y Y
Reorder Auto X
No Reorder X X X X
Quality Assurance Institute 43
Sample Decision TableAutomobile Product Reorder:
1 2 3 5
Have 20 Units Y Y Y N
Sold Average of 4/week
Y Y N Y
Profit 7% Y N Y Y
Reorder Auto X
No Reorder X X X
Quality Assurance Institute 44
Decision Tables to Test Cases… After creating the decision table…
• Verify the decision table• Create one test case for each unique Rule• Cross-reference with Equivalence Partitioning
test cases and eliminate redundancy• Use boundary-analysis to determine values
for the condition entries• Vary the order of the input variables• Mix the sequence of test orders on different
test executions
Quality Assurance Institute 45
Test Case Design – Wrap Up• Evaluate risk, complexity, frequency of use,
inheritance, dependencies, variables, combinations, test time and budget
• Prioritize critical & complex components
• Use equivalence partitioning, boundary analysis, decision tables and other test design strategies and models to determine test cases
• Combine several test design strategies to ensure acceptable test suite coverage
Quality Assurance Institute 46
Test Case Design Strategies
• Risk-based– Results of risk analysis
• State-based– State of the objects– Transition paths– State of the environment
Quality Assurance Institute 47
Test Case Design Strategies
• Interaction-based– Interactions between components
• Specification-based– Functional; requirements; black box
• Implementation-based– Structural; code-based; white box
Quality Assurance Institute 48
Test Case Design Strategies• Use Case Scenarios
– Primary flow– Alternate flows– Exceptions– Extensions
• CRUD– Create, Read, Update, Delete
Quality Assurance Institute 49
In Summary• Designing good test cases is hard work
• With a systematic approach, we can design fewer test cases while still ensuring a high probability of detecting errors
• A combination of strategies, techniques and models must be utilized
• Utilize checklists to support various types of testing
Quality Assurance Institute 50
In Summary• Use Equivalence Partitioning and Boundary
Analysis for input and output variations • Use Decision Tables for combinations of
conditions• Use State Charts for testing combinations of
message sequencing & instance variables• Use UML diagrams to identify relationships• Use Cases are an excellent source of
information for functional/system testing
Thank You