Software Testing

CHAPTER-1CHAPTER-1INTRODUCTION TOINTRODUCTION TO

SOFTWARE TESTINGSOFTWARE TESTING

INSIDE THIS CHAPTERINSIDE THIS CHAPTER

IntroductionIntroduction The Testing ProcessThe Testing Process What is Software Testing?What is Software Testing? Why Should We Test? What is the Purpose?Why Should We Test? What is the Purpose? Who Should Do Testing?Who Should Do Testing? What Should We Test?What Should We Test? Selection of Good Test CasesSelection of Good Test Cases Measurement of the Progress of TestingMeasurement of the Progress of Testing Incremental Testing ApproachIncremental Testing Approach Basic Terminology Related to Software TestingBasic Terminology Related to Software Testing Testing Life CycleTesting Life Cycle When to Stop Testing?When to Stop Testing? Principles of TestingPrinciples of Testing Limitations of TestingLimitations of Testing Available Testing Tools, Techniques and MetricsAvailable Testing Tools, Techniques and Metrics

INTRODUCTIONINTRODUCTION

Testing is the processes of executing the program Testing is the processes of executing the program with the intent of finding faults. Who should do with the intent of finding faults. Who should do this testing and when should it start are very this testing and when should it start are very important questions that are answered in the important questions that are answered in the text. As we know that software testing is the text. As we know that software testing is the fourth phase of Software Development Life Cycle fourth phase of Software Development Life Cycle (SDLC). About 70% of development time is spent (SDLC). About 70% of development time is spent on testing. We explore this and many other on testing. We explore this and many other interesting concepts in this chapter.interesting concepts in this chapter.

THE TESTING PROCESSTHE TESTING PROCESS

Please note that testing starts from Please note that testing starts from requirements analysis phase only and requirements analysis phase only and goes till the last maintenance phase.goes till the last maintenance phase.

Static Testing wherein the SRS is Static Testing wherein the SRS is tested to check whether it is as per tested to check whether it is as per user requirements or not.user requirements or not.

Dynamic Testing starts when the code Dynamic Testing starts when the code is ready or even a unit (or module) is is ready or even a unit (or module) is ready.ready.

WHAT IS SOFTWARE TESTING?WHAT IS SOFTWARE TESTING? The concept of software testing has evolved from simple program The concept of software testing has evolved from simple program

“check-out” to a broad set of activities that cover the entire “check-out” to a broad set of activities that cover the entire software life-cycle.software life-cycle.

There are five distinct levels of testing that are given below:There are five distinct levels of testing that are given below: Debug: Debug: It is defined as the successful correction of a failure.It is defined as the successful correction of a failure. Demonstrate:Demonstrate: The process of showing that major features work The process of showing that major features work

with typical input.with typical input. Verify: Verify: The process of finding as many faults in the application The process of finding as many faults in the application

under test(AUT) as possible.under test(AUT) as possible. Validate: Validate: The process of finding as many faults in requirements, The process of finding as many faults in requirements,

design and AUT.design and AUT. Prevent: Prevent: To avoid errors in development of requirements, design To avoid errors in development of requirements, design

and implementation by self-checking techniques, including “test and implementation by self-checking techniques, including “test before design”. before design”.

““Testing is the process of exercising or Testing is the process of exercising or evaluating a system or system evaluating a system or system component or automated means to component or automated means to verify that it satisfies specified verify that it satisfies specified requirements”.requirements”.

““Software testing is the process of Software testing is the process of executing a program or system with executing a program or system with intent of finding errors.”intent of finding errors.”

““It involves any activity aimed at It involves any activity aimed at evaluating an attribute or capability of evaluating an attribute or capability of a program or system and determining a program or system and determining that it meets its required results.” that it meets its required results.”

““Testing Is Basically A Task Of Testing Is Basically A Task Of Locating Errors”Locating Errors”

It may be:It may be: Positive Testing: Positive Testing: Operate Operate

application it should be operated.application it should be operated. Negative Testing: Negative Testing: Testing for Testing for

abnormal operations.abnormal operations. Positive View of Negative Positive View of Negative

Testing: Testing: The job of testing is to The job of testing is to discover errors before the user does.discover errors before the user does.

SOFTWARE VERIFICATIONSOFTWARE VERIFICATION

““It is the process of evaluating a system or It is the process of evaluating a system or component to determine whether the products component to determine whether the products of a given development phase satisfy the of a given development phase satisfy the conditions imposed at the start of that phase.”conditions imposed at the start of that phase.”

““It is the process of evaluating, reviewing, It is the process of evaluating, reviewing, inspecting and doing desk checks of work inspecting and doing desk checks of work products such as requirement specifications, products such as requirement specifications, design specifications and code.”design specifications and code.”

““It is human testing activity as it involves It is human testing activity as it involves looking at the documents on paper.” looking at the documents on paper.”

SOFTWARE VALIDATIONSOFTWARE VALIDATION

““It is the process of evaluating a It is the process of evaluating a system or component during or at system or component during or at the end of development process to the end of development process to determine whether it satisfies the determine whether it satisfies the specified requirements. It involves specified requirements. It involves executing the actual software. It is executing the actual software. It is a computer based testing process.” a computer based testing process.”

LEAP YEAR FUNCTIONLEAP YEAR FUNCTION CREATE FUCTION f_is_leap_year (@ ai_year small CREATE FUCTION f_is_leap_year (@ ai_year small

int)int)

RETURNS small int RETURNS small int

ASAS

BEGINBEGIN

-- if year is illegal(null or –ve), return -1-- if year is illegal(null or –ve), return -1

IF (@ ai_year IS NULL) orIF (@ ai_year IS NULL) or

(@ ai_year <= 0) RETURN -1(@ ai_year <= 0) RETURN -1

IF (((@ ai_year % 4) = 0) ANDIF (((@ ai_year % 4) = 0) AND

((@ ai_year %100)< > 0)) OR((@ ai_year %100)< > 0)) OR

((@ ai_year %400) = 0)((@ ai_year %400) = 0)

RETURN 1 –leap yearRETURN 1 –leap year

RETURN 0 – Not a leap yearRETURN 0 – Not a leap year

ENDEND

DATABASE TABLE: TEST_LEAP_YEARDATABASE TABLE: TEST_LEAP_YEAR

Serial No.Serial No. Year(year to Year(year to test)test)

Expected Expected resultresult

Observed Observed resultresult

MatchMatch

11 -1-1 -1-1 -1-1 YesYes

22 -400-400 -1-1 -1-1 YesYes

33 100100 00 00 YesYes

44 10001000 00 00 YesYes

55 18001800 00 00 YesYes

66 19001900 00 00 YesYes

77 20102010 00 00 YesYes

88 400400 11 11 YesYes

99 16001600 11 11 YesYes

1010 20002000 11 11 YesYes

1111 24002400 11 11 YesYes

1212 44 11 11 YesYes

1313 12041204 11 11 YesYes

1414 19961996 11 11 YesYes

1515 20042004 11 11 YesYes

WHY SHOULD WE TEST? WHAT WHY SHOULD WE TEST? WHAT IS THE PURPOSE?IS THE PURPOSE?

The Technical CaseThe Technical Case The Business CaseThe Business Case The Professional CaseThe Professional Case The Economics CaseThe Economics Case To Improve QualityTo Improve Quality For Verification and Validation (V&V)For Verification and Validation (V&V) For Reliability EstimationFor Reliability Estimation

WHO SHOULD DO TESTING?WHO SHOULD DO TESTING?

Testing starts right from the very Testing starts right from the very beginning.beginning.

This implies that testing is everyone’s This implies that testing is everyone’s responsibility.responsibility.

It is a It is a Team EffortTeam Effort.. Even Developers are responsible.Even Developers are responsible. They build the code but do not indicate They build the code but do not indicate

any errors as they have written their own any errors as they have written their own code.code.

WHAT SHOULD WE TEST?WHAT SHOULD WE TEST?

Consider that there is a while loop that has Consider that there is a while loop that has three paths. If this loop is executed twice, three paths. If this loop is executed twice, we have (3*3) paths and so on. So, the total we have (3*3) paths and so on. So, the total number of paths through such a code will number of paths through such a code will be:be:

= 1+3+(3*3)+(3*3*3)+….= 1+3+(3*3)+(3*3*3)+….= 1+∑3= 1+∑3nn

This means an infinite number of test This means an infinite number of test cases. Thus, testing is not 100% cases. Thus, testing is not 100% exhaustive.exhaustive.

SELECTION OF GOOD TEST SELECTION OF GOOD TEST CASESCASES

According to Brian Marick, ”A test idea is a According to Brian Marick, ”A test idea is a brief statement of something that should be brief statement of something that should be tested.”tested.”

Cem Kaner said-”The best cases are the one Cem Kaner said-”The best cases are the one that find bugs.”that find bugs.”

A test case is a question that you ask of the A test case is a question that you ask of the program. The point of running the test is to program. The point of running the test is to gain information like whether the program gain information like whether the program will pass or fail the test. will pass or fail the test.

MEASUREMENT OF TESTINGMEASUREMENT OF TESTING

A good project manager(PM) wants that worse A good project manager(PM) wants that worse conditions should occur in the very beginning of conditions should occur in the very beginning of the project only than in the later phases.the project only than in the later phases.

There is no standard to measure our testing There is no standard to measure our testing process.process.

But metrics can be computed at the organizational, But metrics can be computed at the organizational, process project and product levels.process project and product levels.

Metrics is assisted by four core components- Metrics is assisted by four core components- schedule, quality, resources and size. schedule, quality, resources and size.

INCREMENTAL TESTING APPROACH INCREMENTAL TESTING APPROACH Stage 1:Stage 1: Exploration. Exploration. Purpose:Purpose: To gain familiarity with the application. To gain familiarity with the application. Stage 2:Stage 2: Baseline test. Baseline test. Purpose:Purpose: To devise and execute a simple test case. To devise and execute a simple test case. Stage 3:Stage 3: Trends analysis. Trends analysis. Purpose: Purpose: To evaluate whether the application performs as expected when To evaluate whether the application performs as expected when

actual output can be predetermined.actual output can be predetermined. Stage 4: Stage 4: Inventory.Inventory. Purpose:Purpose: To identify the different categories of data and create a test for To identify the different categories of data and create a test for

each category item.each category item. Stage 5:Stage 5: Inventory combinations. Inventory combinations. Purpose:Purpose: To combine different input data. To combine different input data. Stage 6: Stage 6: Push the boundaries.Push the boundaries. Purpose: Purpose: To evaluate application behavior at data boundaries.To evaluate application behavior at data boundaries. Stage 7:Stage 7: Devious data. Devious data. Purpose: Purpose: To evaluate system response when specifying bad data.To evaluate system response when specifying bad data. Stage 8: Stage 8: Stress the environmentStress the environment.. Purpose:Purpose: To attempt to break the system. To attempt to break the system.

BASIC TERMINOLOGY RELATED TO SOFTWARE BASIC TERMINOLOGY RELATED TO SOFTWARE TESTINGTESTING

Error(or Mistake or Bugs): Error(or Mistake or Bugs): When people make mistakes while coding, we When people make mistakes while coding, we call these mistakes bugs.call these mistakes bugs.

Fault (or Defect): Fault (or Defect): A missing or incorrect statement(s) in a program resulting A missing or incorrect statement(s) in a program resulting from an error is a fault.from an error is a fault.

Failure: Failure: A failure occurs when a fault executes.A failure occurs when a fault executes. Incident: Incident: An incident is a symptom associated with a failure that alerts the An incident is a symptom associated with a failure that alerts the

user to the occurrence of a failure.user to the occurrence of a failure. Test: Test: A test is the act of exercising software with test cases.A test is the act of exercising software with test cases. Test Case: Test Case: The essence of software testing is to determine a set of test cases The essence of software testing is to determine a set of test cases

for the item to be tested.for the item to be tested. Test Suite: Test Suite: A collection of test scripts or test cases that is used for validating A collection of test scripts or test cases that is used for validating

bug fixes within a logical or physical area of a product.bug fixes within a logical or physical area of a product. Test Script: Test Script: The step-by-step instructions that describe how a test case is to The step-by-step instructions that describe how a test case is to

be executed.be executed. Test Ware: Test Ware: It includes all testing documentation created during testing It includes all testing documentation created during testing

process.process. Test Oracle: Test Oracle: Any means used to predict the outcome of a test.Any means used to predict the outcome of a test. Test Log:Test Log: A chronological record of all relevant details about the execution of a A chronological record of all relevant details about the execution of a

test.test. Test Report:Test Report: A document describing the conduct and results of testing carried A document describing the conduct and results of testing carried

out for a system. out for a system.

Requirements Specification

Design

Coding

Testing

Fault Resolution

Fault Isolation

Fault Classification

Error

Error

Error

Error

Fault

Fault

Fault

Incident

FixTESTING LIFE CYCLE

WHEN TO STOP TESTING?WHEN TO STOP TESTING?

The The pessimistic approach pessimistic approach to stop to stop testing is whenever some or any of the testing is whenever some or any of the allocated resources- time, budget or test allocated resources- time, budget or test cases are exhausted.cases are exhausted.

TheThe optimistic stopping rule optimistic stopping rule is to stop is to stop testing when either reliability meets the testing when either reliability meets the requirement, or the benefit from requirement, or the benefit from continuing testing cannot justify the continuing testing cannot justify the testing cost. testing cost.

PRINCIPLES OF TESTINGPRINCIPLES OF TESTING Testing should be based on User Testing should be based on User

Requirements.Requirements. Testing Time and Resources are Limited.Testing Time and Resources are Limited. Exhaustive Testing is impossible.Exhaustive Testing is impossible. Use Effective Resources to Test.Use Effective Resources to Test. Test Planning Should be Done Early.Test Planning Should be Done Early. Testing should begin “in small” and Progress Testing should begin “in small” and Progress

Toward Testing “in large”.Toward Testing “in large”. All tests should be traceable to customer All tests should be traceable to customer

requirements.requirements. Prepare test reports including test cases and Prepare test reports including test cases and

test results to summarize the results of test results to summarize the results of testing.testing.

LIMITATIONS OF TESTINGLIMITATIONS OF TESTING

Testing can show presence of errors-not their Testing can show presence of errors-not their absence.absence.

No matter how hard you try, you would never find No matter how hard you try, you would never find the last bug in an application.the last bug in an application.

The domain of possible inputs is too large to test.The domain of possible inputs is too large to test. There are too many possible paths through the There are too many possible paths through the

programs to test.programs to test. In short, maximum coverage through minimum In short, maximum coverage through minimum

test-cases. That is the challenge of testing.test-cases. That is the challenge of testing. Various testing techniques are complementary in Various testing techniques are complementary in

nature and it is only through their combined use nature and it is only through their combined use that one can hope to detect most errors. that one can hope to detect most errors.

AVAILABLE TESTING TOOLS, AVAILABLE TESTING TOOLS, TECHNIQUES AND METRICS TECHNIQUES AND METRICS

Mothora: Mothora: It is an automated mutation testing tool-It is an automated mutation testing tool-set developed at Purdue university. The tester can set developed at Purdue university. The tester can create and execute test cases, measure test case create and execute test cases, measure test case adequacy.adequacy.

NuMega’s Bounds Checker, Rational’s Purify: NuMega’s Bounds Checker, Rational’s Purify: They are run-time checking and debugging aids.They are run-time checking and debugging aids.

Ballista COTS Software Robustness Testing Ballista COTS Software Robustness Testing Harness(Ballista):Harness(Ballista): It is full-scale automated It is full-scale automated robustness testing tool. The goal is to robustness testing tool. The goal is to automatically test and harden commercial off-the-automatically test and harden commercial off-the-shalf (COTS) software against robustness failures.shalf (COTS) software against robustness failures.

SUMMARYSUMMARY Software testing is an art. Most of the testing methods Software testing is an art. Most of the testing methods

and practices are not very different from 20 years and practices are not very different from 20 years ago. Good Testing also requires a tester’s creativity, ago. Good Testing also requires a tester’s creativity, experience and intuition together with proper experience and intuition together with proper techniques.techniques.

Testing is more than just debugging. It is also used in Testing is more than just debugging. It is also used in validation, verification process and reliability validation, verification process and reliability measurement.measurement.

Testing is expensive Automation is a good way to act Testing is expensive Automation is a good way to act down cost and time.down cost and time.

Complete testing is infeasible. Complexity is the root Complete testing is infeasible. Complexity is the root of the problem.of the problem.

Testing may not be the most effective method to Testing may not be the most effective method to improve software quality. improve software quality.

CHAPTER-2CHAPTER-2SOFTWARE VERIFICATION AND SOFTWARE VERIFICATION AND

VALIDATIONVALIDATION

INSIDE THIS CHAPTERINSIDE THIS CHAPTER IntroductionIntroduction Differences Between Verification and ValidationDifferences Between Verification and Validation Differences between QA and QC?Differences between QA and QC? Evolving Nature of AreaEvolving Nature of Area V&V LimitationsV&V Limitations Categorizing V&V TechniquesCategorizing V&V Techniques Role of V&V in SDLC- Tabular FormRole of V&V in SDLC- Tabular Form Proof of Correctness (Formal Verification)Proof of Correctness (Formal Verification) Simulation and PrototypingSimulation and Prototyping Requirements TracingRequirements Tracing Software V&V Planning (SVVP)Software V&V Planning (SVVP) Software Technical Reviews (STRs)Software Technical Reviews (STRs) Independent V&V Contractor (IV & V)Independent V&V Contractor (IV & V) Positive and Negative Effect of Software V&V on ProjectsPositive and Negative Effect of Software V&V on Projects

INTRODUCTIONINTRODUCTION

The evolution of software that satisfies its user The evolution of software that satisfies its user expectations is a necessary goal of a expectations is a necessary goal of a successful software development organization.successful software development organization.

To achieve this goal, software engineering To achieve this goal, software engineering practices must be applied throughout the practices must be applied throughout the evolution of the software product.evolution of the software product.

Most of these practices attempt to create and Most of these practices attempt to create and modify software in a manner that maximizes modify software in a manner that maximizes the probability of satisfying its user the probability of satisfying its user expectations.expectations.

DIFFERENCES BETWEEN DIFFERENCES BETWEEN VERIFICATION AND VERIFICATION AND

VALIDATIONVALIDATION Software V&V is “a systems engineering Software V&V is “a systems engineering

process employing a rigorous process employing a rigorous methodology for evaluating the methodology for evaluating the correctness and quality of software correctness and quality of software product through the software life cycle”.product through the software life cycle”.

VerificationVerification ValidationValidation

It is a static process of verifying It is a static process of verifying documents, design and code. documents, design and code.

It is a dynamic process of It is a dynamic process of validating/testing the actual validating/testing the actual project.project.

It does not involve executing the It does not involve executing the code.code.

It involves executing the code.It involves executing the code.

It is human based checking of It is human based checking of documents/files.documents/files.

It is computer based execution of It is computer based execution of program.program.

Target is requirements Target is requirements specification, application specification, application architecture, high level and detailed architecture, high level and detailed design, database design.design, database design.

Target is actual product- a unit, a Target is actual product- a unit, a module, a set of integral modules, module, a set of integral modules, final product.final product.

It uses methods like inspections, It uses methods like inspections, walk throughs, Desk-checking etc.walk throughs, Desk-checking etc.

It uses methods like black box, gray It uses methods like black box, gray box, white box testing etc.box, white box testing etc.

It, generally, comes first- done It, generally, comes first- done before validation. before validation.

It generally follows verification.It generally follows verification.

It answers to the question- Are we It answers to the question- Are we building the product right? building the product right?

It answers to the question- Are we It answers to the question- Are we building the right product?building the right product?

It can catch errors that validation It can catch errors that validation cannot catch.cannot catch.

It can catch errors that verification It can catch errors that verification cannot catch.cannot catch.

DIFFERENCES BETWEEN DIFFERENCES BETWEEN QA & QC?QA & QC?

Quality Assurance: Quality Assurance: The planned and The planned and systematic activities, implemented in a systematic activities, implemented in a quality system so that the quality quality system so that the quality requirements for a product or service will requirements for a product or service will be fulfilled, is known as quality assurance.be fulfilled, is known as quality assurance.

Quality Control: Quality Control: The observation The observation techniques and activities used to fulfill techniques and activities used to fulfill requirements for quality is known as requirements for quality is known as quality control.quality control.

Quality Assurance Quality Assurance (QA)(QA)

Quality Control (QC)Quality Control (QC)

It is process related. It is process related. It is product related. It is product related.

It focuses on the It focuses on the process use d to process use d to develop a product.develop a product.

It focuses on testing of It focuses on testing of a product developed or a product developed or a product under a product under development. development.

It involves the quality It involves the quality of the processesof the processes

It involves the quality It involves the quality of the products of the products

It is a preventive It is a preventive controlcontrol

It is a detective control. It is a detective control.

Allegiance is to Allegiance is to development development

Allegiance is not to Allegiance is not to development.development.

Evolving Nature of Area Evolving Nature of Area

As the complexity and diversity of software As the complexity and diversity of software products continue to increase, the products continue to increase, the challenge to develop new and more challenge to develop new and more effective V&V strategies continues. The effective V&V strategies continues. The V&V approaches that were reasonably V&V approaches that were reasonably effective on small batch – oriented products effective on small batch – oriented products are not sufficient for concurrent, distributed are not sufficient for concurrent, distributed or embedded products. Thus, this area will or embedded products. Thus, this area will continue to evolve as new research results continue to evolve as new research results emerged in response to new V&V emerged in response to new V&V challenges.challenges.

V&V LimitationsV&V Limitations

Theoretical foundationsTheoretical foundations Impracticality of testing all dataImpracticality of testing all data Impracticality of testing all paths Impracticality of testing all paths No absolute proof of correctness. No absolute proof of correctness.

Categorizing V&V Technique Categorizing V&V Technique

Role of V&V in SDLC Role of V&V in SDLC Traceability analysisTraceability analysis Interface analysisInterface analysis Criticality analysis Criticality analysis

Step 1: Construct a block diagram or control flow Step 1: Construct a block diagram or control flow diagram (CFD) of the system and its elements. Each diagram (CFD) of the system and its elements. Each block will represent one software function (or module) block will represent one software function (or module) only.only.

Step 2: Trace each critical function or quality Step 2: Trace each critical function or quality requirement through CFD.requirement through CFD.

Step 3:Classify all traced software functions as critical toStep 3:Classify all traced software functions as critical to Proper execution of critical software functions.Proper execution of critical software functions. Proper execution of critical quality requirements.Proper execution of critical quality requirements.

Step 4: Focus additional analysis on these traced critical Step 4: Focus additional analysis on these traced critical software functions. software functions.

Step 5: Repeat criticality analysis for each lifecycle Step 5: Repeat criticality analysis for each lifecycle process to determine whether the implementation process to determine whether the implementation details shift the emphasis of the criticality. details shift the emphasis of the criticality.

Hazard and Risk Analysis Hazard and Risk Analysis V&V V&V

ActivityActivityV&V Tasks V&V Tasks Key IssuesKey Issues

RequirementRequirements V&Vs V&V

Traceability analysisTraceability analysis Software requirements Software requirements

evaluation.evaluation. Interface analysisInterface analysis Criticality analysis Criticality analysis System V&V test plan System V&V test plan

generation.generation. Acceptance V&V test Acceptance V&V test

Plan generationPlan generation Configuration Configuration

management management assessmentassessment

Hazard analysis Hazard analysis Risk Analysis Risk Analysis

Evaluates the correctness, Evaluates the correctness, completeness, accuracy, completeness, accuracy, constituency, testability and constituency, testability and readability of software readability of software requirements. requirements.

Evaluates the software Evaluates the software interfacesinterfaces

Identifies the criticality of each Identifies the criticality of each software function.software function.

Initiates the V&V test planning Initiates the V&V test planning for V &V system test. for V &V system test.

Ensures completeness and Ensures completeness and adequacy of S CM process. adequacy of S CM process.

identifies potential hazards, identifies potential hazards, based on the product data based on the product data during the specified during the specified development activity.development activity.

Identifies potential risks, based Identifies potential risks, based on the product data during the on the product data during the specified development activity.specified development activity.

V&V V&V ActivityActivity

V&V Tasks V&V Tasks Key IssuesKey Issues

Design V&VDesign V&V Traceability analysisTraceability analysis Software design Software design

evaluationevaluation Interface analysis.Interface analysis. Criticality analysisCriticality analysis Component V&V test Component V&V test

plan generation and plan generation and verification.verification.

Integration V&V test Integration V&V test plan generation and plan generation and verificationverification

Hazard analysis Hazard analysis

Risk analysis Risk analysis

Evaluates software design Evaluates software design modules for correctness, modules for correctness, completeness, accuracy, completeness, accuracy, consistency, testability and consistency, testability and readability.readability.

Initiates the V&V test planning Initiates the V&V test planning for V&V component test. for V&V component test.

Initiates the V&V test planning Initiates the V&V test planning for V&V integration test. for V&V integration test.

ImplementatiImplementation V&Von V&V

Traceability analysisTraceability analysis Source code and source Source code and source

code documentation code documentation evaluationevaluation

Interface analysisInterface analysis Criticality analysisCriticality analysis V&V test case V&V test case

generation and generation and verification verification

V&V test procedure V&V test procedure generation and generation and verification.verification.

Component V&V test Component V&V test execution and execution and verification.verification.

Hazard analysisHazard analysis Risk analysisRisk analysis

Verifies the correctness, Verifies the correctness, completeness, consistency, completeness, consistency, accuracy, testability and accuracy, testability and readability of source code. readability of source code.

V&V V&V ActivityActivity

V&V Tasks V&V Tasks Key IssuesKey Issues

Test V&VTest V&V Traceability analysis Traceability analysis Acceptance V&V test Acceptance V&V test

procedureprocedure Integration V&V Test Integration V&V Test

execution and execution and verificationverification

System V&V test System V&V test execution and execution and verificationverification

Maintenance Maintenance V&VV&V

SVVP (software SVVP (software verification and verification and validation plan) revisionvalidation plan) revision

Propose3d change Propose3d change assessmentassessment

Anomaly evaluationAnomaly evaluation Criticality analysis Criticality analysis Migration assessmentMigration assessment Retirement assessmentRetirement assessment Hazard analysisHazard analysis Risk analysis Risk analysis

Modifies the SVVP. Modifies the SVVP. Evaluates the effect of software Evaluates the effect of software

of operation anomaliesof operation anomalies Verifies the correctness of Verifies the correctness of

software when migrated to a software when migrated to a different operational different operational environment.environment.

Ensures that the existing Ensures that the existing system continues to function system continues to function correctly when specific correctly when specific software elements are retired. software elements are retired.

Proof of Correctness (Formal Proof of Correctness (Formal Verification)Verification)

A proof of correctness is a mathematical proof that a A proof of correctness is a mathematical proof that a computer program or a part thereof will, when executed, computer program or a part thereof will, when executed, yield correct results i.e., results fulfilling specific yield correct results i.e., results fulfilling specific requirements.requirements.

Hypothesis: The hypothesis of such a correctness theorem is Hypothesis: The hypothesis of such a correctness theorem is typically a condition that the relevant program variables typically a condition that the relevant program variables must satisfy immediately before the program is executed. must satisfy immediately before the program is executed. This condition is called the This condition is called the preconditionprecondition. .

Thesis: The thesis of the correctness theorem is typically a Thesis: The thesis of the correctness theorem is typically a condition that the relevant programme variables must condition that the relevant programme variables must satisfy immediately after execution of the program. This satisfy immediately after execution of the program. This latter condition is called the latter condition is called the post conditionpost condition. .

So the correctness theorem is stated as follows: So the correctness theorem is stated as follows: If the condition, V, is true before execution of the program, If the condition, V, is true before execution of the program,

S, then the condition, P, will be true after execution of S”.S, then the condition, P, will be true after execution of S”. Notation: such a correctness theorem is usually written as Notation: such a correctness theorem is usually written as

{V}S{P}, where V, S and P have been explained above. {V}S{P}, where V, S and P have been explained above.

Simulation and Prototyping Simulation and Prototyping

Simulation and prototyping are techniques for Simulation and prototyping are techniques for analyzing the expected behavior of a product. There analyzing the expected behavior of a product. There are many approaches to constructing simulations and are many approaches to constructing simulations and prototypes that are well documented in the literature. prototypes that are well documented in the literature.

For V&V purposes, simulations and prototypes are For V&V purposes, simulations and prototypes are normally used to analyze requirements and normally used to analyze requirements and specifications to insure that they reflect the user’s specifications to insure that they reflect the user’s needs. Since they are executable, they offer additional needs. Since they are executable, they offer additional insight into the completeness and correctness of insight into the completeness and correctness of these documents. these documents.

Simulations and prototypes can also be used to Simulations and prototypes can also be used to analyze predicted product performance, especially for analyze predicted product performance, especially for candidate product designs, to insure that they candidate product designs, to insure that they conform to requirements. conform to requirements.

Requirements Tracing Requirements Tracing

It is a technique for insuring that the product as It is a technique for insuring that the product as well as the testing of the product, addresses each well as the testing of the product, addresses each of its requirements. of its requirements. One type of matrix maps requirements to One type of matrix maps requirements to

software modules.software modules. Another type of matrix maps requirements to Another type of matrix maps requirements to

test cases. Construction and analysis of this test cases. Construction and analysis of this matrix can help insure that all requirements are matrix can help insure that all requirements are properly tested. properly tested.

A third type of matrix maps requirements to A third type of matrix maps requirements to their evaluation approach. The evaluation their evaluation approach. The evaluation approaches may consist of various levels of approaches may consist of various levels of testing, reviews, simulations etc. testing, reviews, simulations etc.

Software V&V PlanningSoftware V&V Planning Step 1: Identification of V&V GoalsStep 1: Identification of V&V Goals Step 2: Selection of V&V techniques Step 2: Selection of V&V techniques

During requirements phase: The applicable techniques for During requirements phase: The applicable techniques for accomplishing the V&V objectives for requirements are – accomplishing the V&V objectives for requirements are – technical reviews, prototyping and simulations. The review technical reviews, prototyping and simulations. The review process is often called as a System Requirement Review (SRR).process is often called as a System Requirement Review (SRR).

During Specifications Phase: the applicable techniques for this During Specifications Phase: the applicable techniques for this phase are technical reviews, requirements tracing, prototyping phase are technical reviews, requirements tracing, prototyping and simulations. The requirements must be traced to the and simulations. The requirements must be traced to the specifications.specifications.

During Design Phase: The techniques for accomplishing the During Design Phase: The techniques for accomplishing the V&V objectives for designs are technical reviews, requirements V&V objectives for designs are technical reviews, requirements tracing, prototyping, simulation and proof of correctness. We tracing, prototyping, simulation and proof of correctness. We can go for 2 types of design reviews: can go for 2 types of design reviews:

High level designs High level designs Detailed designsDetailed designs

During implementation phaseDuring implementation phase During maintenance phase During maintenance phase

Step 3: Organizational responsibilitiesStep 3: Organizational responsibilities Developmental OrganizationDevelopmental Organization Independent Test Organization (ITO)Independent Test Organization (ITO) Software quality assurance (SQA) Software quality assurance (SQA)

organizationsorganizations Independent V&V ContractorIndependent V&V Contractor

Step 4: integrating V&V ApproachesStep 4: integrating V&V Approaches Step 5: Problem TrackingStep 5: Problem Tracking Step 6: Tracking Test Activities Step 6: Tracking Test Activities Step 7: Assessment Step 7: Assessment

Software Technical Software Technical Reviews Reviews

Error prone software development and Error prone software development and maintenance process maintenance process

Inability to test all softwareInability to test all software Reviews are a form of testingReviews are a form of testing Reviews are away of tracking a projectReviews are away of tracking a project Reviews provide feedback Reviews provide feedback Educational aspects of reviews Educational aspects of reviews

Types of STRsTypes of STRs

Informal Reviews Informal Reviews Formal Reviews Formal Reviews

It is a type of review that typically It is a type of review that typically occurs spontaneously among peers occurs spontaneously among peers

It is a planned meeting It is a planned meeting

Reviewers have no responsibilityReviewers have no responsibility Reviewers are held accountable for Reviewers are held accountable for their participation in the review. their participation in the review.

No review reports are generated No review reports are generated Review reports containing action Review reports containing action items is generated and acted upon. items is generated and acted upon.

Review Methodologies Review Methodologies

There are three approaches to There are three approaches to reviewsreviews Walk ThroughWalk Through Inspection (or work product reviews), Inspection (or work product reviews),

andand Audits Audits

Independent V&V Contractor Independent V&V Contractor

Technical Independence: Technical Independence: It requires that It requires that members of the IV & V team (organization or members of the IV & V team (organization or group) may not be personnel involved in the group) may not be personnel involved in the development of the software. development of the software.

Managerial Independence: Managerial Independence: It means that the It means that the responsibility for IV & V belongs to an organization responsibility for IV & V belongs to an organization outside the contractor and program organizations outside the contractor and program organizations that develop the software. that develop the software.

Financial Independence: Financial Independence: It means that control of It means that control of the IV & V budget is retained in an organization the IV & V budget is retained in an organization outside the contractor and program organization outside the contractor and program organization that develop the software. that develop the software.

Positive Effects of Software V&V on Positive Effects of Software V&V on Projects Projects

Better quality of software. This includes factors like Better quality of software. This includes factors like completeness, consistency, readability and completeness, consistency, readability and testability of the software. testability of the software.

More stable requirementsMore stable requirements More rigorous development planning, at least to More rigorous development planning, at least to

interface with the software V&V organization.interface with the software V&V organization. Better adherence by the development organization Better adherence by the development organization

to programming language and development to programming language and development standards and configuration management practices. standards and configuration management practices.

Early error detection and reduced false starts. Early error detection and reduced false starts. Better schedule compliance and progress Better schedule compliance and progress

monitoring.monitoring. Greater project management visibility into interim Greater project management visibility into interim

technical quality and progress. technical quality and progress. Better criteria and results for decision making at Better criteria and results for decision making at

formal reviews and audits. formal reviews and audits.

Negative Effects of Software V&VNegative Effects of Software V&Von Projectson Projects

Additional project cost of software V&V (10-30% extra) Additional project cost of software V&V (10-30% extra) Additional interface involving the development team, user Additional interface involving the development team, user

and software V&V organization.and software V&V organization. Additional documentation requirements, beyond the Additional documentation requirements, beyond the

deliverable products, if software V&V is receiving deliverable products, if software V&V is receiving incremental program and documentation releases. incremental program and documentation releases.

Need to share computing facilities with and to provide Need to share computing facilities with and to provide access to, classified data for the software V&V organization. access to, classified data for the software V&V organization.

Lower development staff productivity if programmers and Lower development staff productivity if programmers and engineers spend time explaining spend time explaining the engineers spend time explaining spend time explaining the system to software V&V analysis, especially if explanations system to software V&V analysis, especially if explanations are not documented. are not documented.

Increased paper work to provide written responses to Increased paper work to provide written responses to software V&V error reports and other V&V data software V&V error reports and other V&V data requirements. requirements.

Standard for Software Test Standard for Software Test Documentation (IEEE 829)Documentation (IEEE 829)

The IEEE829 standard for software test The IEEE829 standard for software test documentation describes a set of basic documentation describes a set of basic software test documents. It defines the software test documents. It defines the content and form of each test document. content and form of each test document.

In this addendum we give a summary of the In this addendum we give a summary of the structure of the most important IEEE829 structure of the most important IEEE829 defined test documents. defined test documents.

This addendum is based on the course This addendum is based on the course materials by Jukka Paakki (and the IEEE829 materials by Jukka Paakki (and the IEEE829 standard) standard)

TEST PLAN TEST PLAN Test plan identifierTest plan identifier IntroductionIntroduction Test itemsTest items Features to be testedFeatures to be tested Features not to be testedFeatures not to be tested Approach Approach Item pass / fail criteriaItem pass / fail criteria Suspension criteria and resumptionSuspension criteria and resumption Test deliverablesTest deliverables Testing tasksTesting tasks Environmental needsEnvironmental needs ResponsibilitiesResponsibilities Staffing and training needsStaffing and training needs ScheduleSchedule Risks and contingencies Risks and contingencies Approvals Approvals

Test Case Specification Test Case Specification

Test Case Specification IdentifierTest Case Specification Identifier Test ItemsTest Items Input SpecificationsInput Specifications Output SpecificationsOutput Specifications Environmental NeedsEnvironmental Needs Special Procedural requirements Special Procedural requirements Intercase Dependencies Intercase Dependencies

Test Incident Report (Bug Test Incident Report (Bug Report)Report)

Bug report identifierBug report identifier SummarySummary Bug description Bug description

Inputs Inputs Expected results Expected results Actual resultsActual results Data and timeData and time Test procedure step Test procedure step EnvironmentEnvironment Repeatability Repeatability TestersTesters Other observers Other observers Additional information Additional information

Impact Impact

Test Summary Report Test Summary Report

Test Summary Report Identifier Test Summary Report Identifier SummarySummary VariancesVariances Comprehensiveness AssessmentComprehensiveness Assessment Summary of Result Summary of Result EvaluationEvaluation Summary of Activities Summary of Activities Approvals Approvals

CHAPTER-3CHAPTER-3BLACK BOX BLACK BOX

TESTING TECHNIQUESTESTING TECHNIQUES

INSIDE THIS CHAPTER INSIDE THIS CHAPTER

Introduction to Black Box (or functional Introduction to Black Box (or functional testing)testing)

Boundary Value analysis (BVA)Boundary Value analysis (BVA) Equivalence class testingEquivalence class testing Decision table based testingDecision table based testing Cause effect graphing techniqueCause effect graphing technique Comparison on black box (for functional) Comparison on black box (for functional)

testing techniquestesting techniques Kiviat Charts Kiviat Charts

INTRODUCTION INTRODUCTION

The term black box refers to the software The term black box refers to the software which is treated as a black box. which is treated as a black box.

The system or source code is not The system or source code is not checked at all. It is done from customer’s checked at all. It is done from customer’s viewpoint.viewpoint.

The test engineer engaged in black box The test engineer engaged in black box testing only knows the set of inputs and testing only knows the set of inputs and expected outputs and is unaware of how expected outputs and is unaware of how those inputs are transformed into outputs those inputs are transformed into outputs by the software. by the software.

Boundary Value Analysis (BVA)Boundary Value Analysis (BVA) It is a black box testing technique that believes and extends It is a black box testing technique that believes and extends

the concept that the density defect is more towards the the concept that the density defect is more towards the boundaries. This is done to the following reasons: boundaries. This is done to the following reasons:

Programmers usually are not bale to decide whether they Programmers usually are not bale to decide whether they have to use <= operator or < operator when trying to make have to use <= operator or < operator when trying to make comparisons.comparisons.

Different terminating conditions of for-loops, while loops and Different terminating conditions of for-loops, while loops and repeat loops may cause defects to move around the repeat loops may cause defects to move around the boundary conditions. boundary conditions.

The requirements themselves may not be clearly The requirements themselves may not be clearly understood, especially around the boundaries thus causing understood, especially around the boundaries thus causing even the correctly coded program to not perform the even the correctly coded program to not perform the correct way. correct way.

What is BVA?What is BVA?

The basic idea of BVA is to use input variable values at The basic idea of BVA is to use input variable values at

their minimum, just above the minimum, a nominal their minimum, just above the minimum, a nominal

value, just below their maximum and at their value, just below their maximum and at their

maximum. maximum. BVA is based upon a critical assumption that is known BVA is based upon a critical assumption that is known

as single fault assumption theory. as single fault assumption theory. We derive the test cases on the basis of the fact that We derive the test cases on the basis of the fact that

failures are not due to simultaneous occurrence of two failures are not due to simultaneous occurrence of two

faults. So, we derive test case by holding the values of faults. So, we derive test case by holding the values of

all but one variable at their nominal values and letting all but one variable at their nominal values and letting

that variable assume its extreme values. that variable assume its extreme values.

Limitations of BVA Limitations of BVA

Boolean and logical variables present a Boolean and logical variables present a problem for Boundary value analysis. problem for Boundary value analysis.

BVA assumes the variables to be truly BVA assumes the variables to be truly independent which is not always possible. independent which is not always possible.

BVA test cases have been found to be BVA test cases have been found to be rudimentary because they are obtained rudimentary because they are obtained with very little insight and imagination. with very little insight and imagination.

Robustness Testing Robustness Testing

Another variant to BVA is robustness testing. In BVA, Another variant to BVA is robustness testing. In BVA,

we are within the legitimate boundary of our range. we are within the legitimate boundary of our range.

That is, we consider the following values for testing: That is, we consider the following values for testing:

{min, min+, nom, max-, max, max+}{min, min+, nom, max-, max, max+} Again, with robustness testing, we can focus on Again, with robustness testing, we can focus on

exception handling. With strongly typed languages, exception handling. With strongly typed languages,

robustness testing may be very awkward. For example robustness testing may be very awkward. For example

in, pascal, if a variable is defined to be within a certain in, pascal, if a variable is defined to be within a certain

range, values outside that range result in run time range, values outside that range result in run time

errors that abort normal execution. errors that abort normal execution.

WORST CASE TESTING WORST CASE TESTING

We reject our basic assumption of single fault We reject our basic assumption of single fault assumption theory and focus on what assumption theory and focus on what happens when we reject this theory – it happens when we reject this theory – it simply means that we want to see that what simply means that we want to see that what happens when more than one variable has an happens when more than one variable has an extreme value. extreme value.

This is a multiple path assumption theory. In This is a multiple path assumption theory. In electronic circuit analysis, this is called as electronic circuit analysis, this is called as worst case analysis. worst case analysis.

Test case for the triangle Test case for the triangle problem problem

Before we generate the test case, firstly we need Before we generate the test case, firstly we need to give the problem domain: to give the problem domain:

Problem Domain: the triangle program accepts Problem Domain: the triangle program accepts three integers, a, b and c as input. These are three integers, a, b and c as input. These are taken to be the sides of a triangle. The integers taken to be the sides of a triangle. The integers a, b and c must satisfy the following conditions: a, b and c must satisfy the following conditions:

CC11 : 1 : 1 << a a << 200 200 CC44 : a < b + c : a < b + c CC22 : 1 : 1 << b b << 200 200 CC55 : b < a + c : b < a + c CC33 : 1 : 1 << c c << 200 200 CC66 : c < a + b : c < a + b

The output of the program may be: Equilateral, The output of the program may be: Equilateral, Isosceles, Scalene or “not a triangle”Isosceles, Scalene or “not a triangle”

How to generate BVA Test How to generate BVA Test Cases? Cases?

We know that our range is [1,200] where 1 is We know that our range is [1,200] where 1 is the lower bound and 200 being the upper the lower bound and 200 being the upper bound also, we find that this program has bound also, we find that this program has three inputs – a, b and c. So, for our casethree inputs – a, b and c. So, for our case

BVA yields (4n+1) test cases, so we can say BVA yields (4n+1) test cases, so we can say that the total number of test cases will be that the total number of test cases will be (4=3+1) = 12 + 1 = 13.(4=3+1) = 12 + 1 = 13.

We draw the table now which shows those 13 We draw the table now which shows those 13 test-cases.test-cases.

BVA test cases for triangle BVA test cases for triangle problemproblem

Case IdCase Id aa bb cc Expected Expected output output

11 100100 100100 11 Isosceles Isosceles

22 100100 100100 22 IsoscelesIsosceles

33 100100 100100 100100 EquilateralEquilateral


55 100100 100100 200200 Not a triangleNot a triangle











Guidelines for BVAGuidelines for BVA

the normal versus robust values and the the normal versus robust values and the single fault versus the multiple-fault single fault versus the multiple-fault assumption theory result in better testing. assumption theory result in better testing. These methods can be applied to both input These methods can be applied to both input and output domain of any program.and output domain of any program.

Robustness testing is a good choice for Robustness testing is a good choice for testing internal variables.testing internal variables.

Keep in mind that you can create extreme Keep in mind that you can create extreme boundary results from non-extreme input boundary results from non-extreme input values.values.

EQUIVALENCE CLASS EQUIVALENCE CLASS TESTINGTESTING

The use of equivalence classes as the basis The use of equivalence classes as the basis for functional testing has two motivations:for functional testing has two motivations:

We want exhaustive testing and We want exhaustive testing and We want to avoid redundancy.We want to avoid redundancy. This is not handled by BVA technique as we This is not handled by BVA technique as we

can see massive redundancy in the tables can see massive redundancy in the tables of test cases.of test cases.

The idea of equivalence class testing is to The idea of equivalence class testing is to identify test cases by using one element identify test cases by using one element from each equivalence class.from each equivalence class.

Weak Normal Equivalence Weak Normal Equivalence Class TestingClass Testing

The word ‘weak’ means ‘single The word ‘weak’ means ‘single fault assumption’. This type of fault assumption’. This type of testing is accomplished by using testing is accomplished by using one variable from each one variable from each equivalence class in a test case.equivalence class in a test case.

Strong Normal Equivalence Class Strong Normal Equivalence Class TestingTesting

This type of testing is based on the multiple fault This type of testing is based on the multiple fault

assumption theory. So, now we need test cases from assumption theory. So, now we need test cases from

each element of the cartesian product of the equivalence each element of the cartesian product of the equivalence

classesclasses Just like we have truth tables in digital logic, we have Just like we have truth tables in digital logic, we have

similarities between these truth tables and our pattern of similarities between these truth tables and our pattern of

test cases. The cartesian product guarantees that we test cases. The cartesian product guarantees that we

have a notion of “completeness” in two ways:have a notion of “completeness” in two ways: We cover all equivalence classes.We cover all equivalence classes. We have one of each possible combination of inputs.We have one of each possible combination of inputs.

Weak Robust Equivalence Class Weak Robust Equivalence Class TestingTesting

The word The word ‘weak’ means single fault assumption ‘weak’ means single fault assumption theory and the word ‘robust’ refers to invalid values. theory and the word ‘robust’ refers to invalid values.

Two problems occur with robust equivalence testing. Two problems occur with robust equivalence testing. They are listed below:They are listed below:

Very often the specification does not define what the Very often the specification does not define what the expected output for an invalid test case should be. expected output for an invalid test case should be. Thus, testers spend a lot of time defining expected Thus, testers spend a lot of time defining expected outputs for these cases.outputs for these cases.

Also, strongly typed languages like Pascal, Ada, Also, strongly typed languages like Pascal, Ada, Eliminate the need for the consideration of invalid Eliminate the need for the consideration of invalid inputs. Traditional equivalence testing is a product inputs. Traditional equivalence testing is a product of the time when languages such as FORTRAN, C of the time when languages such as FORTRAN, C and COBOL were dominant. Thus this type of error and COBOL were dominant. Thus this type of error was common.was common.

Strong Robust Equivalence Strong Robust Equivalence Class TestingClass Testing

Robust means consideration of Robust means consideration of invalid values and the ‘strong’ means invalid values and the ‘strong’ means multiple fault assumption.multiple fault assumption.

Guidelines for Equivalence Class Guidelines for Equivalence Class testingtesting

The following guidelines have been found out after studying The following guidelines have been found out after studying

above examples :above examples : The weak forms of equivalences class testing (normal or robust) The weak forms of equivalences class testing (normal or robust)

are not as comprehensive as the corresponding strong forms.are not as comprehensive as the corresponding strong forms. If the implementation language is strongly typed and invalid If the implementation language is strongly typed and invalid

values cause run-time error then it makes no sense to use the values cause run-time error then it makes no sense to use the

robust form.robust form. If error conditions are a high priority, the robust forms are If error conditions are a high priority, the robust forms are

appropriate.appropriate. Equivalence class testing is strengthened by a hybrid approach Equivalence class testing is strengthened by a hybrid approach

with boundary value testing (BVA).with boundary value testing (BVA). Several tries may be needed before the “right” equivalence Several tries may be needed before the “right” equivalence

relation is established.relation is established. The different between the strong and weak forms of The different between the strong and weak forms of

equivalence class testing is helpful in the distinction between equivalence class testing is helpful in the distinction between

progression and regression testing. progression and regression testing.

DECISION TABLE BASED DECISION TABLE BASED TESTINGTESTING

Of all the functional testing method, Of all the functional testing method, those based on decision tables are those based on decision tables are the most rigorous because decision the most rigorous because decision tables enforce logical rigour.tables enforce logical rigour.

What are Decision Tables?What are Decision Tables?

Decision tables are a precise and compact way to Decision tables are a precise and compact way to model complicate logic. They are ideal for describing model complicate logic. They are ideal for describing situations in which a number of combinations of situations in which a number of combinations of actions are taken under varying sets of conditions.actions are taken under varying sets of conditions.

It is another popular black box testing technique. A It is another popular black box testing technique. A decision table has four portions:decision table has four portions:

Stub portionStub portion Entry portionEntry portion Condition portion, and Condition portion, and Action portionAction portion

Advantages of Decision Advantages of Decision Tables Tables

This type of testing also works interactively. This type of testing also works interactively. The table that is drawn in the first iteration, The table that is drawn in the first iteration, acts as a stepping stone to derive new decision acts as a stepping stone to derive new decision table, if the initial table is unsatisfactory.table, if the initial table is unsatisfactory.

These tables guarantee that we consider every These tables guarantee that we consider every possible combination of condition values. This possible combination of condition values. This is known as its “completeness property”. This is known as its “completeness property”. This property promises a form of complete testing property promises a form of complete testing as compared to other techniques.as compared to other techniques.

Decision tables are declarative. There is no Decision tables are declarative. There is no particular order for conditions and actions particular order for conditions and actions occur. occur.

Disadvantages of Decision Disadvantages of Decision TablesTables

Decision tables do not scale up well. Decision tables do not scale up well. We need to “factor” large tables into We need to “factor” large tables into smaller remove redundancy.smaller remove redundancy.

Applications of Decision Applications of Decision TablesTables

This technique is useful for applications This technique is useful for applications characterized by any of the following:characterized by any of the following:

Prominent if then else logicProminent if then else logic Logical relationship among input variables.Logical relationship among input variables. Calculations involving subsets of the input Calculations involving subsets of the input

variables.variables. Cause and effect subsets of the inputs and Cause and effect subsets of the inputs and

outputs.outputs. High cyclomatic complexity.High cyclomatic complexity.

Test Cases for the Commission Test Cases for the Commission ProblemProblem

The commission problem is not suitable for The commission problem is not suitable for it to be solved using this technique of it to be solved using this technique of decision analysis. This is because very little decision analysis. This is because very little decisional logic is used in the problem. decisional logic is used in the problem. Because the variable the equivalence Because the variable the equivalence classes are truly independent, no classes are truly independent, no impossible rules will occur in a decision impossible rules will occur in a decision which conditions correspond to the which conditions correspond to the equivalence classes. Thus, we will have the equivalence classes. Thus, we will have the same test cases we did for equivalence same test cases we did for equivalence class testing.class testing.

Guidelines for Decision Table Based Guidelines for Decision Table Based TestingTesting

The following conclusions are derived from these The following conclusions are derived from these examples:examples:

This techniques works well where lot of decision This techniques works well where lot of decision making takes place such as the triangle problem and making takes place such as the triangle problem and next date problem.next date problem.

The decision table technique is indicated for The decision table technique is indicated for applications characterized by any of the following:applications characterized by any of the following:

Prominent if then else logic.Prominent if then else logic. Logical relationships among input variables.Logical relationships among input variables. Calculations involving subsets of the input variables.Calculations involving subsets of the input variables. High cyclomatic complexity.High cyclomatic complexity.

CAUSE-EFFECT GRAPHING CAUSE-EFFECT GRAPHING TECHNIQUETECHNIQUE

Cause effect graphing is basically a Cause effect graphing is basically a hardware testing technique adapted hardware testing technique adapted to software testing it is a black box to software testing it is a black box method. It considers only the desired method. It considers only the desired external behaviour of a system. external behaviour of a system. Testing technique that aids in Testing technique that aids in selecting test cases that logically selecting test cases that logically relate causes (input) to effect relate causes (input) to effect (outputs) to produce test cases.(outputs) to produce test cases.

Causes and effectsCauses and effects

A ‘cause’ represents a distinct A ‘cause’ represents a distinct input condition that brings about input condition that brings about an internal change in the system. an internal change in the system. An ‘effect’ represents an output An ‘effect’ represents an output condition, a system transformation condition, a system transformation or a state resulting from a or a state resulting from a combination of causes. combination of causes.

Guidelines for Cause-Effect Guidelines for Cause-Effect Functional Testing TechniqueFunctional Testing Technique

If the variables refer to physical quantities, domain testing If the variables refer to physical quantities, domain testing and equivalence class test are indicated.and equivalence class test are indicated.

If the variables are independent, domain testing and If the variables are independent, domain testing and equivalence class testing are indicate.equivalence class testing are indicate.

If the variables, decision table testing s indicated.If the variables, decision table testing s indicated. If the single fault assumption is warranted, worst case If the single fault assumption is warranted, worst case

testing, robust worst case testing and decision table testing, robust worst case testing and decision table testing are identical.testing are identical.

If the variables refer to logical quantities, equivalence If the variables refer to logical quantities, equivalence class testing and decision testing are indicated.class testing and decision testing are indicated.

If the program contains significant exception handling, If the program contains significant exception handling, robustness testing and decision table testing are robustness testing and decision table testing are indicated.indicated.

COMPARISON ON BLACK BOXCOMPARISON ON BLACK BOX(OR FUNCTIONAL) TESTING (OR FUNCTIONAL) TESTING

TECHNIQUESTECHNIQUES

Testing EffortTesting Effort

The function method that we have studied so The function method that we have studied so far, vary both in terms of the number of test far, vary both in terms of the number of test cases generated and the effort to develop cases generated and the effort to develop these test cases. To compare the three these test cases. To compare the three techniques, namely, boundary value analysis techniques, namely, boundary value analysis (BVA)(BVA)

Also note that from the graph, that the Also note that from the graph, that the decision table based technique is most decision table based technique is most sophisticated because it requires the tester to sophisticated because it requires the tester to consider both data and logical dependencies.consider both data and logical dependencies.

Testing EfficiencyTesting Efficiency

What we found in all these functional testing What we found in all these functional testing

strategies is that either the functionality is untested strategies is that either the functionality is untested

or the test cases are redundant. So, gaps do occur or the test cases are redundant. So, gaps do occur

in functional test cases and these gaps are reduced in functional test cases and these gaps are reduced

by using more sophisticated techniques.by using more sophisticated techniques. We can develop ratios of total number of test cases We can develop ratios of total number of test cases

generated by method A to those generated by generated by method A to those generated by

method-B or even ratios on a test case basis. This is method-B or even ratios on a test case basis. This is

more difficult but sometimes management demands more difficult but sometimes management demands

numbers even when they have little real meaning. numbers even when they have little real meaning.

Testing EffectivenessTesting Effectiveness How can we find out the effectiveness How can we find out the effectiveness

of the testing techniques? The easy of the testing techniques? The easy choice is choice is

To be dogmatic :We can select a To be dogmatic :We can select a method, use it to generate test cases method, use it to generate test cases and then run the test cases.and then run the test cases.

The second choice can be the The second choice can be the structural testing techniques for the structural testing techniques for the test effectiveness will be discussed in test effectiveness will be discussed in subsequent chapter.subsequent chapter.

What is the Best Solution?What is the Best Solution? The best solution is to work backward The best solution is to work backward

from fault types. Given a particular from fault types. Given a particular kind of fault, we choose testing kind of fault, we choose testing methods that are likely to reveal fault methods that are likely to reveal fault of that type. If we couple this with of that type. If we couple this with knowledge of the most likely kinds of knowledge of the most likely kinds of faults, we end up with a pragamatic faults, we end up with a pragamatic approach to testing effectiveness. This approach to testing effectiveness. This is improved if we track the kinds of is improved if we track the kinds of faults and their freqnecies software we faults and their freqnecies software we develop.develop.

Guidelines for Functional Guidelines for Functional TestingTesting

If the variables refer to physical quantities then domain testing If the variables refer to physical quantities then domain testing and equivalent class testing are used.and equivalent class testing are used.

If the variables are independent then domain testing and If the variables are independent then domain testing and equivalence class testing are used.equivalence class testing are used.

If the variables are dependent, decision table testing is indicated.If the variables are dependent, decision table testing is indicated. If the single-fault assumption is warranted and robustness testing If the single-fault assumption is warranted and robustness testing

are used.are used. If the multiple-fault assumption is warranted then worst case If the multiple-fault assumption is warranted then worst case

testing, robust worst case testing and decision table based testing, robust worst case testing and decision table based testing are indicated.testing are indicated.

If the variable refers to logical quantities, equivalence class If the variable refers to logical quantities, equivalence class testing and decision table testing are indicated.testing and decision table testing are indicated.

KIVIAT CHARTSKIVIAT CHARTS

A Kiviat chart displays a set of metrics A Kiviat chart displays a set of metrics that provides easy viewing of multiple that provides easy viewing of multiple metrics against minimum and maximum metrics against minimum and maximum thresholds. Each radial of Kiviat chart is thresholds. Each radial of Kiviat chart is a metric. All metrics are scaled so that a metric. All metrics are scaled so that all maximums are on a common circle all maximums are on a common circle and all minimums are on a common and all minimums are on a common circle.circle.

SUMMARYSUMMARY We summarize the scenarios under which each of We summarize the scenarios under which each of

these techniques will be useful :these techniques will be useful :When we want to test scenarios When we want to test scenarios that have-that have-

The most effective black The most effective black box testing techniques to box testing techniques to useuse

Outputs values dictated by certain Outputs values dictated by certain conditions depending upon values of conditions depending upon values of input variables.input variables.

Decision tables.Decision tables.

Input values in ranges, with each Input values in ranges, with each range showing a particular range showing a particular functionality.functionality.

Boundary Value Analysis (BVA).Boundary Value Analysis (BVA).

Input values divided into classes.Input values divided into classes. Equivalence partitioning.Equivalence partitioning.

Checking for expected and Checking for expected and unexpected input values.unexpected input values.

Positive and negative testing.Positive and negative testing.

Workflows, process flows or Workflows, process flows or languages processors.languages processors.

Graph based testing.Graph based testing.

To ensure that requirements are To ensure that requirements are tested and met properly.tested and met properly.

Requirements based testing. Requirements based testing.

CHAPTER-4CHAPTER-4WHITE-BOX (OR WHITE-BOX (OR STRUCTURAL) STRUCTURAL)

TESTING TECHNIQUESTESTING TECHNIQUES

Introduction to white box Introduction to white box testing or structural testing or testing or structural testing or clear box or glass box or open clear box or glass box or open box testing box testing

It is a methodology that tests It is a methodology that tests software using the code.software using the code.

Introduction to white box testing or structural testing or clear Introduction to white box testing or structural testing or clear

box or glass box or open box testing.box or glass box or open box testing. Static versus dynamic white box testing.Static versus dynamic white box testing. Dynamic white box testing techniques.Dynamic white box testing techniques. Mutation testing versus error seeding – differences in tabular Mutation testing versus error seeding – differences in tabular

form.form. Comparison of black box and white box testing in tabular form.Comparison of black box and white box testing in tabular form. Practical challenges in white box testing Practical challenges in white box testing Comparison on various white box testing techniques. Comparison on various white box testing techniques. Advantages of white box testing.Advantages of white box testing.

The code works according to the The code works according to the functional requirements. functional requirements.

The code has been written in The code has been written in accordance with the design developed accordance with the design developed earlier in the project life cycle. earlier in the project life cycle.

The code for any functionality has The code for any functionality has been missed out.been missed out.

The code handles errors properly. The code handles errors properly.

In dynamic testing, we test a running In dynamic testing, we test a running program. So, now binaries and program. So, now binaries and executables are desired. We try to executables are desired. We try to test the internal logic of the program test the internal logic of the program now. It entails running the actual now. It entails running the actual product against some pre designed product against some pre designed test cases to exercise as much of the test cases to exercise as much of the code as possible. code as possible.

At the initial stages, the developer or tester can At the initial stages, the developer or tester can perform certain tests based on the input variables and perform certain tests based on the input variables and the corresponding expected output variables. This can the corresponding expected output variables. This can be a quick test. If we repeat these tests for multiple be a quick test. If we repeat these tests for multiple values of input variables also then the confidence level values of input variables also then the confidence level of the developer to go to the next level increases. of the developer to go to the next level increases.

For complex modules, the tester can insert some print For complex modules, the tester can insert some print statements in between to check whether the program statements in between to check whether the program control passes through all statements and loops. It is control passes through all statements and loops. It is important remove the intermediate print statements important remove the intermediate print statements after the defects are fixed. after the defects are fixed.

Another method is to run the product under a Another method is to run the product under a debugger or an integrated development environment debugger or an integrated development environment (IDE). These tools involve single stepping of (IDE). These tools involve single stepping of instructions, setting break points at any function or instructions, setting break points at any function or instruction.instruction.

Code coverage testing involves designing and executing Code coverage testing involves designing and executing

test cases and finding out the percentage of code that is test cases and finding out the percentage of code that is

covered by testing. covered by testing. The percentage of code covered by a test is found by The percentage of code covered by a test is found by

adopting a technique called as the instrumentation of code. adopting a technique called as the instrumentation of code. These tools rebuild the code, do product linking with a set of These tools rebuild the code, do product linking with a set of

libraries provided by the tool, monitor the portions of code libraries provided by the tool, monitor the portions of code

covered, reporting on the portions of the code that are covered, reporting on the portions of the code that are

covered frequently, so that the critical or most often covered frequently, so that the critical or most often

portions of code can be identified. portions of code can be identified.

Statement coverage refers to writing test cases that Statement coverage refers to writing test cases that execute each of the program statements. We assume execute each of the program statements. We assume that “more the code covered, the better is the testing that “more the code covered, the better is the testing of the functionality”of the functionality”

If there are asynchronous exceptions in the code, like If there are asynchronous exceptions in the code, like divide by zero, then even if we start a test case at the divide by zero, then even if we start a test case at the beginning of a section, the test case may not cover all beginning of a section, the test case may not cover all the statements in that section. Thus, even in case of the statements in that section. Thus, even in case of sequential statements, coverage for all statements may sequential statements, coverage for all statements may not be achieved. not be achieved.

A section of code may be entered from multiple points. A section of code may be entered from multiple points.

In path coverage technique, we split a In path coverage technique, we split a program in to a number of distinct paths. A program in to a number of distinct paths. A program or a part of a program can start program or a part of a program can start from the beginning and take nay of the paths from the beginning and take nay of the paths to its completion. The path coverage of a to its completion. The path coverage of a program may be calculated based on the program may be calculated based on the following formula:-following formula:-

Functions (like functions in C) are easier to identify in a Functions (like functions in C) are easier to identify in a

program and hence it is easier to write test cases to provide program and hence it is easier to write test cases to provide

function coverage. function coverage. Since functions are at higher level of abstraction than code, it Since functions are at higher level of abstraction than code, it

is easier to achieve 100% function coverage. is easier to achieve 100% function coverage. It is easier to prioritize the functions for testing. It is easier to prioritize the functions for testing. Function coverage provides a way of testing traceability, that Function coverage provides a way of testing traceability, that

is tracing requirements through design, coding and testing is tracing requirements through design, coding and testing

phases. phases. Function coverage provides a natural transition to black box Function coverage provides a natural transition to black box

testing. testing.

Which of the paths are independent? Which of the paths are independent? If two paths are not independent, If two paths are not independent, then we may be able to minimize the then we may be able to minimize the number of tests. number of tests.

Is there any limit on the number of Is there any limit on the number of tests that must be run to ensure that tests that must be run to ensure that all the statements have been all the statements have been executed at least once? executed at least once?

The answer to the above questions is a metric The answer to the above questions is a metric that quantifies the complexity of a program that quantifies the complexity of a program and is known as cyclomatic complexity. and is known as cyclomatic complexity.

It is also known as structural complexity It is also known as structural complexity because it gives the internal view of the code.because it gives the internal view of the code.

It is a number which provides us with an upper It is a number which provides us with an upper bound for the number of tests that must be bound for the number of tests that must be conducted to ensure that all statements have conducted to ensure that all statements have been executed at least once. been executed at least once.

Different metrics are as follows:-Different metrics are as follows:- McCabe metrics based on cyclomatic complexity, V(G).McCabe metrics based on cyclomatic complexity, V(G). Execution Coverage Metrics based on any of branch, path Execution Coverage Metrics based on any of branch, path

or Boolean Coverage. or Boolean Coverage. OO metrics based on the work of Chidamber and Kemerer. OO metrics based on the work of Chidamber and Kemerer. Derived metrics based on abstract concepts such as Derived metrics based on abstract concepts such as

understandability, maintainability comprehension and understandability, maintainability comprehension and

testability.testability. Custom metrics imported from third party software / Custom metrics imported from third party software /

system e.g. defect count. system e.g. defect count.

There are three categories of metrics There are three categories of metrics McCabe MetricsMcCabe Metrics OO MetricsOO Metrics Grammar Metrics Grammar Metrics

The Hawthorne Effect which states that The Hawthorne Effect which states that when you collect metrics on people, the when you collect metrics on people, the people being measured will change their people being measured will change their behavior. Either or these practices will behavior. Either or these practices will destroy the efficiency of any metrics destroy the efficiency of any metrics program. program.

Cyclomatic Complexity, V(G) : It is the measure of the amount of Cyclomatic Complexity, V(G) : It is the measure of the amount of logic in a code module of 3logic in a code module of 3rdrd and 4 and 4thth generation languages. If generation languages. If V(G) is excessively high thenit leads to impenetrable code i.e. a V(G) is excessively high thenit leads to impenetrable code i.e. a code which is at higher risk due to difficulty in testing. The code which is at higher risk due to difficulty in testing. The threshold value is 10. When V(G) > 10; then the likelihood of threshold value is 10. When V(G) > 10; then the likelihood of code being unreliable is much higher.code being unreliable is much higher.

Please remember that a high V(G) shows a decreased quality in Please remember that a high V(G) shows a decreased quality in the code resulting in higher defects that become costly to fix. the code resulting in higher defects that become costly to fix.

Essential complexity: it is a measure of the degree which a code Essential complexity: it is a measure of the degree which a code module contains unstructured constructs .module contains unstructured constructs .

A high essential complexity indicates increase maintenance A high essential complexity indicates increase maintenance costs which decreased code quality. Some organizations have costs which decreased code quality. Some organizations have used the essential density metric (EDM) and is defined as :-used the essential density metric (EDM) and is defined as :-

Integration complexity: it is a measure of the Integration complexity: it is a measure of the interaction between the modules of code within a interaction between the modules of code within a program. Say S0, S1 are to derivatives of this program. Say S0, S1 are to derivatives of this complexity.complexity.

Where Where S0S0 -- Provides an overall measure of Provides an overall measure of size and complexity of a program’s design. It will not size and complexity of a program’s design. It will not reflects the internal calculations of each module. It is reflects the internal calculations of each module. It is the sum of all integration complexity in a program. the sum of all integration complexity in a program.

S1 S1 == (S0 – Number of Methods + 1)(S0 – Number of Methods + 1) This is primarily used to determine the number of tests This is primarily used to determine the number of tests

for the some test that is designed to ensure that the for the some test that is designed to ensure that the application would execute without issues in module application would execute without issues in module interaction. interaction.

Pathological complexity: it represents an Pathological complexity: it represents an extremely unstructured code which shows extremely unstructured code which shows a poor design and hence a suggestion for a poor design and hence a suggestion for code’s reengineering. A value greater than code’s reengineering. A value greater than one indicates poor coding practices like one indicates poor coding practices like branching into a loop or into a decision. branching into a loop or into a decision.

Branch Coverage: It is a measure of how Branch Coverage: It is a measure of how many branches or decisions in a module many branches or decisions in a module have been executed during testing. If the have been executed during testing. If the branch coverage is <95% for new code or branch coverage is <95% for new code or 75% for code under maintenance then the 75% for code under maintenance then the test scripts require review and test scripts require review and enhancement. enhancement.

Basis path coverage: A measure of how many Basis path coverage: A measure of how many of the basis (Cyclomatic, V (G)) paths in a of the basis (Cyclomatic, V (G)) paths in a module have been executed. Path coverage is module have been executed. Path coverage is the most fundamental of McCabe design. It the most fundamental of McCabe design. It indicates how much logic in the code is indicates how much logic in the code is covered or not covered. This technique covered or not covered. This technique requires more through testing than branch requires more through testing than branch Coverage. Coverage.

Boolean Coverage: A technique used to Boolean Coverage: A technique used to estbalish that each condition within a decision estbalish that each condition within a decision is shown by execution to independently and is shown by execution to independently and correctly affect the outcome of the decision. correctly affect the outcome of the decision. The major application of this technique is in The major application of this technique is in safety critical systems and projects. safety critical systems and projects.

Combining McCabe Metrics : Cyclomatic complexity is the basic Combining McCabe Metrics : Cyclomatic complexity is the basic

indicator for determining the complexity of logic in a unit of indicator for determining the complexity of logic in a unit of

code. It can be combined with other metrics also. For example code. It can be combined with other metrics also. For example Code Review Candidate Code Review Candidate If V(G) > 10 and Essential complexity / Essential Density If V(G) > 10 and Essential complexity / Essential Density

exceeds 4, then the unit needs a review. exceeds 4, then the unit needs a review. Code refactoring Code refactoring

If V(G) > 10 and the condition If V(G) > 10 and the condition V(G) – EV (g) V(G) – EV (g) << V(g) is true V(g) is true

Then, the code is a candidate for refactoring. Then, the code is a candidate for refactoring.

Inadequate comment contentInadequate comment content If the graph between V(G) against If the graph between V(G) against

comment % (in terms of LOC) does not comment % (in terms of LOC) does not show a linear increase then the show a linear increase then the comment content need to be reviewed. comment content need to be reviewed.

Test CoverageTest Coverage If the graph between V(G) against Path If the graph between V(G) against Path

coverage does not show a linear coverage does not show a linear increase then a test scripts need to be increase then a test scripts need to be reviewed. reviewed.

Average V(G) for a classAverage V(G) for a class Average essential complexity for a classAverage essential complexity for a class Number of parentsNumber of parents Response for class (RFC) Response for class (RFC) Weighted methods for class (WMC) Weighted methods for class (WMC) Coupling between objects (CBO) Coupling between objects (CBO) Class hierarchy levelClass hierarchy level Number of methods (n) Number of methods (n) Lack of cohesion between methods (LOCM) Lack of cohesion between methods (LOCM) Combined OO Metrics .Combined OO Metrics .

Line count .Line count . Nesting levels.Nesting levels. Counts of decision types.Counts of decision types. Maximum number of predicates.Maximum number of predicates. Comment lines.Comment lines.

McCabe’s cyclomatic metric, V(G) McCabe’s cyclomatic metric, V(G) of a graph G with n vertices and e of a graph G with n vertices and e edges is given by the formula : edges is given by the formula :

V(G) = e-n+2V(G) = e-n+2

The cyclomatic complexity, V(G) provides us two The cyclomatic complexity, V(G) provides us two things: things:

To find the number of independent paths through the program.To find the number of independent paths through the program. To provide an upper bound for the number of test cases that To provide an upper bound for the number of test cases that

must be executed in order to test the program thoroughly. The must be executed in order to test the program thoroughly. The complexity measure is defined in terms of independent paths. It complexity measure is defined in terms of independent paths. It is defined as any path through the program that introduces at is defined as any path through the program that introduces at least one new set of processing statements or a new condition. least one new set of processing statements or a new condition. The following steps are usually followed: The following steps are usually followed:

Step 1: Construction of flow graph from the source code or flow chartsStep 1: Construction of flow graph from the source code or flow charts Step 2: identification of independent paths.Step 2: identification of independent paths. Step 3: Computation of cyclomatic complexity .Step 3: Computation of cyclomatic complexity . Step 4: Test cases are designed .Step 4: Test cases are designed .

PROPERTIES OF CYCLOMATIC COMPLEXITY- PROPERTIES OF CYCLOMATIC COMPLEXITY- The following are the properties of cyclomatic The following are the properties of cyclomatic

complexity represented as V (G) : complexity represented as V (G) :

1.1. V (G) ≥ 1.V (G) ≥ 1.

2.2. V (G) is the maximum number of independent V (G) is the maximum number of independent

paths in graph, G.paths in graph, G.

3.3. Inserting and deleting functional statements to Inserting and deleting functional statements to

G does not affect, G.G does not affect, G.

4.4. G has only one path if and only if V (G) =1 . If V G has only one path if and only if V (G) =1 . If V

(G) =1 then the graph, G has one path only.(G) =1 then the graph, G has one path only.

5.5. Inserting a new now in G, increases V (G) by Inserting a new now in G, increases V (G) by

unity. unity.

For small programs cyclomatic complexity For small programs cyclomatic complexity can be calculated manually, but can be calculated manually, but automated tools are essential as several automated tools are essential as several thousands of lines of code are possible in thousands of lines of code are possible in each program in a project. It will be very each program in a project. It will be very difficult to manually create flow graphs for difficult to manually create flow graphs for large programs. There are several tools large programs. There are several tools that are available in the market which can that are available in the market which can compute cyclomatic complexity. compute cyclomatic complexity.

Complexity Complexity What it means?What it means?1-101-10 Well-written code, testability is high , Well-written code, testability is high ,

cost/ effort to maintain is low.cost/ effort to maintain is low.

1-201-20 Moderately complex, testability is Moderately complex, testability is medium, cost/effort to maintain is medium, cost/effort to maintain is medium. medium.

20-4020-40 Very complex, testability is low/ Cost/ Very complex, testability is low/ Cost/ effort to maintain is high.effort to maintain is high.

>40>40 Not testable, any amount of money/ Not testable, any amount of money/ effort to maintain may not be enougheffort to maintain may not be enough

Basic path testing helps a tester to compute Basic path testing helps a tester to compute logical complexity measure, V (G), of the code. logical complexity measure, V (G), of the code. This value of V (G), defines the maximum number This value of V (G), defines the maximum number of test cases to be designed by identifying basis of test cases to be designed by identifying basis set of execution paths to ensure that all set of execution paths to ensure that all statements are executed at least once. The statements are executed at least once. The following steps are usually followed:following steps are usually followed:

- Construct the flow graph the source code or - Construct the flow graph the source code or flow charts.flow charts.

- Identify independent paths.- Identify independent paths.

- Calculate cyclomatic complexity, V (G).- Calculate cyclomatic complexity, V (G).

- Design that test cases. - Design that test cases.

We have 2 types of def-use pairs:We have 2 types of def-use pairs:

(a) That correspond to a (a) That correspond to a definition and its- use (dcu). (C’ definition and its- use (dcu). (C’ stands for computational).stands for computational).

(b) That correspond to a (b) That correspond to a definition and its p-use (dpu). (‘p’ definition and its p-use (dpu). (‘p’ stands for predicate conditions).stands for predicate conditions).

As shown in program, there is a p-use of x at As shown in program, there is a p-use of x at node-2 with outgoing edges (2, 3) and (2,5) node-2 with outgoing edges (2, 3) and (2,5) The existence of def-clear paths from node-1 The existence of def-clear paths from node-1 to each of these tow edges is easily seen in to each of these tow edges is easily seen in DD-path graph. There is not other p-use of x. DD-path graph. There is not other p-use of x. So,So,

dpu (x, 1)= { (2,3), (2, 5) }dpu (x, 1)= { (2,3), (2, 5) }

We can now compute for all other variable as We can now compute for all other variable as shown below and find the trouble-spots: shown below and find the trouble-spots:

VariablVariable e

Defined at node Defined at node (n)(n)

Dcu (v n)Dcu (v n) Dpu (v,n)Dpu (v,n)

XX 11 (5,6)(5,6) {(2,3), (2,5)}{(2,3), (2,5)}

yy 11 (4,6)(4,6) {(3,4), (3,6)}{(3,4), (3,6)}

YY 66 (4,6)(4,6) {(3,4), (3,6)}{(3,4), (3,6)}

ZZ 11 (4,6,7)(4,6,7) { }{ }

ZZ 44 (4,6,7)(4,6,7) { }{ }

ZZ 55 (4,6,7)(4,6,7) { }{ }

CountCount 11 (6)(6) {(6,2), (6,7)}{(6,2), (6,7)}

CountCount 66 (6)(6) {(6,2), (6,7)}{(6,2), (6,7)}

Data flow testing use the sequence of variable access to Data flow testing use the sequence of variable access to

select paths from a control graph. The main focus of this select paths from a control graph. The main focus of this

structural or white box testing technique is on the usage of structural or white box testing technique is on the usage of

variables. With a program data definition faults are nearly variables. With a program data definition faults are nearly

as frequent (22% of all faults) as control flow faults (24 as frequent (22% of all faults) as control flow faults (24

percent). Some common data related faults variable.percent). Some common data related faults variable.

- Using an undefined or uninitialized variable- Using an undefined or uninitialized variable

- Unreachable code segments. - Unreachable code segments.

- Deal locating or re-initializing a variable before its - Deal locating or re-initializing a variable before its

last use. last use.

1.1. Consider the problem of finding out the roots of Consider the problem of finding out the roots of a given quadratic equation, axa given quadratic equation, ax22+ bx + c= 0. + bx + c= 0. Write its program in C and perform the followingWrite its program in C and perform the following

(a) Draw flow graph.(a) Draw flow graph.

(b) Draw DD path graph(b) Draw DD path graph

(c) Find V (G) by all the three methods.(c) Find V (G) by all the three methods.

(d) Perform its data flow testing and find all du (d) Perform its data flow testing and find all du and dc paths.and dc paths.

The steps that are usually followed to develop data The steps that are usually followed to develop data flow test cases are as follows:flow test cases are as follows:

1. 1. Layout a control graph for the program.Layout a control graph for the program.

2.2. For each variable, note the defined (d), used (u) For each variable, note the defined (d), used (u)

and killed (k) actions by node.and killed (k) actions by node.

3. 3. Trace all possible du and dk paths for each Trace all possible du and dk paths for each variable.variable.

4. 4. Merge the paths.Merge the paths.

5. 5. Test/Check path for testability.Test/Check path for testability.

6. 6. Select test data for feasible paths.Select test data for feasible paths.

It can show the ambiguities in code.It can show the ambiguities in code. It leads to move reliable product.It leads to move reliable product. A comprehensive testing can be A comprehensive testing can be

done.done.

It is difficult to identify and kill equivalent It is difficult to identify and kill equivalent mutants.mutants.

Stubborn mutants are difficult to kill.Stubborn mutants are difficult to kill. It is timer consuming technique, so It is timer consuming technique, so

automated tools are required.automated tools are required. Each mutation will have the same size as Each mutation will have the same size as

that of the original program. So a large that of the original program. So a large number mutant programs may need to be number mutant programs may need to be tested a against the candidate test suite.tested a against the candidate test suite.

CHAPTER-5CHAPTER-5GRAY BOX TESTINGGRAY BOX TESTING

Introduction to Gray Box testingIntroduction to Gray Box testing What is Gray Box Testing?What is Gray Box Testing? Various other Definitions of Gray Box Various other Definitions of Gray Box

TestingTesting Comparison of White Box, Black Box Comparison of White Box, Black Box

and Gray Box Testing Approaches in and Gray Box Testing Approaches in Tabular Form.Tabular Form.

Code Coverage testing involves “dynamic testing” methods of executing the Code Coverage testing involves “dynamic testing” methods of executing the product with pre-written test cases and finding out how much of code has product with pre-written test cases and finding out how much of code has been covered.been covered.

If better coverage of a code is desired, several iterations of testing may be If better coverage of a code is desired, several iterations of testing may be required.required.

For each covering those portions of code that were not covered by earlier For each covering those portions of code that were not covered by earlier tests cases.tests cases.

To do such type of testing, not only does not one need to understand the code To do such type of testing, not only does not one need to understand the code and logic test cases that can cover good portions of the code.and logic test cases that can cover good portions of the code.

Understanding of code and logic means white box or structural testing Understanding of code and logic means white box or structural testing whereas writing effective test cases mans black box testing. So, we need a whereas writing effective test cases mans black box testing. So, we need a combination of white box and black box techniques for test effectiveness. This combination of white box and black box techniques for test effectiveness. This type of testing is known as “gray box testing.” We must thus understand that:type of testing is known as “gray box testing.” We must thus understand that:

White + Black = Gray White + Black = Gray

Black box testing focuses on software external attributes and Black box testing focuses on software external attributes and behavior. Such testing looks at an application’s expected behaviour behavior. Such testing looks at an application’s expected behaviour from the users point. from the users point.

White box testing/glass-box testing, however, tests software with White box testing/glass-box testing, however, tests software with knowledge of internal data structures, physical logic, flow, and knowledge of internal data structures, physical logic, flow, and architecture at the source code level.architecture at the source code level.

White box testing looks at testing from the developer’s point of view.White box testing looks at testing from the developer’s point of view. Both black-box and white- box testing are critically important Both black-box and white- box testing are critically important

complements of a complete testing effort.complements of a complete testing effort. Individually, they do not allow for balanced testing. Black box testing Individually, they do not allow for balanced testing. Black box testing

can be less effective at uncovering certain error types as data- flow can be less effective at uncovering certain error types as data- flow errors or boundary condition errors at the source level. White box errors or boundary condition errors at the source level. White box testing does not readily highlight macro level quality risks in testing does not readily highlight macro level quality risks in operating environmental comp ability, time-related errors and operating environmental comp ability, time-related errors and usability. usability.

Gray box testing incorporate the elements of both black box Gray box testing incorporate the elements of both black box and white box testing. and white box testing.

It consider the outcome on the user end, system-specific It consider the outcome on the user end, system-specific technical knowledge and the operating environment. It technical knowledge and the operating environment. It evaluates the application design in the context of the inter-evaluates the application design in the context of the inter-operability of system components.operability of system components.

The gray box testing approach is integral to the effective The gray box testing approach is integral to the effective testing of we application comprise numerous components, both testing of we application comprise numerous components, both software and hardware.software and hardware.

These components must be tested in the context of system These components must be tested in the context of system design to evaluate their functionality and compatibility.design to evaluate their functionality and compatibility.

Listed below are some more definitions of gray box testing.Listed below are some more definitions of gray box testing.

Before tabulating the differences Before tabulating the differences between black box, gray box and between black box, gray box and white box testing techniques we must white box testing techniques we must first understand that when say test first understand that when say test granularity, we mean the level of granularity, we mean the level of details. details.

And when we say the highest. It And when we say the highest. It means that tall internals are known.means that tall internals are known.

Black Box Black Box TestingTesting

Gray Box Gray Box TestingTesting

White Box White Box TestingTesting

1. Low granularity1. Low granularity Medium GranularityMedium Granularity High granularity.High granularity.

2. Internals NOT 2. Internals NOT known.known.

Internals partially Internals partially knownknown

Internals fully known.Internals fully known.

Internals not required Internals not required to be known.to be known.

Internals relevant to Internals relevant to testing are knowntesting are known

Internal code of the Internal code of the application and application and database known.database known.

4 Also known as4 Also known asOpaque box testingOpaque box testingClosed box testingClosed box testingInput output testingInput output testingData driven testingData driven testingBehavioralBehavioralFunctional testingFunctional testing

Also known as Also known as translucent box translucent box testing.testing.

Also known asAlso known asGlass box testingGlass box testingClear box testingClear box testingDesign based Design based testingtestingLogic based testingLogic based testingStructural testingStructural testingCode based testingCode based testing

5. It is done by end-5. It is done by end-user (user acceptance user (user acceptance testing). Also done by testing). Also done by tester, developers.tester, developers.

It is done by end-It is done by end-users (user users (user acceptance testing). acceptance testing). Also done by testers, Also done by testers, developers.developers.

Normally done by Normally done by testers and testers and developers.developers.

6. Testing method 6. Testing method where where System is viewed System is viewed as a black boxas a black boxInternal Internal behaviour of the behaviour of the program is ignoredprogram is ignoredTesting is based Testing is based upon external upon external specifications. specifications.

Here, internals Here, internals partly known partly known (gray), but not fully (gray), but not fully known (white). known (white). Test design is Test design is based on the based on the knowledge of knowledge of algorithm, interval algorithm, interval states, architecture states, architecture or other high level or other high level descriptions of the descriptions of the program behaviour.program behaviour.

Internals fully Internals fully known.known.

7. It is likely to be 7. It is likely to be least exhaustive of least exhaustive of the three.the three.

It is some where in It is some where in betweenbetween

Potentially most Potentially most exhausitve of the exhausitve of the three.three.

8. Requirements 8. Requirements based. Test cases based. Test cases based on the based on the functional functional specifications, as specifications, as internals not known.internals not known.

Better variety/depth Better variety/depth in test cases on in test cases on account of high level account of high level knowledge of knowledge of internals.internals.

Ability to exercise Ability to exercise code with relevant code with relevant variety of data.variety of data.

9. Being 9. Being specification based specification based it would not suffer it would not suffer from the deficiency from the deficiency as described for as described for white box testing.white box testing.

It would also not It would also not suffer from the suffer from the deficiency as deficiency as described for white described for white box testing.box testing.

Since test cases are Since test cases are written based on the written based on the code, specifications code, specifications missed out in coding missed out in coding would not be would not be revealed.revealed.

10. It is suited for 10. It is suited for functional/business functional/business domain testing.domain testing.

It is suited for It is suited for functional/business functional/business domain testing bit in domain testing bit in depth.depth.

It is suited for all.It is suited for all.

Not suited to algorithm Not suited to algorithm testing.testing.

Not suited to algorithm Not suited to algorithm testing.testing.

Appropriate for Appropriate for algorithm testing. algorithm testing.

It is concerned with It is concerned with validating outputs for validating outputs for given inputs, the given inputs, the application being application being treated as a black box.treated as a black box.

Here in, we have a Here in, we have a better variety of inputs better variety of inputs and the ability to and the ability to extract test results extract test results from data base for from data base for comparison with comparison with expected results.expected results.

It facilitates structural It facilitates structural testing. It enables logic testing. It enables logic coverage, coverage of coverage, coverage of statements, decisions, statements, decisions, conditions, path and conditions, path and control flow within the control flow within the code.code.

13. It can test 13. It can test only by trial and only by trial and error data error data domains, domains, internal internal boundaries and boundaries and overflow.overflow.

It can test data It can test data domains, domains, internal internal boundaries and boundaries and overflow, if overflow, if knownknown

It can determine It can determine and therefore and therefore test better : test better : data domain,s data domain,s internal internal boundaries and boundaries and overflow.overflow.

1. As tester, we get ideas for test cases from a wide range 1. As tester, we get ideas for test cases from a wide range of knowledge areas. This is partially because testing is of knowledge areas. This is partially because testing is much more effective when we know what types of buds much more effective when we know what types of buds we are looking for as testers of complex systems, we we are looking for as testers of complex systems, we should strive to attain a broad balance in our knowledge should strive to attain a broad balance in our knowledge learning enough about many aspects of the software learning enough about many aspects of the software and systems being tested to create a battery of tests and systems being tested to create a battery of tests that can challenge the software as deeply as it will be that can challenge the software as deeply as it will be challenged in the rough and tumble of day to day use. challenged in the rough and tumble of day to day use.

2. Every tester in a test team need no be a gray box 2. Every tester in a test team need no be a gray box tester, more is the mix of different types of testers in a tester, more is the mix of different types of testers in a team, better is the success.team, better is the success.

CHAPTER-6CHAPTER-6REDUCING THE NUMBERREDUCING THE NUMBER

OF TEST CASESOF TEST CASES

Prioritization GuidelinesPrioritization Guidelines

Priority category schemePriority category scheme

Risk AnalysisRisk Analysis

Regression Testing – overview Regression Testing – overview

Prioritization of test cases for regression Prioritization of test cases for regression TestingTesting

Regression Testing Technique- A case Regression Testing Technique- A case StudyStudy

Slice Based TestingSlice Based Testing

The goal of prioritization of test cases is The goal of prioritization of test cases is to reduce the set of test cases based on to reduce the set of test cases based on some rational, non-arbitrary, criteria, some rational, non-arbitrary, criteria, while aiming to select the most run with a while aiming to select the most run with a risk that some of the application features risk that some of the application features will not be tested at all.will not be tested at all.

The prioritization schemes basically The prioritization schemes basically address these key concepts:address these key concepts:

(A)(A)What features must be tested?What features must be tested?

(B)(B)What are the consequences if some What are the consequences if some features are not tested?features are not tested?

The most simplest scheme for categorizing The most simplest scheme for categorizing tests is to assign a priority code directly to tests is to assign a priority code directly to each test description. One possible scheme each test description. One possible scheme is a three-level priority categorization is a three-level priority categorization scheme. It is given below:scheme. It is given below:

Priority 1 : this test must be executedPriority 1 : this test must be executed

Priority 2 : if time permits, execute this test.Priority 2 : if time permits, execute this test.

Priority 3 : if this test is not executed, the Priority 3 : if this test is not executed, the team won’t be upset.team won’t be upset.

The second scheme can be new five-level scale to The second scheme can be new five-level scale to further classify the tests. This scheme is given further classify the tests. This scheme is given below:below:

Priority 1a: this test must pass, else the delivery will Priority 1a: this test must pass, else the delivery will be out of date.be out of date.

Priority 2a: before final delivery, this test must be Priority 2a: before final delivery, this test must be executed.executed.

Priority 3a: if time permits, execute this test.Priority 3a: if time permits, execute this test.

Priority 4a: this test can wait after the delivery date.Priority 4a: this test can wait after the delivery date.

Priority 5a: we will probably never execute this test.Priority 5a: we will probably never execute this test.

We try to divide the tests, for example, say tests We try to divide the tests, for example, say tests from priority 2 are now divided between priorities from priority 2 are now divided between priorities 3a, 4a and 5a. We can downgrade or upgrade any 3a, 4a and 5a. We can downgrade or upgrade any test in the similar fashion.test in the similar fashion.

All software projects benefits from risk All software projects benefits from risk analysis. Even non-critical software, analysis. Even non-critical software, using risk analysis at the beginning of using risk analysis at the beginning of a project highlight the potential a project highlight the potential problem area. This helps developers problem area. This helps developers and managers to mitigate the risks. and managers to mitigate the risks. The tester uses the results of risk The tester uses the results of risk analysis to select the most crucial analysis to select the most crucial tests. tests.

Regression testing is done to ensure that Regression testing is done to ensure that enhancements or defect fixes made to the enhancements or defect fixes made to the software works properly and does not affect the software works properly and does not affect the existing functionality. It is usually done during existing functionality. It is usually done during maintenance phase.maintenance phase.

As a software system ages, the cost of maintaining As a software system ages, the cost of maintaining the software dominates the overall cost of the software dominates the overall cost of developing the software. Regression testing is a developing the software. Regression testing is a testing process that is used to determine if a testing process that is used to determine if a modified program still meets its specifications or if modified program still meets its specifications or if new error have been introduced. new error have been introduced.

Normal TestingNormal Testing Regression TestingRegression Testing

It is usually done during It is usually done during fourth phase of SDLC.fourth phase of SDLC.

It is done during It is done during maintenance phase.maintenance phase.

It is basically software’s It is basically software’s verification and validation.verification and validation.

It is also called as program It is also called as program revalidation.revalidation.

New test suites are used to New test suites are used to test our code.test our code.

Both old and new test cases Both old and new test cases can be used for testing.can be used for testing.

It is done on the original It is done on the original software.software.

It is done on modified It is done on modified softwaresoftware

It is cheaper.It is cheaper. It is a costlier testing.It is a costlier testing.

Four types of regression testing Four types of regression testing techniques are discussed one by one. techniques are discussed one by one. They are They are

Corrective regression testingCorrective regression testing Progressive regression testing.Progressive regression testing. Retest all regression testing.Retest all regression testing. Selective regression testing.Selective regression testing.

Prioritization of tests requires a Prioritization of tests requires a suitable criterion. Please understand suitable criterion. Please understand that tests with lower costs are placed that tests with lower costs are placed at the top while those with higher at the top while those with higher cost at the bottom.cost at the bottom.

Regression testing is used to confirm Regression testing is used to confirm that fixed bugs have been fixed and that that fixed bugs have been fixed and that new bugs have not been introduced. new bugs have not been introduced. How many cycles of regression testing How many cycles of regression testing are requires will depend upon the project are requires will depend upon the project size. Cycles of regression testing may be size. Cycles of regression testing may be performed once per milestone or once performed once per milestone or once per build. Regression tests can be per build. Regression tests can be automated.automated.

Static slicing may lead to an unduly Static slicing may lead to an unduly large program slice. So, Korel and Laski large program slice. So, Korel and Laski proposed a method for obtaining proposed a method for obtaining dynamic slices from program executions. dynamic slices from program executions. They used a method to extract They used a method to extract executable and smaller slices and to executable and smaller slices and to allow more precise handling of arrays allow more precise handling of arrays and other structures. So, we discuss and other structures. So, we discuss dynamic slicing.dynamic slicing.

CHAPTER-7CHAPTER-7LEVELS OF TESTINGLEVELS OF TESTING

When we talk of levels of testing then we are When we talk of levels of testing then we are actually talking of three levels of testing:actually talking of three levels of testing:

(a) Unit Testing(a) Unit Testing (b) Integration Testing, and (b) Integration Testing, and (c) System Testing(c) System Testing Generally, system testing is functional rather Generally, system testing is functional rather

than structural testing, we shall now study than structural testing, we shall now study these testing techniques one by one.these testing techniques one by one.

Unit testing – “is the process of Unit testing – “is the process of taking a module and running it in taking a module and running it in isolation from the rest of the isolation from the rest of the software product by using prepared software product by using prepared test cases and comparing the actual test cases and comparing the actual results with the results predicted by results with the results predicted by the specification and design the specification and design module.”module.”

A system is composed of multiple components or A system is composed of multiple components or modules that comprise hardware and software. modules that comprise hardware and software. Integration is defined as the set of interactions among Integration is defined as the set of interactions among components. Testing the interaction between the components. Testing the interaction between the modules and interaction with other systems externally is modules and interaction with other systems externally is called as integration testing.called as integration testing.

It is both a type of testing and a phase of testing. The It is both a type of testing and a phase of testing. The architecture and design can give the details of architecture and design can give the details of interaction within systems. However, testing the interaction within systems. However, testing the interaction between one system and integration interaction between one system and integration depends on many modules and system. This ensuing depends on many modules and system. This ensuing phase is called the integration testing phase. phase is called the integration testing phase.

The goal of decomposition based integration is to test The goal of decomposition based integration is to test

the interfaces among separately tested units.the interfaces among separately tested units. Types of decomposition based techniques Top-Down Types of decomposition based techniques Top-Down

integration Approachintegration Approach Bottom Up Integration ApproachBottom Up Integration Approach Sandwich Integration ApproachSandwich Integration Approach Big Bang StrategyBig Bang Strategy Pros and cons of decomposition Based TechniquesPros and cons of decomposition Based Techniques Guidelines to choose integration Method and conclusions Guidelines to choose integration Method and conclusions

One of the drawbacks of decomposition-One of the drawbacks of decomposition-based integration is that the basis is the based integration is that the basis is the functional decomposition tree. But if we functional decomposition tree. But if we use call graph based technique instead, use call graph based technique instead, we can remove this problem. Also, we we can remove this problem. Also, we will move in the direction of structural will move in the direction of structural testing. Because call graph is a directed testing. Because call graph is a directed graph and thus we can use it as a graph and thus we can use it as a program graph also. program graph also.

Pairwise IntegrationPairwise Integration The main idea behind pairwise integration is The main idea behind pairwise integration is

to eliminate the stub/driver development to eliminate the stub/driver development effort. The end result is what we have one effort. The end result is what we have one integration test session for each edge in the integration test session for each edge in the call graph.call graph.

Neighborhood IntegrationNeighborhood Integration The neighbourhood of a node in a graph is The neighbourhood of a node in a graph is

the set of nodes that are one edge away from the set of nodes that are one edge away from the given node. the given node.

Pros and consPros and cons the call graph based integration the call graph based integration

techniques move away from a purely techniques move away from a purely structural basis toward behavioural structural basis toward behavioural matches well with the developments matches well with the developments characterized by builds.characterized by builds.

We already know that the combination of We already know that the combination of structural and functional is highly desirable at structural and functional is highly desirable at the unit level and it would be nice to have a the unit level and it would be nice to have a similar capability for integration and system similar capability for integration and system testing. Our goal for integration testing is – testing. Our goal for integration testing is – “instead of testing interfaces among separately “instead of testing interfaces among separately developed and tested units, we focus on developed and tested units, we focus on interaction among these units.”interaction among these units.”

We now discuss some basic terminologies that We now discuss some basic terminologies that are used in this technique.are used in this technique.

Statement Fragment.Statement Fragment. Source Node.Source Node. Sink Node.Sink Node. Module Execution Path (MEP)Module Execution Path (MEP) MessageMessage ModuleModule Module-to-Module path (MM-Path)Module-to-Module path (MM-Path) Module to Module Path GraphModule to Module Path Graph

The testing that is conducted on the The testing that is conducted on the complete integrated products and solution complete integrated products and solution to evaluated system compliance with to evaluated system compliance with specified requirements on functional and specified requirements on functional and non functional aspects is called as system non functional aspects is called as system testing.testing.

System Level tests consists of batteries of System Level tests consists of batteries of tests that are designed to fully exercise a tests that are designed to fully exercise a program as a whole and check that all program as a whole and check that all elements of the integrated system function elements of the integrated system function properly.properly.

System testing = functional testing + Non-System testing = functional testing + Non-functional testingfunctional testing

Functional TestingFunctional Testing Non-functional Non-functional TestingTesting

It involves the product’s It involves the product’s functionality.functionality.

It involves the product’s It involves the product’s quality factors.quality factors.

Failures, here, occur due Failures, here, occur due to code.to code.

Failures occur due to Failures occur due to either architecture, either architecture, design or due to code.design or due to code.

It is done during unit, It is done during unit, component, integration component, integration and system testing and system testing phase.phase.

It is done in our system It is done in our system testing phase.testing phase.

Configuration remains Configuration remains same for a test suitssame for a test suits

Test configuration is Test configuration is different for each test different for each test suite.suite.

““A test case moved from a later A test case moved from a later phase to an earlier phase is a batter phase to an earlier phase is a batter option than delaying a test case from option than delaying a test case from an earlier phase to a later phase, as an earlier phase to a later phase, as the purpose of testing is to find the purpose of testing is to find defects as early as possible.” defects as early as possible.”

S.NS.No.o.

Integration TestingIntegration Testing System TestingSystem Testing

1.1. The test cases are created The test cases are created by looking at interfaces.by looking at interfaces.

The test cases are The test cases are created first and verified created first and verified with design and with design and architecture.architecture.

2.2. The integration test cases The integration test cases focus on interactions focus on interactions between modules or between modules or components.components.

The functional system The functional system test focuses on the test focuses on the behavior of the behavior of the complete product. complete product.

Using and testing the product for different business Using and testing the product for different business verticals such as banking, insurances, asset verticals such as banking, insurances, asset management etc. and verifying the business operations management etc. and verifying the business operations and usage is called as business vertical testing.and usage is called as business vertical testing.

Some operations that can only be done by some user Some operations that can only be done by some user objects is called as role based operations BVT involves objects is called as role based operations BVT involves three aspects. They arethree aspects. They are

A) customizationA) customization B) terminologyB) terminology C) syndication C) syndication

System testing is the final phase System testing is the final phase before the product is delivered to the before the product is delivered to the customer. customer.

We will now discuss, onsite We will now discuss, onsite deployment testing in detail. It is deployment testing in detail. It is done in two stages:done in two stages:

1 the first stage1 the first stage 2 the second stage2 the second stage

In the first stage- In the first stage- the actual data from the actual data from the live system is taken and similar the live system is taken and similar machines and configurations are machines and configurations are mirrored and the operations from the mirrored and the operations from the user are rerun on the mirrored machineuser are rerun on the mirrored machine

In the second stage –In the second stage – after a after a successful first stage, the mirrored successful first stage, the mirrored system is made a live system that runs system is made a live system that runs the new productthe new product

Non functional testing differs from Non functional testing differs from the function testing in terms of the function testing in terms of complexity, knowledge requirement complexity, knowledge requirement effort needed and number of times effort needed and number of times the test cases are repeated.the test cases are repeated.

Due to varied types of customers, Due to varied types of customers, resources availability, time involved in resources availability, time involved in getting the exact setup and so on, getting the exact setup and so on, setting up of a scenario that is exactly setting up of a scenario that is exactly real life is difficult. Due to several real life is difficult. Due to several complexities involved simulated setup complexities involved simulated setup is used for non-functional testing is used for non-functional testing where actual configuration is difficult where actual configuration is difficult to get.to get.

Types of testTypes of test ParametersParameters Sample entry Sample entry criteriacriteria

Sample exit Sample exit criteriacriteria

ScalabilityScalability Maximum Maximum limitslimits

Product should Product should scale upto one scale upto one million records or million records or 1000 users 1000 users

Product should Product should scale upto 10 scale upto 10 million records million records or 5000or 5000

Performance Performance testtest

Response Response time time ThroughputThroughputLatencyLatency

Query for 1000 Query for 1000 records should records should have a response have a response time less than 3 time less than 3 secondsseconds

Query for Query for 10,000 records 10,000 records should have should have response time response time less than 3 less than 3 seconds.seconds.

StressStress System System when when stressed stressed beyond the beyond the limits.limits.

25 clients login 25 clients login should be possible should be possible in a configuration in a configuration that can take only that can take only 20 clients.20 clients.

Product should Product should be able to be able to withstand 100 withstand 100 clients logic clients logic simultaneously.simultaneously.

There are four key resources namely,There are four key resources namely, CPUCPU DiskDisk MemoryMemory NetworkNetwork

Guidelines for Scalability Testing:Guidelines for Scalability Testing: Scalability should increase by 50% when the number of Scalability should increase by 50% when the number of

CPUs is doubled. This test is applicable for CPU intensive CPUs is doubled. This test is applicable for CPU intensive product.product.

Scalability should increase by 40% when memory is Scalability should increase by 40% when memory is doubled. This test is applicable for memory intensive doubled. This test is applicable for memory intensive product.product.

Scalability should increase by 30% when the number of NIC Scalability should increase by 30% when the number of NIC cards are doubled. This task is useful for network-intensive cards are doubled. This task is useful for network-intensive product.product.

Scalability should increase by 50% when the I/O bandwidth Scalability should increase by 50% when the I/O bandwidth is doubled. This test is useful for I/O intensive products.is doubled. This test is useful for I/O intensive products.

Reliability testing is done to find out if Reliability testing is done to find out if the software will work in the expected the software will work in the expected environment for an acceptable amount environment for an acceptable amount of time without degradation.of time without degradation.

The tests are executed when specific The tests are executed when specific events occur like:events occur like:

A) major software releasesA) major software releases B) completion of a time-box B) completion of a time-box C) at regular time intervals C) at regular time intervals

Stress testing is a combination of Stress testing is a combination of several types of tests like capacity or several types of tests like capacity or volume testing, reliability and stability volume testing, reliability and stability testing, error handling and so on.testing, error handling and so on.

In stress testing, the load is generally In stress testing, the load is generally increased through various means such increased through various means such as increasing the number of clients, as increasing the number of clients, users and transaction till and beyond users and transaction till and beyond the resources are completely utilized.the resources are completely utilized.

It is a phase after system testing that is done It is a phase after system testing that is done by the customers.by the customers.

A user acceptance test is:A user acceptance test is: A chance to complete test softwareA chance to complete test software A chance to completely test business A chance to completely test business

processes.processes. A comparison of actual test result against A comparison of actual test result against

expected results.expected results. A discussion forum to evaluate the process.A discussion forum to evaluate the process.

The primary goal of performance The primary goal of performance testing is to develop effective testing is to develop effective enhancement strategies for enhancement strategies for maintaining acceptable system maintaining acceptable system performance. It is an information performance. It is an information gathering and analyzing process in gathering and analyzing process in which measurement data are collected which measurement data are collected to predict when load levels will exhaust to predict when load levels will exhaust system resources.system resources.

Step 1. Step 1. Collecting requirements.Collecting requirements. Step 2. Step 2. Writing test cases.Writing test cases. Step 3.Step 3. Automating performance test cases.Automating performance test cases. Step 4.Step 4. Executing performance test cases.Executing performance test cases. Step 5. Step 5. Analyzing performance test cases.Analyzing performance test cases. Step 6.Step 6. Performance tuning.Performance tuning. Step 7.Step 7. Performance benchmarking.Performance benchmarking. Step 8.Step 8. Capacity planning.Capacity planning.

Functional performance tools Functional performance tools help in help in recording and playing back the transaction recording and playing back the transaction and obtaining performance numbers. This test and obtaining performance numbers. This test generally involves very few machines.generally involves very few machines.

Load testing toolsLoad testing tools simulate the load simulate the load condition for performance testing without condition for performance testing without having to keep that many users or machines. having to keep that many users or machines. These tools simplify the complexities involves These tools simplify the complexities involves in creating the load.in creating the load.

Thus, we can say that we start with unit or module testing. Then we go Thus, we can say that we start with unit or module testing. Then we go in for integration testing which is then followed by system testing. in for integration testing which is then followed by system testing. Then, we go in for acceptance testing and regression testing Then, we go in for acceptance testing and regression testing acceptance testing may involve alpha and beta testing while regression acceptance testing may involve alpha and beta testing while regression testing is done during maintenance.testing is done during maintenance.

System testing can comprise of “n” different tests. That it is could System testing can comprise of “n” different tests. That it is could mean:mean:

End to end integration testing.End to end integration testing. User interface testing.User interface testing. Stress testingStress testing Testing of availability Testing of availability Performance testing is a type of testing that is easy to understand but Performance testing is a type of testing that is easy to understand but

difficult to perform due to amount of information and effort needed.difficult to perform due to amount of information and effort needed.

CHAPTER-8CHAPTER-8OBJECT ORIENTED TESTINGOBJECT ORIENTED TESTING

Basic unit for testing, inheritance and testing.Basic unit for testing, inheritance and testing. Basic concepts of state machinesBasic concepts of state machines Testing object oriented systemsTesting object oriented systems Heuristics for class testing.Heuristics for class testing. Levels of subjects oriented testingLevels of subjects oriented testing Unit testing a classUnit testing a class Integration testing of classes Integration testing of classes System testing with case studySystem testing with case study Regression and acceptance testingRegression and acceptance testing Managing the test processManaging the test process Design for testability (DFT)Design for testability (DFT) GUI testingGUI testing Comparison of conventional and object oriented testingComparison of conventional and object oriented testing Testing using orthogonal arraysTesting using orthogonal arrays Test execution issuesTest execution issues Case study- currency converter application Case study- currency converter application

The class is the smallest unit for testing. The class is the smallest unit for testing. This is due to the following reasons.This is due to the following reasons.

A) methods are meaningless apart form A) methods are meaningless apart form their class.their class.

B) we can design, method tests, only if we B) we can design, method tests, only if we take a class into consideration.take a class into consideration.

C) class behavior should be verified before C) class behavior should be verified before a class is reused or promoted to a library. a class is reused or promoted to a library.

The goal is to provide some design The goal is to provide some design paradigms that help us to perform paradigms that help us to perform object oriented testing (OOT).object oriented testing (OOT).

Object oriented software centers Object oriented software centers on a class and the inheritance and on a class and the inheritance and encapsulation that affect a class. encapsulation that affect a class.

The class clusters are the practical unit for The class clusters are the practical unit for testing.testing.

A) it is typically impossible to test a class in A) it is typically impossible to test a class in total isolation.total isolation.

B) it is not cost effective to develop a B) it is not cost effective to develop a complete set of drivers and stubs for every complete set of drivers and stubs for every class.class.

C) a smaller testing budget usually means C) a smaller testing budget usually means larger clusters. larger clusters.

Suppose we change some methods in class A. Suppose we change some methods in class A. clearly,clearly,

We need to retest the changed methods.We need to retest the changed methods. We need to retest interaction among changed We need to retest interaction among changed

and unchanged methods.and unchanged methods. We need to retest unchanged methods, if data We need to retest unchanged methods, if data

flow exists between statements, in changed and flow exists between statements, in changed and unchanged methods.unchanged methods.

The changed methods form A also need to be The changed methods form A also need to be exercised in the unique context of the subclass. exercised in the unique context of the subclass.

Suppose we add a subclass method B-1 Suppose we add a subclass method B-1 which now overrides super class method which now overrides super class method A.1. clearly, the new method should be A.1. clearly, the new method should be tested.tested.

Cases will probably not provide the same Cases will probably not provide the same coverage.coverage.

This means that additional test cases are This means that additional test cases are needed at each level of the inheritance needed at each level of the inheritance heirarchy.heirarchy.

A state machine can model the A state machine can model the behavior of such classes. They are behavior of such classes. They are also called as sequential machines.also called as sequential machines.

The state determined message The state determined message response patterns are called the response patterns are called the behaviour of an objects.behaviour of an objects.

State machines are an engineering State machines are an engineering application of mathematical models application of mathematical models known as finite automata.known as finite automata.

A state machine accepts only certain A state machine accepts only certain sequences of input and rejects all other. sequences of input and rejects all other. State based behaviour means that the State based behaviour means that the same input is not necessarily always same input is not necessarily always accepted and when accepted, does not accepted and when accepted, does not necessarily produce the same output.necessarily produce the same output.

A state machine is a model that A state machine is a model that describes behaviour of the system under describes behaviour of the system under test in terms of events, states and test in terms of events, states and actions.actions.

There are two main variants of state models There are two main variants of state models Moore MachinesMoore Machines Transitions do no have outputTransitions do no have output An output action is associated with each state.An output action is associated with each state. Mealy MachineMealy Machine Transitions have outputTransitions have output No output action is associated with stateNo output action is associated with state In software engineering models, the output action In software engineering models, the output action

often represents activation of a process, program often represents activation of a process, program method or modulemethod or module

One state must be designated as the initial state.One state must be designated as the initial state. At least, one state must be designated as the final At least, one state must be designated as the final

state. If not, assumptions about termination should state. If not, assumptions about termination should be made explicit.be made explicit.

There are no equivalent states. (equivalent state There are no equivalent states. (equivalent state produce exactly the same response for all events produce exactly the same response for all events visible to the machine in question)visible to the machine in question)

Every defined event and every defined action must Every defined event and every defined action must appear in at least one transition.appear in at least one transition.

There should be an explicit definition of the There should be an explicit definition of the mechanism to handle events which are implicity mechanism to handle events which are implicity rejected. rejected.

State diagrams can be easily drawn or read up to 20 State diagrams can be easily drawn or read up to 20 states. Tabular representation helps but is not states. Tabular representation helps but is not suitable for large states.suitable for large states.

Concurrency cannot be modeled. Only a single state Concurrency cannot be modeled. Only a single state may be active, so the basic model cannot may be active, so the basic model cannot accommodate two or more simultaneous transition accommodate two or more simultaneous transition processes. Only one state may client/server system.processes. Only one state may client/server system.

With even a moderately large number of states, a With even a moderately large number of states, a state diagram becomes a spaghetti ball.state diagram becomes a spaghetti ball.

Basic models do not offer any mechanisms for Basic models do not offer any mechanisms for partitioning or hierarchic abstraction .partitioning or hierarchic abstraction .

Concatenation involves the Concatenation involves the formation of a subclass that has formation of a subclass that has no locally defined features other no locally defined features other than the minimum requirement of than the minimum requirement of a class definition.a class definition.

Class hierarchies that do not meet Class hierarchies that do not meet these condition are very likely to these condition are very likely to be buggy.be buggy.

Conventional test case designs are Conventional test case designs are based on the process they are to test based on the process they are to test and its inputs and outputs. Object and its inputs and outputs. Object oriented test cases need to oriented test cases need to concentrate on the state of a class. concentrate on the state of a class. To examine the different states, the To examine the different states, the cases have to follow the appropriate cases have to follow the appropriate sequence of operations in the class.sequence of operations in the class.

Implementation based tests are Implementation based tests are developed by analyzing how a class developed by analyzing how a class meets its responsibilities.meets its responsibilities.

The main approached to The main approached to implementation-based testing are :implementation-based testing are :

Control flow testing.Control flow testing. Path coverage.Path coverage. Data flow testing.Data flow testing.

A predicate is the condition in a control A predicate is the condition in a control statement. If, case, do while, do until or statement. If, case, do while, do until or for. The evaluation of a predicate selects a for. The evaluation of a predicate selects a segment of code to be executed.segment of code to be executed.

There are four levels of predicate average:There are four levels of predicate average: Decision coverageDecision coverage Condition coverageCondition coverage Decision/condition averageDecision/condition average Multiple condition coverage.Multiple condition coverage.

We can improve on statement We can improve on statement coverage by requiring that each coverage by requiring that each decision branch be taken at least decision branch be taken at least once. once.

Loops also pose some unique test problems. Loops Loops also pose some unique test problems. Loops may be simple loops, nested loops, serial loops, may be simple loops, nested loops, serial loops, spaghetti loops. We shall discuss each one by one spaghetti loops. We shall discuss each one by one

A) simple loops :A) simple loops : all loops should get at least all loops should get at least two iterations. This is the minimum needed to two iterations. This is the minimum needed to detect data initialization/use faultsdetect data initialization/use faults

B) Nested loops : B) Nested loops : a nested loop is a simple loop a nested loop is a simple loop contained in another simple loop. contained in another simple loop.

C) serial loops : C) serial loops : two or more loops on the same two or more loops on the same control path are serial loops.control path are serial loops.

D) Spaghetti loops : D) Spaghetti loops : it is a loop which has more it is a loop which has more than one entry or exit point. Such loop , if are than one entry or exit point. Such loop , if are nested, can result in very complex control flow nested, can result in very complex control flow paths. paths.

Responsibility-based class testing is the Responsibility-based class testing is the process of devising tests to see that a process of devising tests to see that a class correctly fulfills its class correctly fulfills its responsibilities. These tests are derived responsibilities. These tests are derived from specifications and requirements. from specifications and requirements. They are not derived from They are not derived from implementation details. This is also implementation details. This is also called black-box, functional or called black-box, functional or specification based testing. specification based testing.

Test case data is identified by domain Test case data is identified by domain analysis. For each testable function, we must:analysis. For each testable function, we must:

A) identify all parameters, arguments, A) identify all parameters, arguments, variables, database conditions, set points, variables, database conditions, set points, toggles etc.toggles etc.

B) identify equivalence classes or partitions B) identify equivalence classes or partitions for each variables of interest.for each variables of interest.

C) identify boundary values for the partition.C) identify boundary values for the partition. Domain analysis can reveal design faults. It is Domain analysis can reveal design faults. It is

good at revealing missing logic faults – holes good at revealing missing logic faults – holes in the specification or the implementation.in the specification or the implementation.

Behavior is a specified or observed sequence of Behavior is a specified or observed sequence of accepted messages and responses. There are accepted messages and responses. There are three behavior patterns of interest for testing. –three behavior patterns of interest for testing. –

A) independent : A) independent : the method output is totally the method output is totally independent of previous activations. We can use independent of previous activations. We can use domain, numerical or combination techniques.domain, numerical or combination techniques.

B) Sequential : B) Sequential : the method output depends on the method output depends on specific prior sequences of activation and can be specific prior sequences of activation and can be medaled by a finite state machine.medaled by a finite state machine.

C) Recursive : C) Recursive : method output depends on method output depends on previous activations but there are no constraints previous activations but there are no constraints on order of activates. on order of activates.

An abstract class is typically not directly An abstract class is typically not directly instantiated. Instead, subclasses specialize or instantiated. Instead, subclasses specialize or provide an implementation. A strategy for testing provide an implementation. A strategy for testing C++ abstract classes can be as given below:C++ abstract classes can be as given below:

Develop specification based tests from abstract Develop specification based tests from abstract class specificationclass specification

Test all member functions that don’t call pure Test all member functions that don’t call pure virtual methods.virtual methods.

Test all member functions that do call pure virtual Test all member functions that do call pure virtual methods. This may requires stubs for the derived methods. This may requires stubs for the derived class implementation.class implementation.

Do state/sequential test of new and inherited Do state/sequential test of new and inherited method.method.

A sequential collection contains a A sequential collection contains a variables number of items. Items are variables number of items. Items are added to the “end”, e.g., strings, added to the “end”, e.g., strings, buffers, ASCII files etc. you must buffers, ASCII files etc. you must “walk” the entire collection in “walk” the entire collection in sequence to access a particular sequence to access a particular element. For an sequential collection element. For an sequential collection like a stack or queue. like a stack or queue.

In an ordered collection, elements are added, In an ordered collection, elements are added, accessed and removed using some ordering necessary accessed and removed using some ordering necessary changes in expected results. The following test changes in expected results. The following test patterns verify position-dependent behavior:- patterns verify position-dependent behavior:-

CollectiCollection on operatioperationon

InputInput Element Element positionposition

CollectiCollection stateon state

ExpecteExpected resultd result

All All operatiooperationsns

Single Single itemitemSingle Single itemitem

FirstFirstLastLast

Not Not EmptyEmptyNot Not EmptyEmpty

AddedAddedAddedAdded

Delete/Delete/removeremove

Single Single itemitemSingle Single itemitem

DcDcDcDc

Single Single elementelementEmptyEmpty

Deleted Deleted rejectreject

Operations may be defined which Operations may be defined which use collection as operands. Suppose use collection as operands. Suppose there are two collections, A and B there are two collections, A and B then,then,

A and B should be individually A and B should be individually verified firstverified first

A and B are populated before A and B are populated before running the testrunning the test

Some collection, provide a unique Some collection, provide a unique identifier for each member of a collection.identifier for each member of a collection.

Application classes that provide file or Application classes that provide file or database wrappers may also rely on keyed database wrappers may also rely on keyed access.access.

Keyed collection must provide a wide Keyed collection must provide a wide range of paired operation. Item to item range of paired operation. Item to item processing is often faulty.processing is often faulty.

Collection classes are well-suited to Collection classes are well-suited to sequential constraint or recursive sequential constraint or recursive equivalence test patterns on operations. equivalence test patterns on operations.

Test cases are needed to force Test cases are needed to force exceptions.exceptions.

File errors (empty, overflow, missing)File errors (empty, overflow, missing) I/O errors (device not ready, parity I/O errors (device not ready, parity

check)check) Arithmetic over/under flows.Arithmetic over/under flows. Memory allocationMemory allocation Task communication/creation.Task communication/creation.

There are many situation that indicate additional There are many situation that indicate additional testing may be valuable. Some of those situation testing may be valuable. Some of those situation are given below:are given below:

A module written by the least experienced A module written by the least experienced programmer.programmer.

A module with a high failure rate, in either the A module with a high failure rate, in either the filed or in development.filed or in development.

A module that failed an inspection and needed A module that failed an inspection and needed big changes at the last minute.big changes at the last minute.

A module that was subject to a late or large A module that was subject to a late or large change order, after most of the coding was done.change order, after most of the coding was done.

A module about which a designer or programmer A module about which a designer or programmer feels uneasy. feels uneasy.

Experience, hunches or educated Experience, hunches or educated guesses can suggest good test guesses can suggest good test cases. There is no systematic cases. There is no systematic procedure for guessing error. procedure for guessing error. According to Beizer, “logic errors and According to Beizer, “logic errors and fuzzy thinking are inversely fuzzy thinking are inversely proportional to the probability of a proportional to the probability of a path’s execution”. path’s execution”.

Metrics from past projects or previous Metrics from past projects or previous releases may suggest possible trouble releases may suggest possible trouble spots. We have already noted that C spots. We have already noted that C metric was a good predictor of faults. metric was a good predictor of faults. Coupling and cohesion are also good fault Coupling and cohesion are also good fault predictors. Modules with high coupling predictors. Modules with high coupling and low cohesion are 7 times more likely and low cohesion are 7 times more likely to have defects compared to modules to have defects compared to modules with low coupling and high cohesion.with low coupling and high cohesion.

If individual operations or methods are considered If individual operations or methods are considered to be units, we have four levels: operation/method, to be units, we have four levels: operation/method, class, integration and system testing.class, integration and system testing.

With this choice, operation/method testing is With this choice, operation/method testing is identical to unit testing of procedural software.identical to unit testing of procedural software.

Class and integration testing can be well renamed Class and integration testing can be well renamed as intraclass testing. as intraclass testing.

Integration testing is the major issue of object Integration testing is the major issue of object oriented testing and must be concerned with oriented testing and must be concerned with testing interactions among previously tested testing interactions among previously tested classes.classes.

Finally, system testing is conducted at the port Finally, system testing is conducted at the port event level and is identical to system testing of event level and is identical to system testing of traditional software. The only difference is where traditional software. The only difference is where system level test cases originate. system level test cases originate.

Classes are the Building Blocks for an Classes are the Building Blocks for an entire object oriented system. Just as entire object oriented system. Just as the building blocks of a procedure-the building blocks of a procedure-oriented system have to be unit oriented system have to be unit tested individually before being put tested individually before being put together, so also the classes have to together, so also the classes have to be unit tested.be unit tested.

Of the three main levels of software testing, Of the three main levels of software testing, integration testing is the least understood. This is integration testing is the least understood. This is true for both traditional and object-oriented true for both traditional and object-oriented software. As with traditional procedural software, software. As with traditional procedural software, object-oriented integration testing presumes object-oriented integration testing presumes complete unit level testing.complete unit level testing.

Both unit choices have implications for object-Both unit choices have implications for object-oriented integration testing. If the oriented integration testing. If the operation/method choice is taken, tow levels of operation/method choice is taken, tow levels of integration are required.integration are required.

A) one to integrate operations into a full class, andA) one to integrate operations into a full class, and B) one to integrate the class with other classes.B) one to integrate the class with other classes.

There are three basic strategies for integration:There are three basic strategies for integration: i) i) client/supplier client/supplier : the structure of class/supplier : the structure of class/supplier

classes can be used to guide integration.classes can be used to guide integration. Thread Based :Thread Based : the thread of messages activated the thread of messages activated

to service a single user input or external event to service a single user input or external event studied to decide the order of integration.studied to decide the order of integration.

Configuration Based Configuration Based : the thread of messages : the thread of messages activated to service a single user input or external activated to service a single user input or external event studied to decide the order of integration.event studied to decide the order of integration.

Hybrid Strategy :Hybrid Strategy : A mix of top down, bottom up or A mix of top down, bottom up or Big-Bang integration can be used. Big-Bang integration can be used.

According to Myer, if you do not have According to Myer, if you do not have written objectives for your product, written objectives for your product, or if your objectives are or if your objectives are immeasurable, then you cannot immeasurable, then you cannot perform system test.perform system test.

A requirement is a capability, a features or A requirement is a capability, a features or function that the system must provide. function that the system must provide.

We must, however, take care of the followingWe must, however, take care of the following Traceability from requirements to test cases Traceability from requirements to test cases

is necessary.is necessary. Each test case, at all levels, should indicate Each test case, at all levels, should indicate

which requirement it implements.which requirement it implements. The test is simple : can you meet the The test is simple : can you meet the

requirement?requirement?

The notice of the “average” user or usage The notice of the “average” user or usage scenario can be misleading or too gague to scenario can be misleading or too gague to construct useful tests. We need to define specific construct useful tests. We need to define specific scenarios to explore dynamic behavior for scenarios to explore dynamic behavior for exampleexample

Customer/user oriented definitionCustomer/user oriented definition Revalidates the productRevalidates the product Focuses development on customer requirementFocuses development on customer requirement Early customer involvement can build Early customer involvement can build

relationshipsrelationships Real-world orientation. Real-world orientation.

It is defined as the selective retesting of It is defined as the selective retesting of system or component after changes system or component after changes have been made.have been made.

OrOr It is defined as the process to verify It is defined as the process to verify

absence of unintended effects.absence of unintended effects. It is defined as the process of verifying It is defined as the process of verifying

compliance with all old and new compliance with all old and new requirements.requirements.

A comprehensive test plan package is described in A comprehensive test plan package is described in IEEE 23b, which is an accepted industry standard for IEEE 23b, which is an accepted industry standard for test planning. It recommends the following main test planning. It recommends the following main documents.documents.

The Test PlanThe Test Plan: it define the features to be tested and : it define the features to be tested and excluded from testing. excluded from testing.

The Test Design The Test Design : it defines the features/functions to : it defines the features/functions to test, pass fail criterion.test, pass fail criterion.

The Test Cases The Test Cases : it defines the items to be tested : it defines the items to be tested and provides traceability to SRS, SDD.and provides traceability to SRS, SDD.

Test Procedures Test Procedures : it describes the defines the : it describes the defines the procedures necessary to perform each test.procedures necessary to perform each test.

Design for testability is a strategy to align the Design for testability is a strategy to align the development process for maximum development process for maximum effectiveness under either a reliability – drives effectiveness under either a reliability – drives or resource limited regime.or resource limited regime.

A reliability driven process A reliability driven process uses testing to uses testing to produce evidence that a pre-release reliability produce evidence that a pre-release reliability goal has been met.goal has been met.

A resources limited process A resources limited process views testing views testing as a way to remove as many rough edges as a way to remove as many rough edges from a system as time or money permits.from a system as time or money permits.

The main characteristics of any The main characteristics of any graphical user interface application is graphical user interface application is that it is event driven. Users can that it is event driven. Users can cause any of several events in any cause any of several events in any order. Many GUIs have event order. Many GUIs have event sequence that is not guided. One sequence that is not guided. One benefit of GUI applications to testers benefit of GUI applications to testers is that there is little need for is that there is little need for integration testing.integration testing.

Although there is basically no Although there is basically no difference in the method of testing, difference in the method of testing, whether one is testing a non-object whether one is testing a non-object oriented application or an object oriented application or an object oriented applications; still the following oriented applications; still the following aspects applicable mostly for object-aspects applicable mostly for object-oriented applications may be kept in oriented applications may be kept in mind and testing tactics accordingly mind and testing tactics accordingly devised. devised.

The goal is to pair each state with The goal is to pair each state with every other state at least once. every other state at least once.

Orthogonal array testing is a Orthogonal array testing is a powerful technique to use any time a powerful technique to use any time a system presents a small input system presents a small input domain with a large number of domain with a large number of permutations.permutations.

After defining the test cases, the next step is After defining the test cases, the next step is to formulate an environment In which to to formulate an environment In which to execute the tests. This requires special test execute the tests. This requires special test software, known as a driver. It invokes the software, known as a driver. It invokes the software under test. The typical driver consist software under test. The typical driver consist of the following features.:of the following features.:

A) Packaging input and data.A) Packaging input and data. B) invoking the software-under-testB) invoking the software-under-test C) capturing the resulting outcome.C) capturing the resulting outcome.

Problem statement: the currency converter Problem statement: the currency converter application coverts U.S. dollars to any of the four application coverts U.S. dollars to any of the four currencies : Brazilian real, Canadian dollars, currencies : Brazilian real, Canadian dollars, European euros and Japanese yen. The user can European euros and Japanese yen. The user can revise inputs and perform repeated currency revise inputs and perform repeated currency conversion. The user selects the country whose conversion. The user selects the country whose currency equivalent is desired using radio buttons currency equivalent is desired using radio buttons which are mutually exclusive. When the compute which are mutually exclusive. When the compute button is clicked, the result is displayed. Clear button is clicked, the result is displayed. Clear button will clear the screen. And clicking on the button will clear the screen. And clicking on the quit button ends the application.quit button ends the application.

Now we will perform the following on this GUI Now we will perform the following on this GUI application:application:

GUI Testing on this Application : GUI Testing on this Application : to test a GUI to test a GUI application, we begin by identifying all the user application, we begin by identifying all the user input events and all the system output events.input events and all the system output events.

Unit Testing of Currency Conversion Unit Testing of Currency Conversion Program.: Program.: the command buttons on the form the command buttons on the form have an event-driven code attached to them. have an event-driven code attached to them. Hence, they are sensible places to perform unit-Hence, they are sensible places to perform unit-level testing. level testing.

Integration Testing of Currency Conversion Program Integration Testing of Currency Conversion Program : : whether integration testing is suitable for this problem whether integration testing is suitable for this problem depends on how it is implemented. Three main choices depends on how it is implemented. Three main choices are available for the compute button.are available for the compute button.

First choice concentrates all logic in one place and simply First choice concentrates all logic in one place and simply uses the status of the option buttons as in IF tests. Such IF uses the status of the option buttons as in IF tests. Such IF tests would be thoroughly done at the unit level. tests would be thoroughly done at the unit level.

The second and more object oriented choice would have The second and more object oriented choice would have methods in each option button object that send the methods in each option button object that send the exchange rate value for the corresponding country in exchange rate value for the corresponding country in response to a click event.response to a click event.

The third choice is in the visual basic style. The code for The third choice is in the visual basic style. The code for VB includes a global variable for the exchange rate. There VB includes a global variable for the exchange rate. There are event produces for each option button.are event produces for each option button.

As we have already discussed that As we have already discussed that unit and integration testing are unit and integration testing are minimally needed for small GUI minimally needed for small GUI applications. The onus therefore shifts applications. The onus therefore shifts to system testing.to system testing.

System testing may beSystem testing may be A) UML-based may beA) UML-based may be B) State-chart based system testing.B) State-chart based system testing.

Various key object oriented concepts can be Various key object oriented concepts can be testing using some testing methods and tools. testing using some testing methods and tools. They are summarized below:They are summarized below:

Key object Key object oriented oriented conceptsconcepts

Testing methods and toolsTesting methods and tools

Object-Object-orientationorientation

Tests need to integrate data and Tests need to integrate data and methods more tightly.methods more tightly.

Unit testing of Unit testing of classesclasses

BVA, equivalence partitioning to test BVA, equivalence partitioning to test variables.variables.Code coverage methods for Code coverage methods for methods of a classmethods of a classAlpha omega method of exercising Alpha omega method of exercising methods.methods.State diagram to test states of a State diagram to test states of a classclassStress testing to detect memory Stress testing to detect memory leaks.leaks.

Abstract Abstract classesclasses

Requires retesting for every new Requires retesting for every new implementation of the abstract class.implementation of the abstract class.

Encapsulation Encapsulation and and inheritanceinheritance

Requires unit testing at class level and Requires unit testing at class level and incremental class testing when encapsulating incremental class testing when encapsulating Inheritance introduces extra context; each Inheritance introduces extra context; each combination of different contexts has to be combination of different contexts has to be tested.tested.

PolymorphismPolymorphism Each of the different methods of the same Each of the different methods of the same name should be tested separately.name should be tested separately.Maintainability of code may suffer.Maintainability of code may suffer.

Dynamic Dynamic bindingbinding

Conventional code coverage has to be Conventional code coverage has to be modified to be applicable for dynamic modified to be applicable for dynamic binding.binding.Possibility of unanticipated runtime defects Possibility of unanticipated runtime defects higher.higher.

Inter-object Inter-object communicatiocommunication via n via messages.messages.

Message sequencingMessage sequencingSequence diagrams.Sequence diagrams.

Object reuse and parallel Object reuse and parallel development of objects.development of objects.

Needs more frequent Needs more frequent integration tests and integration tests and regression tests.regression tests.Integration testing and Integration testing and unit testing are not as unit testing are not as clearly separated as in clearly separated as in the case of a procedure the case of a procedure oriented language.oriented language.Errors in interfaces Errors in interfaces between objects likely to between objects likely to be more common in be more common in object oriented systems object oriented systems and hence needs and hence needs thorough interface thorough interface testing.testing.

CHAPTER-9CHAPTER-9AUTOMATED TESTINGAUTOMATED TESTING

Automated testingAutomated testing Consideration during automated testingConsideration during automated testing Types of testing tool-static V/s dynamicTypes of testing tool-static V/s dynamic Problems with manual testingProblems with manual testing Benefits of automated testingBenefits of automated testing Disadvantages of automated testing Disadvantages of automated testing Skills needed for using auto mated tools.Skills needed for using auto mated tools. Test automation : “No silver bullet” Test automation : “No silver bullet” DebuggingDebugging Criteria for selection of test toolsCriteria for selection of test tools Steps for tools selectionSteps for tools selection Characteristics of modern toolsCharacteristics of modern tools Case study on automated tools, Namely, Rational Robot, Win Case study on automated tools, Namely, Rational Robot, Win

Runner, silk test and load Runner Runner, silk test and load Runner

Automated testing is automating the Automated testing is automating the manual testing process. It is used to manual testing process. It is used to replace or supplement manual testing replace or supplement manual testing with a suite of testing tools.with a suite of testing tools.

Manual testing is used to document Manual testing is used to document tests, produce test guides based on data tests, produce test guides based on data queries, provide temporary structures to queries, provide temporary structures to help run tests and measure the result of help run tests and measure the result of the test.the test.

While performing testing with automated tools, the While performing testing with automated tools, the following points should be noted:following points should be noted:

Clear and reasonable expectations should be established in Clear and reasonable expectations should be established in order to know what can and what cannot be accomplished order to know what can and what cannot be accomplished with automated testing in the organization.with automated testing in the organization.

There should be clear understanding of the requirements There should be clear understanding of the requirements that should be met in order to achieve successful that should be met in order to achieve successful automated testing. This requires that the technical automated testing. This requires that the technical personnel should use the tools effectively.personnel should use the tools effectively.

The organization should have detailed, reusable test cases The organization should have detailed, reusable test cases which contain exact expected results and a stand alone test which contain exact expected results and a stand alone test environment with a restorable database.environment with a restorable database.

Testing tool should be cost effective. The tool must ensure Testing tool should be cost effective. The tool must ensure that test cases developed for manual testing are also useful that test cases developed for manual testing are also useful for automated testing. for automated testing.

Since testing is of two typesSince testing is of two types Static testingStatic testing Dynamic testingDynamic testing And also that the tools used during these And also that the tools used during these

testing are accordingly named astesting are accordingly named as Static testing tools.Static testing tools. Dynamic testing tools.Dynamic testing tools.

Static testingStatic testing Dynamic testingDynamic testingStatic testing does not require the Static testing does not require the actual execution of softwareactual execution of software

Dynamic testing involves testing Dynamic testing involves testing the software by actually the software by actually executing it.executing it.

It is more cost effectiveIt is more cost effective It is less cost effectiveIt is less cost effective

It may achieve 100% statement It may achieve 100% statement coverage in relatively short timecoverage in relatively short time

It achieves less than 50% It achieves less than 50% statement coverage because it statement coverage because it finds bugs only in part of codes finds bugs only in part of codes that are actually executed.that are actually executed.

It usually takes shorter time.It usually takes shorter time. It may involve running several It may involve running several test case, each of which may take test case, each of which may take longer then compilation.longer then compilation.

It may uncover variety of bugs.It may uncover variety of bugs. It uncovers limited type of bugs It uncovers limited type of bugs that are exportable through that are exportable through execution.execution.

It can be done before compilationIt can be done before compilation It can take place only after It can take place only after executables are ready.executables are ready.

The main problems with manual testing are listed below:The main problems with manual testing are listed below: Not reliable : manual testing is not reliable as there is no yardstick Not reliable : manual testing is not reliable as there is no yardstick

available to find out whether the actual and expected result have available to find out whether the actual and expected result have been compared. We just rely on the tester’s words.been compared. We just rely on the tester’s words.

High Risk : a manual testing process is subject to high risks of High Risk : a manual testing process is subject to high risks of oversights and mistakes. oversights and mistakes.

Incomplete coverage: testing is quite complex when we have mix of Incomplete coverage: testing is quite complex when we have mix of multiple platforms, O.S servers, clients, channels, business processes multiple platforms, O.S servers, clients, channels, business processes etc.etc.

Time consuming : limited test resources makes manual testing simply Time consuming : limited test resources makes manual testing simply too time consuming. As per a study done, 90% of all IT project are too time consuming. As per a study done, 90% of all IT project are delivered late due to manual testing.delivered late due to manual testing.

Fact and Fiction : the fiction is that manual testing is done while the Fact and Fiction : the fiction is that manual testing is done while the fact is only some manual testing is done depending upon the fact is only some manual testing is done depending upon the feasibility.feasibility.

Automated testing is the process of Automated testing is the process of automating the manual testing process. It is automating the manual testing process. It is used to replace or supplement manual used to replace or supplement manual testing with a suite of testing tools. testing with a suite of testing tools. Automated testing tools assist software Automated testing tools assist software testers to evaluate the quality of the software testers to evaluate the quality of the software by automating the mechanical aspects of the by automating the mechanical aspects of the software testing task. The benefits of software testing task. The benefits of automation include increased software automation include increased software quality, improved time to market, repeatable quality, improved time to market, repeatable test procedure and reduced testing costs.test procedure and reduced testing costs.

Despite of many benefits, pace of test-automation is slow. Some of Despite of many benefits, pace of test-automation is slow. Some of its disadvantages are given below:its disadvantages are given below:

An average automated test suite development is normally 3-5 times An average automated test suite development is normally 3-5 times the cost of a complete manual test cycle.the cost of a complete manual test cycle.

Automation is too cumbersome. Who would automate? Who would Automation is too cumbersome. Who would automate? Who would train? Who would maintain? This complicates the matter.train? Who would maintain? This complicates the matter.

In many organizations, test automation is not even a discussion In many organizations, test automation is not even a discussion issue.issue.

There are some organizations where there is practically no There are some organizations where there is practically no awareness or only some awareness on test automation.awareness or only some awareness on test automation.

Automation is not an item of higher priority for managements. It Automation is not an item of higher priority for managements. It does not make much difference to many organization.does not make much difference to many organization.

Automation would require additional trained staff. There is no staff Automation would require additional trained staff. There is no staff for the purpose. for the purpose.

The skills required depends on what The skills required depends on what generation of automation the company is generation of automation the company is in.in.

Capture/playback and test harness tools Capture/playback and test harness tools (first generation)(first generation)

Data driven tools (second generation)Data driven tools (second generation) Action driven (third generation)Action driven (third generation)

Test automation is a partial solution and not a Test automation is a partial solution and not a complete solution. One does not go in for complete solution. One does not go in for automation because it is easy. It is painful and automation because it is easy. It is painful and resource-consuming exercise but once it is done. resource-consuming exercise but once it is done. It has numerous benefits. For example, It has numerous benefits. For example, developing a software to automate inventory developing a software to automate inventory management may be time-consuming, painful management may be time-consuming, painful and costly resource-intensive exercise but once and costly resource-intensive exercise but once done, inventory management becomes relatively done, inventory management becomes relatively a breeze. a breeze.

Debugging occurs as a consequence of successful Debugging occurs as a consequence of successful testing. That is, when a test case uncovers an testing. That is, when a test case uncovers an error, debugging is the process that result in the error, debugging is the process that result in the removal of the error. After testing, we begin an removal of the error. After testing, we begin an investigation to locate the error to find out which investigation to locate the error to find out which module or interface is causing it. Then that section module or interface is causing it. Then that section of the code is to be studied to determine the cause of the code is to be studied to determine the cause of the problem. This process is called debugging. of the problem. This process is called debugging. The debugging process will always have one of the The debugging process will always have one of the two outcomes. two outcomes.

The cause will be found, corrected and removed, or The cause will be found, corrected and removed, or The cause will not be found.The cause will not be found.

During debugging, errors are encountered that During debugging, errors are encountered that range from less damaging to catastrophic. range from less damaging to catastrophic.

The debugging process attempts to match with The debugging process attempts to match with cause, thereby leading to error correction.cause, thereby leading to error correction.

Several approaches have been discussed in Several approaches have been discussed in literature for debugging software under test (SUT).literature for debugging software under test (SUT).

Some of them are discussed below:Some of them are discussed below: Brute force Method : this method is most common Brute force Method : this method is most common

and least efficient for isolating the cause of a and least efficient for isolating the cause of a software error. software error.

Back tracking Method : it is a fairly common Back tracking Method : it is a fairly common debugging approach that can be used successful in debugging approach that can be used successful in small programs. Beginning at the site where a small programs. Beginning at the site where a symptom has been uncovered, the source code is symptom has been uncovered, the source code is traced backward until the site of the cause is found.traced backward until the site of the cause is found.

Cause Elimination : the third approach to Cause Elimination : the third approach to debugging, cause elimination, is manifested debugging, cause elimination, is manifested by induction or deduction and introduces by induction or deduction and introduces the concept of binary partitioning. This the concept of binary partitioning. This approach is also potential causes.approach is also potential causes.

The main criteria for selection of test The main criteria for selection of test tools are given below:tools are given below:

Meeting requirementsMeeting requirements Technology expectations.Technology expectations. Training skills and Training skills and Management aspects.Management aspects.

There are many tools available in the market There are many tools available in the market today but rarely do they meet all the requirements today but rarely do they meet all the requirements of given product or a given organization.of given product or a given organization.

Test tools may not provide backward or forward Test tools may not provide backward or forward compatibility with the product-under-test (PUT).compatibility with the product-under-test (PUT).

Test tools may not go through the same amount of Test tools may not go through the same amount of evaluation for new requirements.evaluation for new requirements.

A number of test tools cannot distinguish between A number of test tools cannot distinguish between a product failure and a test failure this increases a product failure and a test failure this increases of trouble shooting messages to help in analysis. of trouble shooting messages to help in analysis.

In general, test tools may not allow test developers to In general, test tools may not allow test developers to extend/modify the functionality of the framework. So, extend/modify the functionality of the framework. So, it involves going back to the tool vendor with it involves going back to the tool vendor with additional cost and effort.additional cost and effort.

A good number of test tools require their libraries to A good number of test tools require their libraries to be linked with product binaries. When these libraries be linked with product binaries. When these libraries are linked with the source code of the product. It is are linked with the source code of the product. It is called as the “instrumented code”. called as the “instrumented code”.

Finally, test tools are not 100% cross-platform. They Finally, test tools are not 100% cross-platform. They are supported only on some O.S. platform and the are supported only on some O.S. platform and the scripts generated from these tools may not be scripts generated from these tools may not be compatible on other platforms. compatible on other platforms.

Test tools requires plenty of training, but very few Test tools requires plenty of training, but very few vendors provide the training to the required level. vendors provide the training to the required level.

Organization level training is needed to deploy Organization level training is needed to deploy the test tools, as the users of the test suite are the test tools, as the users of the test suite are not only the test team but also the development not only the test team but also the development team and other areas like SCM. Test tools expect team and other areas like SCM. Test tools expect the users to learn new language and may not use the users to learn new language and may not use standard languages. This increases skill standard languages. This increases skill requirements for automation and increases the requirements for automation and increases the need for a learning curve the organization.need for a learning curve the organization.

A test tool increases the system requirement A test tool increases the system requirement and requires the hardware and software to be and requires the hardware and software to be upgraded. upgraded.

This increases the cost of the already-This increases the cost of the already-expensive test tool. When selecting the test expensive test tool. When selecting the test tool. When selecting the test tool, it is tool. When selecting the test tool, it is important to note the system requirements important to note the system requirements and the cost involved in upgrading the and the cost involved in upgrading the software and hardware needs to be included software and hardware needs to be included with the cost of the tool.with the cost of the tool.

There are seven steps and deploy a test tool in an There are seven steps and deploy a test tool in an organization. These steps are :organization. These steps are : Step 1 : identify your test suite requirements among the generic Step 1 : identify your test suite requirements among the generic

requirement discussed. Add other requirements, if any.requirement discussed. Add other requirements, if any. Step 2 : make sure experience discussed in previous section are taken care Step 2 : make sure experience discussed in previous section are taken care

of.of. Step 3 : collect the experiences of other organizations which used similar Step 3 : collect the experiences of other organizations which used similar

test tools.test tools. Step 4: keep a checklist of questions to be asked to the vendors on Step 4: keep a checklist of questions to be asked to the vendors on

cost/effro/suppors.cost/effro/suppors. Step 5 : identify list of tools that meet the above requirements and give Step 5 : identify list of tools that meet the above requirements and give

priority for the tool which is available with the source code.priority for the tool which is available with the source code. Step 6: evaluate and shortlist one/set of tools and train all test developers Step 6: evaluate and shortlist one/set of tools and train all test developers

on the tool.on the tool. Step 7 : deploy the tool across test teams after training all potential users of Step 7 : deploy the tool across test teams after training all potential users of

the tool.the tool.

The modern testing tools available today have some The modern testing tools available today have some salient features that are discussed below :salient features that are discussed below : It should use one or more testing strategy for performing It should use one or more testing strategy for performing

testing on host and target platforms.testing on host and target platforms. It should support GUI based test preparation.It should support GUI based test preparation. It should provide complete code coverage and create test It should provide complete code coverage and create test

documentation in various formats like .doc, .html,.rtf etc.documentation in various formats like .doc, .html,.rtf etc. These tools should be able to adopt the underlying These tools should be able to adopt the underlying

hardware.hardware. It should be easy to use.It should be easy to use. It should provide a clear and correct report on test case, It should provide a clear and correct report on test case,

test case status etc.test case status etc.

It’s main features are :It’s main features are : ““Rational” is for the name of the company, “Rational software Rational” is for the name of the company, “Rational software

cooperation (IBM)”. It is called as a “Robot” because like a robot cooperation (IBM)”. It is called as a “Robot” because like a robot it operates applications and does data entry on screensit operates applications and does data entry on screens

Rational robot is highly useful and effective in automating Rational robot is highly useful and effective in automating screen based testingscreen based testing

Operations on window/web GUI, entry of data on screens and Operations on window/web GUI, entry of data on screens and other screen operations can be automatically captured by robot other screen operations can be automatically captured by robot into a script.into a script.

The recorded scripts can be played back.The recorded scripts can be played back. Rational robot and SQA basic provide excellent, user-friends Rational robot and SQA basic provide excellent, user-friends

development environment.development environment.

Testing is an expensive and laborious phase of the Testing is an expensive and laborious phase of the software process. As a result, testing tools were among software process. As a result, testing tools were among the first software tools to be developed these tools now the first software tools to be developed these tools now offer a range of facilities and their use. Significantly offer a range of facilities and their use. Significantly reduces the cost of the testing process. Different reduces the cost of the testing process. Different testing tools may be integrated into testing workbench.testing tools may be integrated into testing workbench.

These tools are :These tools are : Test Manage : it manages the running of program tests. It keeps Test Manage : it manages the running of program tests. It keeps

track of test data, expected results and program facilities tested.track of test data, expected results and program facilities tested. Test Data Generator : it generates test data for the program to be Test Data Generator : it generates test data for the program to be

tested. This may be accomplished by selecting data from a tested. This may be accomplished by selecting data from a database.database.

Oracle : it generates predictions of expected test results.Oracle : it generates predictions of expected test results.

Report generator : it provides report definition Report generator : it provides report definition and generation facilities for test results.and generation facilities for test results.

Dynamic Analyzer : it adds code to a program Dynamic Analyzer : it adds code to a program to count the number of times each statement to count the number of times each statement has been executed. After the tests have been has been executed. After the tests have been run, an execution profile is generated showing run, an execution profile is generated showing how often each program statement has been how often each program statement has been executed.executed.

Testing workbenches invariably have to be Testing workbenches invariably have to be adapted to suite the test plan of each system. adapted to suite the test plan of each system. A significant amount of efforts and time is A significant amount of efforts and time is usually needed to create a comprehensive usually needed to create a comprehensive testing workbench. Most testing work benches testing workbench. Most testing work benches are open systems because testing needs are are open systems because testing needs are organization specific.organization specific.

CHAPTER-10CHAPTER-10TEST POINT ANALYSIS (TPA)TEST POINT ANALYSIS (TPA)

IntroductionIntroduction MethodologyMethodology Case studyCase study TPA for case studyTPA for case study Phase wise breakup over testing life Phase wise breakup over testing life

cyclecycle Path analysisPath analysis Path analysis processPath analysis process

The goal of this technique is to outline all major The goal of this technique is to outline all major factors that affect testing projects and to ultimately factors that affect testing projects and to ultimately do an accurate test efforts estimation. do an accurate test efforts estimation.

On time project delivery cannot be achieved without On time project delivery cannot be achieved without an accurate and reliable test effort estimate.an accurate and reliable test effort estimate.

TPA is one such method which can be applied for TPA is one such method which can be applied for estimating test effort in black box testing. It is a 6 estimating test effort in black box testing. It is a 6 step approach to test estimation and planning. We step approach to test estimation and planning. We believe that our approach has a good potential for believe that our approach has a good potential for providing test estimation for various projects.providing test estimation for various projects.

TPA PhilosophyTPA Philosophy The effort estimation technique TPA, is based on The effort estimation technique TPA, is based on

three fundamental elements.three fundamental elements. Size of the information system to be tested.Size of the information system to be tested. Test strategy.Test strategy. Productivity Productivity

Size denotes the size of the information system to be Size denotes the size of the information system to be tested.tested.

Test strategy implies the quality characteristics that Test strategy implies the quality characteristics that are to be tested on each subsystem.are to be tested on each subsystem.

Calculation of static test points :Calculation of static test points : St depends upon total FP of the information system St depends upon total FP of the information system

and static quality characteristics of the system.and static quality characteristics of the system. ISO 9126 has listed the following quality ISO 9126 has listed the following quality

characteristics as static :characteristics as static : FunctionalityFunctionality UsabilityUsability EfficiencyEfficiency PortabilityPortability MaintainabilityMaintainability ReliabilityReliability

Method of calculation of Static Points:-Method of calculation of Static Points:-

SSTT = (FP * Q = (FP * Qii))

Calculation of dynamic test points :Calculation of dynamic test points : DDT T = FP = FPff * D * Dff * Q * QDD

DDTT = Dynamic test points.= Dynamic test points. FPFPff = Number of function points = Number of function points

assigned to assigned to the function . the function . DDff = Weighing factor for function = Weighing factor for function

dependent dependent factors. factors. QQDD = Weighing factor for dynamic = Weighing factor for dynamic

quality quality characteristics characteristics..

It implies how important is the function to the It implies how important is the function to the users related to other system functions.users related to other system functions.

Rule : “about 25% of functions should be Rule : “about 25% of functions should be placed in the high category, 50% in normal placed in the high category, 50% in normal category and 25% in low category.”category and 25% in low category.”

WeightWeight CategoryCategory DescriptionDescription

22 LowLow The function is only used a few The function is only used a few times per day or per week.times per day or per week.

44 NominalNominal The function is being used a great The function is being used a great many times per day.many times per day.

1212 HighHigh The function is used continuously The function is used continuously throughout the day.throughout the day.

It implies how much does one function affect It implies how much does one function affect other parts of the system. The degree of other parts of the system. The degree of interfacing is determined by first ascertaining interfacing is determined by first ascertaining the logical data sets which the function in the logical data sets which the function in question can modify, then the other functions question can modify, then the other functions which access these LDSs. An interface rating is which access these LDSs. An interface rating is assigned to a function by reference to a table in assigned to a function by reference to a table in which the number of LDSs affected by the which the number of LDSs affected by the function are arranged vertically and the number function are arranged vertically and the number of the other functions accessing LDSs are of the other functions accessing LDSs are arranged horizontally. arranged horizontally.

The complexity of a function is The complexity of a function is determined on the basis of its determined on the basis of its algorithm i.e., how complex is the algorithm i.e., how complex is the algorithm in a specific function.algorithm in a specific function.

The complexity rating of the function The complexity rating of the function depends on the number of conditions depends on the number of conditions in the functions algorithm.in the functions algorithm.

It checks the reusability of the code. A It checks the reusability of the code. A uniformity factor of 0.6 is assigned in case of uniformity factor of 0.6 is assigned in case of 22ndnd occurrence of unique function, clone and occurrence of unique function, clone and dummy functions. Otherwise in all cases a dummy functions. Otherwise in all cases a uniformity factor 1 is assigned.uniformity factor 1 is assigned.

Method of calculation of Df : the factor is Method of calculation of Df : the factor is calculated by adding together the rating of calculated by adding together the rating of first-four functions dependent variables i.e., first-four functions dependent variables i.e., Up, Ui I and C and then dividing it by 20 (sum Up, Ui I and C and then dividing it by 20 (sum of median/nominal weight of these factors).of median/nominal weight of these factors).

Step - 1Step - 1 System study by KR V and V company : KR V and V System study by KR V and V company : KR V and V

requests a 2 day systems and requirements study to requests a 2 day systems and requirements study to understand the scope of testing work and assess the understand the scope of testing work and assess the testing requirements to arrive at TPA estimate. testing requirements to arrive at TPA estimate. User important : it was observed that 20% of the functionality of User important : it was observed that 20% of the functionality of

the product is of low importance to user, 60% is of medium the product is of low importance to user, 60% is of medium importance and 20% is of high importance.importance and 20% is of high importance.

Usage intensity of the functions : 10% of functions are less used, Usage intensity of the functions : 10% of functions are less used, 70% are medium used and 20% are extensively used.70% are medium used and 20% are extensively used.

Interfacing of Functions with other function : 50% of function are Interfacing of Functions with other function : 50% of function are almost independent and do not interface with other functions. The almost independent and do not interface with other functions. The remaining 50 % are highly dependent . remaining 50 % are highly dependent .

Step -2 Step -2 Environments Assessment by KR V and V company : KR V Environments Assessment by KR V and V company : KR V

and V carried out environment assessment and noted the and V carried out environment assessment and noted the following:following:

KR V and V would use query language, records and KR V and V would use query language, records and playback tools in testing. playback tools in testing.

Development test plan is available from DCM data Development test plan is available from DCM data systems Ltd. But the test team is not familiar with the systems Ltd. But the test team is not familiar with the earlier test cases executed and results.earlier test cases executed and results.

Product is developed in 4GL with integrated databases Product is developed in 4GL with integrated databases Test environment is now for KR V and V but is similar to Test environment is now for KR V and V but is similar to

earlier used test environment.earlier used test environment.

Dynamic test points Dt: from Eqn. 2, we Dynamic test points Dt: from Eqn. 2, we have:have:

DDTT == FPFPff * D * Dff* Q* QDD

Where FPWhere FPff = Transaction FP = 600 = Transaction FP = 600 DDff = dependency factor = weighted = dependency factor = weighted

rating on importance to user, usagerating on importance to user, usage Intensity, interfacing of functions, Intensity, interfacing of functions,

complexity of functions.complexity of functions.

Total test points (TP):Total test points (TP): TP = DT + ST = 2444.4 + 160 = 2604.4TP = DT + ST = 2444.4 + 160 = 2604.4

Productivity :Productivity :1.4 Test hours per test point1.4 Test hours per test pointEnvironment factor : = (Rating on 6 Environment factor : = (Rating on 6

environmentenvironmentfactors)/21factors)/21Planning and control allowance : Planning and control allowance : = Rating on team size factor + Rating on = Rating on team size factor + Rating on

management tools factormanagement tools factor= 6% + 2% = 8%= 6% + 2% = 8%

Total test hours :Total test hours : = primary test hours + 8% of = primary test hours + 8% of

primary test hoursprimary test hours = 2953 +8% of 2953 = 3189 hours= 2953 +8% of 2953 = 3189 hours

For this case study, the breakup is an For this case study, the breakup is an follows:follows:PhasesPhases Time required (hours)Time required (hours)

Plan (10%)Plan (10%) 319319

Development (40%)Development (40%) 12761276

Execute (45%)Execute (45%) 14351435

Management (5%)Management (5%) 159159

Path analysis is a process specially developed for turning Path analysis is a process specially developed for turning use cases into test cases. This is a tool for the testers, who use cases into test cases. This is a tool for the testers, who are required to test applications that have use cases as are required to test applications that have use cases as required to test applications that have use cases as required to test applications that have use cases as requirement. requirement.

Each use case path is derived from a possible combination Each use case path is derived from a possible combination of following use case elements:of following use case elements:

Basic CourseBasic Course Alternate CourseAlternate Course Exception CourseException Course Extends and Uses Relationships Extends and Uses Relationships

The advantages can be summarized asThe advantages can be summarized as Better coverage through scientific analysisBetter coverage through scientific analysis Both positive and negative test cases can be Both positive and negative test cases can be

easily identified through path analysiseasily identified through path analysis Easier coupling of path and dataEasier coupling of path and data Reduction of test cases.Reduction of test cases. Better management of testing risksBetter management of testing risks Extremely effective in identifying and correction Extremely effective in identifying and correction

use case errors.use case errors.

The path analysis process consists of The path analysis process consists of following four major steps:-following four major steps:-

Draw flow diagram for the use caseDraw flow diagram for the use case Determine all possible pathsDetermine all possible paths Analyze and rank the pathsAnalyze and rank the paths Decide which paths to use for Decide which paths to use for

testing.testing.

Use case is a textual document and hence cannot Use case is a textual document and hence cannot be easily analyzed. The first task is to convert this be easily analyzed. The first task is to convert this into a graphical format called flow diagram. Let us into a graphical format called flow diagram. Let us define some conceptsdefine some concepts

Flow Diagram Flow Diagram : Graphical depiction of the use case : Graphical depiction of the use case

Node Node : the point where flow deviates or converges: the point where flow deviates or converges Branch Branch : connection between two nodes. Each : connection between two nodes. Each

branch has a direction attached to it. branch has a direction attached to it. Node is also where more than one branch meets.Node is also where more than one branch meets.

Beginning from the start each time, list all possible ways Beginning from the start each time, list all possible ways

to reach the end, keeping the direction of flow in mind.to reach the end, keeping the direction of flow in mind. Path ID designation Path ID designation : each path should be suitably : each path should be suitably

designated as P1, P2 etc.designated as P1, P2 etc. Path Name Path Name : this is the sequence of branch numbers : this is the sequence of branch numbers

that the path comprises of. For the example given above that the path comprises of. For the example given above

path ID P2 has a path name of 2, 3, 4, 5.path ID P2 has a path name of 2, 3, 4, 5. Path Description Path Description : this is a textual description of the : this is a textual description of the

complete sequence of user actions and system responses complete sequence of user actions and system responses

taking place in that path. taking place in that path.

For simple use cases, which have about 10 separate paths, For simple use cases, which have about 10 separate paths,

it is straight forward to select all paths for testing and hence it is straight forward to select all paths for testing and hence

this step can be skipped. However for some complex use this step can be skipped. However for some complex use

cases, the number of possible paths can easily exceed 50, cases, the number of possible paths can easily exceed 50,

in which case it may be necessary to select only limited in which case it may be necessary to select only limited

paths for testing.paths for testing. FrequencyFrequency : this attribute can have a value of 1 to 10, with : this attribute can have a value of 1 to 10, with

1 being least frequent and 10 most frequent. 1 being least frequent and 10 most frequent. Criticality Criticality : this attribute describes how critical the failure : this attribute describes how critical the failure

of this path could be with 1 being least and 10 being most.of this path could be with 1 being least and 10 being most.

In order to have adequate test coverage, and In order to have adequate test coverage, and minimize the risk of not testing while balancing the minimize the risk of not testing while balancing the resources there is a need to follow some guidelines. resources there is a need to follow some guidelines. They are as follows:They are as follows:

Always select flow for testing asAlways select flow for testing as It is a critical functionalityIt is a critical functionality If basic flow fails; there may not be much of a If basic flow fails; there may not be much of a

point in testing other pathspoint in testing other paths Basic flow should be included in sanity test suites Basic flow should be included in sanity test suites

and also in Acceptance testingand also in Acceptance testing

Some other paths are too important to be Some other paths are too important to be ignored from functionality point of view. ignored from functionality point of view. These are generally obvious from the table These are generally obvious from the table of paths.of paths.

The path factor should be used for The path factor should be used for selecting a path among several possibilities selecting a path among several possibilities that do not meet the above criteria.that do not meet the above criteria.

The aim here is to select a minimum set of The aim here is to select a minimum set of paths which would adequately test the paths which would adequately test the critical functionality while ensuring that all critical functionality while ensuring that all branches have been included in at least branches have been included in at least one path.one path.

The table of paths provides us with adequate The table of paths provides us with adequate information for testing a use case. information for testing a use case.

The path description is in fact a summary description The path description is in fact a summary description of the test case itself and even a tester with very little of the test case itself and even a tester with very little experience can write test cases from that description.experience can write test cases from that description.

Before proceeding further, let me introduce one more Before proceeding further, let me introduce one more concept here called test scenario.concept here called test scenario.

Any paths with a specific test data becomes a test Any paths with a specific test data becomes a test scenario. Each path is generally associated with scenario. Each path is generally associated with number of test scenarios, to adequately test the data number of test scenarios, to adequately test the data width.width.

In many opinion, one of the most difficult and critical activities in IT In many opinion, one of the most difficult and critical activities in IT is the estimation process. I believe that is occurs because when we is the estimation process. I believe that is occurs because when we say that one project will be accomplished in a such time by a such say that one project will be accomplished in a such time by a such cost, it must happen. If it does not happen, several things may cost, it must happen. If it does not happen, several things may follow : from peers comments and senior management warnings to follow : from peers comments and senior management warnings to being fired depending on the reasons and seriousness of the failure.being fired depending on the reasons and seriousness of the failure.

Before even thinking of moving to systems test at my organization, Before even thinking of moving to systems test at my organization, I always heard from the development team, were: the size of the I always heard from the development team, were: the size of the development team and the number of working days needed for development team and the number of working days needed for building a solution before starting systems tests.building a solution before starting systems tests.

To exemplify how the process worked, if one development team To exemplify how the process worked, if one development team said that to deliver a solution for systems testing it would need 4 said that to deliver a solution for systems testing it would need 4 engineers and 66 working days then, the systems test would need 2 engineers and 66 working days then, the systems test would need 2 engineers and 21 working days. engineers and 21 working days.

CHAPTER-11CHAPTER-11TESTING YOUR WEBSITES-TESTING YOUR WEBSITES-

FUNCTIONAL ANDFUNCTIONAL ANDNON-FUNCTIONAL TESTINGNON-FUNCTIONAL TESTING

AbstractAbstract IntroductionIntroduction Methodology Methodology

Today everyone depends upon websites Today everyone depends upon websites for business, education and trading for business, education and trading purpose. Websites are related to the purpose. Websites are related to the internet. It is believed that no work is internet. It is believed that no work is possible without internet today. There possible without internet today. There are so many types of users connected to are so many types of users connected to the websites who need different type of the websites who need different type of information. So, websites should respond information. So, websites should respond according to the user requirements.according to the user requirements.

The client end of the system is The client end of the system is represented by a browser, which represented by a browser, which connect to the website server via the connect to the website server via the internet. The centerpiece of all web internet. The centerpiece of all web applications is a relational database applications is a relational database which stored dynamic contents. A which stored dynamic contents. A transaction server controls the transaction server controls the interaction between the database and interaction between the database and other servers. other servers.

Typical software projectTypical software project Web application projectWeb application projectGathering user requirements Gathering user requirements ::What are we going to build? How What are we going to build? How does it compare to products does it compare to products currently available ?currently available ?

Gathering user requirements:Gathering user requirements:What services are we going to What services are we going to offer our customers? What is the offer our customers? What is the best user interface best user interface

Planning Planning ::How long will it take our available How long will it take our available resources to build this product? resources to build this product? How will we test product How will we test product

PlanningPlanningWe need to get this product out We need to get this product out now.now.Purely driven by available time Purely driven by available time window and resources.window and resources.

Analysis and design.Analysis and design.What technologies should we What technologies should we use? Any design patterns we use? Any design patterns we should follow? What kind of should follow? What kind of patterns we should follow?patterns we should follow?

Analysis and design.Analysis and design.How should the site look? What How should the site look? What kinds of logos and graphics will kinds of logos and graphics will we use? we use?

Implementation Implementation Let us decide on the sequence of Let us decide on the sequence of building block hat will optimize building block hat will optimize our integration of a series of our integration of a series of builds.builds.

ImplementationImplementationLet us put in the framework and Let us put in the framework and hang some of the key features. hang some of the key features. We can then show it as a demo or We can then show it as a demo or pilot site to our customers.pilot site to our customers.

Non-functional testingNon-functional testing

(or white box testing)(or white box testing)

This types of test includes-This types of test includes- The operating system platforms used.The operating system platforms used. The type of network connectionThe type of network connection Internet service provider type.Internet service provider type. Browser used (including version)Browser used (including version) The real work for this type of test is ensuring The real work for this type of test is ensuring

that the requirements and assumption are that the requirements and assumption are understood by the development team, and understood by the development team, and that test environment with those choice are that test environment with those choice are put in place to properly test it.put in place to properly test it.

For usability testing, there are standards and For usability testing, there are standards and guidelines that have been established throughout guidelines that have been established throughout the industry. The end users can blindly accept the industry. The end users can blindly accept these sites since the standards are being these sites since the standards are being followed. But the designer shouldn’t completely followed. But the designer shouldn’t completely rely on these standards. While following these rely on these standards. While following these standards and guidelines during the making of standards and guidelines during the making of the websites, he should also consider the learn the websites, he should also consider the learn ability, understandability, and operability features ability, understandability, and operability features so that the user can easily use the website. so that the user can easily use the website.

Performance testing involves testing a program for Performance testing involves testing a program for timely responses .timely responses .

Typical the software processes that directly influence Typical the software processes that directly influence the overall performance of the system.the overall performance of the system.

For each of the identified processes, identify only the For each of the identified processes, identify only the essential input parameters that influence system essential input parameters that influence system performance.performance.

Created usage scenarios by determining realistic values Created usage scenarios by determining realistic values for the parameters based on past use. for the parameters based on past use.

If there is no historical data to base the parameter If there is no historical data to base the parameter values on, use estimates based on requirements, an values on, use estimates based on requirements, an earlier version, or similar systems.earlier version, or similar systems.

The term “scalability” can be defined as a web The term “scalability” can be defined as a web application’s ability to sustain its required application’s ability to sustain its required number of simultaneous users and transactions. number of simultaneous users and transactions. While maintaining adequate response time to its While maintaining adequate response time to its end users. end users.

When testing scalability, configuration of the When testing scalability, configuration of the server under test is critical. All logging level, server under test is critical. All logging level, server timeouts, etc. need to be configured. server timeouts, etc. need to be configured.

In order to test scalability, the web traffic loads In order to test scalability, the web traffic loads must be determined to know what the threshold must be determined to know what the threshold requirement for scalability should be.requirement for scalability should be.

Probably the most critical criterion for a web application Probably the most critical criterion for a web application is that of security.is that of security.

The test cases for security testing can be derived as The test cases for security testing can be derived as follows:follows: The web server should be setup so that unauthorized The web server should be setup so that unauthorized

users cannot browse directories users cannot browse directories Early in the project, encourage developers to use the Early in the project, encourage developers to use the

POST command wherever possible POST command wherever possible When testing, check URLs to ensure that there are no When testing, check URLs to ensure that there are no

“information leaks” due to sensitive information being “information leaks” due to sensitive information being placed in the URL while using a GET command. placed in the URL while using a GET command.

Website should have backup or redundant Website should have backup or redundant server to which the traffic is rerouted when server to which the traffic is rerouted when the primary server fails. And the rerouting the primary server fails. And the rerouting mechanism for the data must be tested. If a mechanism for the data must be tested. If a user finds your service unavailable for an user finds your service unavailable for an excessive period of time, the user will switch excessive period of time, the user will switch over or browse the competitor’s website. If the over or browse the competitor’s website. If the site can’t recover quickly then inform the user site can’t recover quickly then inform the user when the site will be available and functional.when the site will be available and functional.

Reliability testing is done to evaluate Reliability testing is done to evaluate the product’s ability to perform its the product’s ability to perform its requires functions and give response requires functions and give response under stated conditions for a under stated conditions for a specified period of timespecified period of time

This type of test covers the object and code that executes This type of test covers the object and code that executes within the browser, but does not execute the server based within the browser, but does not execute the server based component. For example, java script and VB script code component. For example, java script and VB script code within HTML that does rollovers, and other special effects.within HTML that does rollovers, and other special effects.

The test cases for web browser testing can be derived as The test cases for web browser testing can be derived as follows:follows: If all mandatory fields on the form are not filled then it If all mandatory fields on the form are not filled then it

will display a message on pressing a submit button.will display a message on pressing a submit button. Hidden passwordHidden password It should check the limits of all the fields given in the It should check the limits of all the fields given in the

form.form.

In this testing, test cases are designed to In this testing, test cases are designed to confirm that information entered by the confirm that information entered by the user at the web page level makes it to the user at the web page level makes it to the database, in the proper way, and that database, in the proper way, and that when database calls are made from the when database calls are made from the page, the proper data is returned to the page, the proper data is returned to the user.user.

It is clear from this chapter that for the failure It is clear from this chapter that for the failure free operation of a website we must follow free operation of a website we must follow both non-functional and functional testing both non-functional and functional testing methods. With these methods one can test methods. With these methods one can test the performance. Security, reliability, user the performance. Security, reliability, user interfaces etc. which are the critical issue interfaces etc. which are the critical issue related to the website. Web testing is a related to the website. Web testing is a challenging exercise and by following the challenging exercise and by following the methods described in this chapter. Some of methods described in this chapter. Some of those challenges may be mitigated.those challenges may be mitigated.

CHAPTER-12CHAPTER-12REGRESSION TESTING OF A REGRESSION TESTING OF A

RELATIONAL DATABASERELATIONAL DATABASE

IntroductionIntroduction Why test an RDBMS?Why test an RDBMS? What should we test?What should we test? When should we test?When should we test? How to test?How to test? Who should test?Who should test?

Relational databases are tabular databases that Relational databases are tabular databases that are used to store target related data that can be are used to store target related data that can be easily reorganized and queried. They are used in easily reorganized and queried. They are used in many applications by millions of end users. many applications by millions of end users. Testing database involves three aspects:Testing database involves three aspects: Testing of the actual dataTesting of the actual data Database integrityDatabase integrity Functionality testing of database application.Functionality testing of database application.

Extensive testing of RDBMS is done due to the Extensive testing of RDBMS is done due to the following reasons:following reasons:

Quality data is an important assetQuality data is an important asset Target related business functions are implemented Target related business functions are implemented

in RDBMSin RDBMS Present approaches of RDBMS testing are inefficient Present approaches of RDBMS testing are inefficient Testing provides a concrete test suite to regression Testing provides a concrete test suite to regression

test an RDBMStest an RDBMS Verification of all modificationsVerification of all modifications

We will be discussing both black-box and white-box testing We will be discussing both black-box and white-box testing approaches on relational database. Black box testing involve:approaches on relational database. Black box testing involve:

I/O validation I/O validation : regression testing will help us in validating : regression testing will help us in validating incoming data-values; outgoing data values from queues, incoming data-values; outgoing data values from queues, stored-function and views.stored-function and views.

Error handling Error handling : Regression testing of an RDBMS allows us to : Regression testing of an RDBMS allows us to test quasi-nulls that is, empty strings are not allowed.test quasi-nulls that is, empty strings are not allowed.

Table Structure can be easily validated : Table Structure can be easily validated : we can validate we can validate the relationship between the rows in different tables.the relationship between the rows in different tables.

Testing of stored data Testing of stored data : data stored in the form of tables can : data stored in the form of tables can be tested extensively by regression testing.be tested extensively by regression testing.

Testing of modified data : updating the tables might introduce Testing of modified data : updating the tables might introduce new errors which can be easily detected by regression testing.new errors which can be easily detected by regression testing.

Testing databases involves initial testing of Testing databases involves initial testing of database and database refactoring. This strategy database and database refactoring. This strategy can be applied concurrently to both the application can be applied concurrently to both the application code and the database schema. code and the database schema.

Step by step approach is as follows:Step by step approach is as follows: Step 1- a test is added for just enough code to failStep 1- a test is added for just enough code to fail Step 2- test are then run to make sure that the new Step 2- test are then run to make sure that the new

tests does in fact fail.tests does in fact fail. Step 3- functional code is then updated so that it Step 3- functional code is then updated so that it

passes the new testspasses the new tests Step 4- tests are run again.Step 4- tests are run again. Step 5- if tests fail, update functional code again Step 5- if tests fail, update functional code again

and retest.and retest. Step 6- once the tests pass, next step is start again.Step 6- once the tests pass, next step is start again.

Database testing involves the need of a copy Database testing involves the need of a copy of databases which are called sandboxes.of databases which are called sandboxes.

Functionality sandboxFunctionality sandbox: In this we check the : In this we check the new functionality of database and reactor the new functionality of database and reactor the existing functionality. Then we pass the existing functionality. Then we pass the tested sandbox to the next stage which is tested sandbox to the next stage which is integrated sandbox.integrated sandbox.

Integrated sandboxIntegrated sandbox: In this we integrate all : In this we integrate all the sandboxes and then test the system.the sandboxes and then test the system.

Step 1- setting up of the test cases : set up the Step 1- setting up of the test cases : set up the database to a known state.database to a known state. External test data.External test data. Test scripts.Test scripts. Test data with known values.Test data with known values. Real world data.Real world data.

Step 2- running the test cases : the test cases Step 2- running the test cases : the test cases are then run. The running of the databases are then run. The running of the databases test cases is analogous to usual development test cases is analogous to usual development testing. testing.

Advantages : It is simple and no programming Advantages : It is simple and no programming skill is required. It not only addresses the skill is required. It not only addresses the functionality of stored procedures, rules, functionality of stored procedures, rules, triggers and data integrity but also the triggers and data integrity but also the functionality of web application as a whole.functionality of web application as a whole.

Disadvantages : Sometimes the results sent Disadvantages : Sometimes the results sent to the browser after test case execution do to the browser after test case execution do not necessarily indicate that the data itself is not necessarily indicate that the data itself is properly written to a record in the table.properly written to a record in the table.

Preparation for database testingPreparation for database testing Generate a list of database tables, stored procedures, triggers, Generate a list of database tables, stored procedures, triggers,

defaults, rules and so on. defaults, rules and so on. The points which we can follow are :The points which we can follow are : Generation data schemata for tables. Analyzing the schema will Generation data schemata for tables. Analyzing the schema will

help us determine:help us determine: Can a certain field value be null?Can a certain field value be null? What are the allowed or disallowed values?What are the allowed or disallowed values? Is the value dependent upon values in another table?Is the value dependent upon values in another table? Will the values of this field be in the look up table?Will the values of this field be in the look up table? What are primary key and foreign key relationship among What are primary key and foreign key relationship among

tables? tables?

Main people responsible for doing Main people responsible for doing database testing are application database testing are application developers and agile database developers and agile database administrators. administrators.

The database testers are also responsible The database testers are also responsible for procuring database testing tools for the for procuring database testing tools for the organization. organization.

Some of the dataset testing case tools are:Some of the dataset testing case tools are:

Category of Category of testingtesting

MeaningMeaning ExamplesExamples

UNIT TESTING TOOLSUNIT TESTING TOOLS Tools which enables Tools which enables you to regression test you to regression test your database.your database.

DBUnit, SQL, UnitDBUnit, SQL, Unit

LOAD TESTING LOAD TESTING TOOLSTOOLS

These tools will test These tools will test whether our system whether our system will be able to stand will be able to stand high conditions of high conditions of load.load.

Mercury interactive., Mercury interactive., rational suite, test rational suite, test studio. studio.

TEST DATA TEST DATA GENERATORGENERATOR

They help to They help to generate large generate large amounts of data for amounts of data for stress and load stress and load testing.testing.

Data factory, turbo Data factory, turbo data.data.

In this chapter, we have studied about the In this chapter, we have studied about the regression testing of relational databases. regression testing of relational databases. We have also done the black box testing of We have also done the black box testing of a database code example.a database code example.

Some popular websites-Some popular websites- www.stickyminds.com www.testingcenter.com www.owasp.org www.ieee.org www.iiste.org www.ijcsi.orgwww.ijcsi.org

BY: BY: ER. RAJIV CHOPRAER. RAJIV CHOPRA

M. TECH.(IT); MIT; BE(CSE)M. TECH.(IT); MIT; BE(CSE)

CREZONE CERTIFIED [CCEH]CREZONE CERTIFIED [CCEH]

http://www.stickyminds.com/

http://www.testingcenter.com/

http://www.owasp.org/

http://www.ieee.org/

http://www.iiste.org/

Software Testing

Documents

Transcript of Software Testing