Post on 22-Dec-2015
2
Software Testing StrategyStrategy
Integration of software test case design Integration of software test case design methods into a well-planned series of stepsmethods into a well-planned series of steps
Successful software constructionSuccessful software construction Provides a road map that describes Provides a road map that describes
The steps to be conducted as part of testingThe steps to be conducted as part of testing How much effort, time and resources will be How much effort, time and resources will be
requiredrequired Must incorporate test planning, test case Must incorporate test planning, test case
design, test execution, and resultant data design, test execution, and resultant data collection and evaluationcollection and evaluation
3
Software Testing StrategyStrategy
Should be flexible enough to promote a Should be flexible enough to promote a customized testing approachcustomized testing approach
Must be rigid enough to promote reasonable Must be rigid enough to promote reasonable planning and management tracking as the planning and management tracking as the project progressesproject progresses
Different test methods are beginning to cluster Different test methods are beginning to cluster themselves into distinct approaches and themselves into distinct approaches and philosophiesphilosophies
4
Software Testing What is it?What is it?
To uncover errorsTo uncover errors How do we conduct the test?How do we conduct the test?
Do we develop a formal plan?Do we develop a formal plan? Should we test the entire program as a whole or run Should we test the entire program as a whole or run
tests on a small part of it?tests on a small part of it? Should we rerun tests we have already conducted as Should we rerun tests we have already conducted as
we add new components to a large system?we add new components to a large system? When should we involve customer?When should we involve customer? These questions must be answered when we develop a These questions must be answered when we develop a
software testing strategy.software testing strategy.
5
Software Testing Who does it?Who does it?
The strategy is developed by the project The strategy is developed by the project manager, software engineers, and testing manager, software engineers, and testing specialists.specialists.
Why is it importantWhy is it important Testing often accounts for more project effort Testing often accounts for more project effort
than any other software engineering activitythan any other software engineering activity If conducted haphazardlyIf conducted haphazardly
time is wastedtime is wasted Unnecessary effort is expendedUnnecessary effort is expended Errors sneak through undectedErrors sneak through undected
6
Software Testing What are the steps?What are the steps?
Begins in the small and progresses to the Begins in the small and progresses to the largelarge
Focus on a single component or a small group of Focus on a single component or a small group of related componentsrelated components
After components are tested they may be integrated After components are tested they may be integrated until the complete system is constructeduntil the complete system is constructed
As errors are uncovered, they must be diagnosed As errors are uncovered, they must be diagnosed and corrected using a process that is called and corrected using a process that is called debuggingdebugging
7
Software Testing What is the work product?What is the work product?
A test specification documentA test specification document Defining a plan that describes overall strategyDefining a plan that describes overall strategy A procedure that defines specific testing steps and A procedure that defines specific testing steps and
the tests that will be conductedthe tests that will be conducted
How do I ensure that I’ve done it right?How do I ensure that I’ve done it right? By reviewing the Test specification prior to By reviewing the Test specification prior to
testingtesting Assess the completeness of test cases and Assess the completeness of test cases and
testing taskstesting tasks
8
Software Testing A Strategic ApproachA Strategic Approach
Testing is a set of activities that can be planned Testing is a set of activities that can be planned in advance and conducted systematicallyin advance and conducted systematically
Conduct effective formal technical reviewsConduct effective formal technical reviews Begin at the component level and work “outward” Begin at the component level and work “outward”
toward the integration of the entire computer-based toward the integration of the entire computer-based systemsystem
Different testing techniques at different points in timeDifferent testing techniques at different points in time Usually conducted by the software developer and for Usually conducted by the software developer and for
large projects by an independent test grouplarge projects by an independent test group Testing and debugging are different activities, but Testing and debugging are different activities, but
debugging must be accommodated in any testing debugging must be accommodated in any testing strategystrategy
9
Software Testing Verification and ValidationVerification and Validation
Software testing is often referred to as V&VSoftware testing is often referred to as V&V Verification refers to the set of activities that Verification refers to the set of activities that
ensures that software correctly implements a ensures that software correctly implements a specific functionspecific function
Are we building the product right?Are we building the product right?
Validation is a different set of activities that ensure Validation is a different set of activities that ensure that the software that has been built is traceable to that the software that has been built is traceable to customer requirementscustomer requirements
Are we building the right product?Are we building the right product?
10
Software Testing Verification and ValidationVerification and Validation
Encompass a wide array of Software Quality Encompass a wide array of Software Quality Assurance (SQA) activities includingAssurance (SQA) activities including
Formal technical reviewsFormal technical reviews Quality and configuration auditsQuality and configuration audits Performance monitoringPerformance monitoring SimulationSimulation Feasibility studyFeasibility study Documentation reviewDocumentation review Database reviewDatabase review Algorithm analysisAlgorithm analysis Development, usability, installation testingDevelopment, usability, installation testing
11
Software Testing Organizing for Software TestingOrganizing for Software Testing
Psychological point of viewPsychological point of view The software engineer analyzes, models, and then The software engineer analyzes, models, and then
creates a computer program and its documentationcreates a computer program and its documentation From the software engineers perspective testing is From the software engineers perspective testing is
destructive as it tries to break the thing that SE has destructive as it tries to break the thing that SE has builtbuilt
There are often a number of misconceptionsThere are often a number of misconceptions The developer of the software should no do testThe developer of the software should no do test Software should be tossed over the wall to strangers Software should be tossed over the wall to strangers
who will test it mercilesslywho will test it mercilessly Testers get involved with the project only when the Testers get involved with the project only when the
testing steps are about to begintesting steps are about to begin
12
Software Testing Organizing for Software TestingOrganizing for Software Testing
The role of the independent test groupThe role of the independent test group Remove the inherent problems associated with Remove the inherent problems associated with
letting the builder test the thing that has been builtletting the builder test the thing that has been built Removes the conflict of interestRemoves the conflict of interest The software engineer, however, does turn the The software engineer, however, does turn the
program over to the ITG and walk away; both have program over to the ITG and walk away; both have to work closely throughout the projectto work closely throughout the project
The developer must be available to correct The developer must be available to correct uncovered errorsuncovered errors
13
Software Testing Strategy for conventional software Strategy for conventional software
architecturearchitecture Unit TestingUnit Testing
Concentrates on each unit (i.e. component) of the Concentrates on each unit (i.e. component) of the software as implemented in the source codesoftware as implemented in the source code
Integration TestingIntegration Testing Focus is on design and the construction of the software Focus is on design and the construction of the software
architecture architecture Validation TestingValidation Testing
Requirements are analysis are validated against the Requirements are analysis are validated against the software that has been constructedsoftware that has been constructed
System TestingSystem Testing Software and other elements are tested as a wholeSoftware and other elements are tested as a whole
14
Software Testing Strategy for Object Oriented ArchitectureStrategy for Object Oriented Architecture
Focus on “testing in the small” and work Focus on “testing in the small” and work outward toward “testing in the large”outward toward “testing in the large”
Testing in the small changes from an individual Testing in the small changes from an individual module to a class encompassing attributes and module to a class encompassing attributes and operations; implies communication and collaborationoperations; implies communication and collaboration
A series of regression tests are run to uncover errors A series of regression tests are run to uncover errors due to communication and collaboration between due to communication and collaboration between classesclasses
15
Software Testing Strategic IssuesStrategic Issues
Specify product requirements in a quantifiable manner Specify product requirements in a quantifiable manner long before testinglong before testing
A good strategy not only looks for errors, but also asses A good strategy not only looks for errors, but also asses other quality characteristics such as other quality characteristics such as
PortabilityPortability MaintainabilityMaintainability UsabilityUsability
State testing objectives explicitlyState testing objectives explicitly The specific objectives of testing should be stated in The specific objectives of testing should be stated in
measurable termsmeasurable terms Test effectivenessTest effectiveness The cost to find and fix defectsThe cost to find and fix defects Test coverageTest coverage
16
Software Testing Strategic IssuesStrategic Issues
Understand the user of the software and Understand the user of the software and develop a profile for eachdevelop a profile for each
Use-cases that describe the interaction scenario for Use-cases that describe the interaction scenario for each class of user can reduce overall testing effort by each class of user can reduce overall testing effort by focusing testing on actual use of the projectfocusing testing on actual use of the project
Build robust software that is designed to test Build robust software that is designed to test itselfitself
Software should be designed in a manner that uses Software should be designed in a manner that uses anti-bugging techniques. Software should be capable anti-bugging techniques. Software should be capable of diagnosing certain classes of errors. The design of diagnosing certain classes of errors. The design should be accommodated automated testing and should be accommodated automated testing and regression testingregression testing
17
Software Testing Strategic IssuesStrategic Issues
Use effective formal technical reviews as a Use effective formal technical reviews as a filter prior to testingfilter prior to testing
Reviews can reduce the amount of testing effort that Reviews can reduce the amount of testing effort that is required to produce high-quality softwareis required to produce high-quality software
Develop a continuous improvement approach Develop a continuous improvement approach The testing strategy should be measured. The The testing strategy should be measured. The
metrics collected during testing should be used as metrics collected during testing should be used as part of a statistical process control approach for part of a statistical process control approach for testingtesting
18
Software Testing Test Strategies for Conventional SoftwareTest Strategies for Conventional Software
There are many strategies for testing software. There are many strategies for testing software. A software team could wait until the system is fully A software team could wait until the system is fully
constructed and then conduct tests on the overall system. constructed and then conduct tests on the overall system.
At the other extreme software engineer can conduct tests At the other extreme software engineer can conduct tests on a daily basis, whenever any part of the system is on a daily basis, whenever any part of the system is constructedconstructed
A testing strategies falls between the two extremesA testing strategies falls between the two extremes Begin with the testing of individual program unitsBegin with the testing of individual program units Moving to tests designed to facilitate the integration of the Moving to tests designed to facilitate the integration of the
unitsunits
19
Software Testing Test Strategies for Conventional SoftwareTest Strategies for Conventional Software
Unit TestingUnit Testing Focuses verification effort on the smallest unit of Focuses verification effort on the smallest unit of
software design – the software component or modulesoftware design – the software component or module Integration TestingIntegration Testing
““If they all work individually, why do you doubt that If they all work individually, why do you doubt that they’ll work when we put them together?”they’ll work when we put them together?”
The problem is “putting them together” – interfacingThe problem is “putting them together” – interfacing Data may be lostData may be lost One module can have an inadvertent, adverse One module can have an inadvertent, adverse
effect on another; sub-function when combinedeffect on another; sub-function when combined Global data structure can present problemsGlobal data structure can present problems
20
Software Testing Test Strategies for Conventional SoftwareTest Strategies for Conventional Software
Regression TestingRegression Testing Each time a new module is added as part of Each time a new module is added as part of
integration testing, the software changesintegration testing, the software changes New data flow paths are establishedNew data flow paths are established New I/O and control logic is invokedNew I/O and control logic is invoked Regression testing is the re-execution of some Regression testing is the re-execution of some
subset of tests that have already been conducted to subset of tests that have already been conducted to ensure that changes have not propagated ensure that changes have not propagated unintended side effectsunintended side effects
Smoke testingSmoke testing An integration testing An integration testing
21
Software Testing Test Strategies for Conventional SoftwareTest Strategies for Conventional Software
Smoke testingSmoke testing An integration testing An integration testing Software components that have been translated into Software components that have been translated into
code are integrated into a “build” – data files, code are integrated into a “build” – data files, libraries, reusable modules, and engineered libraries, reusable modules, and engineered components that are required to implement one or components that are required to implement one or more productmore product
The intent of testing here is to uncover show The intent of testing here is to uncover show stoppers – errors that have the highest likelihood of stoppers – errors that have the highest likelihood of throwing the software project behind schedulethrowing the software project behind schedule
The build is integrated with other builds and the The build is integrated with other builds and the entire product is smoke tested dailyentire product is smoke tested daily
22
Software Testing Test Strategies for OBJECT-ORIENTED Test Strategies for OBJECT-ORIENTED
SOFTWARESOFTWARE Unit TestingUnit Testing
An encapsulated class is the focus of unit testingAn encapsulated class is the focus of unit testing Operations within the class are, however, the smallest Operations within the class are, however, the smallest
unitunit Integration TestingIntegration Testing
Thread based testingThread based testing Integrates a set of classes required to respond to one input Integrates a set of classes required to respond to one input
or event for the systemor event for the system Used based testingUsed based testing
Begins the construction of the system by testing Begins the construction of the system by testing independent classes – use very few if any server classesindependent classes – use very few if any server classes
Testing dependent classes – which use independent classesTesting dependent classes – which use independent classes
23
Software Testing Test Strategies for OBJECT-ORIENTED Test Strategies for OBJECT-ORIENTED
SOFTWARESOFTWARE Integration TestingIntegration Testing
Cluster TestingCluster Testing A cluster of collaborating classes (determined by A cluster of collaborating classes (determined by
examining the CRC) to uncover errors in collaborationexamining the CRC) to uncover errors in collaboration
24
Software Testing Validation TestingValidation Testing
At this level the distinction between conventional and At this level the distinction between conventional and object-oriented software disappearsobject-oriented software disappears
Testing focuses on user visible actions and user Testing focuses on user visible actions and user recognizable output from the systemrecognizable output from the system
Can be defined in many waysCan be defined in many ways A simple definition is that validation succeeds when A simple definition is that validation succeeds when
software functions in a manner that can be reasonably software functions in a manner that can be reasonably expected by the customer.expected by the customer.
Alpha TestingAlpha Testing Conducted at the developer site by end-usersConducted at the developer site by end-users
Beta TestingBeta Testing Conducted at end user sites – the developer is generally Conducted at end user sites – the developer is generally
not presentnot present
25
Software Testing System TestingSystem Testing
Ultimately, software is incorporated with other system Ultimately, software is incorporated with other system elements elements
Hardware, people, informationHardware, people, information A series of system integration and validation tests are A series of system integration and validation tests are
conductedconducted A classic system testing problem – finger pointingA classic system testing problem – finger pointing
Software engineers should anticipate potential interfacing Software engineers should anticipate potential interfacing problemsproblems
Design error handling paths that test all information Design error handling paths that test all information coming from other elements of the systemcoming from other elements of the system
Conduct series of tests to simulate bad dataConduct series of tests to simulate bad data Record the results of testing as evidenceRecord the results of testing as evidence Participate in planning and design of systems tests to Participate in planning and design of systems tests to
ensure that software is adequately testedensure that software is adequately tested
26
Software Testing System TestingSystem Testing
Recovery TestingRecovery Testing Is a system test that forces the software to fail in a Is a system test that forces the software to fail in a
variety of ways and verifies that recovery is properly variety of ways and verifies that recovery is properly performedperformed
If automatic recoveryIf automatic recovery Re-initializationRe-initialization Check-pointing mechanismsCheck-pointing mechanisms Data recoveryData recovery Restart are evaluated for correctnessRestart are evaluated for correctness
If manualIf manual Mean time to repair (MTTR) is evaluatedMean time to repair (MTTR) is evaluated
27
Software Testing Security TestingSecurity Testing
It verifies that protection mechanisms built into a It verifies that protection mechanisms built into a system will, in fact protect it from improper system will, in fact protect it from improper penetrationpenetration
Stress TestingStress Testing It executes a system in a manner that demands It executes a system in a manner that demands
resources in abnormal quantity, frequency, or volumeresources in abnormal quantity, frequency, or volume Performance TestingPerformance Testing
Are often coupled with stress testing and usually Are often coupled with stress testing and usually require both hardware and software instrumentation. require both hardware and software instrumentation.
To measure resource utilizationTo measure resource utilization