SOFTWARE ENGINEERING UNIT 6 Ch 13
-
Upload
azhar-shaik -
Category
Education
-
view
286 -
download
0
Transcript of SOFTWARE ENGINEERING UNIT 6 Ch 13
3/18/2009 S.Sreenivasa Rao 1
Chapter 13Chapter 13Software Testing StrategiesSoftware Testing Strategies
ww
w.jntuw
orld.com
ww
w.jntuw
orld.com
ww
w.jw
jobs.net
3/18/2009 S.Sreenivasa Rao 2
Software TestingSoftware Testing
Testing is the process of exercising aTesting is the process of exercising aprogram with the specific intent of findingprogram with the specific intent of findingerrors prior to delivery to the end user.errors prior to delivery to the end user.
ww
w.jntuw
orld.com
ww
w.jntuw
orld.com
ww
w.jw
jobs.net
3/18/2009 S.Sreenivasa Rao
Test strategy provides a road map;Test strategy provides a road map;Steps to be conductedSteps to be conductedWhenWhenHow much effort, time, resourcesHow much effort, time, resources
Test strategyTest strategyTest planningTest planningTest case designTest case designTest executionTest executionResultant data collection and evaluationResultant data collection and evaluation
3
ww
w.jntuw
orld.com
ww
w.jntuw
orld.com
ww
w.jw
jobs.net
3/18/2009 S.Sreenivasa Rao
Verification and ValidationVerification and ValidationV&VV&V
Testing is part of V&VTesting is part of V&V
Verification: Are we building the product right?Verification: Are we building the product right?Software correctly implements a specific Software correctly implements a specific functionfunction
Validation: Are we building the right product?Validation: Are we building the right product?Software is traceable to customer Software is traceable to customer requirementsrequirements 4
ww
w.jntuw
orld.com
ww
w.jntuw
orld.com
ww
w.jw
jobs.net
3/18/2009 S.Sreenivasa Rao 5
What Testing ShowsWhat Testing Showserrorserrors
requirements conformancerequirements conformance
performanceperformance
an indicationan indicationof qualityof quality
ww
w.jntuw
orld.com
ww
w.jntuw
orld.com
ww
w.jw
jobs.net
3/18/2009 S.Sreenivasa Rao 6
Who Tests the Software?Who Tests the Software?
developerdeveloper independent testerindependent tester
Understands the system Understands the system
but, will test "gently"but, will test "gently"and, is driven by "delivery"and, is driven by "delivery"
Must learn about the system,Must learn about the system,but, will attempt to break itbut, will attempt to break itand, is driven by qualityand, is driven by quality
ww
w.jntuw
orld.com
ww
w.jntuw
orld.com
ww
w.jw
jobs.net
3/18/2009 S.Sreenivasa Rao 7
Testing StrategyTesting Strategyunit testunit test integrationintegration
testtest
validationvalidationtesttest
systemsystemtesttest
ww
w.jntuw
orld.com
ww
w.jntuw
orld.com
ww
w.jw
jobs.net
3/18/2009 S.Sreenivasa Rao
Levels of Testing
Client Needs
Requirements
Design
Code
Acceptance Testing
System Testing
Integration Testing
Unit testing
ww
w.jntuw
orld.com
ww
w.jntuw
orld.com
ww
w.jw
jobs.net
3/18/2009 S.Sreenivasa Rao
Requirement Requirement --> high> high--order tests: order tests: validation test, System testvalidation test, System testDesign Design --> Integration test> Integration testCode Code --> unit test> unit test
9
ww
w.jntuw
orld.com
ww
w.jntuw
orld.com
ww
w.jw
jobs.net
3/18/2009 S.Sreenivasa Rao 10
Testing StrategyTesting Strategy
We begin by We begin by ‘‘testingtesting--inin--thethe--smallsmall’’ and move toward and move toward ‘‘testingtesting--inin--thethe--largelarge’’For conventional softwareFor conventional software
The module (component) is our initial focusThe module (component) is our initial focusIntegration of modules followsIntegration of modules follows
For OO softwareFor OO softwareour focus when our focus when ““testing in the smalltesting in the small”” changes from an changes from an individual module (the conventional view) to an OO class individual module (the conventional view) to an OO class
that encompasses attributes and operations and implies that encompasses attributes and operations and implies communication and collaborationcommunication and collaboration
ww
w.jntuw
orld.com
ww
w.jntuw
orld.com
ww
w.jw
jobs.net
3/18/2009 S.Sreenivasa Rao 11
Strategic IssuesStrategic IssuesSpecify product requirements in a quantifiable manner long befoSpecify product requirements in a quantifiable manner long before re testing.testing.State testing objectives explicitly. State testing objectives explicitly. Understand the users of the software and develop a profile for eUnderstand the users of the software and develop a profile for each ach user category.user category.Develop a testing plan that emphasizes Develop a testing plan that emphasizes ““rapid cycle testing.rapid cycle testing.””Build Build ““robustrobust”” software that is designed to test itselfsoftware that is designed to test itselfUse effective formal technical reviews as a filter prior to testUse effective formal technical reviews as a filter prior to testingingConduct formal technical reviews to assess the test strategy andConduct formal technical reviews to assess the test strategy and test test cases themselves. cases themselves. Develop a continuous improvement approach for the testing Develop a continuous improvement approach for the testing process. process.
ww
w.jntuw
orld.com
ww
w.jntuw
orld.com
ww
w.jw
jobs.net
3/18/2009 S.Sreenivasa Rao 12
Unit TestingUnit Testing
modulemoduleto beto be
testedtested
test casestest cases
resultsresults
softwaresoftwareengineerengineer
ww
w.jntuw
orld.com
ww
w.jntuw
orld.com
ww
w.jw
jobs.net
3/18/2009 S.Sreenivasa Rao 13
Unit TestingUnit Testing
interface interface local data structureslocal data structuresboundary conditionsboundary conditionsindependent pathsindependent pathserror handling pathserror handling paths
modulemoduleto beto be
testedtested
test casestest cases
ww
w.jntuw
orld.com
ww
w.jntuw
orld.com
ww
w.jw
jobs.net
3/18/2009 S.Sreenivasa Rao 14
Unit Test EnvironmentUnit Test Environment
ModuleModule
stubstub stubstub
driverdriver
interface interface
local data structureslocal data structures
boundary conditionsboundary conditions
independent pathsindependent paths
error handling pathserror handling paths
RESULTSRESULTS
test casestest cases
ww
w.jntuw
orld.com
ww
w.jntuw
orld.com
ww
w.jw
jobs.net
3/18/2009 S.Sreenivasa Rao 15
Integration Testing StrategiesIntegration Testing Strategies
Options:Options:•• the the ““big bangbig bang”” approachapproach•• an incremental construction strategyan incremental construction strategy
ww
w.jntuw
orld.com
ww
w.jntuw
orld.com
ww
w.jw
jobs.net
3/18/2009 S.Sreenivasa Rao 16
Top Down IntegrationTop Down Integration
top module is tested with top module is tested with stubsstubs
stubs are replaced one at stubs are replaced one at a time, "depth first"a time, "depth first"
as new modules are integrated, as new modules are integrated, some subset of tests is resome subset of tests is re--runrun
AA
BB
CC
DD EE
FF GG
ww
w.jntuw
orld.com
ww
w.jntuw
orld.com
ww
w.jw
jobs.net
3/18/2009 S.Sreenivasa Rao 17
BottomBottom--Up IntegrationUp Integration
drivers are replaced one at a drivers are replaced one at a time, "depth first"time, "depth first"
worker modules are grouped into worker modules are grouped into builds and integratedbuilds and integrated
AA
BB
CC
DD EE
FF GG
clustercluster
ww
w.jntuw
orld.com
ww
w.jntuw
orld.com
ww
w.jw
jobs.net
3/18/2009 S.Sreenivasa Rao 18
Sandwich TestingSandwich Testing
Top modules areTop modules aretested with stubstested with stubs
Worker modules are grouped into Worker modules are grouped into builds and integratedbuilds and integrated
AA
BB
CC
DD EE
FF GG
clustercluster
ww
w.jntuw
orld.com
ww
w.jntuw
orld.com
ww
w.jw
jobs.net
3/18/2009 S.Sreenivasa Rao
Regression testingRegression testing
to ensure that changes do not introduce to ensure that changes do not introduce unintended behavior or additional errors.unintended behavior or additional errors.
In context of an integration test: reIn context of an integration test: re--execution of some subset of testsexecution of some subset of testsAny correction Any correction --> change of software> change of software
19
ww
w.jntuw
orld.com
ww
w.jntuw
orld.com
ww
w.jw
jobs.net
3/18/2009 S.Sreenivasa Rao 20
Smoke TestingSmoke TestingA common approach for creating A common approach for creating ““daily buildsdaily builds”” for product softwarefor product softwareSmoke testing steps:Smoke testing steps:
Software components that have been translated into code are inteSoftware components that have been translated into code are integrated grated into a into a ““build.build.””
A build includes all data files, libraries, reusable modules, anA build includes all data files, libraries, reusable modules, and d engineered components that are required to implement one or moreengineered components that are required to implement one or moreproduct functions.product functions.
A series of tests is designed to expose errors that will keep thA series of tests is designed to expose errors that will keep the build e build from properly performing its function. from properly performing its function.
The intent should be to uncover The intent should be to uncover ““show stoppershow stopper”” errors that have the errors that have the highest likelihood of throwing the software project behind schedhighest likelihood of throwing the software project behind schedule.ule.
The build is integrated with other builds and the entire productThe build is integrated with other builds and the entire product (in its (in its current form) is smoke tested daily. current form) is smoke tested daily.
The integration approach may be top down or bottomThe integration approach may be top down or bottom up.up.
ww
w.jntuw
orld.com
ww
w.jntuw
orld.com
ww
w.jw
jobs.net
3/18/2009 S.Sreenivasa Rao
Identify critical modulesIdentify critical modules
Address several software requirementsAddress several software requirementsHas a high level of control (high in the program Has a high level of control (high in the program structure)structure)Is complex or error proneIs complex or error proneHas definite performance requirementsHas definite performance requirements
21
ww
w.jntuw
orld.com
ww
w.jntuw
orld.com
ww
w.jw
jobs.net
3/18/2009 S.Sreenivasa Rao 22
ObjectObject--Oriented TestingOriented Testing
testing strategy changestesting strategy changesthe concept of the the concept of the ‘‘unitunit’’ broadens due to encapsulationbroadens due to encapsulationintegration focuses on classes and their executionintegration focuses on classes and their executionvalidation uses conventional black box methodsvalidation uses conventional black box methods
test case design draws on conventional methods, but test case design draws on conventional methods, but also encompasses special featuresalso encompasses special features
ww
w.jntuw
orld.com
ww
w.jntuw
orld.com
ww
w.jw
jobs.net
3/18/2009 S.Sreenivasa Rao 23
OOT StrategyOOT Strategy
class testing is the equivalent of unit testingclass testing is the equivalent of unit testingoperations within the class are testedoperations within the class are testedthe state behavior of the class is examinedthe state behavior of the class is examined
integration applied three different strategiesintegration applied three different strategiesthreadthread--based testingbased testing——integrates the set of classes integrates the set of classes required to respond to one input or eventrequired to respond to one input or eventuseuse--based testingbased testing——integrates the set of classes required integrates the set of classes required to respond to one use caseto respond to one use casecluster testingcluster testing——integrates the set of classes required to integrates the set of classes required to demonstrate one collaborationdemonstrate one collaboration
ww
w.jntuw
orld.com
ww
w.jntuw
orld.com
ww
w.jw
jobs.net
3/18/2009 S.Sreenivasa Rao
Validation TestingValidation Testing
Validation succeeds when software functions in a Validation succeeds when software functions in a manner that can be reasonably expected by the manner that can be reasonably expected by the customer.customer.Reasonable expectation are defined in the Software Reasonable expectation are defined in the Software Requirement SpecificationRequirement Specification
FunctionalFunctionalBehavioralBehavioralPerformancePerformanceDocumentationDocumentationUsability Usability ……. .
Alpha/Beta testingAlpha/Beta testingFocus is on customer usageFocus is on customer usage
24
ww
w.jntuw
orld.com
ww
w.jntuw
orld.com
ww
w.jw
jobs.net
3/18/2009 S.Sreenivasa Rao 25
High Order Testing: High Order Testing: System testingSystem testing
System testingSystem testingFocus is on system integrationFocus is on system integrationRecovery testingRecovery testing
forces the software to fail in a variety of ways and forces the software to fail in a variety of ways and verifies that recovery is properly performedverifies that recovery is properly performedAutomatic: reAutomatic: re--initialization, check pointing, data initialization, check pointing, data recovery, restartrecovery, restartBy human intervention ; meanBy human intervention ; mean--timetime--toto--repairrepair
Security testingSecurity testingverifies that protection mechanisms built into a system verifies that protection mechanisms built into a system will, in fact, protect it from improper penetrationwill, in fact, protect it from improper penetration
Stress testingStress testingexecutes a system in a manner that demands resources executes a system in a manner that demands resources in abnormal quantity, frequency, or volumein abnormal quantity, frequency, or volume
Performance TestingPerformance Testingtest the runtest the run--time performance of software within the time performance of software within the context of an integrated systemcontext of an integrated system
ww
w.jntuw
orld.com
ww
w.jntuw
orld.com
ww
w.jw
jobs.net
3/18/2009 S.Sreenivasa Rao 26
Debugging: Debugging: A Diagnostic ProcessA Diagnostic Process
ww
w.jntuw
orld.com
ww
w.jntuw
orld.com
ww
w.jw
jobs.net
3/18/2009 S.Sreenivasa Rao 27
The Debugging ProcessThe Debugging Processtest casestest cases
resultsresults
DebuggingDebugging
suspectedsuspectedcausescauses
identifiedidentifiedcausescauses
correctionscorrections
regressionregressionteststests
new testnew testcasescases
ww
w.jntuw
orld.com
ww
w.jntuw
orld.com
ww
w.jw
jobs.net
3/18/2009 S.Sreenivasa Rao 28
Debugging EffortDebugging Effort
time requiredtime requiredto diagnose theto diagnose thesymptom andsymptom anddetermine thedetermine thecausecause
time requiredtime requiredto correct the errorto correct the errorand conductand conductregression testsregression tests
ww
w.jntuw
orld.com
ww
w.jntuw
orld.com
ww
w.jw
jobs.net
3/18/2009 S.Sreenivasa Rao 29
Symptoms & CausesSymptoms & Causes
symptomsymptomcausecause
symptom and cause may be symptom and cause may be geographically separated geographically separated
symptom may disappear when symptom may disappear when another problem is fixedanother problem is fixed
cause may be due to a cause may be due to a combination of noncombination of non--errors errors
cause may be due to a system cause may be due to a system or compiler erroror compiler error
cause may be due to cause may be due to assumptions that everyone assumptions that everyone believesbelieves
symptom may be intermittentsymptom may be intermittent
ww
w.jntuw
orld.com
ww
w.jntuw
orld.com
ww
w.jw
jobs.net
3/18/2009 S.Sreenivasa Rao
Debugging tacticsDebugging tactics
Brute forceBrute forceDump the memoryDump the memory
BacktrackingBacktrackingStart from presence of error, go backward in the code manuallyStart from presence of error, go backward in the code manually
Cause eliminationCause eliminationCause hypothesis Cause hypothesis ……
30
ww
w.jntuw
orld.com
ww
w.jntuw
orld.com
ww
w.jw
jobs.net
3/18/2009 S.Sreenivasa Rao 31
Consequences of BugsConsequences of Bugs
damagedamage
mildmild annoyingannoying
disturbingdisturbingseriousserious
extremeextremecatastrophiccatastrophic
infectiousinfectious
Bug TypeBug Type
Bug Categories:Bug Categories: functionfunction--related bugs, related bugs, systemsystem--related bugs, data bugs, coding bugs, related bugs, data bugs, coding bugs, design bugs, documentation bugs, standards design bugs, documentation bugs, standards violations, etc.violations, etc.
ww
w.jntuw
orld.com
ww
w.jntuw
orld.com
ww
w.jw
jobs.net
3/18/2009 S.Sreenivasa Rao 32
Debugging: Final ThoughtsDebugging: Final Thoughts
thinkthink about the about the symptom you're seeing.symptom you're seeing.
Use toolsUse tools (e.g., dynamic debugger) to gain (e.g., dynamic debugger) to gain more insight.more insight.
get helpget help from someone else.from someone else.
Be absolutely sure to Be absolutely sure to conduct regression testsconduct regression testswhen you do "fix" the bug.when you do "fix" the bug.
1.1.
2.2.
3.3.
4.4.
ww
w.jntuw
orld.com
ww
w.jntuw
orld.com
ww
w.jw
jobs.net