Software Testing Slides
-
Upload
sheetal-giriya -
Category
Documents
-
view
146 -
download
1
Transcript of Software Testing Slides
E&R Infosys 1ER/CORP/CRS/SE04/003
Software Testing
2E&R InfosysER/CORP/CRS/SE04/003
Objectives
Quality – a cultureSoftware Testing Life CycleSoftware Testing Techniques Types of testingIntroduction to Test automation and Test toolsTest Planning and Strategizing
3E&R InfosysER/CORP/CRS/SE04/003
Agenda
What is quality?What is testing?Testing life cycleTesting TechniquesTypes of TestingTest AutomationTest Tools
4E&R InfosysER/CORP/CRS/SE04/003
What is Quality?
“The degree to which a system, component or process meets requirements”
or“The degree to which a system, component or process meets customer or user needs or expectations”
-I.E.E.E.
5E&R InfosysER/CORP/CRS/SE04/003
Quality is…
What the customer says it is… Features Cost Time
Not a goal, but a means for continuous improvement
6E&R InfosysER/CORP/CRS/SE04/003
Dimensions of Software Quality
Functionality Completeness Correctness Compatibility
Performance Time Resources
Maintainability Correctability Expandability Testability
7E&R InfosysER/CORP/CRS/SE04/003
Dimensions of Software Quality (contd.)
Portability Hardware independence Interoperability
Reliability Error tolerance Availability
Usability Understandability Learnability Operability Communicativeness
8E&R InfosysER/CORP/CRS/SE04/003
Software Quality Issues
Scope Creep / Churning requirementsFuzzy RequirementsLack of Skilled ResourcesLack of a most suitable processFlaws in DesignIncomplete Test PlanningLate TestingUnreliable Data for TestingImproper Defect Tracking
User or Business needs not MetPoor End-user ExperienceDifficult-to-Maintain SystemsBuild and Release IssuesModules Don’t IntegratePoor Performance under LoadFlaws Discovered Very LateDefect Leakage
Challenges Problem Symptoms
9E&R InfosysER/CORP/CRS/SE04/003
Testing – What Is it?
A process of demonstrating that errors are not present?
ORA way of establishing confidence that a program does what it is supposed to do?
ORA means of achieving an error-free program by finding all errors?
10E&R InfosysER/CORP/CRS/SE04/003
Testing is…
A process of executing a program with the intent of finding errors
A “DESTRUCTIVE”, yet creative process
11E&R InfosysER/CORP/CRS/SE04/003
Why Testing?
Verifies that all requirements are implemented correctly (both for positive and negative conditions)Identifies defects before software deploymentHelps improve quality and reliabilityMakes software predictable in behaviorReduces incompatibility and interoperability issuesHelps marketability and retention of customers
12E&R InfosysER/CORP/CRS/SE04/003
Cost of Ineffective Testing
Time Late Releases Projects need to be reworked or abandoned
Money Budget over-runs Defects are 100 to 1000 times more costly to
find and repair after deploymentQuality
Developers unsure of product quality Products released with undiscovered or
unresolved defects
13E&R InfosysER/CORP/CRS/SE04/003
Testing LifecycleIteration
nDevelopment LifecycleProject Planning
Requirements Capture
Implementation
Build 0 Build 1 Build 2
Test Lifecycle
Plan Test
Analysis & Design
Design Test
Setup Test Bed
Execute TestEvaluate Test
Defect Tracking/Enhancement Request
14E&R InfosysER/CORP/CRS/SE04/003
Testing LifecycleRequirements
Capture
Analysis
Test planning & Scenario Design
Test Case Development
Test Execution
Test Result Analysis
Test Cycle Closure
Defect Fixing Cycle
Defects
New Version
The Scenarios Design and the Test Case Development could normally start in parallel with the Development Cycle.Test Execution Synchs up with the Development Cycle during the functional Testing Phases
15E&R InfosysER/CORP/CRS/SE04/003
Testing Lifecycle
Test requirements identification Validate for testability
Test Planning and Scenario design Develop Test Objectives Identify Test Items Resources and Schedules
Test Case Development Test Case Specification Pre-requisites, post-requisites and Acceptance
criteria
16E&R InfosysER/CORP/CRS/SE04/003
Testing Lifecycle (contd.)
Prepare Test Bed Test Data Preparation Test Environment Setup
Test Execution Run Tests and Validate Results Bug Reporting Bug fixes and retesting
Test Result Analysis Defect Analysis Determining Test Set Coverage and
Effectiveness
17E&R InfosysER/CORP/CRS/SE04/003
Start Testing Early
There are two basic forms of testing Verification Testing Validation Testing
Verification: “Are we building it right?”
Validation: “Are we building the right thing?”
18E&R InfosysER/CORP/CRS/SE04/003
Testing Stages – the V-ModelRequirement Analysis
Functional Specifications
High-Level Design
Detailed Design
Code
User Acceptance Testing
System Testing
Integration Testing
Unit Testing
Verification ValidationReviews
19E&R InfosysER/CORP/CRS/SE04/003
Testing Techniques
Static TestingDynamic TestingWhite Box TestingBlack Box Testing
20E&R InfosysER/CORP/CRS/SE04/003
Testing Techniques (contd.)
Static Testing/Reviews Scope
Requirements FeasibilityTechnical ArchitectureDesignProgram specificationsTest PlansTest Cases and DataUser Documents
21E&R InfosysER/CORP/CRS/SE04/003
Testing Techniques (contd.)
Static Testing Advantages
Capture defects early, so saves rework costChecklist-based approachFocuses on coverageGroup perspectiveHighest probability of finding defects
DisadvantagesTime-consumingCannot test data dependenciesHigh skill levels required
22E&R InfosysER/CORP/CRS/SE04/003
Testing Techniques (contd.)
Dynamic Testing White-box/Glass-box Testing
Unit TestingIntegration/Interface Testing
Black-box TestingSystem TestingAcceptance testing
23E&R InfosysER/CORP/CRS/SE04/003
White-Box/Glass-box Testing
Structural Testing processProgram Logic-driven TestingDesign-based TestingExamines the internal structure of program
24E&R InfosysER/CORP/CRS/SE04/003
White-Box Testing – Advantages
High Code coverage – Exhaustive (thorough) path testingProgram logic is testedInternal boundaries are testedTesting is algorithm specific
25E&R InfosysER/CORP/CRS/SE04/003
Black-box Testing
Specifications-based TestingBusiness Transaction-driven TestingNo concern to internal behavior and program structure
26E&R InfosysER/CORP/CRS/SE04/003
Black-box Testing – Advantages
User’s perspectiveFocus on features, not implementationBig-picture approach
27E&R InfosysER/CORP/CRS/SE04/003
Levels of Testing
Unit TestingIntegration TestingFunction Testing Performance Testing SystemAcceptance Testing TestingInstallation Testing}
E&R Infosys 28
Unit Testing
29E&R InfosysER/CORP/CRS/SE04/003
Testing Stages – the V-modelRequirement Analysis
Functional Specifications
High-Level Design
Detailed Design
Code
User Acceptance Testing
System Testing
Integration Testing
Unit Testing
Verification ValidationReviews
30E&R InfosysER/CORP/CRS/SE04/003
What is Unit Testing?
Testing of individual units or groups of related code is known as Unit Testing Unit tested with a Unit Test Plan
31E&R InfosysER/CORP/CRS/SE04/003
Unit Test Plan should test for:
Standards specified (Ex. GUI related)Field validations, boundary valuesBasic functionality of the component (add/delete/modify/query activities)Code coverage and exception handlingNegative or destructive testingMemory leaksData computation/integrity
32E&R InfosysER/CORP/CRS/SE04/003
Unit Test Plan should test for:
Standards specified (Ex. GUI related)Field validations, boundary valuesBasic functionality of the component (add/delete/modify/query activities)Code coverage and exception handlingNegative or destructive testingMemory leaksData computation/integrity
Usability
Functionality
Performance
33E&R InfosysER/CORP/CRS/SE04/003
Unit Testing – White Box Technique
Activities Identification of Testable units/modules Focus on code and logic Design logic-based Test cases and prepare Test
data Run tests and verify output through path
traversing
34E&R InfosysER/CORP/CRS/SE04/003
Unit Testing – White Box Technique
Logic Coverage method includes: Statement coverage Decision (branch) coverage Condition Coverage Path Coverage
35E&R InfosysER/CORP/CRS/SE04/003
Unit Testing - Black Box Technique
Commonly used: Equivalence partitioning Boundary value analysis Error Guessing
Less commonly used: Cause effect graphing State-transition testing
36E&R InfosysER/CORP/CRS/SE04/003
Equivalence Partitioning
Systematic process of identifying a set of input conditions to be tested
The two distinct steps involved: Identify equivalence classes Identify test cases
37E&R InfosysER/CORP/CRS/SE04/003
Equivalence Partitioning
Equivalence Classes – Valid Data User supplied command Responses to system prompts File names Computational data Physical parameters, bounding values,
initialization values Output data formatting commands Responses to error messages Graphical data (e.g. mouse picks)
38E&R InfosysER/CORP/CRS/SE04/003
Equivalence Partitioning
Equivalence Classes – invalid Data Data outside bounds of the program Physically impossible data Proper value supplied in the wrong place
39E&R InfosysER/CORP/CRS/SE04/003
Boundary Value Analysis
Check edge values of all Output domain User queries Numerical data Mouse picks on menu Output format requests Responses to prompts Command key input
40E&R InfosysER/CORP/CRS/SE04/003
Error Guessing
Ad Hoc method to identify tests likely to expose errors based on experience and intuitionSome areas to guess are:
Empty or Null lists / strings Zero instances / Occurrences Blanks or null characters in strings Negative numbers
41E&R InfosysER/CORP/CRS/SE04/003
Advantage/Disadvantage
Equivalence Partitioning Significantly reduces number of input
conditions to be tested Does not test combinations of input conditions
Boundary Value Analysis Boundary conditions sometimes subtle and
difficult to identify Does not test combinations of input conditions
42E&R InfosysER/CORP/CRS/SE04/003
Exercise
Writing a unit test plan…
43E&R InfosysER/CORP/CRS/SE04/003
Recap
What is qualityDimensions of QualityTesting TechniquesUnit Test
E&R Infosys 44
Integration Testing
45E&R InfosysER/CORP/CRS/SE04/003
Testing Stages – the V-modelRequirement Analysis
Functional Specifications
High-Level Design
Detailed Design
Code
User Acceptance Testing
System Testing
Unit Testing
Integration Testing
Verification ValidationReviews
46E&R InfosysER/CORP/CRS/SE04/003
Integration Testing
Combining and testing multiple components togetherIntegration of modules, programs and functionsTests Internal Program interfacesTests External interfaces for modules
47E&R InfosysER/CORP/CRS/SE04/003
Integration TestingBanking System
CRM HR Financial
Java Based Based on C, C++ VB based
Online banking Loans Account Opening
External interface
Inter module interface
Intra module interface
Unit Module
External System
48E&R InfosysER/CORP/CRS/SE04/003
Approach for testing interfaces Intra-Module Inter-Module External
Test Techniques
White Box Black Box Black Box
Test Environment Setup
Development environment
Isolated development environment
Simulate live environment
Test Approach Check for parameter / procedure called.Database updation
Identify pre-process and approach to Interface.Identify the post-process to validate the interface
Identify external / internal interface and validate against external system
49E&R InfosysER/CORP/CRS/SE04/003
Techniques based on order of Integration
Big-bang Integration (non-incremental)Random Incremental IntegrationTop-Down IntegrationBottom-up Integration
50E&R InfosysER/CORP/CRS/SE04/003
Top-down vs. Bottom-up
Top-down Integration Advantage
Key Interface defects trapped earlier Disadvantage
Core functionality tested late in the cycleBottom-up Integration
AdvantageCore functionality tested early
DisadvantageKey Interface defects trapped late in the cycle
E&R Infosys 51
System Testing
52E&R InfosysER/CORP/CRS/SE04/003
Testing Stages – the V-modelRequirement Analysis
Functional Specifications
High-Level Design
Detailed Design
Code
User Acceptance Testing
Integration Testing
Unit Testing
System Testing
Verification ValidationReviews
53E&R InfosysER/CORP/CRS/SE04/003
System Testing – What is it?
A Black-box Testing techniqueSpecifications-based TestingTypically independent team testingSimulated environment testingLive/Simulated user dataTests the whole systemFunctional and non-functional requirements tested Business transaction-driven testingCompatibility errors uncoveredPerformance limitations uncovered
54E&R InfosysER/CORP/CRS/SE04/003
System Testing – Different Types
Functional testing (Sanity / Regression )Performance and Scalability testingLoad/Stress testingVolume TestingUsability testingConfiguration testingInstallability testingDisaster and Recovery testingReliability / Availability testingStatistical Testing based on Usage Models
55E&R InfosysER/CORP/CRS/SE04/003
Functional Testing – Sanity Test
Very basic minimal number of tests to verify the product for the feature / protocol complianceCould be a subset of regression test used to validate only high level functionalityTypically an initial testing effort to determine if a new software version is performing well enough to accept it for a major testing effortSanity testing will be performed in scenarios when cursory testing is sufficient to prove that the application is functioning properly
56E&R InfosysER/CORP/CRS/SE04/003
Why Sanity Testing?
Regression testing is more time taking and more effort is required whereas Sanity testing is quicker focusing on important functionality of softwareSaves time as only selected high level functionality areas are only tested. Bugs found this way would take almost triple the time when found through regression testingEfficient in scenarios wherein releases comes in pretty often with very few changes
57E&R InfosysER/CORP/CRS/SE04/003
Functional Testing – Regression
Re-execution of one or more tests in subsequent build of the application/product to ensure
Revisiting and testing all prior bug-fixes in response to a new fix/enhancement
Re-testing all programs that might be affected by the fix/enhancement
Hidden Bugs are uncoveredThe baseline for Regression Testing grows with every build
58E&R InfosysER/CORP/CRS/SE04/003
Regression Testing
Should Cover Business Process depending on criticality User friendliness Cross functional dependencies
Activities System appreciation Preparation of Test case repository Automate test cases Execute Regression test suite
59E&R InfosysER/CORP/CRS/SE04/003
Without Regression Testing – Hidden Bugs
Fix
PASS PASS? PASS?
FAIL FAIL PASS
PASS BUG? BUG?
Test Requirement 1
Unknown
Test Requirement 2
Test Requirement 3
New Problem
Hidden Bug
Unknown
Fix
CBA
Software Builds
60E&R InfosysER/CORP/CRS/SE04/003
With Regression Testing – No Hidden Bugs
Fix
PASS PASS PASS
FAIL FAIL PASS
PASS BUG PASS
Test Requirement 1
Test Requirement 2
Test Requirement 3
New Problem
Fix
CBA
Software Builds
Fix
61E&R InfosysER/CORP/CRS/SE04/003
Acceptance Testing
Demonstrates that the system meets mutually agreed Acceptance criteria
Critical Requirements Minimum Performance level Maximum Defect Detection Rate
Typically, a sub-set of System testing
62E&R InfosysER/CORP/CRS/SE04/003
Alpha / Beta Testing
Forms of Acceptance testingTesting in the production environmentAlpha testing is performed by end users within a company but outside development groupBeta testing is performed by a sub-set of actual customers outside the company
63E&R InfosysER/CORP/CRS/SE04/003
Performance Testing
Number of concurrent users accessing at any point in given time System’s performance under high volume of data Stress testing for systems, which are being scaled up to larger environments or implemented for the first time Operational intensive transactions. (Most frequently used transactions) Volume intensive transactions (for both volume and stress testing)
64E&R InfosysER/CORP/CRS/SE04/003
Installation Testing
Basic installationInstallation of various configurationsInstallation on various platformsRegression testing of basic functionality
65E&R InfosysER/CORP/CRS/SE04/003
System TestingA Banking System
CRM HR Financial
Java Based Based on C, C++ VB based
Online banking Loans Account Opening
66E&R InfosysER/CORP/CRS/SE04/003
When do you stop testing?
When time runs out Specific test case design methodsTarget test coverage attainedCertain number of errors found*Error detection rate drops below a threshold*Predicted density of errors left drops below a threshold
Sequence test cases in order of priority
* Infosys Process Capability Baseline
67E&R InfosysER/CORP/CRS/SE04/003
Test Artifacts
Test PlanTest Environment/BedTest CaseTest DataTest ToolsTest ScriptsTest LogBug Reports
68E&R InfosysER/CORP/CRS/SE04/003
Traits of a good Tester
Destructive creativity Detective skills Appreciating the users’ perspective Adapt to and Understand Requirements change A skeptical, but not hostile attitude An eagerness to embrace new technologies
69E&R InfosysER/CORP/CRS/SE04/003
Test Engineers
Role - Execute tests, record test scripts, maintain statistics and metrics, check test data setup, test environment setup, execute re-testsKnowledge - Understanding of the importance of testing, awareness of tools, how to progress against a planSkills - Observation, accuracy, methodical, co-ordination, problem solverExperience - Following instructions, problem reporting and solving, and relevant testing tools
70E&R InfosysER/CORP/CRS/SE04/003
Test Automation
Good candidates: Complex and time-consuming tests Tests requiring a great deal of precision Tests involving many simple, repetitive tests Tests involving many data combinations
Lower priority candidates: One-time only tests Testing peripheral devices Subjective assessment tests (look and feel
based)
71E&R InfosysER/CORP/CRS/SE04/003
Test Automation
Evolvable
Effective
Exemplary
Economic
Manual Test
First Run of Automated Test
Automated Test (After many runs)
72E&R InfosysER/CORP/CRS/SE04/003
Simple Vs Sophisticated Automation
COST
Simple Automation
Sophisticated Automation
Maintenance CostImplementation Cost
E&R Infosys 73
Testing Tools
74E&R InfosysER/CORP/CRS/SE04/003
Types of Testing Tools
The software testing tools broadly of the following type:
Functional / regression Load /Stress: Performance testing under high
load / stress Web Testing: testing the Web Sites Data Generation: Generation of test data
75E&R InfosysER/CORP/CRS/SE04/003
Benefits
Reduces manpower and timeMore coverage within the same testing time.Testing products which are very difficult to test manually. Example, Load TestsCan look inside the software for memory leaks, redundant code etc.
76E&R InfosysER/CORP/CRS/SE04/003
Test Tools (Client Server)Category Tool Vendor
Source Code Testing BoundsChecker NumegaPure Coverage RationalPurify RationalJprobe Sitraka SoftwareATTOLCoverage ATTOL Software
Functional Testing QA Run CompuwareWin Runner MercuryTeam Test RationalQA Center Compuware
Performance Testing OptimizeIT VM Gear
Load Runner Mercury
QA Load Compuware
77E&R InfosysER/CORP/CRS/SE04/003
Test Tools (Web Applications)
Category Tool VendorFunctional Testing Rational Robot Rational
Silk Test Segue SoftwarePerformance Testing Silk Performer Segue Software
Web Load RadviewLink & HTML Testing NetMechanic Monte Sano
Linkbot TetranetSite InspectorDoctor HTML Imageware
78E&R InfosysER/CORP/CRS/SE04/003
Test Tools (Network & Security)
Category Tool Vendor
Security & Communication
SNMP Test Suite Interworking Labs Inc.
Sniffer Pro Network Associates Inc.
Lophtcrack
Saint, Satan, Sara SwTech
TCP Wrappers
TCP Dump
E&R Infosys 79
Going Forward…
80E&R InfosysER/CORP/CRS/SE04/003
Testing LifecycleIteration
nDevelopment LifecycleProject Planning
Requirements Capture
Implementation
Build 0 Build 1 Build 2
Test Lifecycle
Plan Test
Analysis & Design
Design Test
Setup Test Bed
Execute TestEvaluate Test
Defect Tracking/Enhancement Request
81E&R InfosysER/CORP/CRS/SE04/003
Test Planning/Strategizing
Includes – Testing Objectives and Goals Test Strategy/Approach based on customer priorities Test Environment (Hardware, Software, Network,
Communication etc.) Features to test with priority/criticality Test Deliverables Test Procedure – Activities and tools Test Entry and Exit criteria Test Organization and Scheduling Testing Resources and Infrastructure Test Measurements/Metrics
82E&R InfosysER/CORP/CRS/SE04/003
Test Planning / Strategizing
Benefits Sets clear and common objectives Helps prioritize tests Facilitates Technical tasks Helps improve coverage Provides structure to activities Improves communication Streamlines tasks, roles and responsibilities Improves test efficiency Improves test measurability
83E&R InfosysER/CORP/CRS/SE04/003
Optimizing TestingCategorize entire gamut of testing into Sanity / Regression /Performance and StressEfficient Review Process Use Tools to do the testing (both Automation and Simulators )Look out for Automation if it helps ( leverage 24 hours a day ) Use Traceability MatrixTracking and Reporting of bugs and defects – Defining a process Revisit the Test coverage and categorization on an ongoing basisRisk strategy Retest strategy
84E&R InfosysER/CORP/CRS/SE04/003
Recap
Integration TestingSystem TestingTest AutomationTesting Tools
E&R Infosys 85