Software Testing Process, Testing Automation and Software Testing Trends
Chap 1- Introduction to Software Testing
Transcript of Chap 1- Introduction to Software Testing
-
8/3/2019 Chap 1- Introduction to Software Testing
1/38
BITP 3213Software Testing & Quality Assurance
byMadam Nor Haslinda Hj Ismail
Email:[email protected]
Phone: 331 6521Room: Ground Floor FTMKbuilding
Semester 1, 2010/2011 Fakulti Teknologi Maklumat dan Komunikasi
mailto:[email protected]:[email protected] -
8/3/2019 Chap 1- Introduction to Software Testing
2/38
Fakulti Teknologi Maklumat dan Komunikasi
Week 1
Introduction to Software Testing
-
8/3/2019 Chap 1- Introduction to Software Testing
3/38
Fakulti Teknologi Maklumat dan Komunikasi
Human Error Fault Failure
can lead to can lead to
-
8/3/2019 Chap 1- Introduction to Software Testing
4/38
Overview
What is Software Testing? Objectives of Software Testing
General Failure Terminology
Impact of Software Error
Real-life cases Critical Choice in Software Testing
Fundamental of Software Process
Type of Software Testing
Testing and Quality
Career in Software Testing
Fakulti Teknologi Maklumat dan Komunikasi
-
8/3/2019 Chap 1- Introduction to Software Testing
5/38
Software Testing OverviewStudies have observed that the cost of correcting
software bugs has tremendously increased thetotal cost of a software project.
Defect. Fault. Problem. Error. Incident.Variance. Failure. Inconsistency
It is crucial to detect the bugs at an early stage.
Software Testing
Fakulti Teknologi Maklumat dan Komunikasi
-
8/3/2019 Chap 1- Introduction to Software Testing
6/38
When to consider that a softwarehas Bugs?
(The 5 rules)1. The software doesnt do something that the product
specification says.
2. The software does something that the product specification says
it shouldnt do.3. The software does something that the product specification
doesnt mention
4. The software doesnt do that the product specification doesntmention but should
5. The software is difficult to understand, hard to use, or-anything which a tester think that will be viewed as not right bythe end user.
Fakulti Teknologi Maklumat dan Komunikasi
-
8/3/2019 Chap 1- Introduction to Software Testing
7/38
The 5 Rules : Rule 1
The software doesnt do something that the
product specification says.
Example :-
The + key of a calculator should do the addition.But it doesnt do any calculation or instead it
works as subtraction.
Fakulti Teknologi Maklumat dan Komunikasi
-
8/3/2019 Chap 1- Introduction to Software Testing
8/38
The 5 Rules : Rule 2
The software does something that the productspecification says it shouldnt do.
Example:-
The product specification specifies that thecalculator should never crash, lock up or
freeze. Yet, when you press Enter key, itstops responding to any of your next input.
Fakulti Teknologi Maklumat dan Komunikasi
-
8/3/2019 Chap 1- Introduction to Software Testing
9/38
The 5 Rules : Rule 3
The software does something that the productspecification doesnt mention
Example:-It is specified that the calculator should only do
addition, subtraction, multiplication and division.Nevertheless, there is an additional of a squareroot function, which being added by programmerbecause he/she things the function is nice tohave.
Fakulti Teknologi Maklumat dan Komunikasi
-
8/3/2019 Chap 1- Introduction to Software Testing
10/38
The 5 Rules : Rule 4
The software doesnt do that the product
specification doesnt mention but should
Example:-
The calculation of a calculator is wrongly calculatedif the battery is weak. This may not be specifiedin the specification.
Fakulti Teknologi Maklumat dan Komunikasi
-
8/3/2019 Chap 1- Introduction to Software Testing
11/38
The 5 Rules : Rule 5
The software is difficult to understand, hard touse, or- anything which a tester think that willbe viewed as not right by the end user.
Example:-
As a tester, you will be the first person to use thesoftware product. Thus, if the tester feels thereis something not right for whatever reason, thenits consider as a bug.
Fakulti Teknologi Maklumat dan Komunikasi
-
8/3/2019 Chap 1- Introduction to Software Testing
12/38
Why bugs occur in SoftwareDevelopment?
Missing requirements or therequirements/specifications are not welldocumented. e.g. Contradiction, ambiguity,incomplete requirement
Volatile requirementsThe requirements areconstantly changing.
Communication problem among team members.
No team work
Fakulti Teknologi Maklumat dan Komunikasi
-
8/3/2019 Chap 1- Introduction to Software Testing
13/38
What is Software Testing?Definition
Process of exercising software to confirm that itsatisfies requirements and identify differences
between expected and actualresults(Khanapi, 2004)
Testing is the process of executing a programwith intention of finding errors
(Myers, 1979)
Fakulti Teknologi Maklumat dan Komunikasi
-
8/3/2019 Chap 1- Introduction to Software Testing
14/38
Definition
The measurement of software quality
(Hetzel, 1983)
The process of operating a system or component
under specified conditions, observing or
recording the results, and making an evaluationof some aspect of the system or component
(IEEE, 1990)
Fakulti Teknologi Maklumat dan Komunikasi
-
8/3/2019 Chap 1- Introduction to Software Testing
15/38
Definition : Summary
Software testing is a process by which people,
methods, measurements, tools and equipmentare integrated to test a software product.
Fakulti Teknologi Maklumat dan Komunikasi
-
8/3/2019 Chap 1- Introduction to Software Testing
16/38
Process A quality process and planning determines the
success of the test effort
People Software testing is a professional discipline , thus
need trained and skilled people with the right andpositive attitudes.
Methods
There are different methods and techniques to beapplied in testing. E.g White-box, black box,
Fakulti Teknologi Maklumat dan Komunikasi
-
8/3/2019 Chap 1- Introduction to Software Testing
17/38
Measurements
- Used to calculate the quality of a softwareproduct.
Tools and Equipments
- A wide variety of tools to make the testing moreefficient and faster.
- E.g. Using automated tool to re-run the test for
a modified software product. Save time and ableto test thoroughly.
Fakulti Teknologi Maklumat dan Komunikasi
-
8/3/2019 Chap 1- Introduction to Software Testing
18/38
Basic forms of Testing
VerificationAre we building the product right?
Is the process of evaluating a system orcomponent to determine whether the products of
a given development phase satisfy the conditionimposed at the start of the phase.
ValidationAre we building the right product
Is the process of evaluating a system or
component during or at the end of thedevelopment process to determine whether itsatisfied specified requirements.
Fakulti Teknologi Maklumat dan Komunikasi
-
8/3/2019 Chap 1- Introduction to Software Testing
19/38
Objectives of Software Testing
To identify and reveal as many errorsas possiblein the tested software
To produce a quality software product, which the
software can be used, and used successfully To bring the tested software, after correction of
the identified errors and retesting, to anacceptable level of quality
Fakulti Teknologi Maklumat dan Komunikasi
-
8/3/2019 Chap 1- Introduction to Software Testing
20/38
Objectives of Software Testing
To perform the required tests efficiently andeffectively, within budgetary and schedulinglimitations
To compile a record of software errors for use inerror prevention
Fakulti Teknologi Maklumat dan Komunikasi
-
8/3/2019 Chap 1- Introduction to Software Testing
21/38
General Failure Terminology
Error :
An incorrect implementation of a program
Fault :
An incorrect or unexpected behavior of a programin response to a real-world condition resulting fromerrors
Failure : Occurs when a program is prevented from
achieving its mission resulting from faults
Fakulti Teknologi Maklumat dan Komunikasi
-
8/3/2019 Chap 1- Introduction to Software Testing
22/38
Relationship : Error-Fault-Failure
Error made by a person.- Avoidable through training,process improvement, or similar
Leading to a fault in the program
- Detectable in review
Causing a noticeable failure inthe system
- Detectable trough testing
Fakulti Teknologi Maklumat dan Komunikasi
-
8/3/2019 Chap 1- Introduction to Software Testing
23/38
Impact of software error
Case 1
The first Venus probe missed its target in 1979
because a full stop(.) was used in place ofcomma(,) in a Fotran program.
The company lost several hundred million dollarsworth of effort.
Fakulti Teknologi Maklumat dan Komunikasi
-
8/3/2019 Chap 1- Introduction to Software Testing
24/38
Impact of software error
Case 2
In 1984, the southern French Tarn Valley was
flooded due to the failure of an automatic locksystem at Requista where it couldnt recognize
the faulty flood message.
A massive flood which caused catastrophic.
Fakulti Teknologi Maklumat dan Komunikasi
-
8/3/2019 Chap 1- Introduction to Software Testing
25/38
Impact of software error
Case 3
Years before year 2000, most systems were using
the 2-digit date format. (E.g. 1973 as 73). Whenit hits year 2000, problems occurred when thesystems recognized the 00 as year 1900 insteadof 2000.
Estimated that several hundred billions were spent,worldwide to correct the date format.
Fakulti Teknologi Maklumat dan Komunikasi
-
8/3/2019 Chap 1- Introduction to Software Testing
26/38
Critical choices in SoftwareTesting
What, when and how to test?What to test?
Test the right things! ensure the most critical
things are tested and not waste limited testingresources on less important items.
When to test?
Test early! focus on detecting the error closer tothe phase where it introduce. E.g. V modelTesting
Fakulti Teknologi Maklumat dan Komunikasi
-
8/3/2019 Chap 1- Introduction to Software Testing
27/38
Fakulti Teknologi Maklumat dan Komunikasi
How to test?
Test effectively and cost-effectively! Identifythe critical/important area to be tested byconsidering the worthwhile from financialpoint of view.
The more effective the error detection, the greaterthe savings in development and maintenance costsover the life of the product. Several exampleshave indicated that partial testing can yield saving
of 1.5 times its cost; full testing can yields savingsup to 2 times its cost
(Lewis, 1992: p.280)
-
8/3/2019 Chap 1- Introduction to Software Testing
28/38
RISKS and Prioritization
To determine : What to test first?
What to test most?
How thoroughly to test each item?
Example of general risks to be considered : risk of missing important faults
risk of incurring failure costs
risk of releasing untested or under-tested software
risk of losing credibility and market share risk of missing a market window
risk of over-testing, ineffective testing
Fakulti Teknologi Maklumat dan Komunikasi
-
8/3/2019 Chap 1- Introduction to Software Testing
29/38
Fundamental of Test Process
Test planning and control
Test analysis and test design
Test implementation and test execution
Test evaluation of the test exit criteria
Post testing activities : Review
Fakulti Teknologi Maklumat dan Komunikasi
-
8/3/2019 Chap 1- Introduction to Software Testing
30/38
Types of Software Testing
Unit Testing Test particular function or codemodule : micro level testing
Integration Testing Test the integrated/combined
components of code modules, individualapplication, client and server application andetc.
Performance Testing Test the performance of a
software productSystem Testing Test the system as a whole.
Fakulti Teknologi Maklumat dan Komunikasi
-
8/3/2019 Chap 1- Introduction to Software Testing
31/38
Regression Testing Is used to check that thechanges made to the program have notintroduce any new faults in the system
User Acceptance Testing Test the final endproduct by a group of end users to determine thesatisfaction of the customers
Usability TestingUsed to test user-friendliness
of the software product
Fakulti Teknologi Maklumat dan Komunikasi
-
8/3/2019 Chap 1- Introduction to Software Testing
32/38
What makes a good Tester?
They are explorers As a tester, you are notbound to only one specific application/product.You must not afraid to venture into unknownsituations.
They are troubleshooters You must be able tofigure out the reason why something is not
working.
Fakulti Teknologi Maklumat dan Komunikasi
-
8/3/2019 Chap 1- Introduction to Software Testing
33/38
What makes a good Tester?
They are relentless Persistence and a strongsense of mission to systematically locate theweaknesses in a complex structure anddemonstrates it failures. There are bugs that
quickly vanish or difficult to re-create. Ratherthan dismiss it as fluke, they will try everypossible way to find it.
They are creative Imaginative. Able to come outwith different possibleways/scenarios/approaches in finding the bugs.
Fakulti Teknologi Maklumat dan Komunikasi
-
8/3/2019 Chap 1- Introduction to Software Testing
34/38
What makes a good Tester?
They exercise good judgment - A tester need tomake fine decision about what they will test, howlong will it take, and whether the problem theyre
looking at is really a bug.
They are tactful and diplomatic A tester is alwaysa bearers of bad news. He/she has to conveythe bugs to appropriate party. This must be
done in professional and diplomatic manner sothat, the other party will not feel offended orpersonally being attacked. A team player
attitude.Fakulti Teknologi Maklumat dan Komunikasi
-
8/3/2019 Chap 1- Introduction to Software Testing
35/38
Testing and Quality
Testing measures the quality of software product.
Testing can find faults; when they are removed,
software quality (and possibly reliability) isimproved.
Fakulti Teknologi Maklumat dan Komunikasi
-
8/3/2019 Chap 1- Introduction to Software Testing
36/38
Career in Software Testing
Fakulti Teknologi Maklumat dan Komunikasi
Why choose careerin Software Testing?
Increase
productivity
Wide range ofcareer option
Career Path/Options
Test Engineer Test > Test
Manager
QA Lead QAManager
Option byspecialized intesting tool
Key areas to moveahead
Domain
knowledge
ToolsKnowledgeAutomatedTesting
CertificationsE.g. ISTQB,QAI, ASQ
-
8/3/2019 Chap 1- Introduction to Software Testing
37/38
Find the solution! How much more does it cost to fix a bug
found after the product is released that itdoes from the very start of the project?
Give three reason why the productspecification is usually the largest source ofbugs in a software product
Find 2 article from ACM & IEEE Journal
regarding Software Testing from Year 2009until 2011.
Fakulti Teknologi Maklumat dan Komunikasi
-
8/3/2019 Chap 1- Introduction to Software Testing
38/38
Next Chapter
Week 2: Software Testing & SDLC
Fakulti Teknologi Maklumat dan Komunikasi