MTAT.03.159: Software Testing - utMTAT.03.159 / Lecture 01 / © Dietmar Pfahl 2014 What is software...

MTAT.03.159 / Lecture 01 / © Dietmar Pfahl 2014

MTAT.03.159: Software Testing

Lecture 01:

Introduction to Software

Testing

(Textbook Ch. 1-3) Dietmar Pfahl email: [email protected] Spring 2014


Structure of Lecture 1

• Introduction and Motivation

• Course Information

• Definitions and Principles


Test?


What is software testing?

The process of evaluating a program or a system



• Verifying that the

product is developed

right

• Validating that the

right product is

developed




• Verifying that the

product is developed

right

• Validating that the

right product is

developed

• Find defects

• Confirm quality

(Testing paradox)



Testing Takes Time

System development:

•1/3 planning

•1/6 coding

•1/4 component test

•1/4 system test

[Brooks75]


Testing in the process

See Fig 1.6

Textbook

Requ. Spec.

Design

Unit test

Integration test

System test

Acceptance test

Code


Agile methods increase testing

[http://www.extremeprogramming.org]


Product Lines increase testing

[Pohl05]


Quality attributes – ISO 9126


Test complexity

Example

• 40 variables, 2 levels

-> 240 ≈1012

combinations

• 3 tests/second ->

3000 years of test!


Course Information/Overview

• Level: Course at bachelor's level (in English)

• Credits: 3 ECTS, 2 CP

• Prerequisite:

– Compulsory: MTAT.03.094 Software Engineering (6 ECTS, 4 CP)

– Recommended: MTAT.03.130 Object-oriented Programming (6 ECTS, 4 CP)

• Work load:

– Lectures (incl. practical work): 32 hours – incl. labs and exam

– Independent work: 46 hours

• Assessment:

– 5 Lab Assignment (work in pairs) – 40% of grade (~6 h per lab)

– Project (in groups): Report(s) and presentation (Lab 6) – 20% of grade (~17 h)

– Exam (written) – 40% of grade (~31 h)

• Grade scale: A (90%+), B(80%+), C(70%+), D(60%+), E(50%+), F


Letter Grades

• A - An excellent performance, clearly outstanding. The candidate demonstrates

excellent judgement and a high degree of independent thinking.

• B - A very good performance. The candidate demonstrates sound judgement and

a very good degree of independent thinking.

• C - A good performance in most areas. The candidate demonstrates a reasonable

degree of judgement and independent thinking in the most important areas.

• D - A satisfactory performance, but with significant shortcomings. The candidate

demonstrates a limited degree of judgement and independent thinking.

• E - A performance that meets the minimum criteria, but no more. The candidate

demonstrates a very limited degree of judgement and independent thinking.

• F - A performance that does not meet the minimum academic criteria. The

candidate demonstrates an absence of both judgement and independent thinking.


Course Objectives

• The course addresses the essential

concepts of software quality control and

testing and introduces various testing

strategies and types of testing.

• It will also give an overview of different

software defects, software defect

management, and organizational

aspects of software testing.


Learning Outcomes

• On successful completion of this course, students

will be able to demonstrate knowledge of:

– The role of testing in the software development

process

– Test planning and documentation

– Different types of testing techniques

– Static testing and defect estimation


Goals:

Knowledge & understanding

For a passing grade the student must

• be able to define basic concepts and principles within

software testing

• give an account of the most common techniques for

software testing

• be able to describe the relation between the software

testing process and other processes in the product

lifecycle


Goals: Skills and abilities


• be able to plan the testing of a small software

system

• be able to test a software system based on a

chosen test strategy

• be able to apply a systematic approach for

software testing


Goals:

Judgement and approach


• understand the complexity of the task to test a

software system and be aware of the costs and

the impact of different testing activities during the

development of a software product.

• be able to value and synthesize the information

given in published scientific articles about

software testing.


Lectures (J. Liivi 2-111)

Theory + discussions

Cover the basic parts of software testing

• Lecture 1 (10.04) - Introduction to Software Testing

• Lecture 2 (17.04) - Black-Box Testing Techniques

• Lecture 3 (24.04) - White-Box Testing Techniques

• Lecture X (02.05) - Industry Presentation

• Lecture 4 (08.05) - Static Testing Techniques

• Lecture 5 (15.05) - Lifecycle, Documentation,

Organisation, Tools

• Lecture 6 (22.05) - Metrics and Test Process

Improvement (Test Maturity Model)

Technical

Overview

Technical /

Managerial

Managerial

E

c

o

n

o

m

i

c

s


Lab Sessions (J. Liivi 2-207 / 2-20x)

Preparation, Execution, Report – Work in Pairs

1. Issue reporting (8%) / Bonus: Ad-hoc testing lifecycle (4%)

2. Black-box testing (8%)

3. White-box testing (8%)

4. Static code analysis (8%)

5. Static testing (inspection) and defect estimation (8%)

6. Project presentation (20%)

Send reports via submission button on course wiki before

your next lab starts. Only PDF files will be accepted.


GO TO LABS !!!!

(it’s mandatory)


Project

– Learn about a specific area of software testing

– Collect and summarize research information

– Critical thinking beyond the written information

– Present information in a structured way

– Details are provided on the course wiki


Project – Tasks

1. Form project groups

2. Decide on a subject

3. Find literature

4. Read literature

5. Outline the report

6. Write the report

7. Get help from peers and

course instructor

8. Submit the report

9. Present the report

10. Submit presentation


Project – Schedule


Project – To Do Soon!

– Per Lab Group, form groups of three and fill-in the

Project Group Set-Up table (Google docs link on course wiki)

• the names of the students in your group

– Do this at the latest in your lab next week!!

– Select the topic you decide to work on

– Do this at the latest by Friday, Apr 18!!

– Rule for selection of topic:

• At most two project groups per lab group can pick the

same topic (first come first serve principle)


FORM PROJECT TEAMS!


Final Exam

Written exam (40%)

• Based on textbook, lectures and lab sessions

• Open book

Dates:

• Exam 1: 29-May-2013 10:15-11:45 (room to be announced)

• Exam 2: 09-June-2013 14:15-15:45 (room to be announced)


Literature

– Textbook:

• Ilene Burnstein:

Practical Software

Testing

– Additional reading

materials posted on

course web


Further Information https://courses.cs.ut.ee/2014/SWT/spring/Main/HomePage


Types of Testing

unit

integration

system

efficiency

maintainability

functionality

white box black box

Level of detail

Accessibility

Characteristics

usability

reliability

(module)

portability


Definitions (Ch 2)

• Failure is an event, fault is a state

of the software (code) caused by an error

– Error – human mistake

– Fault / Defect – anomaly in the software (incorrect code)

– Failure – inability to perform its required functions (correctly)

• Debugging / Fault localization – or: localizing,

repairing, re-testing


Fault model (Ch 3)

Defect sources

Lack of education

Poor communication

Oversight

Transcription

Immature process

Impact on software

artifacts

Errors

Faults / Defects

Failures Impact from user’s view

Poor quality software

User dissatisfaction

Fault model

Fig 3.1


Why do faults occur in software? (Ch 3)

• Software is written by humans

– Who know something, but not everything

– Who have skills, but aren’t perfect

– Who don’t usually use rigorous methods

– Who do make mistakes (errors)

• Under increasing pressure to deliver to strict deadlines

– No time to check, assumptions may be wrong

– Systems may be incomplete

• Software is complex, abstract and invisible

– Hard to understand

– Hard to see if it is complete or working correctly

– No one person can fully understand large systems

– Numerous external interfaces and dependencies


Types of defects

Requirements defects

Design defects

Coding defects

Testing defects

See fig 3.2


Definitions (cont)

• A test case consists of:

– A set of inputs

– Execution conditions

– Expected outputs


Definitions (cont)

• A test oracle is the principle or mecha-

nism by which you recognize a problem

• It provides the expected result for a test, e.g.

– Specification document

– Formula

– Computer program

– Person

• Often it is hard to find an oracle

– Even the customer and end user might not be able to

tell which is the correct behaviour


Principle 1 – Purpose of testing

• Testing is the process of exercising a software

component using a selected set of test cases, with

the intent of

– Revealing defects

– Evaluating quality

• “The testing paradox”

Testing can’t show that defects don’t exist

--

The more you test software, the more

immune it becomes to your tests


Principles (cont.)

2: A good test case – When the test objective is to detect defects, then a good test case is one that has high probability of revealing a yet undetected defect(s)

3: Test result – The results should be inspected meticulously

4: Expected output – A test case must contain the expected output


Principles (cont.)

5: Input – Test cases should be developed for both

valid and invalid input conditions

6: Fault content estimation – The probability of the

existence of additional defects in a software

component is proportional to the number of

defects already detected in that component

7: Test organization – Testing should be carried out

by a group that is independent of the

development group


Principles (cont.)

8: Repeatable – Tests must be repeatable and

reusable

9: Planned – Testing should be planned

10: Life cycle – Testing activities should be

integrated into the software life cycle

11: Creative – Testing is a creative and challenging

task


Recommended

Textbook Exercises

• Chapter 1

– 2, 3

• Chapter 2

– 1, 3, 5

• Chapter 3

– 1, 2, 3, 6


Next Week

• Lecture 2:

– Black-Box Testing Techniques

• Lab 1:

– Issue Reporting

– Bonus: Ad-hoc testing lifecycle (BugHunt)

• In addition to do:

– Form project groups, select topic, and send to me

– Read textbook chapters 1-3 (available via OIS)

MTAT.03.159: Software Testing - utMTAT.03.159 / Lecture 01 / © Dietmar Pfahl 2014 What is software...

Documents

Transcript of MTAT.03.159: Software Testing - utMTAT.03.159 / Lecture 01 / © Dietmar Pfahl 2014 What is software...