Software Testing Course

Shmuel Ur

ur@il.ibm.com

8296250

An interview situation

• How do you know that your program work correctly?

• What part of the total development budget should be spent on testing?

• Do you use coverage?

• Can you achieve zero defects?

• Have you ever used a testing tool?

Administrative details

• There will be an optional exercise

• Class slides will be on the class web-site (hopefully by Monday)

• Office hours upon request

Outline

• Foundation of Modern Testing

• Course topics

Foundation of Modern Testing

• Testing Defined– Why do we test– What do we test

• Modern Themes– Why understanding risks is so important– Testing as an integrated engineering process– What is preventive testing

Why do we test?

• Phase 0 - Testing is debugging (1950’s)• Phase 1 - Show the software works

(1970’s)• Phase 2 - Show that the software does

not work (1975’s)• Phase 3 - Assess quality, evaluation

(1980’s)• Phase 4 - Control quality, prevention

(1990’s)

What do we test?

• Unit testing• Component testing• Integration testing• System testing• Regression testing• Acceptance Testing• Usability Testing

More thing we test

• Field Testing

• Stress or capacity testing

• Platform testing

• Performance Testing

• Viral protection testing

Why understanding risks is so important

• Bug importance = f (frequency, correction_cost, instaltion_cost, consequences)

• Bug consequences scale: mild, moderate, annoying, disturbing, serious, very serious, extreme, intolerable, catastrophic, infectious.

• Quantifying risk leads to cost-effective test sets

What is preventive testing?

• Using tests to influence and control software design and development

• Implicit concurrent engineering of testware and software– Preventive testing is built on the observation that

one effective way to specifying something is to describe (in detail) how you would accept (test) the something if someone gave it to you.

• Bill Hetzel

Side-effects are important

• The side-effect of early test preparation can be more important than later results.

• Side-effects– Better communication– Clearer and better understood requirements– better design definition and understanding– more reliable implementations

• Higher quality software!

Key elements of preventive testing

• Tests are used as requirements models

• Testware design leads software design

• Defects are detected earlier or prevented altogether

• Defects are systematically analyzed

• Tester and developers work together

Why is testing so difficult?

• Testing is a challenge because:– Complexity of software products– Nature of software behavior and problems– Large amounts of information required

• Plus Frustration due to– Attitude, Communication, Schedule,

Resources, Documentation, Requirements,– Changes, .....

Foundation of Modern Testing Summary

• Testing is planning, designing, building, executing &maintaining testware

• We test for many reasons• Good testing drives good quality• testing produces a product• Software and testware are strongly co-

dependent

Topics to be covered

• Coverage

• Regression

• Bug Classifications

• Requirement Check List

• Test Specifications

• Code Inspection

Topics to be covered II

• Software Reliability

• Risks and Bug Taxonomy

• Domain Specific Test Generators

• OO Testing

• Metrics and Complexity

Topics to be covered III

• Test Tools

• Formal Verification

• Automatic Test Generators

• Random Test Generators

• Exploratory Testing