[email protected] Contents Introduction Requirements Engineering Project Management Software...

PVK-HT04 [email protected] 1

ContentsIntroduction Requirements Engineering Project Management Software Design Detailed Design and Coding Quality AssuranceMaintenance

Analysis

Design

Testing

Coding

Operation andMaintenance

Installation

Require-ments

Requirements

Specification

Planning


Quality Assurance

Introduction Testingo Testing Phases and Approacheso Black-box Testingo White-box Testingo System Testing


What is Quality Assurance?

Constructive vs analytic approaches to QAQualitative vs quantitative quality standardsMeasuremento Derive qualitative factors from measurable

quantitative factorsSoftware Metrics

QA is the combination of planned and unplanned activities to ensure the fulfillment of predefined quality standards.


Approaches to QAConstructive Approaches

o Syntax-directed editorso Type systemso Transformational programmingo Coding guidelineso ...

Analytic approaches

o Inspectionso Static analysis tools (e.g. lint)o Testingo ...

Usage of methods, languages, and tools that ensure the fulfillment of some quality factors.

Usage of methods, languages, and tools to observe the current quality level.


Fault vs Failure

?!human error fault failure

can lead to can lead to

Different types of faultsDifferent identification techniquesDifferent testing techniques

Fault prevention and -detection strategies should be based on expected fault profile

[email protected] 6

Specification/requirements

Environment/support

Documen-tation OtherDesign Code

Fault origin: WHERE?

Missing Unclear Wrong Changed Better way

Fault mode: WHY?

Fau

lt t

ype:

WH

AT

? Requirementsor

specifications

Functionality

HW interface

SW interface

User interface

Functionaldescription

Test HW

Test SW

Integration SW

Developmenttools

Logic

Computation

Data handling

Moduleinterface/

implementation

Standards

(Inter-)Processcommunications

Data definition

Module design

Logic description

Error checking

Standards

HP´s Fault Classification


Fault Profile of a HP Division

6%

19%

5%

4%

5%

32%

18%

11%

Data handling

Documentation

Requirements

Hardware

Process/ interprocess

Logic

Computation

Other codeSee [Pfleeger 98].

PVK-HT04 8

Testing Phases

Unittest

Unittest

Unittest

Integrationtest

Functiontest

Performancetest

Acceptance(,) test

Installationtest

Com

ponent

code

Com

ponent

code

.

.

.

Teste

d co

mponent

Test

ed c

om

ponent

Integratedmodules

Functioningsystem

Verifiedsoftware

Accepted,validatedsystem

SYSTEMIN USE!

Designspecifications

Systemfunctional

requirements

Othersoftware

requirements

Customerrequirementsspecification

Userenvironment

Systemtest

Pre-implementationtest

9

Pre-Implementation Testing

Inspectionso evaluation of documents and code prior to technical

review or testingWalkthrougho In teamso Examine source code/detailed design

Reviewso More informalo Often done by document owners

Advantageso Effectiveo High learning effecto Distributing system knowledge

Disadvantageso Expensive


Testing vs “Proofing” Correctness

Verificationo Check the design/code against the requirements

Are we building the product right?Validationo Check the product against the expectations of the

customerAre we building the right product?

Testing

Testing can neither proof that a program is error free, nor that it is correct

[Myers 76]

Testing can only show the presence of errors, never their absence.

[Dijkstra 69]

Testing is the process in which a (probably unfinished) program is executed with the goal to find errors.

11

Testing PrinciplesConstruction of test suiteso Plan tests under the assumption to find errorso Try typical and untypical inputso Build classes of inputs and choose representatives of each

classCarrying out testso Testers implementerso Define the expected results before running a testo Check for superfluous computationo Check test results thoroughlyo Document test thoroughly

Simplify testo Divide programs in separately testable unitso Develop programs test friendly

Each test must be reproducible


Test MethodsStructural testing (white-box, glass-box)o Uses code/detailed design is to develop test caseso Typically used in unit testingo Approaches:

• Coverage-based testing• Symbolic execution• Data flow analysis• ...

Functional testing (black-box)o Uses function specifications to develop test caseso Typically used in system testingo Approaches:

• Equivalence partitioning• Border case analysis• ...

time

developblack-boxtest cases

developwhite-boxtest cases

performwhite-box

testing

performblack-box

testing


Test Preparation

Exhaustive testing is prohibited, because of the combinatorial explosion of test casesChoose representative test data

for i := 1 to 100 do if a = b then X else Y;

i paths to test #tests

1 X, Y 22 XX, XY, YX, YY 43 XXX, XXY, ... 8

...100 2100 2 2100 - 2 > 2,5 1030

With 106 tests/sec this would take 81016 yearsChoose test data (test cases)


Test Case DevelopmentProblems:o Systematic way to develop test caseso Find a satisfying set of test cases

Test case test dataTest data: Inputs devised to test the systemTest case:o Situation to testo Inputs to test this situationo Expected outputs Test are reproducible

Equivalence partitioning Coverage-based testing


Equivalence Partitioning

System

Input data

Output data

Inputs causinganomalous behaviour

Outputs which revealthe presence of faults

Input- and output data can be grouped into classes where all members in the class behave in a comparable way.

Define the classes Choose representatives

Typical element Borderline cases

x [25 .. 100] class 1: x < 25class 2: x >= 25 and x <= 100class 3: x > 100

PVK-HT04 17

Equivalence Partitioning Examples

oModule keeps a running total of amount of money receivedoInput: AmountofContribution, specified as being a value from $0.01 to $99,999.99

oModule prints an appropriate response letteroInput: MemberStatus, specified as having a value of {Regular, Student, Retiree, StudioClub}

o 3 equivalence classes:Values from $0.01 to $99,999.99

(valid)Values less than $0.01 (invalid)Values greater than $99,999.99

(invalid)

o 2 equivalence classes:Values of {Regular, Student, Retiree, StudioClub}

(valid)All other input (invalid)


Statement Coverage

1

3

2 4

5

6 7

8

2 test cases: 12467; 13567

Every statement is at least executed once in some test


Branch Coverage

1

3

2 4

5

6 7

8

2 test cases: 12467; 1358

For every decision point in the graph, each branch is at least chosen once


Condition Coverage

Test all combinations of conditions in boolean expressions at least onceif (X or not (Y and Z) and ... then b;c := (d + e * f - g) div op( h, i, j);


Path Coverage

Assure that every distinct paths in the control-flow graph is executed at least once in some test


Path Coverage

1

3

2 4

5

6 7

8



Path Coverage

1

3

2 4

5

6 7

8

4 test cases: 12467; 1358; 1248; 13567



Data-flow testing

Def-use analysis: match variable definitions (assignments) and uses. Example:

x = 5; … if (x > 0) ...

Does assignment get to the use?


Data Flow CoverageAll-uses coverage

x :=2

x :=1

z := 2*r

x :=3

z := 2*x-y

z :=x+y

y :=2r :=4

Red path covers the defs y :=2; r :=4; x :=1

Blue path covers y :=2; x :=3. Does not cover x :=2


Coverage-based TestingAdvantageso Systematic way to develop test caseso Measurable results (the coverage)o Extensive tool support

• Flow graph generators• Test data generators• Bookkeeping• Documentation support

Disadvantageso Code must be availableo Does not (yet) work well for data-driven

programs


Integration TestingHow to integrate & test the system

Top-downBottom-upCritical units firstFunctionality-oriented (threads)

Implications of build order Top-down => stubs; more thorough top-levelBottom-up => drivers; more thorough bottom-levelCritical => stubs & drivers.

A

DCB

GFE Test E Test F Test G

Test B,E,F Test C Test D,G

Test all


Further phases of testing

System testing: Test the functionality, performance, reliability, security of the entire system.Acceptance testing: Operate system in user environment, with standard user input scenarios. Regression testing: Test of modified versions of previously validated system.

30

Testing Tools / SupportTest data generatorso Input: Program + testing strategyo Output: Sets of input data

Profilerso Instrument code to collect run-time data

• Time spent in operations• Number of calls to operations• Number of loop iterations• ... Find bottle-necks Indicate dead code

Simulatorso Common in hard-/software systems and/or real-time

systemso Emulate critical parts by software


Testing Tools / Support

Debuggerso Manual code instrumentationo Inspect/trace variableso ...

File comparatorso E.g. for regression testing

Test-stub/-driver generatorso Simulate client or server components, which

are not yet availableClient

Server

[email protected] Contents Introduction Requirements Engineering Project Management Software...

Documents

Transcript of [email protected] Contents Introduction Requirements Engineering Project Management Software...