DR. HARIS MOURATIDIS02/11/08

Software Testing and Extreme Programming

1

© H. Mouratidis

© H. Mouratidis

2

What we do today?

Learning Objectives Provide an understanding of

Development and implementation of a testing strategy eXtreme Programming

Lecture Layout Testing

Concepts Testing strategies

Agile Development Extreme Programming

Testing

“Testing is the process of finding differences between the expected behaviour specified by system models and the observed behaviour of the implemented system” [Bruegge, 2004]

Another definition “Testing is a systematic attempt to find faults in a planned

way” Contrary to all other activities: find defects and reveal

problems It is impossible to completely test any nontrivial module or

any system Prohibitive in time and cost

“Testing can only show the presence of bugs, not their absence” (Dijkstra)

© H. Mouratidis

3

Some testing truths

Testing often viewed as dirty/easy work.To develop an effective test, one must have:

Detailed understanding of the system Knowledge of the testing techniques Skill to apply these techniques in an effective and efficient manner

Testing is done best by independent testers We often develop a certain mental attitude that the

program should work in a certain way when in fact it does not.

Programmers often stick to the data set that makes the program work "Don’t mess up my code!"

A program often does not work when tried by somebody else. Don't let this be the end-user.© H. Mouratidis

4

Testing concepts

Component Part (object, subsystem) of the system that can be

isolated for testing Fault (bug, defect)

Design or coding mistake that can cause abnormal component behaviour

Erroneous state A fault during the execution of a system

Failure Deviation between the specification and the

actual behaviour© H. Mouratidis

5

Testing concepts (II)

Test case Set of inputs and expected results that exercises

a component with the purpose of causing failures and detecting faults

Test driver Partial implementation of components that

depends on the tested component (calls the tested component)

Test stub Partial implementation of components on which

the tested component depends (are called by the tested component)

Correction A change to a component whose purpose is to

repair a fault© H. Mouratidis

6

Graphical Representation

© H. Mouratidis

7

Component

Test Driver

Test Stub

Depends on

Depends on

Test Case

Five attributes Name: name of the test case Location: full path name of executable Input: input data or commands Oracle: expected test results Log (output): output produced by the test

You have to specify also the associations!

© H. Mouratidis

8

Test Case Associations

© H. Mouratidis

9

Test Case A

Test Case E

Test Case W

Test Case D

Precede

As less associations as possible

Aggregation

Precede

Classification of test cases

© H. Mouratidis

10

Which aspect of system model is tested?

Blackbox test Focuses on the input/output of the component

Whitebox test Internal structure of the component

InputOutput

Testing technical activities

Component inspection Finds faults in individual components through

manual inspections of codeUsability testing

Differences between the system and the user’s expectations

Unit testing Finds faults by isolating individual components

using a test caseIntegration testing

Finds faults by integrating several components together

System testing Focuses on the complete system© H. Mouratidis

11

Component inspection

Find faults by reviewing source code in a formal meeting

Fagan’s inspection method Overview (Purpose and scope of component) Preparation

Reviewers become familiar with component code Inspection meeting

Identify the presence of faults Rework

Revise the component Follow-up

Check the revisionGood but Time consuming

Parna’s method Find faults during preparation© H. Mouratidis

12

Usability testing

Tests the user’s understanding of the system

Does not compare the system against specifications

Three types Scenario test

Users presented with visionary scenario of the system

Prototype test Users presented with a prototype version of the

system Product test

Users presented with a functional version of the system

Test objectives: identify useful features, compare user interaction styles, type of training information

© H. Mouratidis

13

Unit testing

Motivations Reduces complexity of overall test activities Makes it easier to identify/ correct faults Saves time, since each component can be tested

independently of the others

In principle all components should be tested!Not feasible!

© H. Mouratidis

14

Unit testing techniques

Equivalence testing (blackbox) Systems usually behave in similar ways for all

members of a class Test one member of the class If an object is supposed to accept a negative

number, testing one negative number is enough+ Minimises number of test cases- Do not explore combinations of test input data

Boundary testing Special case of equivalence testing Testing elements from the “edges”

© H. Mouratidis

15

Unit testing techniques (II)

Path testing (whitebox) By exercising all possible paths through the code

at least once, most faults will trigger failures Flow graph Test cases to check transitions

State based testing Recently developed for O-O systems Compares the resulting state of the system with

the expected state Not part of the state of practice

© H. Mouratidis

16

Integration testing

Suitable for finding faults on the interfaces between the components

Focuses on groups of components…whole system

Detects faults that have not been detected during unit testing by focusing on small group of components

Couple of components are integrated and tested…more added to the group

© H. Mouratidis

17

Integration testing strategies

Big bang testing All components tested individually, then all

together as a single system+ Simple strategy- Expensive!

Bottom-up testing Split system into layers Test each component of the bottom layer…then

integrates them with the components of the next layer up

+ Interface faults can be more easily found- Interface components are tested last

© H. Mouratidis

18

Integration testing techniques (II)

Top-down testing Similar to bottom-up…tests the components of

the top layer first…then integrates the components of the next layer down

+ Starts with user interface components…same set of tests as requirements

- Test stubs take time and are prone to errorsSandwich testing

Combination of top-down and bottom-up Systems is decomposed into three layers

Target layer (the meat!) Layer above the target layer (the top slice of bread) Layer below the target layer (the bottom slice)

Top-down and bottom-up can be done in parallel© H. Mouratidis

19

System Testing

Ensures the system complies with its functional and non functional requirements

Different activities Functional testing

Functional requirements Performance testing

Non-functional requirements Pilot testing

Selected set of users test the common functionality Acceptance testing

Tests against acceptance criteria Installation testing

Tests performed in the target environment

© H. Mouratidis

20

Functional testing

Differences between functional requirements and the system

Test cases are derived from the use case model

Quality of use cases determines the ease of functional testing

Different than usability testing Differences between use case model and system

behaviour not user expectations of the system

© H. Mouratidis

21

Performance testing

Finds differences between design goals and the system

Different kind of tests Stress testing

Simultaneous requests Volume testing

Large amount of data Security testing

Security faults Timing testing

Identify time constraints Recovery tests

Ability to recover© H. Mouratidis

22

Pilot/acceptance/installation tests

Pilot System is installed and used by selected users Alpha test: development environment Beta test: target environment

Acceptance Client tests the system

Installation Most times repeats previous tests in the target

environment

© H. Mouratidis

23

Plan testing

Testing should be left for the end of the project…Wrong!!!

Start testing as soon as possible! But remember to update!

Conduct testing activities in parallel But keep them consistent

In summaryStart early, update, keep testing consistent!

© H. Mouratidis

24

Testing life cycle

© H. Mouratidis

25

Establish the test objectives

Design the test cases

Write the test cases

Test the test cases

Execute the tests

Evaluate the test results

Change the system

Documenting Testing

Test Plan Documents the scope, approach, resources and

schedule of testing activities Requirements and components to be tested

Test Case Specification Inputs, drivers, stubs, expected outputs

Test incident report Actual results of tests and differences

Test report summary Lists all failures

© H. Mouratidis

26

Automated testing

Manual testing can be: Costly Error prone Time consuming

Automated testing is focused on executionMany infrastructures to support automated

testing JUnit: writes and executes unit tests for Java classes

© H. Mouratidis

27

A historical view

1963-1968 The emerging software crisis

1968-1970 Birth of software engineering

The most important goals were to get control of the software development process and to improve software performance and software reliability

28

© H. Mouratidis

What changed? Did it help?

The introduction of software engineering led to New approaches to the management of software

engineering New techniques of software development

The introduction of software engineering helped to Better understand the issues involved in the

development of systems Successfully build large, reliable systems

Software Engineering has worked! (??)

29

© H. Mouratidis

Software Development challenges

Shorter development lifecycles

More ambitious requirements

Less certain requirementsMore challenging

environments Dynamic systems Open systems Security issues

30

© H. Mouratidis

Dealing with the challenges

Better processes Faster development Light methods

Better models Abstractions Architectures Components

Agile Development

Agent Oriented Software Engineering

31

© H. Mouratidis

Agile development approaches

AD – Agile Database Techniques The goal of the Agile Data (AD) methodology is to define strategies

that IT professionals can apply in a wide variety of situations to work together effectively on the data aspects of software systems.

AM – Agile Modelling Agile Modeling (AM) is a practice-based methodology for effective

modeling and documentation of software-based systems. Simply put, Agile Modeling (AM) is a collection of values, principles, and practices for modeling software that can be applied on a software development project in an effective and light-weight manner.

XP – Extreme Programming Extreme Programming is a discipline of software development based

on values of simplicity, communication, feedback, and courage. It works by bringing the whole team together in the presence of simple practices, with enough feedback to enable the team to see where they are and to tune the practices to their unique situation

32

© H. Mouratidis

Extreme Programming - XP

“XP is a light weight methodology for small-to-medium-sized teams developing software in the face of vague or rapidly changing requirements” (Beck, 2000)

XP is based on four values Communication Simplicity Feedback Courage

33

© H. Mouratidis

Value 1: Communication

Good communication is very important for the successful completion of projects

XP aims to keep the right communication flowing by employing practices that cannot be done without

communicating employing a coach whose job is to notice when people

aren’t communicating and reintroduce them

34

© H. Mouratidis

Value 2: Simplicity

XP argues for a simplistic point of view“It is betting that it is better to do a simple thing

today and pay a little more tomorrow to change it if it needs it, than to do a more complicated thing today that may never be used anyway” (Beck, 2000)

Simplicity supports communication and vice versa The more you communicate the more you

understand about the system and what it doesn’t need to be done

The simpler your system is the less you have to communicate

35

© H. Mouratidis

Value 3: Feedback

According to XP feedback works at different time scales Scale of minutes/days

Feedback provided to customers, developers, progression

Scale of weeks/months Check the overall velocity of the project

Feedback works together with communication and simplicity The more feedback you have the easier is to

communicate

36

© H. Mouratidis

Value 4: Courage

Within the context of the first three values, courage is also needed

Throw code away when necessary Working on code for whole day, it doesn’t work, five

minutes the next morningCommunication supports courage because it opens

the possibility for more high-riskSimplicity supports courage because you can afford

to be much more courageous with a simple systemCourage supports simplicity because as soon as

you see the possibility of simplifying the system, you try it

37

© H. Mouratidis

The XP principles

The four values are too vague to give us much help in deciding which practices to use

We need principlesXP has a set of fundamental principles and a set of

“less central” principlesFundamental principles

Rapid Feedback Assume Simplicity Incremental change Embracing change Quality Work

38

© H. Mouratidis

Principle 1: Rapid Feedback

Research indicates that the time between an action and feedback is critical to learning

Important to get feedback, interpret it, put what is learned back to the system as soon as possible

39

© H. Mouratidis

Principle 2: Assume Simplicity

“Treat every problem as if it can be solved with simplicity”

“Do a good job of solving today’s job today and trust your ability to add complexity in the future where you need it.”

40

© H. Mouratidis

Principle 3: Incremental change

Making big changes at one, most likely it will not work

Everything should be changed incrementally through a series of smallest changes The plan changes a little at a time The design changes a little at a time The code changes a little at a time

41

© H. Mouratidis

XP practices

1. The planning game2. Small releases3. Metaphor4. Simple design5. Testing6. Refactoring7. Pair Programming8. Collective ownership9. Continuous Integration10.40-hour week11.On-site customer12.Coding standards

42

© H. Mouratidis

Practice 1: The planning game

The goal of the planning game is to maximise the value of software produced by the team

How XP does this? The customer provides business stories Developers estimate the time to build software to

implement the stories The customer decides which stories provide the most

business value The developers implement these stories

43

© H. Mouratidis

Practice 2: Small releases

Put a simple system into production quickly Release early and release often

Releases are not prototypes A release implements a couple of stories A release is thoroughly tested

44

© H. Mouratidis

Practice 3: Metaphor

Figure of speech used to refer to something that it does not literally denote to suggest a similarity

Guide all the development with a simple shared story of how the whole system works

To do this you need To have concrete feedback quickly about whether the

metaphor works Both the customer and the developers are comfortable

talking about the system in terms of the metaphor

45

© H. Mouratidis

Practice 4: Simple Design

The system should be designed as simply as possible at any given moment

Right (simple) design at any given time is one that Runs all the tests Has no duplicated logic States every intention important to programmers Has the fewest possible classes and methods

46

© H. Mouratidis

Practice 5: Testing

Programmers continually write unit tests Unit tests always run 100% If a test is broken, fixing it is the most important job

Customers write tests demonstrating that features are finished. What would have to be checked before I would be

confident that this story was done? Each scenario turns into a test

47

© H. Mouratidis

Practice 6: Refactoring

Programmers restructure the system without changing its behaviour

Used mainly to simplify code to make it easier to understand and maintain

After refactoring all the tests should still run 100%

E.g. If there is an one-minute way to get a test working, and a ten-minute way with a simpler design -> go for the ten-minute!

48

© H. Mouratidis

Practice 7: Pair Programming

All production code is written with two programmers at one machine

What pair programming is not? It is not one person programming and the other

watching

Dialog between two people that analyse, design, program, test and improve the program together!

49

© H. Mouratidis

Practice 8: Collective Ownership

Anyone can change any code anywhere in the system at any time

Without any tests, it would be a dead ideaImportant to run the tests after any changeIt tends to

Prevent complex code writing Spread knowledge of the system

50

© H. Mouratidis

Practice 9: Continuous Integration

Integrate and build the system many times a day, every time a task is completed

Is this possible?It is if:

You run the tests quickly so you know you haven’t broken anything

You program in pairs, so there are half as many streams of changes to integrate

You refactor, so there are more smaller pieces, reducing the chance of conflicts

51

© H. Mouratidis

Practice 10: 40-hour week

Work no more than 40 hours per week as a rule Never work overtime a second week in a row Can this work?

Valuable work is identified by the Planning Game

The combination of the Planning game and the testing reducing the frequency of surprises

The practices help you program at top speed

52

© H. Mouratidis

Practice 11: On site customer

Include a real, live user on a team, available to answer questions

Very difficult to have a full time on-site customer!

But if you can…They can produce value for the project

by writing functional tests By making priorities and scope decisions for

programmers.

53

© H. Mouratidis

Practice 12: Coding standards

Programmers write all code in accordance with rules emphasizing communication through the code

Difficult because Individual programmers Different styles

Be part of a winning team

54

© H. Mouratidis

Roles on an XP project

7 different roles Programmer Customer Tester Tracker Coach Consultant Big Boss

55

© H. Mouratidis

Programmer

The programmer is at the heart of XPLooks like being a programmer within other

software development disciplinesExtra skills required

Pair programming Simplicity Refactor Unit test Set aside the feeling of individual ownership

56

© H. Mouratidis

Customer

Being an XP customer is not easyThere are skills to be learned

Writing good stories The team can help with continuous feedback

Make decisions You have to be confident enough to decide fast for

something Write tests

You have to work close with the team to learn the kind of tests which are helpful

Spend time You need to be available …always

57

© H. Mouratidis

Tester

Mainly help customer choose and write testsRun all the tests regularlyBroadcast the test resultsMake sure the test tools run well

58

© H. Mouratidis

Tracker

Tracks and provides feedback regarding estimates Lots of practice and feedback Usually at the beginning you do a lot of estimations

and see how reality conformed to your guessesKeep an eye on the bigger picture, i.e. the

project as a wholeThe most important skill is to get the

necessary information without disturbing the rest of the team too much!!!

59

© H. Mouratidis

Coach

Responsible for the process as a wholeYou notice when people are deviating from

the team’s processes and bring this to the team’s attention

Working on a thin line Every time you guide the team, you make them that

much less self-reliant Too much steering and they lose the ability to work

with you On the other side, you must be direct sometimes

60

© H. Mouratidis

Consultant

Consultants are not always needed in XP projects

When are needed they provide deep technical knowledge

Not solve the problem alone, but help the team to solve it

61

© H. Mouratidis

Big Boss

Provide courageProvide confidencePersistence to check that the team does what

they say they doCorrect the team, if what they do does not

make sense

62

© H. Mouratidis

References about XP

Extreme Programming explained, Kent Beck, (005.1 BEC – library)

http://www.extremeprogramming.org/index.html

Extreme programming for Web projects, Wallace et al., (005.72 – Library)

Various web pages

63

© H. Mouratidis

Conclusions

Testing Concepts Testing strategies

Agile Development Extreme Programming

© H. Mouratidis

64